Test method and computing device
By grouping and sorting test cases, dynamically creating test environments, solving the problems of complex preparation of operating system test environments and wasting resources, and achieving efficient and accurate test environment configuration and resource management.
Patent Information
- Application Number
- CN202411035882.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-12-03
AI Technical Summary
The operating system test environment is complex and time-consuming to prepare, and it is impossible to accurately predict the number of environments required, resulting in wasted resources or excessive testing time.
By obtaining the hardware resource requirements, grouping and sorting of test cases, determining mutex arrays and containing arrays, dynamically creating test environments, ensuring that each test case collection meets its needs, and optimizing resource configuration and allocation.
Accurately configure the test environment, reduce resource waste, improve testing efficiency, avoid resource shortages and performance bottlenecks, and realize automated management and dynamic adjustments.
Smart Images

Figure CN119065963A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of server technology, and in particular to a testing method and computing equipment. Background Art
[0002] Currently, the environment preparation for operating system testing is more complicated than the environment deployment of application software, because the operating system involves more system-level configuration and settings. Usually, the required operating system test environment needs to be manually prepared before the test case for the operating system is executed. This manual environment preparation method is very time-consuming and cannot accurately prepare the required number of environments. If the number of environments is too small, the test time will be too long; and excessive environment preparation will waste resources. Summary of the invention
[0003] A testing method and a computing device are provided in an embodiment of the present application, which can provide a suitable number of testing environments to reduce the waste of testing environment resources and improve testing efficiency.
[0004] To this end, the following technical solutions are provided in the embodiments of the present application:
[0005] In a first aspect, an embodiment of the present application provides a testing method, comprising: obtaining a test task, wherein the test task includes multiple test cases; obtaining a test environment requirement for each of the test cases; determining multiple first test case sets based on the test environment requirement information, wherein the test environment requirements of each test case included in one of the first test case sets are the same; determining a mutually exclusive array and at least one inclusion array based on the multiple first test case sets; the mutually exclusive array includes at least one first test case set, and there is no inclusion relationship between different first test case sets; the inclusion array includes multiple first test case sets, and there is an inclusion relationship between different first test case sets; creating a corresponding test environment for each first test case set in the mutually exclusive array, and assigning each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set; creating a corresponding test environment for each of the inclusion arrays, and assigning all first test case sets in each inclusion array to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set; and executing the test task.
[0006] In this implementation, first, multiple test cases are obtained according to the test requirements of the operating system, and these test cases require a specific test environment when executed. Test cases with the same test environment requirements are grouped, which helps to preliminarily optimize the utilization of the environment and avoid waste of resources. Optimization is performed for test cases in different groups. For example, the set inclusion relationship of the disk information required is calculated to determine the overlapping relationship of the disk information, thereby accurately determining the number of test environments required to be configured, so as to effectively configure the test environment. In this way, the test cases are reasonably grouped and sorted according to the test environment requirements of the test cases, and the first test case set is optimized. After obtaining the required test environment for the operating system test case, the accurate number of test environments can be obtained to reduce the waste of test environment resources and improve test efficiency.
[0007] As an achievable implementation method, the test environment requirements include processor, memory and disk information.
[0008] In this implementation, the test environment requirements usually involve the configuration of hardware resources (such as processors, memory, and disks) to ensure that the test environment can effectively simulate the operating environment and support the required test activities. For example, by configuring a high-performance processor, a large-capacity memory, and a high-speed disk, the test environment can better simulate the performance level in the system operating environment, thereby more accurately evaluating the performance of the system in actual use.
[0009] As an achievable implementation, determining a mutually exclusive array and at least one inclusive array according to a plurality of the first test case sets includes: determining a mutually exclusive array and at least one inclusive array based on disk information in the test environment required by the first test case.
[0010] As an achievable implementation method, the disk information includes the number of disks and the size of disks; the determination of a mutually exclusive array and at least one inclusion array based on the disk information in the test environment required by the first test case includes: traversing each first test case set; in the case where there is an inclusion relationship between a first test case set and the number of disks and the size of disks required by any first test case set, the first test case set and the first test case set with the inclusion relationship are taken together as an inclusion array; in the case where there is no inclusion relationship between a first test case set and the number of disks and the size of disks required by any first test case set, the first test case set is placed in a mutually exclusive array.
[0011] In this embodiment, in a test environment with multiple test cases, disk information is usually a relatively stable and important factor in the test environment configuration, affecting the storage and data processing capabilities of the test cases. Therefore, screening based on disks can more easily divide a set of test cases that can share the same disk resources, thereby providing a more stable and reliable test environment division, helping to ensure that test cases can run under similar test environment conditions, thereby improving test efficiency and repeatability.
[0012] As an achievable implementation method, the test environment requirements include processor, memory and disk information; the method also includes: when the disk information in the test environment of the test case of the first test case set is 0, based on the processor and / or memory configuration information in the test environment requirements of the first test case set, and the processor and / or memory configuration information of the test environment requirements of the first test case set in the mutex array and the inclusion array, the first test case set with disk information of 0 is allocated to the first test case set in the mutex array and the inclusion array.
[0013] In this embodiment, when the number of disks and the disk size in the test environment are both 0, the first test case set is allocated to other first test case sets based on the number of processor cores and / or memory configuration information in the test environment to ensure that subsequently merged test cases can obtain computing resources (processors and memory) that meet their needs, avoid resource waste and shortages, and at the same time, ensure that all test cases can run efficiently and shorten the overall testing time.
[0014] As an achievable implementation method, the creating a corresponding test environment for each first test case set in the mutually exclusive array includes: creating the number of processor cores of the test environment based on the number of processor cores in the test environment requirements of the first test case set, creating the memory size of the test environment based on the memory size in the test environment requirements of the first test case set, and creating the disk information of the test environment based on the disk information in the test environment requirements of the first test case set.
[0015] In this embodiment, the test environment can configure the number of processor cores, memory size and disk information according to the actual test environment requirements of each first test case set, thereby maximizing performance, thereby ensuring efficient execution of the test and avoiding test execution problems caused by insufficient hardware resources.
[0016] As an achievable implementation method, the creation of a corresponding test environment for each of the included arrays includes: creating disk information for the corresponding test environment based on the disk information most required by the first test case set in the included array, creating the number of processor cores for the corresponding test environment based on the maximum number of processor cores required by the first test case in the included array, and creating the memory size for the corresponding test environment based on the maximum memory size of the first test case in the included array.
[0017] In this implementation, by creating a test environment based on the maximum number of processor cores, memory size, and disk requirements in the first test case set in each inclusion array, it is ensured that the test environment is sufficient to meet the maximum requirements of each test case. In this way, only one test environment needs to be created for each inclusion array, and no additional redundant environment or excessive resource configuration is required, saving costs and resources, thereby avoiding resource waste and performance bottlenecks.
[0018] As an achievable implementation method, creating a corresponding test environment for each of the inclusion arrays includes: determining whether the total number of test cases of multiple first test case sets in each inclusion array is less than or equal to a preset number; when the total number of test cases of multiple first test case sets in each inclusion array is less than or equal to the preset number, creating a test environment for each inclusion array and allocating all first test case sets in each inclusion array to the corresponding test environment; when the total number of test cases of multiple first test case sets in each inclusion array is greater than the preset number, creating a number of test environments corresponding to the ratio of the total number / the preset number for the first test case sets in the inclusion array.
[0019] In this embodiment, by calculating the comparison between the total number of test cases and the preset number, resource allocation can be effectively optimized. When the number of test cases does not exceed the preset number, the number of test environments can be minimized to save resources; when the number of test cases exceeds the preset number, the test environment is dynamically increased according to the ratio to ensure that the load of each test environment is reasonable. Moreover, the number and configuration of the test environment are dynamically adjusted according to the actual situation, and the test requirements of different scales and complexities can be flexibly responded to. This flexibility can improve the efficiency and reliability of test execution. Finally, by accurately calculating and allocating test resources, resource waste and over-configuration can be avoided, thereby ensuring the efficient execution and accuracy of the test process. Each test environment can run its assigned test cases under appropriate conditions, avoiding the problems of resource contention and performance bottlenecks. At the same time, this solution can use automated tools to achieve dynamic management and configuration of the test environment, reduce the complexity and possibility of errors in manual adjustments, and improve overall management efficiency and quality control.
[0020] As an achievable implementation, the method further includes: releasing the test environment when the test case is executed successfully; retaining the test environment when the test case fails to be executed, and releasing the test environment when the retention time reaches a preset time.
[0021] In this implementation, when a test case fails to execute, the test environment will be retained for a preset time. This is done to facilitate subsequent debugging and analysis of the cause of the failure, while avoiding the additional overhead and time consumption caused by frequent creation and destruction of the environment. On the contrary, when the test case is successfully executed, the used test environment will be released immediately. In this way, resources can be reallocated to other test tasks to be executed to maximize resource utilization and ensure that resources are not wasted due to not releasing environments that are no longer needed.
[0022] As an achievable implementation, the test environment includes one or more of network card, GPU or SATA configuration information.
[0023] In this implementation, by combining the network card, GPU, and SATA configuration, the test environment can simulate a variety of usage scenarios to more comprehensively evaluate the performance and stability of the system.
[0024] In a second aspect, an embodiment of the present application further provides a testing device, including:
[0025] Acquisition module: used to acquire a test task, the test task includes multiple test cases, and acquire the test environment requirements of each test case;
[0026] A grouping module: used for determining a plurality of first test case sets according to the test environment requirement information, wherein the test environment requirements of the test cases included in one of the first test case sets are the same;
[0027] Construction module: Determine a result array mutually exclusive array and at least one inclusive array according to a plurality of the first test case sets; the mutually exclusive array includes at least one first test case set, and there is no inclusion relationship between different first test case sets; the inclusive array includes a plurality of first test case sets, and there is an inclusion relationship between different first test case sets; the construction module is also used to create a corresponding test environment for each first test case set in the mutually exclusive array, and assign each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set; create a corresponding test environment for each of the inclusive arrays, and assign all first test case sets in each inclusive array to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set;
[0028] A test module executes the test task.
[0029] As an achievable implementation manner, the construction module determines a mutually exclusive array and at least one inclusive array based on the disk information in the test environment required by the first test case.
[0030] As an achievable implementation method, the construction module is also used to traverse each first test case set; when there is an inclusion relationship between a first test case set and the number of disks and the disk size required by any first test case set, the first test case set and the first test case set with the inclusion relationship are taken together as an inclusion array; when there is no inclusion relationship between a first test case set and the number of disks and the disk size required by any first test case set, the first test case set is placed in a mutually exclusive array.
[0031] As an achievable implementation method, the construction module is also used to create the number of processor cores of the test environment based on the number of processor cores in the test environment requirements of the first test case set, create the memory size of the test environment based on the memory size in the test environment requirements of the first test case set, and create the disk information of the test environment based on the disk information in the test environment requirements of the first test case set.
[0032] As an achievable implementation method, the construction module is also used to create disk information of the corresponding test environment based on the disk information most required by the first test case set in the array, create the number of processor cores of the corresponding test environment based on the maximum number of processor cores required by the first test case in the array, and create the memory size of the corresponding test environment based on the maximum memory size of the first test case in the array.
[0033] As an achievable implementation method, the construction module is also used to determine whether the total number of test cases of each of the multiple first test case sets in the containing array is less than or equal to a preset number; when the total number of test cases of each of the multiple first test case sets in the containing array is less than or equal to the preset number, create a test environment for each containing array, and allocate all the first test case sets in each containing array to the corresponding test environment; when the total number of test cases of each of the multiple first test case sets in the containing array is greater than the preset number, create a test environment for the first test case sets in the containing array according to the ratio of the total number / the preset number.
[0034] As an achievable implementation method, the testing device also includes: a monitoring module: used to monitor the execution results of the test case in real time; when the test case is executed successfully, releasing the test environment; when the test case fails to execute, retaining the test environment, and, when the retention time reaches a preset time, releasing the test environment.
[0035] In a third aspect, an embodiment of the present application further provides a computing device, including: a memory and a processor;
[0036] Memory and processor coupling;
[0037] Memory, used to store programs;
[0038] A processor is used to call a program stored in a memory so that a computing device executes the method involved in the first aspect and possible implementations thereof.
[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions in the computer-readable storage medium are executed by a computing device, the computing device executes the method involved in the first aspect and its possible implementation methods.
[0040] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computing device, the computing device executes the method described in the first aspect and its possible implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 A schematic diagram showing an application scenario of the test method provided in an embodiment of the present application is shown;
[0042] Figure 2 A schematic diagram of a testing method provided in an embodiment of the present application is shown;
[0043] Figure 3 A flowchart for dividing a plurality of first test case sets into mutually exclusive arrays and inclusive arrays is shown;
[0044] Figure 4 A schematic diagram of the process of creating a corresponding test environment for each containing array is shown;
[0045] Figure 5 A flowchart of another testing method provided in an embodiment of the present application;
[0046] Figure 6 A schematic diagram of the test method provided in Example 1 of the present application;
[0047] Figure 7A schematic diagram of the structure of a testing device provided in an embodiment of the present application;
[0048] Figure 8 A schematic diagram of the structure of another computing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0050] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.
[0051] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.
[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.
[0053] In the description of the embodiments of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0054] In the description of the embodiments of the present application, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.
[0055] In the description of the embodiments of the present application, the numbers representing the steps, such as S31, S32, etc., do not necessarily mean that the steps will be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.
[0056] Relevant terms involved in the embodiments of this application:
[0057] Test environment: refers to the description of the operating system and hardware environment on which the test is run, as well as any other software that interacts with the tested operating system, including hardware, software, and network configurations. This environment is usually specially configured so that developers, testers, or automated testing tools can perform various tests in it, including unit testing, integration testing, system testing, etc. A stable and controllable test environment allows testers to complete the execution of test cases in less time, without spending extra time on the maintenance of test cases and test processes, and can ensure that every defect submitted can be accurately reproduced at any time.
[0058] Number of test environments: refers to the number of different independent test environments required for operating system testing. Generally, the number of test environments depends on the following factors: Parallel testing requirements. If multiple independent test tasks need to be run simultaneously (such as parallel testing), multiple test environments are required. The resources required for each test environment (such as CPU, memory, storage space) and their availability will affect the number of test environments. By increasing the number of test environments, the efficiency and speed of testing can be improved, thereby obtaining test results faster. The number of test environments is determined based on the scale, characteristics and resource requirements of the test tasks, and is intended to support effective, efficient and reliable operating system testing activities.
[0059] Test cases: Test cases for operating system testing refer to specific test scenarios or test steps used to verify and evaluate the functions, performance, and stability of an operating system. Specifically, they describe the test tasks for a specific operating system, reflecting the test plans, methods, techniques, and strategies. When using test cases to test an operating system, each test case should specifically describe one or more expected behaviors, as well as the inputs and execution conditions required to trigger these behaviors. Simply put, a test case is a set of inputs, execution conditions, expected results, and execution steps used to verify specific functions, features, or performance during the operating system testing process. Test cases are designed to verify whether the operating system works as expected and can be used to discover potential defects or problems.
[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0061] With the continuous advancement of computer technology, operating systems are becoming more and more complex, and the operating system software and functions involved are becoming more and more complicated. The probability of operating system failures increases accordingly, and it becomes more difficult to discover and solve failures. Therefore, it is necessary to use test cases to test the operating system to verify its functions, compatibility and performance.
[0062] Normally, the required test environment needs to be manually prepared before the test cases for operating system testing are executed. For example, the current mainstream process in the industry is for testers to evaluate the number of test environments required for their own test cases, and then manually deploy and create an operating system test environment, then deploy the test framework and execute the test, and manually delete the operating system and test environment after the test. Manual preparation of the test environment requires testers to perform operations one by one, including installing the operating system, configuring the network, installing software packages, adjusting system settings, etc. This method is time-consuming and error-prone, especially in large-scale test environments, where manual management becomes impractical. Moreover, manually preparing the environment usually cannot accurately predict the number of environments required for testing. If the number of environments is insufficient, it may lead to insufficient test resources and extend the test time; while too many environments will waste resources.
[0063] Based on this, the embodiment of the present application provides a test method and computing equipment. In this method, by obtaining the requirements for the test environment when executing the test case itself, and the hardware resource information (for example, processor (central processing unit, CPU), memory (memory), disk (disk), network card (netcard), etc.) of the test environment required by the test case is arranged and combined to calculate the precise number of test environments required for the test case, and automatically deploy and create an operating system test environment according to the calculated number of test environments. In this way, the number of test environments can be accurately provided in the case of test environment diversity to reduce the waste of resources in the test environment and better improve the test efficiency.
[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0065] Figure 1 The following is a schematic diagram showing an application scenario of the test method provided in the embodiment of the present application. Figure 1 As shown, the application scenario includes a management device 101 and multiple test devices 101. Although the management device 101 and the test device 101 are different electronic devices, they have the same architecture type.
[0066] It should be noted that the management device 101 is interconnected with multiple test devices 101, wherein the management device 101 and the test device 101 establish a communication connection through a wired network or a wireless network to exchange data. For example, the network can be a local area network (LAN) or a wide area network (WAN) (such as the Internet). The specific embodiment of the present application does not limit the communication connection method between the management device 101 and the test device 101.
[0067] Exemplarily, the management device 101 may be a server or a terminal device, and the management device 101 is used to provide multiple test cases based on the test task, and the test cases carry their own required test environment requirements. The management device 101 may obtain the test environment requirements of each test case, and determine multiple first test case sets according to the test environment requirement information, wherein the test environment requirements of each test case included in any first test case set are the same.
[0068] The management device 101 is also used to determine a mutually exclusive array and an inclusive array according to multiple first test case sets; the mutually exclusive array includes at least one first test case set, and there is no inclusive relationship between different first test case sets in the mutually exclusive array; the inclusive array includes multiple first test case sets, and there is an inclusive relationship between different first test case sets in the inclusive array. The management device 101 respectively distributes the first test case sets in the mutually exclusive array and the inclusive array to different test devices 101 according to different test environment requirements of the test cases.
[0069] The test device 101 involved in the embodiment of the present application is used to provide the test environment required by the first test case set, and different test devices 101 are used to provide different test environments to meet the test cases in different first test case sets to execute the test tasks. Exemplarily, the test device 101 can be a hardware server or a virtualized server. For example, the test environment in the test device 101 can be configured by a virtual machine executed on a hardware server including one or more other virtual machines.
[0070] The above is an introduction to the application scenarios of the execution test method involved in this solution. Next, based on the above content, the process of the test method provided in the embodiment of the present application is introduced.
[0071] Figure 2 FIG. 1 is a flow chart of a test method provided by an embodiment of the present application. The scheduling method is suitable for execution in a computing device (such as the aforementioned management device 101). Figure 2 As shown, the test method includes:
[0072] S101. Obtain a test task, where the test task includes multiple test cases.
[0073] In this step, multiple test cases to be executed can be obtained based on the test task of the operating system. It can be understood that in the test task based on the operating system, "test task" usually refers to an overall test process or plan, which includes the execution and management of multiple test cases. Each test task may be a batch job or a group of related test operations, which is intended to evaluate the performance, stability or function of the operating system under specific conditions.
[0074] The test case is the basic execution unit of the test task. Each test case describes a specific test behavior or function point, usually including input data, expected output, execution steps and other information. It should be noted that the test case, test results and other data can be stored in a database such as a relational database (such as MySQL, PostgreSQL) or a NoSQL database (such as MongoDB).
[0075] S102: Obtain the test environment requirement of each test case.
[0076] It should be noted that the test case carries its own test environment requirement information, where the test environment includes hardware environment resource information such as the number of processor cores, memory size, disk information, etc. required to be configured when the test case is executed.
[0077] Exemplarily, each test case carries information about the test environment required for execution. This information may include: operating system configuration: such as operating system type (Windows, Linux, etc., in this embodiment, Linux operating system), version number; hardware configuration: such as the number of CPU cores, memory size, disk information, network card settings, etc.; wherein the disk information describes the number of disks and disk space, read and write speeds, etc. required during test execution. This information is crucial for the execution of certain test cases, such as test cases that require a large amount of data storage or IO operations.
[0078] Specifically, when writing test cases, developers or testers can record the test environment information required for each test case, including the operating system, software version, configuration parameters, network settings, etc. In other words, when writing test cases, the environmental requirements of the operating system are recorded using fixed data results. This information can be embedded in the test case in a binary manner (a language that the computing device can recognize) in the form of comments, documents, or specific tags (such as keywords) for subsequent use. This embodiment can be used to obtain hardware resource configuration information in the test environment by identifying the keywords of each test case, such as the number of CPU cores, memory size, disk information, network card settings, etc. described above. Of course, in some possible implementations, the hardware resource configuration information required for the test environment may also include:
[0079] Network card (netcard): High-quality network card configuration can ensure the stability and speed of network communication, which is crucial for testing distributed systems, network services, or applications that require high bandwidth.
[0080] GPU (graphics processing unit) information: For test cases that require graphics processing or computationally intensive tasks, the GPU model, video memory size, and driver version may be necessary information.
[0081] USB and other interface information: If the test requires the use of specific USB devices or other interfaces, such as SATA (serial advanced technology attachment), Thunderbolt, etc., the relevant interface type and version information needs to be recorded.
[0082] In this way, S102 can obtain the test environment requirement information when the test case is executed based on the test environment bound to the test case.
[0083] S103: Determine multiple first test case sets according to the test environment requirement information of the test cases, wherein the test environment requirements of the test cases included in one first test case set are the same.
[0084] In this step, multiple test cases are arranged and combined, and test cases with the same test environment are grouped together. This is used to preliminarily integrate the test environment required for the operating system test cases. By calculating the arrangement and combination of these test environment information, it can be determined how many operating system test cases can be deployed in a given test environment. Specifically, the test cases with the same test environment requirements can be grouped according to the processor, memory, disk, and network card configuration information required in the test environment, and the test cases can be grouped into the same group, so as to preliminarily determine the number of test environments required for the operating system test. Afterwards, the test cases in the same group are sorted to obtain the number of test cases in the same group.
[0085] For example, if there are 25 test cases, they can be divided into several groups, and the test cases in each group have the same test environment requirements, see Table 1 for details.
[0086] Table 1 Example of test case grouping with the same test environment and the cpu, mem and disk information required in the test environment
[0087]
[0088] As can be seen from Table 1, the 25 test cases are divided into 6 groups of first test case sets according to different test environment requirements, where the test environment information in Table 1 includes processor cpu, memory mem and disk disk. Specifically, in the cpu information, each row represents the cpu configuration of a test environment, where the numbers 2, 4, and 8 represent the number of cpu cores. For example, the first group of test environments has 2 cpu cores, the second group of test environments has 4 cpu cores, and so on. Memory information: Each row represents the memory configuration of a test environment, and each number represents the memory size in GB. For example, the first test environment has 4GB memory, the second test environment has 4GB memory, and so on. Disk information: Each row represents the disk configuration of a test environment. The disk information is separated by commas, and each number represents the size of a disk in GB. For example, the first column of test environments has a 50GB and a 30GB disk, the second column of test environments has a 30GB and a 10GB disk, and so on. Test case information: Each line represents a test case in a test environment. Test cases are listed in order, and each line contains one or more test case names. For example, the first test environment contains two test cases, testcase1 and testcase2, the second test environment contains three test cases, testcase3, testcase4, and testcase5, and so on. The information shown in Table 1 can be used to configure and manage the test environment to preliminarily determine the number of test environments, thereby ensuring that the test cases can be executed and verified in the test environment they require.
[0089] In this way, the test cases with the same processor, memory and disk requirements are classified into the same group, which helps the test cases in the same group to share the same test environment, thereby preliminarily determining the number of test environment configurations required to reduce the waste of resources and improve test efficiency. In some embodiments, the test cases in the same group are sorted according to factors such as their priority, execution time, and dependencies to determine the execution order. At the same time, the number of test cases in each group is counted to facilitate resource allocation and the formulation of test plans. In this way, the test cases of the same test environment can share the same set of resource configurations, so that hardware resources (such as processors, memory and disks) can be managed and allocated more finely, and resource utilization can be improved. Moreover, through reasonable sorting and grouping, the test cases of the same test environment are concentrated and executed in the same batch, reducing the frequent switching of the test environment configuration, thereby saving time and resources. For example, for the test cases of the same group, it can be executed continuously, reducing the preparation time before and after the execution of a single test case, while effectively reducing the waiting time between the test cases, and improving the test execution efficiency as a whole.
[0090] S104. Determine a mutually exclusive array and at least one inclusion array according to multiple first test case sets; wherein the mutually exclusive array includes at least one first test case set, and there is no inclusion relationship between different first test case sets in the mutually exclusive array; and the inclusion array includes multiple first test case sets, and there is an inclusion relationship between different first test case sets in each inclusion array.
[0091] This step is mainly used to integrate the multiple first test case sets preliminarily divided in step S103 to obtain a more accurate number of test environments.
[0092] Exemplarily, when the test environment of the test case in the third test case set includes the test environment of the test case in the fourth test case set, the third test case set and the fourth test case set are merged in the same inclusion array, wherein the third test case set and the fourth test case set are any one of the multiple first test case sets. It should be noted that the inclusion relationship in this embodiment means that the parameters of a certain test environment of a first test case set include the parameters of a certain test environment of another first test case set. Referring to Table 1, exemplarily, the above parameters can be disk information, and the disk information includes the number of disks and the size of the disk. For example, the disk requirement information in the test environment requirements of the fourth test case set is 10GB (the 6th column in Table 1), and the disk information in the test environment requirements of the third test case set is 2 disks and the sizes of the 2 disks are 50GB and 10GB respectively (the 3rd column in Table 1), then at this time, the 50GB and 10G disk information of the test environment requirements of the third test case set includes the 10GB disk information in the test environment requirements of the fourth test case set, because 50GB and 10GB include 10GB. At this time, the third test case set and the fourth test case set are merged into the same inclusion array. Similarly, the disk information required in the first test case set in the 4th column of Table 1 is 50GB, 10GB and 20GB, which includes the disk information 50GB, 10GB in the 3rd column and the disk information 10GB in the 6th column. Then the first test case set in the 4th column of Table 1 is merged into the inclusion array where the 3rd column and the 6th column are merged. That is, the test environment requirements between different first test case sets in the inclusion array have an inclusion relationship.
[0093] In another case, if the test environment of the test cases in the third test case set does not include the test environment of the test cases in the fourth test case set, the third test case set is divided into a mutually exclusive array. The third test case set and the fourth test case set are any one of the first test case sets. In other words, the test environment requirements required between any two first test case sets in the mutually exclusive array do not include each other. For example, the disk information required for the first test case set in the first column of Table 1 is 50GB and 30GB, and the disk information required for the first test case set in the second column is 30GB and 10GB. It can be seen that the disk information of the test environment information in the first test case sets in the first and second columns of Table 1 does not include each other, so the first test case sets in the first and second columns are merged into the same mutually exclusive array. Thus, in this step, the inclusion relationship between the third test case set and the fourth test case set can be determined based on the number and size of disks in the test environment of the test cases in the third test case set and the number and size of disks in the test environment of the test cases in the fourth test case set, and a mutually exclusive array and at least one inclusion array can be determined based on whether there is an inclusion relationship between the third test case set and the fourth test case set.
[0094] In a possible implementation, sorting and screening can be performed mainly based on disk information, and the first test case set requiring different disk information can be divided into mutually exclusive arrays (mutually non-inclusive groups) and inclusive arrays. The reason is that in a test environment with multiple test cases, disk information is usually a relatively stable and important factor in the test environment configuration, affecting the storage and data processing capabilities of the test cases. In contrast, the selection of processor (CPU) and memory (mem) may be more affected by the computing requirements and memory usage of the test cases, and these factors may vary greatly with different test cases. Screening mainly based on processor or memory may result in large changes in the combination, making it difficult to determine which test cases can run stably in the same test environment. Therefore, screening mainly based on disk can more easily divide a set of test cases that can share the same disk resources, thereby providing a more stable and reliable test environment division, helping to ensure that test cases can run under similar test environment conditions, thereby improving test efficiency and repeatability.
[0095] In some embodiments, Figure 3 A flowchart for dividing a plurality of first test case sets into mutually exclusive arrays and inclusive arrays is shown. Figure 3 As shown, multiple first test case sets can be divided into multiple first test case sets by following the steps below:
[0096] S1041. Traverse each first test case set.
[0097] In this step, the disk information in the test environment required by a first test case set is defined as an array. Exemplarily, the disk information in the test environment includes the number of disks and the size of the disks. For details, please refer to Table 1. Exemplarily, the disk information 50, 10, and 20 in the fourth group of test environments in Table 1 can be used as an array, indicating that the disk information requires three disks, and the sizes of the disks are 50GB, 10GB, and 20GB, respectively. All arrays (such as the 6 groups of arrays in Table 1) are used as original arrays, and the original array is used as a collection of all arrays, and is traversed from the first test case set with the largest number of disks, so that the subsequent collections include calculation steps.
[0098] In a possible implementation, the length of the array with the largest number of disks is generally longer, and the traversal starts from the array with the largest length. This is because the more disks there are, the greater the possibility of including the contents of other arrays. For example, the fourth group of disk information data (50, 10, 20) includes the fourth group of disk information data (50, 10) and the sixth group of disk information data (10). Therefore, the merging of arrays can be completed with a smaller number of inclusion judgments, thereby reducing the number of traversals as much as possible when merging arrays, and improving the efficiency of the algorithm. For example, in this step, the array with the fourth group of disk information data (50, 10, 20) is first added to the mutually exclusive array.
[0099] S1042, determine the inclusion relationship between the test environments required by multiple first test case sets. In one case, if the test environments required by multiple first test case sets have an inclusion relationship, execute S1043. In another case, if the test environments required by multiple first test case sets do not have an inclusion relationship, execute S1044. Specifically, the inclusion relationship of the disk information can be determined based on the number and size of disks required for the test environments in different first test case sets. For example, the 4th group of disk information data in Table 1 is (50, 10, 20). By comparing the disk information data of other groups in Table 1, we can obtain that the 3rd group of disk information data (50, 10) and the 6th group of disk information data (10) are both included in the 4th group of data. At this time, the 3rd, 4th, and 6th groups of disk information are said to have an inclusion relationship. In Table 1, although the first set of disk information data (50, 30) and the second set of disk information (30, 10) have a common disk of 30GB, the first set of disk information also requires a 50GB disk and the second set of disk information also requires a 10GB disk, making the two mutually exclusive and not inclusive. Similarly, the first and second sets of disk information also have mutually exclusive relationships with the third, fourth, and sixth sets of disk information because they require a 30GB disk.
[0100] S1043: When there is an inclusion relationship between a first test case set and a test environment required by any first test case set, the first test case set and the first test case set having the inclusion relationship therewith are taken together as an inclusion array.
[0101] Exemplarily, referring to Table 1, the first test case set of the 3rd, 4th and 6th groups having the inclusion relationship is divided into an inclusion array.
[0102] S1044: When there is no inclusion relationship between a first test case set and any test environment required by the first test case set, the first test case set is put into a mutually exclusive array.
[0103] Exemplarily, referring to Table 1, if the first test case sets of Group 1 and Group 2 do not contain the disk information in the test environment required by the first test case sets of other groups, the first test cases of Group 1 and Group 2 are put into a mutually exclusive array.
[0104] In this way, by performing set inclusion calculation on the disk information, we can clearly get the hierarchical relationship and overlap between the data, which helps us to clarify the inclusion relationship between the data and divide the disk information data in the original array into mutually exclusive arrays and inclusive arrays. Among them, after the traversal is completed, the mutually exclusive array contains all arrays that are not completely contained by other arrays (that is, it has a disk that is not required by other arrays), and the arrays in the inclusive array have an inclusion relationship (that is, the disk information required by one array is completely contained in the other array). For example, for the data in Table 1, we can get a mutually exclusive array: (50, 30), (30, 10); inclusive array: (50, 10, 20), (50, 10) and (10). In this way, we get a non-overlapping, independent data set (mutually exclusive array), and the test cases in the inclusive array need to be further processed in the future to facilitate the subsequent determination of the precise number of test environments.
[0105] S105 . Create a corresponding test environment for each first test case set in the mutually exclusive array, and assign each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set.
[0106] Optionally, in this step, a physical machine and / or a virtual machine is used to construct a corresponding test environment according to the test environment requirements required by each first test case set in the mutually exclusive array. It is understandable that the memory configuration information of the created test environment is equal to the memory configuration information cpu of the first test case set, the memory configuration information is equal to the memory configuration information mem required by the first test case set, and the disk information parameter is equal to the disk information disk of the first test case set.
[0107] Exemplary, physical machine deployment: If there are enough physical servers or computer resources available, the test environment can be directly built on these physical machines. It should be noted that, according to the number of test environments required, multiple test environments can be deployed in parallel on different physical machines to support parallel execution of test cases.
[0108] Virtual machine deployment: If physical machine resources are limited or a more flexible test environment management method is required, virtualization technology (such as VMware, VirtualBox, KVM, etc.) can be used to build a test environment. Through virtualization technology, multiple virtual machine instances can be created on a computing device (physical machine) and a separate test environment can be built in each virtual machine. This can make more efficient use of physical machine resources and facilitate the management and maintenance of the test environment.
[0109] S106: Create a corresponding test environment for each inclusion array, and assign all first test case sets in each inclusion array to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case sets.
[0110] In this step, the disk information disk of the created test environment is equal to the disk information disk that is most required by the first test case set in the included array, the processor cpu is equal to the processor cpu that is most required by the first test case in the included array, and the memory mem is equal to the largest mem of the first test case in the included array. In this way, the created test environment can satisfy the test operations of all the test cases in the first test case set in the included array. Exemplarily, the creation of the test environment can use a physical machine and / or a virtual machine, and see the description in S105 above for details.
[0111] For example, the test environment requirement information of the first test case set in the array divided in Table 1 is: Group 3 (corresponding to cpu of 4, mem of 2, disk of 50, 10), Group 4 (corresponding to cpu of 8, mem of 8, disk of 50, 10, 20) and Group 6 (corresponding to cpu of 4, mem of 8, disk of 10). It can be obtained that the most disk information disk is 50, 10, 20, the largest processor cpu is 8, and the largest mem is 8; then the corresponding test environment created is cpu of 8, mem of 8, and disk of 50, 10, 20. In this way, the corresponding test environment created can meet the machine resources required by Groups 3, 4, and 6 in Table 1.
[0112] In one possible implementation, Figure 4 A schematic diagram of the process of creating a corresponding test environment for each containing array is shown. Figure 4As shown, in a possible implementation, creating a corresponding test environment for each containing array can be achieved by the following steps:
[0113] S1061. Determine whether the total number of test cases in each of the multiple first test case sets in the array is less than or equal to a preset number. In one case, the total number of test cases in the multiple first test case sets in the array is less than or equal to the preset number, and execute S1062. In another case, the total number of test cases in the multiple first test case sets in the array is greater than the preset number, and execute S1063.
[0114] It can be understood that the above preset number indicates the number of test cases that can be executed in a group under the same test environment.
[0115] It should be noted that the reason for setting the preset number is that running too many test cases in the same test environment will affect the test efficiency to a certain extent. Therefore, in this implementation, a test environment case limit (preset number) is added. When the environment resources are sufficient, the preset number can be reasonably adjusted according to the test task, so as to better control the test execution efficiency.
[0116] Specifically, the preset number can be set based on available hardware resources, such as the number of available CPU cores, memory size, and disk space. The execution of test cases consumes these resources, so the preset number needs to ensure that it does not exceed the maximum resource limit that the computing device can provide to avoid resource bottlenecks and performance issues.
[0117] The preset number can also be set based on the execution time and frequency of the test case. If the test case takes a long time to execute or needs to be run frequently, the preset number may be adjusted based on these requirements to ensure that the system can complete the test in a reasonable time.
[0118] Sometimes tests need to be executed in parallel in multiple environments to speed up the overall test time. The preset number setting can also take into account this parallelism requirement to make full use of available test resources and maximize test efficiency.
[0119] In actual applications, the preset number may be determined through experiments and performance tests to ensure that the test cases can be run stably and efficiently under given resources. For example, the preset number may be 1-100, and this embodiment does not impose any restrictions thereon.
[0120] S1062. When the total number of test cases of the multiple first test case sets in each inclusion array is less than or equal to a preset number, create a test environment for each inclusion array, and allocate all the first test case sets in each inclusion array to the corresponding test environment.
[0121] In this step, the disk information disk of the created test environment is equal to the disk information disk that is most needed by the first test case set in the included array, the processor cpu is equal to the processor cpu that is largest needed by the first test case in the included array, and the memory mem is equal to the largest mem of the first test case in the included array. The creation of the test environment can use a physical machine and / or a virtual machine, as described in S104 above for details.
[0122] Exemplarily, in this embodiment, the preset number is 20. There are three test cases in the third group, namely testcase7-testcase9. There are four test cases in the sixth group, namely testcase22-testcase25. And there are six test cases in the fifth group, namely testcase10-testcase15. In this way, the total number of test cases in the three groups of the first test case set of the containing array is 13, which is less than the preset number 20, so a test environment is created for the containing array. Specifically, the test environment requirement information of the first test case set in the containing array divided in Table 1 is: Group 3 (corresponding to cpu is 4, mem is 2, disk is 50, 10), Group 4 (corresponding to cpu is 8, mem is 8, disk is 50, 10, 20) and Group 6 (corresponding to cpu is 4, mem is 8, disk is 10), then the corresponding created test environment is cpu is 8, mem is 8, disk is 50, 10, 20. In this way, the corresponding test environment can meet the machine resources required by groups 3, 4, and 6 in Table 1. In this way, a containing array only needs to create one test environment, and the accurate number of test environment creation can be further obtained, saving machine resources used to prepare test environment resources.
[0123] S1063. When the total number of test cases in each of the plurality of first test case sets in the array is greater than a preset number, create a number of test environments corresponding to the ratio of the total number / the preset number for the first test case sets in the array.
[0124] Exemplarily, there are three test cases in the third group in Table 1, namely testcase7-testcase9. There are four test cases in the sixth group, namely testcase22-testcase25. And there are six test cases in the fifth group, namely testcase10-testcase15. In this way, the total number of test cases in the three groups of the first test case set containing the array is 13, which exceeds the preset number 10. The ratio of the total number 13 / the preset number 10 is 1.3, and the comparison value is the rounding of the decimal value, so two test environments need to be created for the containing array. At this time, the fifth group of arrays plus the sixth group of arrays have a total of 10 test cases, and the sum of the test cases of the two groups is equal to the preset number 10, then a test environment is created for the test cases in the first test case sets of the fifth and sixth groups, and the test environment is cpu 8, mem8, disk50, 10, 20. At the same time, a separate test environment is created for the first test case set of group 3, and the test environment is cpu 4, mem2, disk50, 10, which meets the test environment requirements of the test cases in the first test case set of group 3. In this way, although an inclusion array has three first test case sets, only two test environments need to be created to meet the test execution requirements, thereby integrating test resources and reducing the test environment resources required to be configured on the premise of meeting test efficiency.
[0125] In one possible implementation, for the case where no disk information is required in the test environment, that is, the number of disks and the disk size required in the test environment of the first test case set test cases are both 0, for example, the first test case set in the 5th group in Table 1, the test environment does not need to be configured with a disk during execution. At this time, the disk attribute is empty (that is, there is no disk information), and the disk information is recorded as 0. Since the first test case set in the 5th group does not have disk information, the 5th group of test environments cannot include calculation steps through a set based on disk information. The processing method for this group of test cases can be processed and stored separately, and in the subsequent processing process, this group of test cases can be allocated to the group where other test environments are located according to the requirements of other hardware resources (such as CPU, memory), so as to save resources and improve testing efficiency.
[0126] Exemplarily, for a first test case set whose test cases do not require disk information, the test cases in the first test case set that do not require disk information are allocated to a first test case set under a mutually exclusive array and an inclusive array, wherein the processor and memory configuration information required to receive the first test case set of allocated test cases is greater than or equal to the maximum value of the processor and memory configuration information in the required test environment of the allocated test cases.
[0127] For example, for the data in Table 1, after set inclusion partitioning, we can get mutually exclusive arrays: Group 1 (corresponding to cpu 2, mem 4) and Group 2 (corresponding to cpu 4, mem 4); inclusive arrays: Group 3 (corresponding to cpu 4, mem 2), Group 4 (corresponding to cpu 8, mem 8) and Group 6 (corresponding to cpu 4, mem 8). The number of CPU cores required in the test environment of the first test case set of Group 5 in Table 1 is 4, and the memory size is 2. It is acceptable that the test cases in Group 5 are divided into Group 2, Group 3, Group 4 and Group 6. Then the 6 test cases in Group 5 can be merged into Group 2, Group 3, Group 4 and Group 6 in sequence.
[0128] Combined with Table 1, it can be seen that after grouping the test cases, we can identify which test cases do not require disk information, such as the fifth group of arrays. For test cases that do not require disk information, analyze the processor and memory configuration information required by each group to find the maximum demand value, and evenly divide the test cases that do not require disk information into mutually exclusive arrays and inclusive arrays, ensuring that the processor and memory configuration information of the allocated mutually exclusive arrays and inclusive arrays is greater than or equal to the maximum value in the test environment required by the group, so as to ensure that the test cases can be correctly executed under the allocated resources. In this way, through dynamic adjustment and balanced allocation, it is ensured that each test case can obtain computing resources (processors and memory) that meet its needs, avoiding resource waste and shortages. At the same time, avoid arrays that have reached the preset number, configure resources according to the test cases with the greatest demand, ensure that all test cases can run efficiently, and shorten the overall test time.
[0129] In this way, through the dual judgment of the disk resource inclusion relationship and the number of test cases in steps S1061-S1063, the number of the first test case set and the number of the included array in the mutually exclusive array are obtained, which is the number of the test environment to be created, so as to obtain the accurate number of test environments, so as to fully consider the diversity of test cases and the specific side test environment required. In other words, the number of the first test case set in the mutually exclusive array and the number of the included array after processing are the machine physical resource data (test environment) to be created, thereby realizing the management and optimization of the test environment of multiple first test case sets, optimizing and integrating the test environment resources required by multiple first test case sets, avoiding repeated configuration of the same or partially identical test environment, reducing redundant test environments, obtaining the accurate number of test environments, reducing deployment time, and taking into account test efficiency and resource utilization. In addition, by calculating the accurate number of test environment requirements through this algorithm, the problem of excessive waste of resources or long execution time caused by human estimation can be avoided.
[0130] S107: Execute test cases.
[0131] In this step, you can use resource management tools or automation platforms, such as container technology Kubernetes (container orchestration engine), Docker Swarm (Docker cluster), etc., to create resources for the test environment, achieve rapid deployment and resource isolation, and ensure that the test cases can be successfully executed in the required environment. Exemplarily, you can create a test environment in the resource management tool based on the multiple test cases executed to execute multiple test cases concurrently. This can save time and improve overall testing efficiency.
[0132] Alternatively, you can deploy test framework tools to automate the creation and scheduling of test environments, such as JUnit, TestNG, pytest, Selenium, Appium, Ansible, Puppet, Chef and other automation platforms, to automatically create and schedule test environments and execute test cases.
[0133] In some possible implementations, such as Figure 5 As shown, a test method provided in an embodiment of the present application may further include: S108 and S109. Exemplarily, S108 and S109 may be executed after S106.
[0134] S108: When the test case is executed successfully, release the test environment.
[0135] In this step, after the test case is successfully executed, the test environment is released to ensure the cleanliness and stability of the operating system test environment. Developers can dynamically schedule and manage the test environment according to different test requirements without affecting the execution of test cases in the next test environment. At the same time, scheduling the operating system test environment on demand can avoid serious pollution of the test environment due to excessive use. After each test, the operating system environment that is no longer needed can be released in time. The released test environment can also be used to build the next test environment, avoiding resources being idle for a long time, ensuring the rational application of resources, improving resource utilization, and also improving test efficiency.
[0136] S109: When the test case fails to be executed, retain the test environment, and when the retention time reaches a preset time, release the test environment.
[0137] In this step, the environment where the successful use case group is located is cleared, and the environment of the failed use case group is retained for a period of time as needed to provide researchers with positioning and use. This can ensure the reasonable application of the environment. Exemplarily, the preset time can be 5 minutes, 1 hour or 5 hours. Developers can set a reasonable retention time based on historical data and experience to balance troubleshooting and resource recovery. In this way, sufficient troubleshooting time can be guaranteed without occupying resources for a long time. The embodiment of the present application does not make specific restrictions on the preset time.
[0138] In this step, you can use automated tools and logs to perform preliminary fault location and analysis in the retained test environment to identify possible causes of failure. If necessary, testers can manually enter the retained environment to perform detailed problem diagnosis and reproduction operations.
[0139] Specifically, when the tester determines the specific reason for the failure of the test case execution (such as finding a code defect or configuration error) and records the relevant information, the test environment can be released manually or automatically. Timeout release: If the cause of the failure cannot be determined within the preset time, the system will automatically release the test environment to ensure that resources are not occupied for a long time. In this way, through dynamic management of the environment, the environment is retained only when the test fails and diagnosis is required, and it is released in time in other cases to avoid waste of resources. Moreover, releasing the environment of a successful test case allows it to be quickly reallocated to a new test task, reducing environment preparation time and improving parallel testing capabilities.
[0140] The testing method provided in the embodiment of the present application can calculate the precise number of test environments by reasonably grouping, sorting and set inclusion relationship processing of the test environments required for the test cases, thereby solving the problem of resource waste caused by the inability to accurately prepare the number of environments or low execution efficiency caused by too few resources being created.
[0141] Afterwards, physical machines and virtual machines are used to build flexible and controllable sets of different test environments according to the number of test environments, so as to realize the ability of automated test environment preparation. Then an automated test framework (automated test platform) is built, which can dynamically schedule and manage the test environment according to the test requirements. If the test case is successfully executed, the test environment is released immediately to realize the recycling of the test environment, and provide a powerful base capability for the test of the Linux operating system. In this way, this method achieves a balance between cost and resource utilization, avoiding excessive waste or high cost. Finally, this implementation scheme can not only meet the testing business of the operating system, but also add a test case execution quantity limit in each test environment when the environmental resources are sufficient, so as to better control the test execution efficiency.
[0142] It is understandable that the size of the sequence number of each step in the above-mentioned embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in some possible implementations, the steps in the above-mentioned embodiments can be selectively executed according to actual conditions, and can be partially executed or fully executed, which is not limited here. In addition, all or part of any features of any of the above-mentioned embodiments can be freely and arbitrarily combined without contradiction; the combined technical solutions are also within the scope of the present application.
[0143] Example 1
[0144] Embodiment 1 of the present application provides a testing method for scheduling and executing a use case test in a Linux kernel operating system execution environment. Figure 6 As shown, the test method includes the following steps:
[0145] S51. Receive a test task.
[0146] See here Figure 2 Step S101 in will not be repeated here.
[0147] S52. Obtain test case information.
[0148] In this step, the test case information is obtained through the test task. When writing the test case, the developer will record the test environment information required for each test case, including the number of CPU cores, memory size, disk space, network card settings, etc.
[0149] S53, test case permutations and combinations.
[0150] See here Figure 2 In steps S103-S104, after the test cases are arranged and combined and set inclusion calculation is performed, a mutually exclusive array and a processed inclusion array are obtained, and the precise number of test environments is obtained according to the number of the first test case set in the mutually exclusive array and the number of the inclusion array.
[0151] S54. Create a test environment by group.
[0152] In this step, a test environment is created based on a physical machine or a virtual machine according to the exact number of test environments obtained.
[0153] S55. Deploy the test framework.
[0154] In this step, developers can use test framework tools to deploy and schedule automated test environments, such as JUnit, TestNG, pytest, Selenium, Appium, Ansible, Puppet, Chef, and other automation platforms to automatically schedule the test environment.
[0155] S56. Execute test cases concurrently.
[0156] In this step, the test case is executed through the test framework deployed in step S55.
[0157] S57. Determine whether the test case is executed successfully.
[0158] S58. Release the test environment.
[0159] In case the test case is executed successfully, the test environment is released immediately.
[0160] S59. The test environment is reserved.
[0161] In the event of a test case execution failure, the test environment is retained for a period of time. The test environment retention time can be reasonably set based on historical data and experience to balance troubleshooting and resource recovery.
[0162] S60. Positioning analysis.
[0163] During the test environment retention time, the cause of the test case execution failure is located and analyzed to facilitate troubleshooting, so that the test case can be successfully executed and the operating system test can be completed.
[0164] S61, manual release.
[0165] After determining the cause of the test case execution failure, you can manually release the test environment.
[0166] S62. Release upon expiration.
[0167] After determining the cause of the test case execution failure, you can also allow the test environment to be automatically released after a preset time without performing manual operations to ensure the reasonable use of the environment.
[0168] The test method provided in Example 1 of the present application calculates the precise number of environments by arranging and combining environmental information (number of CPUs, mem, disks, netcards, etc.), and creates an operating system test environment according to the calculated environmental information deployment, thereby achieving accurate provision of the number of test environments under the condition of diverse test environments, and by running the environment scheduling plan on the automation platform, it can flexibly create and recycle resources in real time to avoid excessive waste of resources, and at the same time, it can also successfully control the test execution efficiency by limiting the number of test environment use cases.
[0169] like Figure 7 As shown, an embodiment of the present application further provides a testing device, including: an acquisition module 1021 , a grouping module 1022 , a construction module 1023 , a testing module 1024 and a monitoring module 1025 .
[0170] Among them, the acquisition module 1021 is used to acquire a test task, and the test task includes multiple test cases. The acquisition module 1021 is also used to acquire the test environment requirements of each of the test cases. The test case includes the test environment when it is executed on the operating system, wherein the test environment includes the number of processor cores, memory size, disk information, and network card configuration information required when the test case is executed.
[0171] Grouping module 1022: Determine multiple first test case sets according to the test environment requirement information of the test cases, wherein the test environment requirements of the test cases included in one first test case set are the same. In other words, the grouping module 1022 can arrange and combine the test cases based on the test environment, and group the test cases with the same test environment into one group. Specifically, the test cases with the same test environment requirements are grouped into one group according to the number of processor cores, memory size, disk information, and network card configuration information required when executing the test cases.
[0172] In one possible implementation, the grouping module 1022 is further used to determine a mutually exclusive array and at least one inclusion array based on multiple first test case sets; wherein the mutually exclusive array includes at least one first test case set, and there is no inclusion relationship between different first test case sets in the mutually exclusive array; and the inclusion array includes multiple first test case sets, and there is no inclusion relationship between different first test case sets in the inclusion array.
[0173] Construction module 1023: Create a corresponding test environment for each first test case set in the mutually exclusive array, and assign each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set.
[0174] In one possible implementation, the construction module 1023 is further used to create a corresponding test environment for each inclusion array, and assign all first test case sets in each inclusion array to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set.
[0175] The test module 1024 is used to schedule the test environment for executing test cases.
[0176] Optionally, the test module 1025 may automatically schedule the test environments required by the multiple test cases according to the multiple test cases being executed, and execute the multiple test cases concurrently.
[0177] Monitoring module 1025: used to monitor the execution results of the test cases in real time; release the test environment when the test cases are successfully executed; retain the test environment when the test cases fail to execute, and release the test environment when the retention time reaches a preset time.
[0178] As an achievable implementation, the monitoring module 1026 is also used to determine the reason for the failure of the test case execution when the test case execution fails.
[0179] Thus, the computing device provided by the embodiment of the present application has the ability to automatically prepare the environment, and can dynamically schedule and manage the test environment according to the test requirements. In addition, this implementation scheme needs to take into account the balance between cost and resource utilization, and can accurately obtain the number of test environments required for the test through calculation, and can release the test environment in time after the test case is executed, avoiding excessive waste or high cost.
[0180] like Figure 8 As shown, the present application embodiment also provides a computing device, the computing device 600 can be Figure 1 The management device 101 in FIG. 4 . Exemplarily, the computing device 600 may be a server or a terminal.
[0181] The computing device 600 includes: a bus 601, a processor 602, a memory 603 and a communication interface 604. The processor 602, the memory 603 and the communication interface 604 communicate with each other through the bus 601. The computing device 600 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 600.
[0182] The bus 601 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus 601 may include a path for transmitting information between various components of the computing device 600 (e.g., the processor 602, the memory 603, and the communication interface 604).
[0183] The processor 602 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0184] The memory 603 may include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0185] The memory 603 stores executable program instructions, and the processor 102 executes the executable program instructions to respectively implement the test methods involved in the above embodiments.
[0186] The communication interface 604 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.
[0187] The present application also provides a computer-readable storage medium. The computer-readable storage medium is used to store computer program instructions, and when the computer program instructions are executed on a computing device, the computing device executes the test method involved in the above embodiment. The computer-readable storage medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.
[0188] The embodiment of the present application also provides a computer program product including instructions. When the computer program product is run on a computing device, the computing device is caused to execute the test method involved in the above embodiment.
[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application. Those skilled in the art should understand that, although the present application is described in detail with reference to the aforementioned embodiments, the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions in the embodiments of the present application.
Claims
1. A testing method, characterized in that: include: Acquire a test task, wherein the test task includes multiple test cases; Obtaining the test environment requirements for each of the test cases; Determine a plurality of first test case sets according to the test environment requirement information, wherein the test environment requirements of the test cases included in one of the first test case sets are the same; Determine a mutually exclusive array and at least one inclusive array according to a plurality of the first test case sets; the mutually exclusive array includes at least one first test case set, and there is no inclusion relationship between different first test case sets; the inclusive array includes a plurality of first test case sets, and there is an inclusion relationship between different first test case sets; Creating a corresponding test environment for each first test case set in the mutually exclusive array, and assigning each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set; Creating a corresponding test environment for each of the inclusion arrays, and assigning all first test case sets in each inclusion array to the corresponding test environment, wherein each test environment meets the test environment requirements of the assigned first test case set; Execute the test task.
2. The method according to claim 1, characterized in that The test environment requirements include processor, memory, and disk information.
3. The method according to claim 2, characterized in that The determining of a mutually exclusive array and at least one inclusive array according to the plurality of the first test case sets comprises: Based on the disk information in the test environment required by the first test case, a mutually exclusive array and at least one inclusive array are determined.
4. The method according to claim 2, characterized in that: The disk information includes the number of disks and the size of disks; the determining of a mutually exclusive array and at least one inclusive array based on the disk information in the test environment required by the first test case includes: Traverse each first test case set; In the case where a first test case set has an inclusion relationship with the number of disks and the disk size required by any first test case set, the first test case set and the first test case set having the inclusion relationship therewith are taken together as an inclusion array; In the case that there is no inclusion relationship between a first test case set and the number of disks and the disk size required by any first test case set, the first test case set is put into a mutually exclusive array.
5. The method according to any one of claims 1 to 4, characterized in that: The step of creating a corresponding test environment for each first test case set in the mutually exclusive array includes: Create the number of processor cores of the test environment based on the number of processor cores in the test environment requirements of the first test case set, create the memory size of the test environment based on the memory size in the test environment requirements of the first test case set, and create the disk information of the test environment based on the disk information in the test environment requirements of the first test case set.
6. The method according to any one of claims 1 to 4, characterized in that: The step of creating a corresponding test environment for each of the included arrays includes: Create the disk information of the corresponding test environment based on the disk information that is most required by the first test case set in the array, create the number of processor cores of the corresponding test environment based on the largest number of processor cores required by the first test case in the array, and create the memory size of the corresponding test environment based on the largest memory size of the first test case in the array.
7. The method according to claim 6, characterized in that The step of creating a corresponding test environment for each of the included arrays includes: Determine whether the total number of test cases in each of the plurality of first test case sets in the array is less than or equal to a preset number; When the total number of test cases of the plurality of first test case sets in each inclusion array is less than or equal to a preset number, creating a test environment for each inclusion array, and allocating all the first test case sets in each inclusion array to the corresponding test environment; When the total number of test cases in each of the plurality of first test case sets in the array is greater than the preset number, a ratio of the total number to the preset number is used to create a number of test environments corresponding to the ratio for the first test case sets in the array.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: When the test case is executed successfully, releasing the test environment; When the test case fails to be executed, the test environment is reserved, and when the reserved time reaches a preset time, the test environment is released.
9. The method according to any one of claims 1 to 8, characterized in that: The test environment requirements include one or more of network card, GPU or SATA configuration information.
10. A computing device, characterized in that: include: Memory and processor; The memory is coupled to the processor; The memory is used to store program instructions; The processor is used to call the program instructions so that the computing device implements the method according to any one of claims 1-9.
Citation Information
Cited By
A test method, system and device for a test execution environment of a test case
CN122757187A
