Test method, computing device and computer program product

Abstract

The present invention provides a testing method, a computing device and a computer program product. In an embodiment, the method is applied to a device to be tested, including: determining a snapshot of a test environment, the test environment including an operating system and a test product deployed in the operating system; executing a first test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the test environment; restoring the test environment based on the snapshot; executing a second test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the restored test environment. In this solution, the test environment is restored based on the initial snapshot of the test environment, providing a reliable and repeatable test environment, thereby reducing the possibility of kernel contamination during the test process, and further reducing the possibility of affecting other test cases.

Description

Technical Field

[0001] The present invention relates to the field of testing technology, and in particular to a testing method, computing equipment and computer program product. Background Art

[0002] The current application testing process requires the deployment of a test environment and the execution of test cases in the test environment. However, in the process of executing test cases, it may be necessary to operate the kernel of the operating system, but the operation of the kernel of the operating system may cause the operating system to be unable to recover by itself, and may damage the kernel of the operating system, ultimately affecting the accuracy of other test cases. Therefore, a testing method that reduces operating system pollution is urgently needed. Summary of the invention

[0003] Embodiments of the present invention provide a testing method, a computing device, and a computer program product, which restore the testing environment based on an initial snapshot of the testing environment, provide a reliable and repeatable testing environment, thereby reducing the possibility of kernel contamination during the testing process, and further reducing the possibility of affecting other test cases.

[0004] In a first aspect, an embodiment of the present invention provides a testing method, which is applied to a device to be tested, comprising:

[0005] Determine a snapshot of the test environment, the test environment includes an operating system and a test product deployed in the operating system; execute a first test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the test environment; after the first test case is tested, restore the test environment based on the snapshot; execute a second test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the restored test environment.

[0006] In this solution, the test environment is restored based on the initial snapshot of the test environment, providing a reliable and repeatable test environment, thereby reducing the possibility of kernel contamination during the test process, and further reducing the possibility of affecting other test cases.

[0007] In a possible implementation, before determining the snapshot of the test environment, the method further includes:

[0008] Determining initial environment state information of the test environment, the initial environment state information indicating an initial state of the operating system;

[0009] The method also includes: determining whether the test environment has changed based on initial environment status information and operating environment status information; the operating environment status information indicates the state of the operating system after executing the first test case; when it is determined that the test environment has changed, executing the test environment based on the snapshot; when it is determined that the test environment has not changed, executing the second test case corresponding to the object to be tested through the test product.

[0010] In this solution, the test environment after the test is compared with the initial test environment. When the test environments are different, the test environment is restored by creating a snapshot of the initial test environment to ensure that the test environment is the same before each test, thereby reducing the possibility of contamination of the operating system kernel during the test process.

[0011] In a possible implementation, the operating system includes files and processes, the state of the operating system can be characterized by process attributes and file attributes of files, and the files include kernel files and configuration files.

[0012] In a possible implementation, determining whether the test environment has changed based on the initial environment state information and the running environment state information includes:

[0013] When it is determined that the process attributes or the file attributes in the initial environment state information and the running environment state information are different, it is determined that the test environment is changed.

[0014] In a possible implementation, before executing the first test case corresponding to the object to be tested by the test product, the method further includes:

[0015] Back up the configuration files in the operating system and confirm the backed-up configuration files;

[0016] Based on the initial environment status information and the running environment status information, determine whether the test environment has changed, including:

[0017] When it is determined that the file attributes of the configuration files corresponding to the initial environment status information and the running environment status information have changed, and other contents except the file attributes of the configuration files are the same, it is determined that the test environment has not changed, and the configuration files are replaced with the backed-up configuration files; when it is determined that other contents except the file attributes of the configuration files in the initial environment status information and the running environment status information have changed, it is determined that the test environment has changed.

[0018] In a possible implementation, the file includes a binary file, and the state of the operating system further includes an identity of the binary file, where the identity is used to indicate the binary file;

[0019] Determining whether the test environment has changed based on the initial environment state information and the running environment state information includes: determining that the test environment has changed when it is determined that the identities of the binary files corresponding to the initial environment state information and the running environment state information are different.

[0020] In a possible implementation, a virtual machine is deployed on the device to be tested, and the test environment is deployed on the virtual machine.

[0021] In a possible implementation, before restoring the test environment based on the snapshot, the method further includes: when the first test case fails to execute, creating a snapshot of the test environment when the first test case fails to execute.

[0022] In a second aspect, an embodiment of the present invention provides a test device, the test device includes a plurality of modules, each module is used to perform each step in the test method provided in the first aspect of the embodiment of the present invention, and the division of the modules is not limited here. For the specific functions performed by each module of the test device and the beneficial effects achieved, please refer to the functions of each step of the test method provided in the first aspect of the embodiment of the present invention, which will not be repeated here.

[0023] Illustratively, the test device comprises:

[0024] The environment deployment module is used to determine the snapshot of the test environment, which includes the operating system and the test products deployed on the operating system;

[0025] A first test module is used to execute a first test case corresponding to the object to be tested through a test product; wherein the object to be tested is deployed in a test environment;

[0026] An environment recovery module, used to restore the test environment based on the snapshot after the first test case is tested;

[0027] The second test module is used to execute a second test case corresponding to the object to be tested through the test product; the object to be tested is deployed in the restored test environment.

[0028] In a possible implementation, the environment deployment module is used to determine initial environment state information of the test environment, where the initial environment state information indicates an initial state of the operating system;

[0029] The environment recovery module includes a judgment unit and a recovery unit; wherein,

[0030] A judgment unit, configured to determine whether the test environment has changed based on the initial environment state information and the running environment state information; the running environment state information indicates the state of the operating system after executing the first test case;

[0031] The recovery unit is used to recover the test environment based on the snapshot when it is determined that the test environment has changed; and trigger the second test module when it is determined that the test environment has not changed.

[0032] In a possible implementation, the operating system includes files and processes, the state of the operating system can be characterized by process attributes and file attributes of files, and the files include kernel files and configuration files.

[0033] In a possible implementation, the judgment unit is configured to determine that the test environment is changed when it is determined that the process attributes or the file attributes in the initial environment state information and the running environment state information are different.

[0034] In a possible implementation, the environment deployment module is further used to: back up the configuration file in the operating system and determine the backed-up configuration file;

[0035] The judgment unit is used to determine that the test environment has not changed and replace the configuration file with the backed-up configuration file when it is determined that the file attributes of the configuration file corresponding to the initial environment status information and the running environment status information have changed, and other contents other than the file attributes of the configuration file are the same; when it is determined that other contents other than the file attributes of the configuration file in the initial environment status information and the running environment status information have changed, determine that the test environment has changed.

[0036] In a possible implementation, the file includes a binary file, and the state of the operating system further includes an identity of the binary file, where the identity is used to indicate the binary file;

[0037] The judgment unit is used to determine that the test environment is changed when it is determined that the identity identifiers of the binary files corresponding to the initial environment state information and the running environment state information are different.

[0038] In a possible implementation, a virtual machine is deployed on the device to be tested, and the test environment is deployed on the virtual machine.

[0039] In a possible implementation, the first test module is further configured to create a snapshot of the test environment when the first test case fails to be executed when the first test case fails to be executed.

[0040] In a third aspect, an embodiment of the present invention provides a testing device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory, wherein when the programs stored in the memory are executed, the processor is used to execute the method provided in the first aspect.

[0041] In a fourth aspect, an embodiment of the present invention provides a computing device, wherein the device runs computer program instructions to execute the method provided in the first aspect.

[0042] In one example, the computing device may include a processor, the processor may be coupled to a memory, the memory is used to store a program, the processor is used to execute the program stored in the memory, and when the program stored in the memory is executed, the computing device executes the method provided in the first aspect. The memory may be integrated in a chip or a processor, or may be independent of the chip or the processor. The computing device may be a server.

[0043] In a fifth aspect, an embodiment of the present invention provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method provided in the first aspect.

[0044] In a sixth aspect, an embodiment of the present invention provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the framework of the test method provided by the embodiment of the present invention. Figure 1 ;

[0046] Figure 2 is a schematic diagram of the structure of a server cluster provided by an embodiment of the present invention;

[0047] Figure 3 The following is a flow chart of the test method provided by the embodiment of the present invention. Figure 1 ;

[0048] Figure 4 This is a schematic diagram of the framework of the test method provided by the embodiment of the present invention. Figure 2 ;

[0049] Figure 5 The following is a flow chart of the test method provided by the embodiment of the present invention. Figure 2 ;

[0050] Figure 6 This is a schematic diagram of the framework of the test method provided by the embodiment of the present invention. Figure 3 ;

[0051] Figure 7 The following is a flow chart of the test method provided by the embodiment of the present invention. Figure 3 ;

[0052] Figure 8 is a schematic diagram of an application scenario of the test method provided by an embodiment of the present invention;

[0053] Fig. 9 It is a schematic diagram of the structure of a testing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0055] In the description of the embodiments of the present invention, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0056] In the description of the embodiments of the present invention, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0057] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0058] Hereinafter, some terms used in this embodiment are explained. It should be noted that these explanations are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed by the present invention.

[0059] Test case: A set of test inputs, execution conditions, and expected results used to verify whether a specific software requirement is met.

[0060] Test Framework: refers to a collection designed to facilitate the software testing process, which provides a standard way to organize and execute tests. The framework mainly includes guidelines, testing tools, libraries, interfaces, and other elements that help create and manage automated test scripts.

[0061] Testing environment: refers to the description of the software and hardware environment on which the test is run, as well as any other software that interacts with the software under test. Testing environment refers to the computer hardware, software, network equipment, and historical data required to complete software testing work.

[0062] Software testing: refers to the verification and validation of software systems during the software development process to ensure its quality, reliability and integrity. The purpose of software testing is to find defects, problems or errors in the software and ensure that the software meets the expected requirements and specifications.

[0063] Operating system (OS): A built-in program that works with the computer's hardware to interact with the user. Common operating systems include Windows, macOS, and open source Linux.

[0064] Kernel: It is the most basic part of the operating system. It is a part of the software that provides secure access to the computer hardware for many applications. This access is limited, and the kernel determines when and how long a program can operate on a certain part of the hardware.

[0065] Binary File: A file of data and / or instructions that a computer processes.

[0066] Hash value: It is the ID card of the file. It is calculated based on the file size, time, type, creator, etc. It can easily change. No one can predict what the next number will be, and no one has changed his software. The hash value of the file changes. The hash algorithm maps a binary value of any length to a smaller binary value of a fixed length. This small binary value is called a hash value. The hash value is a unique and extremely compact numerical representation of a piece of data.

[0067] Process: It is a running activity of a program in a computer on a certain data set. It is the basic unit for resource allocation in the system and the foundation of the operating system structure. It is a dynamic execution process of a program with certain independent functions on a data set. It is an independent unit for resource allocation and scheduling in the operating system and the carrier for the operation of the application. The process generally consists of three parts: program, data set and process control block (PCB). The program is a static file after code compilation; the process is a dynamic instance in operation; the process control block (processctrl block) is a special structure task_struct set by the operating system to manage the process. The operating system uses it to record the external characteristics of the process and describe the movement and change process of the process. At the same time, the system can use PCB to control and manage the process. The process control block (i.e., task_struct structure) contains the process number pid, process status, and other contents. The process status can be ready (Ready) state, execution (Running) state, blocked (Blocked) state, etc. It can also be running state: R; interruptible sleep state: S; uninterruptible sleep state: D; pause state: T; tracking state: t; zombie state: Z.

[0068] Kernel state: A state in which a computer processor runs at the highest privilege level. In kernel state, kernel code can be executed. During the execution of kernel code, all hardware and all memory of the computer can be accessed, which enables the kernel to perform very low-level and powerful operations, such as managing processes, accessing hardware, and executing system calls.

[0069] In the scenario of automated test development and execution, for example, software testing of applications; the testing process of software testing involves operations on the kernel state (also known as "operating system state" or "privileged state"). However, it should be noted that if an error occurs in the kernel, the entire system may crash. Therefore, operations on the kernel state may cause the operating system to be unable to recover on its own, may contaminate the kernel of the operating system, and affect the accuracy of other test cases. To solve this problem, you can reinstall the operating system or install multiple operating systems, but this method will increase resource consumption.

[0070] Based on this, an embodiment of the present invention proposes a testing method.

[0071] like Figure 1 As shown, the method creates an initial snapshot of the test environment before testing, executes the test case in the test environment, and then restores the test environment based on the initial snapshot of the test environment, and executes the next test case through the restored test environment, thereby reducing the possibility of contaminating the kernel of the operating system during the test process, thereby reducing the possibility of affecting other test cases. In addition, after the test case fails to execute, a snapshot of the test environment after the test is created.

[0072] It should be noted that, by using the initial snapshot of the test environment to restore the test environment, on the one hand, you can quickly roll back to the previous state, which improves the efficiency and flexibility of the test and can reduce the time and resource consumption required to reinstall the test environment or install multiple test environments. On the other hand, the test environment before each test case execution remains unchanged, providing a reliable and repeatable test environment to maintain the consistency and comparability of the test. During the test process, if an unrecoverable error or unexpected situation occurs, you can use the snapshot of the test environment before the test and / or when the test case execution fails to go back to troubleshoot and solve the problem, which improves the efficiency and quality of the test, while reducing the human resources and time required during the test process.

[0073] In summary, by using snapshots of the test environment to unify the test environment before executing test cases, the possibility of kernel contamination of the operating system can be reduced, the efficiency and accuracy of the test can be improved, and the time and resource consumption during the test process can be reduced.

[0074] This is just a brief description of the method. For details about the method, please refer to the description below.

[0075] Next, a test system to which the test method provided in the embodiment of the present invention may be applied is introduced. Figure 2 The present invention provides a test system architecture example diagram. The present invention provides a test method that can be applied to Figure 2 As shown in the system architecture diagram. Figure 2 As shown, the test system includes a management device 201 and a server 202; wherein the server 202 is used to implement the execution of test cases, the management device 201 deploys an automated test platform, and the server 202 and the management device 201 can communicate, such as through network communication, so that the automated test platform is used to manage and control the test of the server 202. The management device 201 can run the automated test platform, and the automated test platform can be a desktop application, a Web application, a command line terminal, etc. In addition, the tester can configure the object to be tested, the test case, etc. through the automated test platform run by the management device 201, so as to perform test configuration. It should be understood that the management device 201 can be a device with a display device, such as a host configured with a display screen, or a computer.

[0076] Among them, server 202 can be a single-node server, such as a rack server; it can also be a computing node in a multi-node server. The multi-node server can be a blade server, a high-density server or a whole cabinet server (an independent product formed by integrating the original architecture that separates the rack and the machine, such as the server node).

[0077] For example, Figure 2 As shown, the server 202 may include a power supply 121 and a mainboard 110 , wherein the power supply 121 is electrically connected to the mainboard 110 and is used to supply power to devices connected to the mainboard 110 .

[0078] In some possible embodiments, the devices connected to the motherboard 110 may include a processor 111, a memory 112, a programmable logic device (PLD) 113, a baseboard management controller (BMC) 114, a PCIE slot 115, a network card 122, a disk 123, and a fan 124.

[0079] The processor 111 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 201 may also be a controller with computing capabilities.

[0080] Among them, the memory 112 can be inserted into a memory slot to connect to the motherboard 110. The memory 112 can be a random access memory (RAM). By way of example but not limitation, many forms of RAM are available. Exemplarily, the memory 112 includes at least two types of memories, for example, the memory can be either a random access memory (RAM) or a read-only memory (ROM). For example, the RAM can include a dynamic random access memory (DRAM) or a storage class memory (SCM) and other memories.

[0081] The programmable logic device 113 may be a complex logic device (CPLD, which is a digital integrated circuit whose logic functions are constructed by users according to their own needs) or a field programmable gate array (FPGA).

[0082] Among them, the PCIE slot 115 is suitable for expanding at least one of a GPU card, a network card, a video capture card, an HBA (Host Bus Adapter) card, a RAID (redundant arrays of independent disks) card, and an SSD, and can also support the expansion of various adapter cards.

[0083] The network card 124 is used to communicate with the management device 201 .

[0084] The disk 123 may be a mechanical hard disk HDD or a solid state disk SSD. It should be understood that the disk 123 is only an example of a non-volatile memory and does not constitute a specific limitation. In actual applications, a non-volatile memory may be selected in combination with actual conditions.

[0085] It should be noted that Figure 2 This is only an example of the server 202 and does not constitute a specific limitation. In practical applications, the server 202 may include Figure 2 More or fewer devices.

[0086] Next, in combination with the test system provided above, a test method provided by an embodiment of the present invention is introduced in detail.

[0087] Figure 3 is a flow chart of a test method provided by an embodiment of the present invention. This embodiment can be applied to a test system. Figure 3 As shown, the testing method provided by the embodiment of the present invention comprises at least the following steps:

[0088] Step 301: The management device 201 sends a deployment instruction and a plurality of test cases of the object to be tested to the server 202. The deployment instruction is used to instruct the deployment of the test environment and the deployment of the object to be tested.

[0089] The object to be tested may be an application or a code, which is only an example and is not limited in the embodiment of the present invention.

[0090] It should be noted that in the embodiments of the present application, there is no intention to limit the order of sending deployment instructions and sending test cases. For example, the deployment instructions may be sent first, and then the test cases; for another example, the test cases may be sent first, and then the deployment instructions; for another example, the deployment instructions and test cases may be sent at the same time.

[0091] Step 302: The server 202 deploys a test environment, where the test environment is an operating system and a test product deployed on the operating system.

[0092] In the embodiment of the present invention, the test environment is an environment that carries the object to be tested. In a specific implementation, the server 202 can obtain the test environment file based on the deployment instruction, and deploy the test environment through the test environment file. The test environment may include an operating system and a test product, etc. This is only an example of deploying the test environment, and the embodiment of the present invention does not limit this.

[0093] The test environment files refer to the relevant files required for deploying the test environment; illustratively, the test environment files may include the installation packages of the operating system and the test product.

[0094] The operating system may include related firmware, related drivers, etc.; wherein firmware (Firmware, FW) refers to a program written into an erasable programmable read-only memory or an electrically erasable programmable read-only memory.

[0095] Among them, the test product can be a product for executing test cases, such as a test tool or a test framework. Among them, the test framework is an infrastructure built to provide an execution environment for automated test cases. The test framework helps to effectively develop, execute and report automated test cases. A test framework refers to a structured method for organizing and managing test cases, which provides a series of libraries and tools for writing, executing and managing test cases, and generating test reports and logs. The test framework can be JUnit, TestNG, PyTeast, etc. A test tool refers to a software tool that can automatically execute test cases, generate test reports, and detect and analyze test results. The test tool can be Selenium, Appium, Jenkins, etc.

[0096] Step 303: Server 202 determines an initial snapshot of the test environment.

[0097] In some possible implementations, the server 202 is deployed with a virtual machine, and a test environment is deployed in the virtual machine. The method for determining the snapshot of the test environment is: shutting down the operating system, creating a snapshot of the test environment, and marking the snapshot with the initial test environment identifier, and then starting the operating system; it should be noted that when the virtual machine is running, the state of the virtual machine may change in real time. At this time, creating a snapshot requires tracking the state of the virtual machine, which may result in low efficiency in creating the snapshot. Shutting down the operating system is equivalent to shutting down the virtual machine, making the state of the virtual machine stable. Creating a snapshot at this time can improve the efficiency of snapshot creation.

[0098] Exemplarily, the data structure recording the initial snapshot of the test environment may be: {"initial snapshot":"snapshot name xx"}.

[0099] Step 304: The server 202 executes a first test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in a test environment.

[0100] In actual applications, the test product can encode multiple test cases, determine the execution order of the test cases, and then execute the test cases in sequence.

[0101] Step 305: When the first test case fails to execute, the server 202 creates a snapshot of the test environment when the first test case fails to execute.

[0102] In the embodiment of the present application, step 305 is an optional step. The server 202 creates a snapshot of the test environment when the test case fails, thereby facilitating subsequent analysis.

[0103] Exemplarily, the data structure of the snapshot of the test environment for the test case that failed to execute may be: {"test case xxx":"snapshot name xxx"}. Step 306: After the first test case is tested, the server 202 restores the test environment based on the initial snapshot.

[0104] In the specific implementation, the test environment is restored to its initial state based on the initial snapshot.

[0105] Step 307: The server 202 executes the second test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the restored test environment.

[0106] It should be noted that the test environment should be the same each time a test case is executed. The initial snapshot of the test environment makes the test environment the same before each test case is executed, reducing the possibility of affecting other test cases.

[0107] In this solution, after each test case is executed, the test environment is restored using the initial snapshot of the test environment, providing a reliable and repeatable test environment, thereby reducing the possibility of kernel contamination during the test process, and further reducing the possibility of affecting other test cases.

[0108] Figure 3 What is shown is only a basic embodiment of the method of the embodiment of the present invention. By performing certain optimization and expansion on this basis, other preferred embodiments of the method can be obtained.

[0109] In one embodiment, based on the above-mentioned embodiment, this embodiment provides a more detailed description and a certain degree of optimization for the test process. Specifically, Figure 4 As shown, an initial snapshot of the test environment is created, test cases are executed in the test environment, and the test environment is compared with the test environment after the test; if the test environment after the test changes, the test environment is restored based on the initial snapshot of the test environment, and the next test case is executed through the restored test environment; if the test environment after the test does not change, the next test case is continued to be executed through the test environment after the test, thereby reducing the possibility of kernel contamination during the test process, thereby reducing the possibility of affecting other test cases. In addition, after the test case fails to execute, a snapshot of the test environment after the test is created. This is just a brief description of the method. For details about the method, please refer to the description below.

[0110] Figure 5is a flow chart of a test method provided by an embodiment of the present invention. This embodiment can be applied to a test system. Figure 5 As shown, the testing method provided by the embodiment of the present invention comprises at least the following steps:

[0111] Step 501: The management device 201 sends a deployment instruction and a plurality of test cases of an object to be tested to the server 202. The deployment instruction is used to instruct the deployment of a test environment and the deployment of an object to be tested.

[0112] Please refer to the description of step 301 for details, which will not be repeated here.

[0113] Step 502: The server 202 deploys a test environment, where the test environment is an operating system and a test product deployed on the operating system.

[0114] Please refer to the description of step 302 for details, which will not be repeated here.

[0115] Step 503: The server 202 determines the initial snapshot and initial environment status information of the test environment; the initial environment status information indicates the status of the operating system.

[0116] In some possible implementations, the server 202 determines the content of the initial snapshot of the test environment. Please refer to the above description of step 303, which will not be repeated here.

[0117] The operating system includes files and processes; illustratively, the files include kernel files, library files, configuration files, etc. Library files include subroutine libraries that allow users to read and execute them but not to modify them, such as C language subroutine libraries, etc. Kernel files are kernel-related files. Configuration files are used to describe system parameters that can be set by the operating system, such as display, sound, notification, power, device, network, application, account, privacy, security, and update.

[0118] Correspondingly, the initial environment state information may include file attributes and process attributes of the file. Exemplarily, the file may be a kernel file and a configuration file.

[0119] Among them, file attributes may include the type of file, the permissions of the file, the owner and group of the file, the metadata of the file, and the action of the file; it should be noted that the description of the file status may be different in different operating systems, and the embodiment of the present invention does not specifically limit this; among them, the type of file may be different in different operating systems. For example, in Linux, the type of file may include ordinary files, directories, symbolic links, device files, and socket files. The permissions of a file refer to the access rights of the file owner, the group to which the file belongs, and other users to the file; the owner of the file indicates the creator of the file, who has the highest authority and can change the attributes and permissions of the file. The group to which the file belongs refers to the user group to which the file belongs. Members of the group can share the permissions of the file. The metadata of the file is used to describe the attributes of the file and the data characteristics of the file. The action of the file is used to describe the reading and writing of the file.

[0120] It is worth noting that the file may include a binary file, and the second progress file is a partial file in the kernel file; the initial environment status information may also record the identity of the second progress file. It should be noted that, considering that binary files are generally unreadable, the file-related content is recorded by recording the identity of the file. In an example, the identity is calculated by an identity algorithm, and the identity algorithm is calculated based on the size, creation time, type, creator, etc. of the file; illustratively, the identity algorithm may be a hash function, and the identity may be a hash value; in the scenario where the same identity algorithm is used to determine the identity algorithm of the second progress file, once the file changes, the identity of the file will also change, so by comparing the identity, it can be determined whether the file has changed.

[0121] It should be noted that in the embodiment of the present invention, it is necessary to first determine the initial environment state information and then determine the snapshot of the test environment.

[0122] Step 504: The server 202 deploys the object to be tested in the test environment.

[0123] Step 505: The server 202 executes the first test case corresponding to the object to be tested through the test product to determine the operating environment status information, where the operating environment status information indicates the state of the operating system after the first test case is executed.

[0124] Step 506: When the first test case fails to execute, the server 202 creates a snapshot of the test environment when the first test case fails to execute.

[0125] In the embodiment of the present invention, step 506 is an optional step, and the server 202 creates a snapshot of the test environment when the test case fails, thereby facilitating subsequent analysis.

[0126] Exemplarily, the data structure of the snapshot of the test environment recording the test case that failed to execute may be: {"test case xxx":"snapshot name xxx"}.

[0127] Step 507 : The server 202 determines whether the test environment has changed based on the initial environment status information and the running environment status information. If yes, execute step 509 ; if no, execute step 508 .

[0128] In some possible implementations, when the server 202 determines that the process attributes and / or file attributes in the initial environment state information and the running environment state information are different, it is determined that the test environment is disturbed.

[0129] In a specific implementation, there may be multiple types of files. For any type of file, if the file attribute of the file changes in the initial environment state information and the operating environment state information, it can be determined that the test environment has changed. For each type of file, there may be multiple files. For each file, if the file attribute of the file changes in the initial environment state information and the operating environment state information, it can be determined that the test environment has changed. There may also be multiple processes. For each process, if the process attribute of the process changes in the initial environment state information and the operating environment state information, it can be determined that the test environment has changed. In some possible implementations, the configuration file in the operating system is backed up on the server 101, and the backed-up configuration file is determined.

[0130] It should be noted that the configuration file will not be very large, and the frequency of changes during the test process is high. In order to reduce the number of times the test environment is restored based on snapshots, the configuration file can be backed up. When it is found later that only the configuration file has changed, it can be directly replaced without restoring the test environment, thereby improving the test efficiency. In a possible embodiment of this implementation method, when the server 202 determines that the file attributes of the corresponding configuration file in the initial environment state information and the running environment state information have changed, and other contents other than the file attributes of the configuration file, such as the file attributes and the process attributes, are the same, it is determined that only the configuration file has changed. At this time, the configuration file is replaced with the backed-up configuration file, and the configuration file is restored to the original state, and it is determined that the test environment has not changed. Among them, the configuration files corresponding to the initial environment state information and the running environment state information can be understood as configuration files with the same name; it should be noted that in actual applications, there can be multiple configuration files, and the names of the multiple configuration files are different. For a configuration file of any name, if the file attributes of the configuration file of the name in the initial environment state information and the running environment state information change, it can be determined that the configuration file has changed.

[0131] In a possible embodiment of this implementation, the server 202 may determine that the test environment has changed when it is determined that other contents other than the attributes of the corresponding configuration files in the initial environment status information and the running environment status information, such as file attributes and / or process attributes, have changed.

[0132] In some possible implementations, the files in the operating system may include binary files; the initial environment status information includes the identity of the binary file; the running environment status information includes the identity of the binary file, and the identity in the running environment status information indicates the binary file in the process of executing the test case in the test environment; in a specific implementation, when the file attributes or the identities of the two progress files in the initial environment status information and the running environment status information of the server 202 are different, it means that the binary file has changed, and it can be determined that the test environment has changed; when the file attributes or the identities of the two progress files in the initial environment status information and the running environment status information are the same, it means that the binary file has not changed.

[0133] In some possible implementations, when the server 202 determines that the process attributes or the file attributes of the kernel file in the initial environment state information and the running environment state information are different, it indicates that the process or the kernel file has changed, and at this time it is determined that the test environment has changed.

[0134] Step 508: The server 202 executes the second test case corresponding to the object to be tested through the test product.

[0135] Step 509: Server 202 restores the test environment based on the initial snapshot.

[0136] During the specific implementation, the test environment is rebuilt based on the initial snapshot.

[0137] Step 510: The server 202 executes a second test case corresponding to the object to be tested through the test product, wherein the object to be tested is deployed in the restored test environment.

[0138] It should be noted that the test environment should be the same each time a test case is executed, so after the test environment changes, the test environment needs to be redeployed.

[0139] In this solution, the test environment after the test is compared with the initial test environment. When the test environments are different, the test environment is restored by creating an initial snapshot of the test environment to ensure that the test environment is the same before each test, thereby reducing the possibility of contamination of the operating system kernel during the test process.

[0140] Figure 3What is shown is only a basic embodiment of the method of the embodiment of the present invention. By performing certain optimization and expansion on this basis, other preferred embodiments of the method can be obtained.

[0141] In one embodiment, based on the above-mentioned embodiment, this embodiment provides a more detailed description and a certain degree of optimization for the test process. Specifically, Figure 6 As shown, create an initial snapshot of the test environment, back up the configuration file (system content that can be configured by users in the operating system) in the test environment, and obtain the backed-up configuration file; after executing the test case in the test environment, compare the test environment with the test environment after the test; if only the configuration file changes, replace the changed configuration file with the backed-up configuration file, and execute the next test case through the test environment after the configuration file is replaced; if the content other than the configuration file changes, determine that the test environment after the test has changed, restore the test environment based on the initial snapshot of the test environment, and execute the next test case through the restored test environment; if the test environment after the test has not changed, execute the next test case through the test environment after the test, thereby reducing the possibility of kernel contamination during the test process, and then reducing the possibility of affecting other test cases. In addition, after the test case fails to execute, create a snapshot of the test environment after the test. This is just a brief description of the method, and the details of the method are described below.

[0142] In one embodiment, based on the above-mentioned embodiment, this embodiment provides a more detailed description and a certain degree of optimization for the test process. Figure 7 As shown, the method in this embodiment includes:

[0143] Step 701: The management device 201 sends a deployment instruction and a plurality of test cases of the object to be tested to the server 202. The deployment instruction is used to instruct the deployment of the test environment and the deployment of the object to be tested.

[0144] Please refer to the description of step 301 for details, which will not be repeated here.

[0145] Step 702: Server 202 deploys a test environment, where the test environment is an operating system and a test product deployed on the operating system.

[0146] Please refer to the description of step 302 for details, which will not be repeated here.

[0147] Step 703: The server 202 determines the initial snapshot and initial environment status information of the test environment; the initial environment status information indicates the status of the operating system.

[0148] Please refer to the description of step 503 for details, which will not be repeated here.

[0149] Step 704: The server 202 backs up the configuration file in the operating system to obtain the backed-up configuration file.

[0150] As mentioned earlier, configuration files are used to describe system parameters that the operating system can set, such as display, sound, notification, power, device, network, application, account, privacy, security and update.

[0151] Step 705: The server 202 deploys the object to be tested in the test environment.

[0152] Step 706: The server 202 executes the first test case corresponding to the object to be tested through the test product to determine the operating environment status information, where the operating environment status information indicates the state of the operating system after the first test case is executed.

[0153] Step 707: When the first test case fails to execute, the server 202 creates a snapshot of the test environment when the first test case fails to execute.

[0154] In the embodiment of the present invention, step 707 is an optional step, and the server 202 creates a snapshot of the test environment when the test case fails, so as to facilitate subsequent analysis. Exemplarily, the data structure of the snapshot of the test environment for the test case that failed to execute can be: {"test case xxx": "snapshot name xxx"}.

[0155] Step 708 , the server 202 determines whether other contents other than the file attributes of the configuration files in the initial environment status information and the running environment status information have changed, if not, executing step 711 , if yes, executing step 709 .

[0156] In a possible embodiment, when the server 202 determines that the file status and / or process status of file types other than the configuration type in the initial environment status information and the operating environment status information are different, it can determine that the test environment has changed; otherwise, it is determined that the test environment has not changed.

[0157] Step 709: Server 202 restores the test environment based on the initial snapshot.

[0158] Step 710: The server 202 executes a second test case corresponding to the object to be tested through the test product, wherein the object to be tested is deployed in the restored test environment.

[0159] Step 711, the server 202 determines whether the file attributes of the configuration files corresponding to the initial environment status information and the running environment status information have changed, if so, execute step 712, if not, execute step 713.

[0160] It should be noted that configuration files are not very large and change frequently during the test process. In order to reduce the number of times the test environment is restored based on snapshots, the configuration files can be backed up. When it is later found that only the configuration files have changed, they can be directly replaced without restoring the test environment, thereby improving the test efficiency. In a possible embodiment, when server 202 determines that the initial environment status information and the operating environment status information have different corresponding configuration file states, and the contents other than the file attributes of the configuration file, such as the file attributes and the process attributes, are the same, it is determined that only the configuration file has changed. At this time, the configuration file is replaced with the backed-up configuration file, and the configuration file is restored to its original state to determine that the test environment has not changed.

[0161] Step 712: The server 202 replaces the configuration file with the backed-up configuration file and executes step 713.

[0162] Step 713: The server 202 executes the second test case corresponding to the object to be tested through the test product.

[0163] In this solution, configuration files that change frequently are backed up, and the test environment after the test is compared with the initial test environment. When only the configuration file is changed, the configuration file can be replaced without restoring the entire environment, thereby improving test efficiency. When content other than the configuration file is changed, the test environment is restored through the initial snapshot created for the test environment, ensuring that the test environment is the same before each test, thereby reducing the possibility of contamination of the operating system kernel during the test process.

[0164] Based on the test method provided above, the specific application of the test method is explained. Figure 8 A flow chart of a specific application of a test method provided for the implementation of the present invention. Figure 8 The specific contents include:

[0165] 1. The test task starts.

[0166] 2. Deploy the test environment.

[0167] 3. The anti-interference mechanism begins to intervene and triggers the following events concurrently:

[0168] a) Create and store an initial snapshot of the test environment;

[0169] b) record environmental status information;

[0170] c)Back up configuration files;

[0171] d) Generate hash values ​​of binary files in the test environment;

[0172] 4. Deploy the object to be tested.

[0173] 5. Execute the test cases.

[0174] 6. When the use case execution ends, the anti-interference mechanism continues to intervene and triggers the following events concurrently:

[0175] a) Determine the test result and obtain the judgment result, which is used to indicate whether the test case is successfully executed;

[0176] b) Compare the test environment to obtain comparison results. The comparison results are used to indicate whether the file attributes of the configuration file have changed, and whether other contents other than the file attributes of the configuration file have changed, such as file attributes, process attributes, and hash values ​​of binary files.

[0177] 7. When a test case fails to execute, a snapshot of the current test environment is created and the snapshot is tagged with the test case's ID.

[0178] 8. Determine whether the comparison result indicates that other contents besides the file attributes of the configuration file have changed. If so, it indicates that the test environment has changed. Initialize the test environment based on the initial snapshot of the test environment and deploy the object to be tested.

[0179] 9. Determine whether the comparison result indicates that the file attributes of the configuration file have changed. If so, replace the existing configuration file with the backed-up one and restart the operating system.

[0180] 10.Determine the next test case.

[0181] Based on the same concept as the embodiment of the method of the present invention, the embodiment of the present invention also provides a test device. The test device includes several modules, each module is used to execute each step in the test method provided by the embodiment of the present invention, and the division of the modules is not limited here. The technicians in the relevant field can clearly understand that in practical applications, the various steps in the test method provided by the embodiment of the present invention can be assigned to different modules as needed, that is, the internal structure of the device is divided into different modules to complete all or part of the functions described above. Each module in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more modules can be integrated in one unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each module are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the module in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0182] For example, the testing device is used to execute the testing method provided by the embodiment of the present invention. Fig. 9Schematic diagram of the structure of the test device provided by the embodiment of the present invention. Fig. 9 As shown, the testing device provided by the embodiment of the present invention includes:

[0183] An environment deployment module 901 is used to determine a snapshot of a test environment, where the test environment includes an operating system and a test product deployed on the operating system;

[0184] A first test module 902 is used to execute a first test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the test environment;

[0185] An environment recovery module 903, used to restore the test environment based on the snapshot;

[0186] The second test module 904 is used to execute a second test case corresponding to the object to be tested through the test product; the object to be tested is deployed in the restored test environment.

[0187] Based on the same concept as the embodiment of the method of the present invention, the embodiment of the present invention further provides a computing device. The computing device may be a server 202. The structure of the computing device may refer to Figure 2 Server 202 in.

[0188] In a specific application, the hard disk 123 can store computer programs. When the computing device is running, the CPU 111 can read the computer programs stored in the hard disk 123 into the memory, and read the computer programs from the memory to implement the steps performed by the server 202 in the above test method, for example Figure 3 , Figure 5 , Figure 7 Steps performed by server 202 in .

[0189] Exemplarily, the computer program may be divided into one or more modules / units, which may be a series of computer program instruction segments capable of completing specific functions, and the one or more modules / units are stored in the hard disk 123 and executed by the CPU 111 to complete the present invention. For example, the computer program may be divided into an environment deployment module 901, a first test module 902, an environment recovery module 903, and a second test module 904. The specific functions of each module refer to the above description.

[0190] In addition to the above methods, apparatuses and computing devices, an embodiment of the present invention may also provide a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps in the test method of various embodiments of the present invention described in the above "Method" section of this specification. Wherein, the computer program product can be written in any combination of one or more programming languages ​​to perform computer program code for performing the operation of the embodiment of the present invention, and the programming language includes an object-oriented programming language, such as Java, C++, etc., and also includes a conventional procedural programming language, such as "C" language or similar programming language. Wherein, the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer program code can be executed completely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on the remote computing device, or completely on the remote computing device or server.

[0191] In addition, an embodiment of the present invention may also provide a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the processor executes the steps in the test method according to various embodiments of the present disclosure described in the above "Method" section of this specification. The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0192] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0193] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0194] The basic principles of the present invention are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0195] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0196] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0197] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0198] It should be understood that the various numerical numbers involved in the embodiments of the present invention are only used for the convenience of description and are not used to limit the scope of the embodiments of the present invention.

Claims

1. A testing method, characterized in that: Applicable to the equipment under test, including: Determine a snapshot of a test environment, wherein the test environment includes an operating system and a test product deployed on the operating system, wherein the operating system includes files and processes, and the test product is a product that executes a test case; Executing a first test case corresponding to the object to be tested through the test product; wherein the object to be tested is deployed in the test environment; After the first test case is tested, determining that the test environment has changed based on an initial state of the operating system and a state of the operating system after executing the first test case, the state of the operating system being characterized by process attributes of the process and file attributes of the file; Restoring the test environment based on the snapshot; The second test case corresponding to the object to be tested is executed by the test product; and the object to be tested is deployed in the restored test environment.

2. The method according to claim 1, characterized in that Before determining the snapshot of the test environment, the method further includes: Determining initial environment state information of the test environment, wherein the initial environment state information indicates an initial state of the operating system; The determining that the test environment has changed based on the initial state of the operating system and the state of the operating system after executing the first test case includes: Based on the initial environment state information and the operating environment state information, it is determined whether the test environment has changed; the operating environment state information indicates the state of the operating system after the execution of the first test case.

3. The method according to claim 2, characterized in that The files include kernel files and configuration files.

4. The method according to claim 3, characterized in that The determining whether the test environment has changed based on the initial environment state information and the running environment state information includes: When it is determined that the process attributes or the file attributes in the initial environment state information and the running environment state information are different, it is determined that the test environment is changed.

5. The method according to claim 3, characterized in that: Before executing the first test case corresponding to the object to be tested through the test product, the method further includes: Backing up the configuration file in the operating system and determining the backed-up configuration file; The determining whether the test environment has changed based on the initial environment state information and the running environment state information includes: When it is determined that the file attributes of the configuration files corresponding to the initial environment state information and the running environment state information have changed, and other contents of the configuration files except the file attributes are the same, it is determined that the test environment has not changed, and the configuration files are replaced with the backed-up configuration files; When it is determined that other contents other than the file attribute of the configuration file in the initial environment state information and the running environment state information have changed, it is determined that the test environment has changed.

6. The method according to claim 4 or 5, characterized in that: The file includes two progress files, and the state of the operating system also includes an identity of a binary file, and the identity is used to indicate the binary file; The determining whether the test environment has changed based on the initial environment state information and the running environment state information includes: When it is determined that the identities of the binary files corresponding to the initial environment state information and the running environment state information are different, it is determined that the test environment is changed.

7. The method according to any one of claims 1 to 5, characterized in that: The device to be tested is deployed with a virtual machine, and the test environment is deployed on the virtual machine.

8. The method according to any one of claims 1 to 5, characterized in that: Before restoring the test environment based on the snapshot, the method further includes: When the first test case fails to execute, a snapshot of the test environment when the first test case fails to execute is created.

9. A computing device, characterized in that comprising a processor and a memory; wherein, The memory is used to store programs; The processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the method described in any one of claims 1 to 8 is performed.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform any one of the methods according to claims 1 to 8.

Images