Test method and device, computing equipment and computer program product

Abstract

The embodiment of the invention provides a test method and device, computing equipment and a computer program product, and the method comprises the steps: setting a test frame to be self-started, and executing a plurality of test cases through the test frame; the plurality of test cases comprise a first test case, and the first test case comprises a test step needing to be restarted; when the test step needing to be restarted is executed, first information is received; the first information indicates that the first test case needs to be restarted; the restart operation is to restart the operating system; based on the first information, contextual information of the first test case process is stored; executing a restarting operation; after the test framework is started, based on the stored context information of the first test case process, the first test case is recovered and executed through the test framework. Therefore, the hardware cost can be reduced, the problem of inconsistent test results caused by hardware and environment differences between two machines is solved, and the accuracy and reliability of the test results are ensured.

Description

Technical Field

[0001] This application relates to the field of testing technologies, and in particular, to a testing method, apparatus, computing device, and computer program product. Background Art

[0002] In the current testing mode, when executing test cases involving operating system restart operations, the test framework and the execution of test cases are respectively deployed on two different computing devices.

[0003] This deployment method results in a large amount of resource waste because both machines need to be independently configured and maintained, increasing the hardware cost and the workload of operation and maintenance. Secondly, if one of the machines fails, such as the control machine where the test framework is deployed, due to network connection problems, it is unable to send correct test instructions to the controlled machine that executes the restart test case, and the entire test process will be forced to interrupt and cannot continue, seriously affecting the test efficiency and progress. In addition, the differences in hardware configuration and operating environment between the two machines may also have a negative impact on the consistency and reliability of test results.

[0004] This test architecture with the separation of the controlled machine and the control machine urgently needs to be optimized to improve resource utilization and enhance the stability and reliability of testing. Summary of the Invention

[0005] Embodiments of this application provide a testing method, apparatus, computing device, and computer program product. By saving the context information of the test case process, test cases including restart operations can be executed on one computing device together with other test cases, which can reduce the hardware cost, solve the problem of inconsistent test results caused by differences in hardware and environment between two machines, and ensure the accuracy and reliability of test results.

[0006] In a first aspect, embodiments of this application provide a testing method. The test framework is set to start automatically when the device boots up before executing test cases, and multiple test cases are executed through the test framework. The method includes: the test framework executes multiple test cases; the multiple test cases include a first test case, and the first test case includes a test step that requires a restart operation; when executing the test step that requires a restart operation, a first piece of information is received; the first piece of information indicates that the first test case requires a restart operation; the restart operation is to restart the operating system; based on the first piece of information, the context information of the first test case process is saved; the restart operation is executed; based on the saved context information of the first test case process, the process context of the first test case is restored through the test framework.

[0007] In the above example, the deployment of the test framework and the execution of test cases are unified on one computing device. By saving the context information of the test case process, test cases including restart operations can be executed on one settlement device together with other test cases, which can reduce the hardware cost, solve the problem of inconsistent test results caused by hardware and environmental differences between two machines, and ensure the accuracy and reliability of test results.

[0008] In a possible implementation, before the test framework receives the first information, the method further includes: the test framework saves the first environmental information, where the first environmental information indicates the first environment before the execution of the test case; the test framework receives the first information, including: the test framework receives the first information and the second information, where the second information indicates whether the first test case needs to perform the first environmental restoration after restart; before the test framework performs the restart operation, the method further includes; when the second information indicates that the test case needs to perform the first environmental restoration after restart, the test framework compares the first environmental information and the second environmental information to obtain the first environmental difference, where the second environmental information indicates the second environment before the test case performs the restart operation; the first environmental difference indicates the difference between the first environmental information and the second environmental information; after the test framework performs the restart operation, the method further includes; the test framework compares the third environmental information and the first environmental difference to obtain the second environmental difference, where the third environmental information indicates the third environment after the test case performs the restart operation; based on the second environmental difference, the test framework restores the third environment.

[0009] In the above example, after the computing device restarts and before restoring the context information of the test case process, the environmental information is restored first. Specifically, by using the environmental information before the execution of the test case including the restart step, the environmental information before the restart and after the restart of this test case are used, and the environmental information that needs to be restored is obtained by comparing less data, making the test results after the restored execution of the test case process more reliable.

[0010] In a possible implementation, when the test framework receives the first information and the second information, the test framework also receives the third information, where the third information indicates that the first test case performs the second environmental restoration after restart; before the test framework compares the third environmental information and the first environmental difference to obtain the second environmental difference, the method further includes: based on the third information, the test framework performs the second environmental restoration.

[0011] In the above example, while automatically performing environmental restoration, the environmental information that needs to be restored can also be defined manually in advance, which is more flexible.

[0012] In a possible implementation, when the test framework receives the first piece of information, the test framework also receives the third piece of information; the third piece of information indicates that the first test case performs a second environment restoration after restart; after the test framework performs a restart operation, the method further includes; based on the third piece of information, the test framework performs a second environment restoration.

[0013] In the above example, by means of the full-volume environmental information comparison method, directly comparing the environmental information before restart and the environmental information after restart to obtain the environmental information that needs to be restored can reduce the steps of comparing environmental information.

[0014] In a possible implementation, the environmental information includes at least one of the following: file system mount point, system process, network configuration, service status, kernel module, Selinux status, kernel parameters, firewall rules.

[0015] In the above example, by collecting the above environmental information, it is possible to make the environmental information as consistent as possible with that before restart when the test case resumes execution. More environmental information can be added as needed.

[0016] In a possible implementation, after the test framework performs a restart operation, the test framework checks whether there is the context information of the first test case process. If there is the context information of the first test case process, based on the saved context information of the first test case process, the process context of the first test case is restored; if there is no test case process context information, the test case is executed.

[0017] In the above example, after the test framework restarts, the test framework first determines whether the context information of the test case including the restart operation is saved to determine whether the test case before restart is the first test case including the restart operation. If it is the first test case, the test framework restores the first test case. If it is not the first test case, then the test case does not include the test steps of the restart operation, and thus the context information of the test case is not saved. Therefore, there is no need to execute the test case restoration process.

[0018] Second aspect, an embodiment of the present application provides a testing device for testing a framework to execute test cases, and the testing framework is set to start automatically when the device is powered on. The device includes: an execution module for executing a plurality of test cases, where the plurality of test cases include a first test case, and the first test case includes a test step that requires a restart operation; a receiving module for receiving a first message, where the first message is sent when the first test case executes to the test step that requires a restart operation, and the first message indicates that the first test case requires a restart operation; the restart operation is to restart the operating system; a saving module for saving the context information of the first test case process based on the first message; a restart module for performing the restart operation; a recovery module for, after the testing framework is started, resuming the execution of the first test case through the testing framework based on the saved context information of the first test case process.

[0019] Third aspect, an embodiment of the present application provides a computing device, including: at least one memory for storing a program; at least one processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is used to execute the method provided in the first aspect.

[0020] Fourth aspect, an embodiment of the present application provides a computing device, characterized in that the device runs computer program instructions to execute the method provided in the first aspect. Exemplarily, the device may be a chip or a processor.

[0021] In one example, the device may include a processor, which may be coupled to a memory, read instructions from the memory, and execute the method provided in the first aspect according to the instructions, or be used to execute the method provided in the second aspect. Wherein, the memory may be integrated in the chip or the processor, or may be independent of the chip or the processor.

[0022] Fifth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored, and when the instructions run on a computer, the computer is caused to execute the method provided in the first aspect, or be used to execute the method provided in the second aspect.

[0023] Sixth aspect, an embodiment of the present application provides a computer program product containing instructions, and when the instructions run on a computer, the computer is caused to execute the method provided in the first aspect, or be used to execute the method provided in the second aspect.

[0024] It can be understood that the beneficial effects of the above second aspect and sixth aspect can refer to the relevant descriptions in the above first aspect, and will not be elaborated here. Description of the Drawings

[0025] Figure 1Schematic flow chart of executing test cases for the test framework provided by the embodiments of the present application;

[0026] Figure 2 Schematic flow chart of a test method for testing on a single machine provided by the embodiments of the present application;

[0027] Figure 3 Schematic flow chart of the test method provided by the embodiments of the present application;

[0028] Figure 4 Schematic flow chart of the saving process context of the test cases provided by the embodiments of the present application;

[0029] Figure 5 Schematic flow chart of the restoration of the process context information of the test cases provided by the embodiments of the present application;

[0030] Figure 6 Schematic flow chart of another test process on a single machine provided by the embodiments of the present application;

[0031] Figure 7 Schematic flow chart of a test method provided by the embodiments of the present application;

[0032] Figure 8 Schematic flow chart of another test process on a single machine provided by the embodiments of the present application;

[0033] Figure 9 Schematic flow chart of a test method provided by the embodiments of the present application;

[0034] Figure 10 Schematic flow chart of the specific application of a test method provided by the present application;

[0035] Figure 11 Schematic structural diagram of the test device provided by the embodiments of the present application;

[0036] Figure 12 Schematic structural diagram of the computing device provided by the embodiments of the present application. Detailed implementation manners

[0037] As used herein, the term "and / or" is an associative relationship describing associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " in this text represents an "or" relationship between associated objects. For example, A / B represents A or B.

[0038] In the specification and claims of this document, terms such as "first" and "second" are used to distinguish different objects, rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages, rather than to describe a specific order of the response messages.

[0039] In the description of this document, it should be understood that terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "circumferential", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. This is only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0040] In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0041] To facilitate the understanding of the technical solutions of the embodiments of this application, the terms involved in this document are explained below.

[0042] Operating System (OS for short): It is the core software of a computer system, used to manage and schedule hardware resources. Through functions such as process management, memory management, device management, and file management, it ensures the efficient, stable, and secure operation of the computer system. Commonly seen ones are Windows, macOS, and the open-source Linux.

[0043] Reboot test: Used to verify the stability and reliability of a system or software during the reboot process. By simulating normal reboot operations or reboots after forced interruptions, it checks the data integrity, service recovery ability, and resource allocation situation during the system startup process, ensuring that the system can correctly recover from various states to the normal operating state while avoiding data loss or system anomalies.

[0044] Test framework: An environment or platform used to support automated testing and test management, enabling test cases to be efficiently reused, maintained, and automated for execution.

[0045] Process: A process is the basic unit for the operating system to allocate and schedule resources, containing resources such as program code, data, stack, register status, etc. It is the basic unit for the system to allocate resources and schedule.

[0046] Test case: A set of inputs, execution conditions, expected results, and execution steps used to verify a specific function, feature, or performance.

[0047] In automated testing, a test framework is usually adopted to manage and execute test cases. All test cases can be executed on a computing device, such as a server, where the computing device serves as both the control machine and the test machine. However, when the test cases involve operations related to restarting the operating system, that is, when test cases related to restart need to be executed, the operating system or some hardware information will be reset. After the restart, the originally running processes, file system status, memory, and cache, etc. will be cleared, and the state of the system will change. It is impossible to automatically restore the process context information of the test cases executed before the restart, so it is impossible to continue executing following the test steps before the restart. Instead, the test cases can only be executed starting from the first test step of the test cases before the restart.

[0048] To address this issue, related technologies use two computing devices for testing. One computing device (control machine) is used to execute test instructions, and the other computing device (controlled machine) is used to execute test cases involving restarts, enabling the controlled machine to independently complete the restart when executing tests involving restarts and continue executing following the test steps after the restart.

[0049] However, because two computing devices are used, remote commands, SSH, or other tools are needed to coordinate the step execution between the control machine and the controlled machine. During this process, if there are problems with the control machine or the controlled machine, the test cannot continue. Moreover, at least two computing devices are required to execute test cases related to restart, increasing additional hardware resources.

[0050] When executing test cases that do not include restart operations on the control machine, in addition, the test environment is the benchmark for the test results. If the restart case and the non-restart case are executed in different environments, the test results may not be directly comparable. For example, the restart case may be run in an environment with fewer resources, while the non-restart case may be run in an environment with sufficient resources, which will lead to deviations in performance test results. Different hardware configurations, software versions, or system settings may cause some functions to function normally in one environment and fail in another environment. A unified environment can avoid such misjudgments caused by environmental differences. If the test environment is not unified, some compatibility issues that may occur in the real environment may be missed. A unified test environment also facilitates automated testing and can reduce script modification and debugging work caused by environmental differences. Automated testing tools can efficiently execute all test cases in a unified environment. However, the control machine and the test machine may use different brands or models of CPU, network card, disk and other hardware, and the operating system may exhibit different behaviors on different hardware, especially in terms of resource allocation, hardware driver loading, etc. Even if the control machine and the test machine use the same operating system version and kernel, the installation and configuration of the software package, network settings, and the startup of system services may still be different, resulting in different results when the same test case is executed on the control machine and the controlled machine. Therefore, the hardware and environment differences between the control machine and the controlled machine may affect the consistency and reliability of the test results.

[0051] Furthermore, in complex test scenarios, especially in terms of system stability and fault tolerance, factors such as fault injection, restart, and environmental interference are usually very important test links. Kdump testing, fault injection, startup parameter modification, and restarting the machine to avoid environmental interference all require restarting. Therefore, conducting accurate and reliable restart testing is a technical problem that needs to be solved urgently.

[0052] An embodiment of the present application provides a testing method, which sets a testing framework to automatically start at boot before executing a test case, and executes multiple test cases through the testing framework; the multiple test cases include a first test case, and the first test case includes a test step that requires a restart operation; when executing the test step that requires a restart operation in the first test case, receiving first information; the first information indicates that the first test case requires a restart operation; the restart operation is to restart the operating system; based on the first information, save the context information of the first test case process; execute the restart operation; when the testing framework is started, based on the saved context information of the first test case process, resume execution of the first test case through the testing framework.

[0053] Figure 1 A flow chart of the test framework provided in the embodiment of the present application executing a test case. Figure 1As shown in the figure, the test framework is deployed on the computing device 100, and the execution of all test cases is also on the computing device 100. The specific type of the computing device 100 is not limited in the embodiments of the present application. The computing device may be, for example, a server, and the physical type of the server may be a rack server, a high-density server, a GPU server, a tower server, or may also be a blade server, a whole cabinet server, etc.

[0054] The test cases include restart test cases and non-restart test cases. The restart test case is a test case whose test steps do not include a restart operation, and the non-restart test case is a test case whose test steps include a restart operation. The test framework can be regarded as a control program and runs as the main process to schedule the execution of test cases. Before executing the test cases, the test framework needs to be configured to start automatically when the system boots. In this way, when the test case contains a system restart operation, the test framework can automatically start after the system restarts and continue to execute the test cases that were not completed before the restart, ensuring the continuity and integrity of the test process.

[0055] When the test framework starts, the main process will sequentially create child processes according to a preset order, and each child process is responsible for executing a test case. Each child process runs in an independent memory space and has its own process context (including memory, CPU registers, etc.). During the execution of the child process, the test framework tracks the execution status of the child process through inter-process communication and can determine the execution status of each test step. After the child process completes each step of the test case, it feeds back the current execution status to the main process. Specifically, after the child process completes the test case, the main process will receive an end execution signal, recycle the system resources occupied by the child process, and destroy its process context.

[0056] For example, after the test framework starts, the main process creates child process 1, and child process 1 executes test case 1. Child process 1 starts to execute, and the test framework obtains the execution status of child process 1. When executing test step 1, the child process feeds back status information to the main process, such as starting to execute test step 1; after test step 1 is completed, the child process feeds back status information to the main process, such as test step 1 is completed. The main process decides the next operation according to the feedback information of the child process. When the child process executes the second step, it continues to feed back status information to the main process. Thus, the test framework can track the specific steps of the test case execution. After child process 1 completes all test steps of the test case, it returns the execution result and end execution information to the main process. After receiving the end execution signal returned by child process 1, the main process creates child process 2 and executes test case 2. Figure 1The test framework can execute test cases including restart test cases and non-restart test cases. For restart test cases, the test framework will save the process context information of the test case and resume the execution of the test case after restart based on the process context information. After the computing device 100 restarts, the test framework first determines whether there is process context information of the test case saved. If there is, it indicates that the test case executed before the restart is a restart test case, and then the execution of the test case is resumed based on the process context information. If not, it indicates a restart for other reasons, such as the user turning on the machine or restarting after a power outage.

[0057] Figure 2 It is a schematic flowchart of a test method for single-machine testing provided by an embodiment of the present application. As Figure 2 shown, the test method for restart testing provided by an embodiment of the present application can be applied to the test execution process shown in the computing device 100 as Figure 2 shown.

[0058] As Figure 2 shown, the test framework 210 executes test cases 1-N in a predefined order. Each test case represents a test task, and each case contains at least one step. The test framework 210 sequentially executes these steps according to the definitions of test cases 1-N.

[0059] Test cases 1-N are assigned to independent process spaces. A process space is a virtual memory area allocated by the operating system for each process, including a code segment, a data segment, a stack, and file descriptors, etc. During the test, test cases 01-0N have their own process contexts, including information such as virtual memory, register status, process identifier (PID), file descriptors, and signal handling.

[0060] The test framework 210 starts the processes of test cases 1-N and can create multiple child processes as needed to process different tasks in parallel. The processes of test cases 1-N and the child processes run in their respective process spaces. The execution of the test framework for the first test case (hereinafter also referred to as the test case) including a restart operation is described below.

[0061] The test framework 210 processes test cases containing reboot operations through the reboot processing module. A first piece of information is defined in the test case, and when the test case is executed, the first piece of information is passed to the test framework 210. The test framework 210 obtains the first piece of information passed by the test case through the information receiving module. Based on the first piece of information, the test framework 210 collects the process context information of the test case through the context collection module. The collected context information is saved in the context information saving module. Exemplarily, the first piece of information is "NeedReboot". It should be noted that the specific composition and implementation manner of the reboot processing module are only examples, and the embodiments of this application aim to demonstrate a possible implementation manner of the test framework. In practical applications, the internal design of the reboot processing module can be flexibly adjusted and optimized according to specific requirements and business scenarios. Figure 2 Only the reboot processing module and its function implementation are described, but this does not mean that the test framework 210 only includes this single module.

[0062] Taking test case 1 as an example for illustration, test case 1 includes three steps, step 1 - step 3, and the test framework 210 executes these steps in sequence. Step 2 includes an operation to reboot the operating system. When executing step 2, the test framework 210 receives the first piece of information passed by test case 1, and the first piece of information indicates that the test case needs to be rebooted at the current test step. Based on the first piece of information, the test framework 210 suspends the process P0 of the current test case 1 and its child processes P1 - Pn, and then saves the context information of the process P0 and its child processes P1 - Pn, including virtual memory: saving the memory mapping information of the process to ensure that its memory layout can be reconstructed during reboot or recovery. Register status: recording the values of the program counter (PC) and other registers. File descriptors: saving the status of all open files, including file pointers, file offsets, etc. Signal handling information: recording the currently registered signal handlers of the process. Shared memory: recording the shared memory segments used by the process. And other resource information: such as timers, thread information, etc. The embodiments of this application do not limit the specific content of the process context information of the test case.

[0063] After the process context information of the process P0 of test case 1 and its child processes P1 - Pn is saved, the test framework 210 performs the reboot operation, and the operating system of the computing device 100 reboots. In the boot startup program, after the operating system of the computing device 100 reboots, the test framework 210 starts. The test framework 210 uses the process context information of the process P0 of test case 1 and its child processes P1 - Pn saved before the reboot to restore the process P0 of test case 1 and its child processes P1 - Pn, so that the test framework 210 can continue to execute step 3 of test case 01.

[0064] It should be noted that the test framework of the embodiments of the present application can also be applicable to automated tests including restart operations in various scenarios such as virtual machines and containers. The test framework can execute the test method for implementing the restart test by being deployed in a virtual machine or a container.

[0065] Next, in combination with the computing device 100 provided above, a test method provided by an embodiment of the present invention will be introduced in detail.

[0066] Figure 3 It is a schematic flowchart of the test method provided by an embodiment of the present application. The embodiments of the present application can be applied in the test framework 210. As Figure 3 shown, the test method provided by the embodiments of the present application at least includes the following steps:

[0067] Step 301: Execute multiple test cases through the test framework; the multiple test cases include a first test case, and the first test case includes a test step that requires a restart operation.

[0068] The test framework 210 is deployed on the computing device 100, and the test cases are automatically executed through a script. The execution of the test cases also takes place on the computing device 100.

[0069] For test cases that do not involve a restart operation, the first information will not be passed to the test framework. The test framework executes the test cases in the normal execution flow order. After executing this test case, the next test case is continued to be executed in the order of the test plan.

[0070] In a possible implementation, the test framework is set to start automatically when the computer is powered on before executing the test cases. In this way, after restarting the computing device 100, the test framework can be self-started to execute the test cases.

[0071] Step 302: When executing the test step that requires a restart operation, receive the first information; the first information indicates that the first test case requires a restart operation; the restart operation is to restart the operating system.

[0072] For test cases that involve a restart operation, when executing the step including the restart operation, the test case passes the first information to the test framework. Exemplarily, the first information is "Reboot".

[0073] In another possible implementation, for a test case including a restart operation, the first information is written into a common configuration file. The test framework 310 reads the first information from the configuration file and, based on the first information, saves the context information of the test case process and performs a restart operation. For example, "test_case_1": step_3: {"Reboot"}, "test_case_5": step_2: {"Reboot"}, the test framework 310 sequentially executes each step of the test case, and reads the parameters in the configuration when the step is executed.

[0074] Step 303: Based on the first information, save the context information of the first test case process.

[0075] The test framework obtains and saves the process context information of the current test case.

[0076] Taking the test method for implementing a restart test as an example, which is executed in a computing device and the operating system in the computing device is a Linux system. In a possible implementation, the test framework suspends the process of the test case through the ptrace system call, pauses the execution of the process, and facilitates saving its context information. The process of the test case is the process for executing the test case. Ptrace is a system call on the Linux operating system, which is used to implement functions such as debugging, controlling, and monitoring of processes.

[0077] Figure 4 This is a schematic diagram of the process for saving the process context of the test case provided by the embodiment of the present application. As Figure 4 shown,

[0078] Step 401: The test framework receives the first information passed by the test case.

[0079] The test case including a restart operation passes the first information to the test framework.

[0080] Step 402: The test framework suspends the test case process and its child processes.

[0081] Before performing the suspension operation, the test framework needs to obtain the process ID (PID) of the target process. The PID is used to identify and operate the target process.

[0082] For example, in the Linux system, the test framework notifies the operating system of the target test case process ID to be controlled through the ptrace system call. At this time, the target process will suspend execution and enter a suspended state. After the ptrace call is successful, the target process will enter a suspended state, and the test framework starts to read the context information of the target test case process.

[0083] Step 403: The test framework collects the process context information of the test case.

[0084] The / proc file system is a virtual file system in the Linux operating system that provides an interface to access kernel and process information. By accessing these files, various internal information of the operating system can be viewed. Each running process has a directory named after its process ID (pid) under / proc (such as / proc / 1234). These directories contain various information related to the process. For example, / proc / pid / mem: the virtual memory of the process, which can be accessed through system calls such as ptrace.

[0085] In a possible implementation, the test framework obtains the memory content of the target test case process by reading / proc / pid / mem. The memory content includes all virtual memory pages of the target test case process, and the virtual memory pages include the stack, heap, data segment, etc. Virtual memory is the address space used by each process in the operating system. Saving the virtual memory of the process can ensure that the memory state of the test case process can be reconstructed during recovery, including the stack, heap, and other dynamically allocated memory.

[0086] In a possible implementation, the test framework obtains the CPU register status of the target test case process by using ptrace (PTRACE_GETREGS), including the program counter (PC), stack pointer (SP), general-purpose registers, etc. PTRACE_GETREGS is used to read the current register status of the process. Saving the register status enables the framework to restore the execution position and context of the process. When the process of the test case is restored, it is necessary to know the position of the program counter in order to continue execution from the correct place.

[0087] In a possible implementation, the test framework determines the file corresponding to each file descriptor by reading the links in the / proc / pid / fd directory, and obtains the file descriptors and status of the target test case process.

[0088] In a possible implementation, the test framework saves the status of TCP / IDP sockets, including port numbers, connection status, buffer content, etc. The test framework obtains network connection information by reading the files in the / proc / pid / tcp and / proc / pid / udp directories.

[0089] In a possible implementation, the test framework records the registered signal handlers. The test framework obtains the current signal handling information of the test case process by reading the files in the / proc / pid / status and / proc / pid / siginfo directories. In the / proc / pid / status file, various status information of the test case process is included, including information such as signals, memory, and threads. This file lists the status of the signal handlers. The / proc / pid / siginfo file contains detailed information about all the received signals of the test case process and can be used to obtain the signal sending history. Each process has signal handlers during runtime, and these handlers define how the process responds to different signals. Saving the information of the signal handlers helps the framework understand how the process currently responds to signals. In this way, during recovery, the reception of signals can be simulated to restore the signal response mechanism of the process and ensure that the test case continues to execute under the same signal conditions.

[0090] In a possible implementation, the test framework saves the shared memory segments of the test case process. Shared memory is a mechanism that allows multiple processes to communicate. It allows different processes to access the same memory area. If a process is using shared memory, the test framework needs to save the state of the shared memory area. The test framework uses proc / sysvipc / shm to view and operate on the shared memory information.

[0091] In a possible implementation, the test framework saves timer, thread information, etc. A timer is a mechanism in the operating system used to delay the execution of tasks and usually triggers a signal or executes a certain callback function after a specified time. A process can contain multiple threads, and these threads share the same address space, but each thread has its own execution stack, registers, and status. The test framework uses proc / pic / timers to obtain timer information and obtains thread information through the subdirectories in the proc / pic / task directory.

[0092] Step 404: The test framework saves the collected process context information to a file.

[0093] After the test case process is paused, the test framework saves information such as the memory, registers, file descriptors, and network connections of the test case process. Through the ptrace call, the content of the process memory, the status of the registers, etc. can be obtained. The file descriptors of the process and other system resources are also recorded and saved in the specified path.

[0094] Step 304: Perform a restart operation.

[0095] After the test framework performs the restart operation, the operating system of the computing device 100 restarts.

[0096] Step 305: After the test framework is started, based on the saved context information of the first test case process, resume the execution of the first test case through the test framework.

[0097] After the operating system restarts, the test framework starts, and the context information recovery of the test case process begins.

[0098] Figure 5 This is a schematic diagram of the test case process context information recovery flow provided by the embodiments of the present application. As Figure 5 shown, the process of saving the context of the test case process includes:

[0099] Step 501: Create a new process.

[0100] Before Figure 4 shown, save the context information of the test case process before restart. After restart, the test framework restores based on the saved context information.

[0101] In a possible implementation, the test framework uses the clone system call to recreate the process. clone is a system call in the Linux system for creating new processes. After the new process is created, put the process in a suspended state.

[0102] Step 502: Load the memory mapping in the new process.

[0103] According to the memory mapping information saved in step 403.

[0104] In a possible implementation, the test framework uses the mmap system call to allocate virtual address spaces for each memory segment and set appropriate permissions (such as read-only, writable, executable), and recreate the memory layout of the new process.

[0105] Step 503: Restore the register state.

[0106] The test framework writes the register state saved in step 403 back to the new process through ptrace (PTRACE_GETREGS).

[0107] In a possible implementation, PTRACE_SETREGS is used to set the register state of the process to the previously saved value. For example, the rip register points to the next instruction to be executed, and the rsp register points to the current stack position.

[0108] Step 504: Reconstruct the file descriptors and network connections.

[0109] Based on the file descriptors and network connection information saved in step 403, the test framework reopens all file descriptors, sets their offsets and flags, and restores the network connection.

[0110] In a possible implementation, the test framework reopens files through the open system call, recreates sockets through the socket system call, and assigns these files to the file descriptors of the new process. If the file descriptors have been released or invalidated after the process restarts, the test framework needs to re - establish the connection. The test framework uses the fcntl system call to set the flags (such as non - blocking mode, file locking, etc.) and offsets of the file descriptors to restore the read - write position or other configurations of the files.

[0111] In a possible implementation, the test framework uses system calls such as connect, bind, listen, etc. to rebind the previously disconnected socket to the specified address and port and restore the connection state of the socket.

[0112] Step 505, start a new process.

[0113] In a possible implementation, the framework sends a SIGCONT signal to the newly created process through the ptrace system call. The SIGCONT signal notifies the operating system to resume the execution of the process from the paused state. After the new process receives the SIGCONT signal, the operating system restores its state to the state before it was paused and continues to execute the original code path.

[0114] In the embodiments of the present application, an automated test involving restart operations and non - restart operations can be completed using a single computing device. By suspending the test process and its child processes before restarting and saving their context information, and restoring the processes through the saved process context information after restarting, the next test step before restarting is continued. The script of the test framework integrates the test method provided by the embodiments of the present application, and does not require many changes in the test cases. This facilitates the consistency and reliability of the automated test results and reduces the cost of hardware resources.

[0115] Figure 3 What is shown is only the basic embodiment of the method in the embodiments of the present application. Based on this, with certain optimizations and expansions, other preferred embodiments of the method can also be obtained.

[0116] On the basis of the foregoing embodiments, the embodiments of the present application describe the test process in more detail and perform a certain degree of optimization. The test cases may have affected the environment (such as file system mount points, network configurations, etc.). The test framework can find the differences caused by the test cases to the environment and restore the environment through the restoration process. Specifically, Figure 6Another schematic diagram of the test process on a single machine provided by the embodiments of this application. As Figure 7 shown, before step 1 of test case 1 is executed, the test framework saves the first environment information, and the first environment information indicates the first environment before the test case is executed. The environment information includes file system mount points, system processes, network configurations, service statuses, kernel modules, se l inux statuses, kernel parameters, firewall rules, etc. It should be noted that the above environment information is only an exemplary description and may also include more or less environment information.

[0117] Before the test framework 310 executes step 1 of test case 1, the test framework 310 first collects the first environment information before the test case is executed through the environment information collection module, and the first environment information indicates the first environment before the test case is executed.

[0118] When the test framework 310 executes step 2 of test case 1, the test case passes the first information and the second information to the test framework 310. The second information indicates whether the test case needs to perform the first environment restoration after restart. When the second information indicates that the test case needs to perform the first environment restoration after restart, for example, when the second information is TRUE, the test framework 310 obtains the second environment information before step 2 is executed through the environment information collection module. The second environment information and the first environment information may be the same or different according to the execution situation of step 1 of test case 1. If step 1 does not modify the first environment information, then the second environment information after step 1 is executed is the same as the first environment information. If step 1 modifies the environment information, then the second environment information after step 1 is executed is different from the first environment information.

[0119] Based on the first information, the test framework suspends the process of test case 1 and its child processes and saves its process context information. When the second information is TRUE, the environment information comparison module compares the first environment information and the second environment information to obtain the first environment difference, and this difference indicates the changes made to the environment information by the test steps before restart.

[0120] The test framework 310 performs a restart operation. After restart, the test framework 310 starts. The test framework 310 checks the second information. For example, when the value of the second information is TRUE, the environment information collection module obtains the third environment information, and the third environment information indicates the third environment after the test case performs the restart operation.

[0121] The environment information comparison module compares the first environment difference and the third environment to obtain the second environment difference. The test framework 310 restores the second environment difference and restores the environment information to the environment information before restart.

[0122] Based on the test case process context information saved before restart, the test framework 310 restores the test case process context information and continues to execute the test case.

[0123] Figure 7 It is a flowchart of a test method provided by an embodiment of the present application. As Figure 6 and Figure 7 shown, this embodiment can be applied to the test framework 310. The test method provided by the embodiment of the present application at least includes the following steps:

[0124] Step 701: Execute multiple test cases through the test framework; the multiple test cases include a first test case, and the first test case includes a test step that requires a restart operation. The test framework saves the first environment information, and the first environment information indicates the first environment before the execution of the first test case.

[0125] For detailed content, refer to the description in Figure 5 and will not be elaborated here.

[0126] In a possible implementation, the test framework is set to start automatically when the device boots.

[0127] Step 702: When executing a test step that requires a restart operation, receive the first information and the second information, and the second information indicates whether the test case needs to perform a first environment restoration after restart.

[0128] For detailed content, refer to the description in Figure 7 and will not be elaborated here.

[0129] Step 703: Based on the first information, save the context information of the first test case process;

[0130] For detailed content, refer to the description in step 303 and will not be elaborated here.

[0131] Step 704: When the second information indicates that the test case needs to perform a first environment restoration after restart, the test framework compares the first environment information and the second environment information to obtain a first environment difference, and the second environment information indicates the second environment before the test case performs the restart operation.

[0132] It should be noted that the present application embodiment does not limit the execution order of steps 703 and 704.

[0133] Exemplarily, taking the restoration of kernel parameter differences as an example, the test framework 310 uses sysctl -a to query the kernel parameters in the current system and obtains the first environment information: A = 1, B = 0, C = 10;

[0134] When executing the step involving the restart operation, the test framework 310 uses sysctl -a to query the kernel parameters in the current system and obtains the second environment information: A = 1, B = 2, C = 10, D = 1;

[0135] Compare the first environment information with the second environment information to obtain the first environment difference, that is, the changes made by the test case to the system parameters are: A = 0, B = 2, C = 1.

[0136] Step 705: The test framework performs a restart operation.

[0137] Step 706: After the test framework starts, the test framework compares the third environment information with the first environment difference to obtain the second environment difference. The third environment information indicates the third environment after the test case performs a restart operation.

[0138] In a possible implementation, after the test framework restarts, it determines again whether the second information indicates that the test case needs to restore the first environment after restart. For example, the test framework saves the second information to a file. After the test framework restarts, it reads the second information from the file to determine whether it is necessary to perform the first environment restoration.

[0139] For example, the test framework 310 uses sysctl -a to query the kernel parameters in the current system and obtains the third environment information: A = 1, B = 0, C = 1, D = 0, E = 1;

[0140] Compare the third environment information with the first environment difference to obtain the second environment difference, B = 2, D = 1. Since the parameters E and C do not belong to the changes made by the test case to the environment, they are not included in the difference.

[0141] Step 707: Based on the second environment difference, the test framework restores the third environment. For example, the test framework 310 uses the sysctl instruction to restore the changes made by the test case to the system parameters: sysctl -n B = 2, sysctl -n D = 1.

[0142] Step 708: Based on the saved context information of the first test case process, the test framework resumes the execution of the first test case.

[0143] For the detailed content, refer to the description in step 303 and will not be elaborated here.

[0144] In the embodiments of the present application, after the computing device is restarted, the environment information is restored first before restoring the test case process context information. Specifically, the environment information before the execution of the test case including the restart step is used. This test case executes the environment information before the restart and the environment information after the restart, and the environment information to be restored is obtained by comparing with less data, making the test results after the restoration and execution of the test case more reliable.

[0145] Figure 3 The shown is only the basic embodiment of the method of the embodiments of the present application. Based on this, with certain optimizations and expansions, other preferred embodiments of the method can also be obtained.

[0146] In the embodiments of the present application, based on the foregoing embodiments, the test process is described in more detail and optimized to a certain extent. Specifically, Figure 8 This is a schematic diagram of another test process on a single machine provided by the embodiments of the present application. As Figure 8 shown, the test framework 310 receives the first parameter, the second parameter, and the third parameter. The third parameter indicates whether the test case needs to perform a second environment restoration after restart. Compared with the first environment restoration, the second environment restoration can be specified by the tester for the specific parameters to be restored, increasing the flexibility of the test.

[0147] When the test framework 310 executes step 2 of test case 1, the test case passes the first parameter, the second parameter, and the third parameter to the information receiving module. After restart, the environment restoration module performs a second environment restoration according to the third parameter. The second environment restoration can be performed before the first environment restoration or after the first environment restoration. The embodiments of the present application do not limit this. Here is only a brief description of the method. For the detailed content of this method, see the following description.

[0148] In one embodiment, based on the foregoing embodiments, the test process is described in more detail and optimized to a certain extent. Figure 9 This is a schematic diagram of a test method provided by the embodiments of the present application. As Figure 9 shown, the method in this embodiment includes:

[0149] Step 901, the test framework executes multiple test cases; the multiple test cases include the first test case, and the first test case includes a test step that requires a restart operation. The test framework saves the first environment information, and the first environment information indicates the first environment before the execution of the first test case.

[0150] The test framework saves the first environment information, and the first environment information indicates the first environment before the execution of the first test case.

[0151] For the detailed content, see the description of step 701 and will not be repeated here.

[0152] Step 902: The test framework receives the first piece of information, the second piece of information, and the third piece of information. The second piece of information indicates that the test case needs to perform a first environment restoration after restart, and the third piece of information indicates that the test case needs to perform a second environment restoration after restart.

[0153] In a possible implementation, the third piece of information can be in the form of key-value pairs. For example, parameter 1 = value 1, parameter 2 = value 2. When the test framework 310 executes the test case, it will parse this information.

[0154] In a possible implementation, the test framework saves the second piece of information and the third piece of information in a file.

[0155] Step 903: Based on the first piece of information, the test framework saves the context information of the test case process and its child processes.

[0156] For the detailed content, please refer to the description in Step 302 and will not be elaborated here.

[0157] Step 904: The test framework determines whether the value of the second piece of information is valid.

[0158] When the value of the second parameter is valid, Step 905 is executed; when the value of the second piece of information is valid, Step 906 is executed.

[0159] Step 905: When the second piece of information indicates that the test case needs to perform a first environment restoration after restart, the test framework

[0160] obtains a first environment difference by comparing the first environment information and the second environment information. The second environment information indicates the second environment before the test case performs the restart operation.

[0161] For the detailed content, please refer to the description in Step 704 and will not be elaborated here.

[0162] Step 906: The test framework performs a restart operation.

[0163] Step 907: The test framework executes the third piece of information to perform a second environment restoration.

[0164] To make the environment information after restart consistent with the environment information before restart, the tester will manually write some parameter information for forced restoration. This can be used as a supplement to the first environment restoration to ensure the accuracy of the test environment.

[0165] Step 908: The test framework determines whether the value of the second piece of information is valid.

[0166] When the value of the second piece of information is valid, Step 909 is executed; when the value of the second piece of information is valid, Step 911 is executed.

[0167] Step 909: The test framework compares the third environment information with the first environment to obtain the second environment difference. The third environment information indicates the third environment after the test case performs a restart operation.

[0168] For the detailed content, please refer to the description in Step 706 and will not be elaborated here.

[0169] Step 910: Based on the second environment difference, the test framework restores the third environment.

[0170] For the detailed content, please refer to the description in Step 807 and will not be elaborated here.

[0171] It should be noted that the execution order of restoring the second environment and the third environment in the embodiments of the present application is not limited.

[0172] Step 911: Based on the saved context information of the first test case process, the first test case is restored and executed through the test framework.

[0173] For the detailed content, please refer to the description in Step 303 and will not be elaborated here.

[0174] In the embodiments of the present application, while automatically performing environment restoration, the environment information to be restored can also be defined manually in advance, which is more flexible.

[0175] Based on the above-provided test method, the specific application of the test method will be described. Figure 10 It is a schematic flowchart of the specific application of a test method provided for the embodiments of the present application. As Figure 10 shown, the specific content includes:

[0176] Step 1001: Add the test framework to the startup program and run the test framework.

[0177] Step 1002: The test framework determines whether there is a process context of the currently running test case.

[0178] When there is a process context of the current test case, Step 1003 is executed. Otherwise, Step 1002 is executed.

[0179] Step 1003: Perform the process of restoring the first test case, and the first test case continues to execute.

[0180] The restoration process of the first test case is Figure 2 the restoration process of the environment information and process context information for the execution of the first test case as described above.

[0181] Step 1004: If the next test step of the first test case also includes a restart operation, send the first message to the test framework.

[0182] Step 1005: The test framework saves the context information of the process.

[0183] Step 1006: The test framework performs a restart operation.

[0184] Step 1007: Resume the test case process, and the test case continues to execute until completion.

[0185] In the embodiment of the present application, there are multiple test steps in the first test case that include restart operations. Then, the test framework will perform the test case recovery process multiple times, that is, for each restart operation, the process context information and / or environment information will be restored.

[0186] Step 1008: Execute the next test case.

[0187] After the first test case is executed, the next test case is executed in the predefined order.

[0188] Based on the same concept as the method embodiment of the present application, the embodiment of the present application also provides a test device. The test device includes several modules, and each module is used to execute each step in the test method provided by the embodiment of the present application. There is no limitation on the division of the modules here. Those skilled in the art can clearly understand that in actual applications, each step in the test method provided by the embodiment of the present application can be allocated to different modules according to needs, that is, the internal structure of the device is divided into different modules to complete all or part of the functions described above. Each module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more modules can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the modules are only for the convenience of distinguishing each other and do not limit the protection scope of the present application. The specific working process of the modules in the above device can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0189] Exemplarily, the test device is used to execute the test method provided by the embodiment of the present application. Figure 11 It is a schematic structural diagram of the test device provided by the embodiment of the present application. As Figure 11As shown, the test device provided by the embodiment of the present application is used to test the framework to execute test cases, and the test framework is set to start automatically when the device is powered on; the device includes: an execution module for executing a plurality of test cases; the plurality of test cases include a first test case, and the first test case includes a test step that requires a restart operation; a receiving module for receiving a first message, the first message is sent when the first test case executes to the test step that requires a restart operation, and the first message indicates that the first test case needs to perform a restart operation; the restart operation is to restart the operating system; a saving module for saving the context information of the first test case process based on the first message; a restart module for performing the restart operation; a recovery module, when the test framework starts, for resuming the execution of the first test case through the test framework based on the saved context information of the first test case process.

[0190] Based on the same concept as the method embodiment of the present application, the embodiment of the present application also provides a computing device, as Figure 12 As shown, the computing device provided by the embodiment of the present application includes a processor 1201, a memory 1202, and a communication interface 1203.

[0191] In the embodiment of the present application, the processor 1201 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0192] The memory 1302 may include a large-capacity memory for data or instructions, thereby providing a storage space. The storage space stores the operating system and executable program code of the network device, and may include, but is not limited to: Windows system (an operating system), Linux system (an operating system), HarmonyOS (an operating system), etc., which are not limited herein.

[0193] By way of example and not limitation, the memory 1202 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 1202 may include removable or non-removable (or fixed) media. In a suitable case, the memory 1202 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1202 is a non-volatile solid-state memory.

[0194] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory may include one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software that may include computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods in the present application.

[0195] Exemplarily, a computer program may be stored on the memory 1202, and when the processor 1201 executes the computer program, the steps in the above method embodiments are implemented. Alternatively, when the processor 1201 executes the computer program, the functions of each module in the above device embodiments are implemented. Exemplarily, the computer program may be divided into one or more modules / units, and the one or more modules / units may be a series of computer program instruction segments capable of completing specific functions. The one or more modules / units are stored in the memory 1202 and executed by the processor 1201 to complete the present application. For example, the computer program may be divided into multiple modules, such as the modules in the device described above.

[0196] The communication interface 1203 is used for sending and receiving data. For example, it sends the data processed by the processor 1201 to other computing devices, or receives the data sent by other computing devices, etc.

[0197] Of course, for simplicity, Figure 12 only some of the components related to the present application in the computing device 1200 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application scenarios, the computing device 1200 may further include any other appropriate components. Additionally, the computing device may be a network device such as a desktop computer, a notebook, a palm computer, and a cloud server, etc. Those skilled in the art can understand that Figure 12This is only an example of the computing device 1200, which does not constitute a limitation on the computing device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the computing device may further include an input device, an output device, a network access device, a bus, etc. Exemplarily, the input device may be a microphone array and may also include, for example, a keyboard, a mouse, and so on. Exemplarily, the output device may output various information to the outside and may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0198] Based on the method in the above embodiments, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program runs on a processor, the processor is caused to execute the method in the above embodiments.

[0199] Based on the method in the above embodiments, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a processor, the processor is caused to execute the method in the above embodiments.

[0200] It should be understood that each step of the above method embodiment can be completed by a logic circuit in hardware form or an instruction in software form in the processor.

[0201] It can be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in some possible implementation manners, the steps in the above embodiments can be selectively executed according to actual situations, can be partially executed, or can be fully executed, and no limitation is made here.

[0202] It can be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0203] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not described in detail or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0204] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0205] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present invention are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0206] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0207] It should also be noted that in the devices, equipment, and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present disclosure.

[0208] The above description has been given for purposes of illustration and description. In addition, this description does not intend to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

[0209] It can be understood that the various numerical numbers involved in the embodiments of the present invention are only for the convenience of description and are not used to limit the scope of the embodiments of the present invention.

Claims

1. A testing method, characterized in that, The test framework is set to start automatically when the device boots up; the method includes: Executing multiple test cases through the test framework; the multiple test cases include a first test case, and the first test case includes a test step that requires a restart operation; when executing the test step that requires a restart operation, receiving a first piece of information; the first piece of information indicates that the first test case requires a restart operation; the restart operation is to restart the operating system; Based on the first piece of information, saving the context information of the first test case process; Performing the restart operation; After the test framework starts, based on the saved context information of the first test case process, resuming the execution of the first test case through the test framework.

2. The method according to claim 1, wherein Before the test framework receives the first piece of information, the method further includes: the test framework saves first environment information, and the first environment information indicates the first environment before the execution of the test case; The test framework receiving the first piece of information includes: The test framework receives the first piece of information and a second piece of information, and the second piece of information indicates whether the first test case needs to perform first environment restoration after restart; Before the test framework performs the restart operation, the method further includes; When the second piece of information indicates that the first test case needs to perform first environment restoration after restart, the test framework compares the first environment information and the second environment information to obtain a first environment difference, and the second environment information indicates the second environment before the first test case performs the restart operation; the first environment difference indicates the difference between the first environment information and the second environment information; After the test framework performs the restart operation, the method further includes; The test framework compares the third environment information and the first environment difference to obtain a second environment difference, and the third environment information indicates the third environment after the first test case performs the restart operation; Based on the second environment difference, the test framework restores the third environment.

3. The method according to claim 2, wherein When the test framework receives the first piece of information and the second piece of information, the test framework also receives a third piece of information, and the third piece of information indicates that the first test case performs second environment restoration after restart; Before the test framework compares the third environment information and the first environment difference to obtain a second environment difference, the method further includes: based on the third piece of information, the test framework performs second environment restoration.

4. The method according to claim 1, wherein When the test framework receives the first piece of information, the test framework also receives a third piece of information; the third piece of information indicates that the first test case performs second environment restoration after restart; After the test framework performs the restart operation, the method further includes; Based on the third piece of information, the test framework performs second environment restoration.

5. The method according to claim 1, wherein The method further includes: When the test framework receives the first piece of information, the test framework also receives a second piece of information, and the second piece of information indicates whether the first test case needs to perform first environment restoration after restart; Before the test framework performs the restart operation, the method further includes: The test framework obtains second environment information, and the second environment information indicates the environment before the first test case performs the restart operation; After the test framework performs a restart operation, the method further includes: The test framework obtains third environment information, where the third environment information indicates the environment after the restart operation of the first test case; The test framework compares the second environment information with the third environment information to obtain a second environment difference; Based on the second environment difference, the test framework restores the third environment.

6. The method according to any one of claims 1-5, characterized in that, The environment information includes at least one of the following: file system mount point, system process, network configuration, service status, kernel module, Selinux status, kernel parameters, firewall rules.

7. The method according to claim 1, wherein After the test framework performs a restart operation, the test framework checks whether there is first test case process context information. If there is first test case process context information, based on the saved context information of the first test case process, restore the process context of the first test case; If there is no first test case process context information, execute the test cases in a predefined order.

8. A testing device, characterized in that, For the test framework to execute test cases, set the test framework to start automatically when the device boots up; the apparatus includes: An execution module, configured to execute multiple test cases; the multiple test cases include a first test case, and the first test case includes test steps that require a restart operation; A receiving module, configured to receive first information, where the first information is sent when the first test case executes to the test step that requires a restart operation, and the first information indicates that the first test case needs to perform a restart operation; the restart operation is to restart the operating system; A saving module, configured to save the context information of the first test case process based on the first information; A restart module, configured to perform the restart operation; A recovery module, when the test framework starts, configured to resume execution of the first test case through the test framework based on the saved context information of the first test case process.

9. A computing device, characterized in that, Includes a processor and a memory; wherein, The memory is used to store programs; The processor is configured to execute the programs stored in the memory, and when the programs stored in the memory are executed, execute the method according to any one of claims 1 to 7.

10. A computer program product comprising instructions, characterized in that, When the instruction is run by a computing device, the computing device is caused to execute the method according to any one of claims 1 to 7.

Images