An automatic testing method based on reverse engineering control extraction technology

Abstract

The application discloses an automatic testing method based on reverse engineering control extraction technology, and relates to the field of software automatic testing, and comprises the following steps: S1, a technical framework of the software to be tested is recognized through static analysis; S2, control distribution, attributes and hierarchical relationships of the software to be tested are recognized through dynamic analysis; S3, an interactive interface of a test analysis monitoring dynamic link library and a test tool is defined; and S4, test cases are applied, and automatic testing is performed; the application can be applied to various desktop application programs under Windows and Linux systems; in the automatic testing process, a tester does not need to manually establish a control mapping relationship, only needs to automatically position the control by the test tool, does not need to simulate mouse and keyboard input, only needs to apply different test cases to different controls, and directly operates on a memory layer of the control by the tool, so that the test target is efficiently completed.

Description

Technical Field

[0001] This invention relates to the field of automated software testing, and more specifically to an automated testing method based on reverse engineering control extraction technology. Background Technology

[0002] The statements in this section are provided only as background information in relation to this disclosure and may not constitute prior art.

[0003] Existing automated software testing solutions mainly include the following:

[0004] 1. Based on operation macros: Testers use macro recording tools to record mouse and keyboard test processes for each interface. After the macros are recorded, test cases are applied to replay the pre-recorded macros for each interface.

[0005] This approach has the following drawbacks: it lacks universality; when the position, size, or occlusion properties of software interface controls change, the macros of the old interface are no longer applicable to the new interface; at the same time, it is prone to human error, and it cannot be guaranteed that the macros recorded by the tester are completely correct, requiring multiple verifications; and the process is cumbersome, as each macro requires manual recording and modification.

[0006] 2. Handle-based: A handle is a unique integer used by the Windows system to identify an object created or used by an application. It is used like a pointer, but it is not a pointer. Through the handle, you can obtain every interface of the software program, every control in the interface, and their hierarchical relationship. Testers can obtain the underlying instance of the control in the system through the handle, manually establish the mapping relationship between the instance and the control, and then apply test cases to write test logic code.

[0007] This approach has the following drawbacks: it lacks universality; it requires manually establishing the mapping relationship between the underlying instance of the handle and the interface control, meaning that each interface has an independent mapping table; and the handle is only applicable to the Windows platform.

[0008] 3. Based on traditional digital image recognition; Image recognition is divided into traditional digital image processing methods and deep learning image recognition methods; Digital image processing methods can use traditional algorithms such as OCR optical character recognition, SVM support vector machine, template matching, etc., to extract the surface features of the control and dynamically recognize the control through the features.

[0009] This approach has the following drawbacks: it requires developers to customize the feature recognition algorithm for the control, and the algorithm may fail or malfunction when the surface features of the control change.

[0010] 4. Deep learning-based image recognition: Deep learning methods require the creation of a large training set and the construction of an algorithm model for training, through which the control object is inferred.

[0011] This solution has the following drawbacks:

[0012] Model training is costly; the process of creating training sets and building models is cumbersome, and the expected results are not guaranteed after training. Model inference is also costly; model inference requires powerful graphics card hardware and consumes a lot of computer performance, while client machines running the software may not have graphics cards and cannot meet the running requirements. Summary of the Invention

[0013] The purpose of this invention is to address the problems existing in the prior art by providing an automated testing method based on reverse engineering control extraction technology, applicable to various desktop applications under Windows and Linux systems. During automated testing, testers do not need to manually establish control mapping relationships; the testing tool automatically locates the controls. There is no need to simulate mouse and keyboard input; different test cases are applied to different controls, and the tool directly operates at the memory level of the controls, efficiently achieving the testing objectives. This solves the aforementioned problems.

[0014] The technical solution of the present invention is as follows:

[0015] An automated testing method based on reverse engineering control extraction technology includes:

[0016] Step S1: Identify the technical framework of the software under test by statically analyzing Windows PE and Linux ELF format files;

[0017] Step S2: Identify the control distribution, attributes, and hierarchical relationships of the software under test by dynamically analyzing program memory data and function calls;

[0018] Step S3: Define the interaction interface between the test analysis and monitoring dynamic link library and the testing tools;

[0019] Step S4: Apply test cases and execute automated tests.

[0020] Further, step S1 includes:

[0021] Step S11: Input the program to be tested;

[0022] Step S12: Obtain all imported dynamic link libraries;

[0023] Step S13: Determine the technical framework;

[0024] Step S14: Based on the results of the technical framework judgment, obtain the relevant parameters used by the program to be tested, providing prerequisites for the subsequent step S2.

[0025] Further, step S1 includes:

[0026] Select the program to be tested and analyze it using testing tools;

[0027] Step S12 includes:

[0028] The testing tool iterates through the import table of the program under test to obtain all referenced dynamic link libraries;

[0029] Step S13 includes:

[0030] Determine the technology framework used by the software interface based on the name of the dynamic link library;

[0031] Step S14 includes:

[0032] The test program and its dependent dynamic link libraries are parsed to obtain the relevant parameters used by the test program; the relevant parameters include: function declarations, declarations of functions imported by dynamic link libraries, and relative offsets.

[0033] Further, step S2 includes:

[0034] Step S21: Start the program to be tested;

[0035] Step S22: Obtain memory operation permissions for the target process;

[0036] Step S23: Monitor the constructors of all controls and record the control types and control pointers;

[0037] Step S24: Obtain control properties and establish hierarchical relationships.

[0038] Further, step S21 includes:

[0039] Use the testing tool to start the program under test, so that the program is loaded by the operating system.

[0040] Further, step S22 includes:

[0041] Step S221: Intercept the main function of the program to be tested. Before the main function is executed, the testing tool obtains the right to execute the program in advance.

[0042] Step S222: Call the system interface to start a remote thread, so that the remote thread loads the test monitoring and analysis dynamic library in the target process's memory space;

[0043] Step S223: Use local sockets to create a process interaction method, enabling the target process to communicate with the testing tool and execute test tool test cases;

[0044] Step S224: Return control to the main function of the program to be tested.

[0045] Further, step S23 includes:

[0046] Step S231: Determine the original constructor of the control based on the technical framework;

[0047] Step S232: The testing tool wraps the original constructor, so that when the program under test calls the original constructor, it calls the constructor wrapped by the testing tool. The wrapped constructor records the control type and control pointer. After recording, the original constructor is called again, returning control of the program.

[0048] Further, step S24 includes:

[0049] Iterate through all the recorded control pointers, obtain the parent control pointer and child control pointer lists in turn, and gradually establish the hierarchical relationship of all controls based on the parent-child relationship.

[0050] Further, step S3 includes:

[0051] The operations used in the test cases are mapped to interfaces, and the control states are described as interfaces, enabling the test tools and dynamic link libraries to interact.

[0052] Further, step S4 includes:

[0053] Based on the business logic, implement different test cases for different controls;

[0054] The testing tools communicate via inter-process communication methods and dynamic link libraries;

[0055] The testing tool breaks down test cases into multiple instructions, issues instructions according to the interface protocol description, the dynamic link library parses and executes the instructions, and reports the control status to the testing tool after execution. The testing tool compares the state of the program under test before and after executing the instructions to determine the completion status of the test cases.

[0056] Compared with existing technologies, the advantages of this invention are:

[0057] 1. An automated testing method based on reverse engineering control extraction technology, comprising: Step S1: Identifying the technical framework of the software under test by statically analyzing Windows PE and Linux ELF format files; Step S2: Identifying the control distribution, attributes, and hierarchical relationships of the software under test by dynamically analyzing program memory data and function calls; Step S3: Defining the interaction interface between the test analysis and monitoring dynamic link library and the testing tool; Step S4: Applying test cases and executing automated testing; it is applicable to various desktop applications under Windows and Linux systems; and during the automated testing process, testers do not need to manually establish control mapping relationships, only that the testing tool automatically locates the controls, without simulating mouse and keyboard input, only applying different test cases to different controls, and the tool directly operates at the memory level of the controls to efficiently complete the testing objectives.

[0058] 2. An automated testing method based on reverse engineering control extraction technology, which, compared with the "record macro" method, directly eliminates the recording and playback steps, thus improving testing efficiency.

[0059] 3. An automated testing method based on reverse engineering control extraction technology, which, compared with the "handle-based" method, eliminates the step of manually mapping the relationship between controls and handles, thus improving efficiency. Attached Figure Description

[0060] Figure 1 A flowchart of an automated testing method based on reverse engineering control extraction technology;

[0061] Figure 2 Here is the flowchart for the static analysis in step S1;

[0062] Figure 3 This is a diagram illustrating the PE file format.

[0063] Figure 4 This is a diagram illustrating the ELF file format.

[0064] Figure 5 Here is the flowchart for the dynamic analysis of step S2;

[0065] Figure 6 This is a sequence diagram illustrating the execution timing of test cases;

[0066] Figure 7 This is a schematic diagram of the directory structure of the status monitoring software in Example 2;

[0067] Figure 8 This is a flowchart illustrating the specific process of testing the status monitoring software in Example 2.

[0068] Figure 9 This is a schematic diagram of the dynamic link library used in step 3 of Example 2;

[0069] Figure 10 This is a graph showing the test results of the status monitoring software in Example 2. Detailed Implementation

[0070] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0071] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0072] Example 1

[0073] Please see Figure 1 An automated testing method based on reverse engineering control extraction technology includes the following steps:

[0074] Step S1: Identify the technical framework of the software under test by statically analyzing Windows PE and Linux ELF format files. It should be noted that static analysis is a method of analyzing program files without starting the application. Identifying the technical framework used by the program under test, such as Qt, MFC, etc., allows us to determine the corresponding analysis method in subsequent dynamic analysis.

[0075] Step S2: By dynamically analyzing program memory data and function calls, identify the distribution, attributes, and hierarchical relationships of controls in the software under test. It should be noted that dynamic analysis is a method of analyzing the memory structure and function call process of a program process during or after startup. Dynamic analysis is a prerequisite for automatically testing control interactions, as it aims to identify the specific distribution, attributes, and hierarchical relationships of controls in the program under test. Furthermore, by combining the results of static and dynamic analysis, the key information required for automatic testing, such as the type, pointer, position, size, and hierarchical relationships of each control, can be reversed. After obtaining the type and pointer of a control, it's like obtaining source code; any operation can be performed on that control, such as automatic input, output acquisition, and command execution.

[0076] Step S3: Define the interaction interface between the test analysis and monitoring dynamic link library and the testing tools;

[0077] Step S4: Apply test cases and execute automated tests.

[0078] In this embodiment, specifically, such as Figure 2 As shown, step S1 includes:

[0079] Step S11: Input the program to be tested;

[0080] Step S12: Obtain all imported dynamic link libraries;

[0081] Step S13: Determine the technical framework;

[0082] Step S14: Based on the results of the technical framework judgment, obtain the relevant parameters used by the program to be tested, providing prerequisites for the subsequent step S2.

[0083] In this embodiment, specifically, step S1 includes:

[0084] Select the program to be tested and analyze it using testing tools;

[0085] Step S12 includes:

[0086] The testing tool iterates through the import table of the program under test to obtain all referenced dynamic link libraries;

[0087] Step S13 includes:

[0088] Determine the technology framework used by the software interface based on the name of the dynamic link library; for example, if the program to be tested contains a dynamic link library named "Qt5Widget.dll", then it uses the Qt technology framework.

[0089] Step S14 includes:

[0090] The test program and its dependent dynamic link libraries are parsed to obtain relevant parameters used by the test program. These parameters include function declarations, declarations of functions imported into the dynamic link libraries, and relative offsets, which provide prerequisites for subsequent dynamic analysis. For example, in a Windows system, the test program and dynamic link libraries exist in PE format, as illustrated in the diagram below. Figure 3 As shown in Figure 4, under the Linux system, the program to be tested and the dynamic link library exist in ELF format.

[0091] In this embodiment, specifically, such as Figure 5 As shown, step S2 includes:

[0092] Step S21: Start the program to be tested;

[0093] Step S22: Obtain memory operation permissions for the target process;

[0094] Step S23: Monitor the constructors of all controls and record the control types and control pointers;

[0095] Step S24: Obtain control properties and establish hierarchical relationships.

[0096] In this embodiment, specifically, step S21 includes:

[0097] Use the testing tool to start the program under test, so that the program is loaded by the operating system.

[0098] In this embodiment, specifically, step S22 includes:

[0099] Step S221: Intercept the main function of the program to be tested. Before the main function is executed, the testing tool obtains the right to execute the program in advance.

[0100] Step S222: Call the system interface to start a remote thread, so that the remote thread loads the test monitoring and analysis dynamic link library in the target process's memory space; it should be noted that the test monitoring and analysis dynamic library is determined by the technical framework.

[0101] Step S223: After loading the test monitoring and analysis dynamic link library, use local sockets to create a process interaction method, enabling the target process to communicate with the test tool and execute test tool test cases;

[0102] Step S224: Return control to the main function of the program to be tested.

[0103] In this embodiment, specifically, step S23 includes:

[0104] Step S231: Determine the original constructor of the control according to the technical framework; for example, Qt controls will use the constructor of QWidget;

[0105] Step S232: The testing tool wraps the original constructor, so that when the program under test calls the original constructor, it calls the constructor wrapped by the testing tool. The wrapped constructor records the control type and control pointer. After recording, the original constructor is called again, returning control of the program.

[0106] In this embodiment, specifically, step S24 includes:

[0107] Iterate through all the recorded control pointers, obtain the parent control pointer and child control pointer lists in turn, and gradually establish the hierarchical relationship of all controls based on the parent-child relationship.

[0108] In this embodiment, specifically, step S3 includes:

[0109] The operations used in the test cases are mapped to interfaces, and the control states are described as interfaces, enabling the test tools and dynamic link libraries to interact.

[0110] In this embodiment, specifically, step S4 includes:

[0111] Based on the business logic, implement different test cases for different controls;

[0112] The testing tools communicate with each other through inter-process communication methods and test monitoring and analysis dynamic link libraries;

[0113] The testing tool breaks down test cases into multiple instructions. According to the interface protocol description, it issues these instructions, which are then parsed and executed by the dynamic link library. Upon completion, the library reports the control's state back to the testing tool. The testing tool compares the state of the program under test before and after instruction execution to determine the test case's completion status. Specifically, a test case execution sequence diagram is shown below. Figure 6 As shown.

[0114] Example 2

[0115] Example 2 is a specific application of an automatic testing method based on reverse engineering control extraction technology proposed in Example 1.

[0116] AutoTestDemo.exe is a status monitoring software in a large system, and its directory structure is as follows: Figure 7 As shown.

[0117] Please see Figure 8 The status monitoring software was tested using the method proposed in Example 1. The specific steps are as follows:

[0118] 1. Select AutoTestDemo.exe, but do not start the program; instead, import it into the testing tool for static analysis.

[0119] 2. Because it's a Windows system, AutoTestDemo.exe is in PE format. Parse the file according to the PE format, obtain its import table, and record it.

[0120] 3. The import table is traversed, comparing the names of the dynamic link libraries used with those of different technology frameworks. Clearly, the program uses two dynamic link libraries from the Qt technology framework: Qt5Core.dll and Qt5Widgets.dll. The technology framework is thus determined to be Qt. The used dynamic link libraries are as follows: Figure 9 As shown.

[0121] 4. Because all controls in the Qt framework inherit from the QWidget base class, the construction of a control will call the QWidget constructor. Therefore, we only need to intercept the QWidget constructor. We continue with static analysis of Qt5Widgets.dll, parsing it according to the PE format to find the relative offset address of its constructor. At this point, the static analysis process is complete.

[0122] 5. Launch the AutoTestDemo.exe program and load it into memory.

[0123] 6. Before the main function runs, intercept the main function, gain control of the program, and delay the construction of Qt controls.

[0124] 7. Because the testing tool process and the target program process are two separate processes and cannot directly access each other's memory, dynamic library injection is needed to gain operational privileges over the target process. In this example, dynamic link libraries from the Qt framework that accompany the testing tool are injected.

[0125] 8. Based on the relative offset address obtained from the static analysis in step 4, intercept the constructor of QWidget and wrap it. When the original program calls this function, call the wrapped function instead. This allows us to obtain the type of the constructed control and the control pointer.

[0126] 9. After injecting the dynamic link library and wrapper constructor, program control can be returned to the original program, which will execute the main function, begin resource initialization, and start normally.

[0127] 10. The original program starts normally and begins calling the constructors of various controls.

[0128] 11. At this point, the constructor function called by the original program has been intercepted by the testing tool and replaced with the wrapped constructor function, allowing the testing tool to indirectly obtain the control type and control pointer.

[0129] 12. Based on the control pointers, call functions such as "Get Parent Object," "Get List of Child Objects," and "Get Control Properties" to obtain the hierarchical relationship and control properties of all controls. This enables the testing tool to manipulate the target program's controls.

[0130] 13. Assign pre-prepared test cases to each control to execute the automated testing process; the test results are as follows: Figure 10 As shown.

[0131] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0132] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. An automated testing method based on reverse engineering control extraction technology, characterized in that, include: Step S1: Identify the technical framework of the software under test by statically analyzing Windows PE and Linux ELF format files; Step S2: Identify the control distribution, attributes, and hierarchical relationships of the software under test by dynamically analyzing program memory data and function calls; Step S3: Define the interaction interface between the test analysis and monitoring dynamic link library and the testing tools; Step S4: Apply test cases and execute automated tests; Step S2 includes: Step S21: Start the program to be tested; Step S22: Obtain memory operation permissions for the target process; Step S23: Monitor the constructors of all controls and record the control types and control pointers; Step S24: Obtain control properties and establish hierarchical relationships; Step S22 includes: Step S221: Intercept the main function of the program to be tested. Before the main function is executed, the testing tool obtains the right to execute the program in advance. Step S222: Call the system interface to start a remote thread, so that the remote thread loads the test monitoring and analysis dynamic library in the target process's memory space; Step S223: Use local sockets to create a process interaction method, enabling the target process to communicate with the testing tool and execute test tool test cases; Step S224: Return control to the main function of the program under test; Step S23 includes: Step S231: Determine the original constructor of the control based on the technical framework; Step S232: The testing tool wraps the original constructor, so that when the program under test calls the original constructor, it calls the constructor wrapped by the testing tool. The wrapped constructor records the control type and control pointer. After recording, the original constructor is called again, returning control of the program. Step S24 includes: Iterate through all the recorded control pointers, obtain the parent control pointer and child control pointer lists in turn, and gradually establish the hierarchical relationship of all controls based on the parent-child relationship.

2. The automatic testing method based on reverse engineering control extraction technology according to claim 1, characterized in that, Step S1 includes: Step S11: Input the program to be tested; Step S12: Obtain all imported dynamic link libraries; Step S13: Determine the technical framework; Step S14: Based on the results of the technical framework judgment, obtain the relevant parameters used by the program to be tested, providing prerequisites for the subsequent step S2.

3. The automatic testing method based on reverse engineering control extraction technology according to claim 2, characterized in that, Step S1 includes: Select the program to be tested and analyze it using testing tools; Step S12 includes: The testing tool iterates through the import table of the program under test to obtain all referenced dynamic link libraries; Step S13 includes: Determine the technology framework used by the software interface based on the name of the dynamic link library; Step S14 includes: The test program and its dependent dynamic link libraries are parsed to obtain the relevant parameters used by the test program; the relevant parameters include: function declarations, declarations of functions imported by dynamic link libraries, and relative offsets.

4. The automatic testing method based on reverse engineering control extraction technology according to claim 3, characterized in that, Step S21 includes: Use the testing tool to start the program under test, so that the program is loaded by the operating system.

5. The automatic testing method based on reverse engineering control extraction technology according to claim 1, characterized in that, Step S3 includes: The operations used in the test cases are mapped to interfaces, and the control states are described as interfaces, enabling the test tools and dynamic link libraries to interact.

6. The automatic testing method based on reverse engineering control extraction technology according to claim 1, characterized in that, Step S4 includes: Based on the business logic, implement different test cases for different controls; The testing tools communicate via inter-process communication methods and dynamic link libraries; The testing tool breaks down test cases into multiple instructions, issues instructions according to the interface protocol description, the dynamic link library parses and executes the instructions, and reports the control status to the testing tool after execution. The testing tool compares the state of the program under test before and after executing the instructions to determine the completion status of the test cases.

Images