Method and apparatus for fuzz testing software using COM technology

Abstract

The application discloses a COM technology software method for fuzzy testing, and first, the method requires software to be tested by using COM technology to realize the following function interfaces: a COM interface for starting the software, and a COM interface for accepting input. The method has the following advantages: the method can significantly improve testing efficiency, the method only needs to restart the program when the first round of testing and program exception occur, and the time of each round of testing is greatly reduced; the method can significantly improve the development efficiency of the fuzzy testing software. The method uses the COM interface provided by the tested software for calling, and does not have the process of constructing binary function interface parameters in the traditional fuzzy testing method, so that the complexity of the fuzzy testing software can be significantly reduced. The method has the advantage of good reusability, and for different tested software, only the code of the COM interface calling part needs to be changed, so that the method can be quickly transplanted to different tested software for testing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a method and device for fuzzy testing COM technology software. BACKGROUND

[0002] Component Object Model (COM) technology is an object-oriented programming technology introduced by Microsoft, mainly used for creating and using reusable software components. These components can be accessed and used by different programming languages and applications, realizing module communication between different languages and processes. Popular office software such as WPS Office, Microsoft Office, LibreOffice, and Kingsoft Office all use COM technology.

[0003] Fuzz testing is a software testing technique commonly used to detect security vulnerabilities in software or computer systems. Its core idea is to input automatically or semi-automatically generated random data into a program and monitor program exceptions such as crashes and assertion failures to find possible program errors such as memory leaks. Popular fuzz testing methods use code coverage-based methods to guide the generation of test cases.

[0004] Breakpoint instrumentation technology is a method of collecting code coverage, which refers to inserting breakpoints or instrumentation code at specific locations in the program code (such as the entry and exit of key functions or key statements), and recording the execution of these breakpoints or instrumentation code to calculate the code coverage. During program execution, when these inserted breakpoints or instrumentation code are executed, the corresponding recording mechanism is triggered, and the information of the executed code line is recorded. Finally, by counting the recorded execution information, the code execution coverage can be obtained, that is, the ratio of the number of executed code lines to the total number of code lines. This technology can help developers understand the test coverage of the code, find out the code areas that have not been fully tested, and then improve the test cases, improve the code quality and reliability.

[0005] Harness program in fuzz testing technology refers to a small program or tool designed to trigger and call the functions that need to be fuzz tested in the program being tested. The Harness program contains functions as target functions, which are the key parts to be tested in the fuzz testing process. In simple terms, Harness is a key component in fuzz testing, which is responsible for converting random or mutated data generated by the fuzzer into a format that the program being tested can understand and process, thereby triggering potential vulnerabilities or errors in the program.

[0006] Current academic and industrial, for the use of COM technology Office software fuzzy testing software method as follows:

[0007] (1) to be tested software some binary interface testing method. The method steps as follows:

[0008] 1) determine the target program.

[0009] 2) attack surface analysis.

[0010] 3) write Harness program, namely a custom program, for calling the binary interface of the software under test.

[0011] 4) start Harness program.

[0012] 5) use the debugger to attach to Harness program.

[0013] 6) input to the Harness program, the Harness program calls the binary interface of the software under test to parse the test sample.

[0014] 7) detect Harness program whether abnormal, if abnormal, collect exception information, and terminate the process of the software under test, execute step 4); otherwise, execute step 8).

[0015] 8) collect the execution information of the software under test, if the software under test has a new execution state, modify the test case to join the test set.

[0016] (2) the whole method of fuzzy testing of software under test. The method steps as follows:

[0017] 1) use the debugger to start the software under test to pass the test sample path to the software under test.

[0018] 2) the software under test parses the test sample.

[0019] 3) detect whether the software under test is abnormal, if abnormal, collect exception information, and terminate the process of the software under test, execute step 1); otherwise, execute step 4).

[0020] 4) collect the execution information of the software under test, if the software under test has a new execution state,

[0021] then modify the test case to join the test set.

[0022] In the traditional fuzzy testing method (1), write Harness program may be the most time-consuming, especially when you want to test large application program. Write Harness step as Figure 1When writing the Harness program, it is often necessary to construct the parameters of the binary interface function; each parameter may depend on many program internal objects, and then the objects dependent on the parameters need to be constructed in reverse. The time spent in this process is a few days to several weeks, and the workload is very large. This leads to the problems of long development cycle and high development difficulty of the traditional fuzz testing method.

[0023] In the traditional fuzz testing method (2), each round of testing needs to start the program to load the shared libraries of the program for initialization, and close the tested program at the end of the test, which takes a long time, for example, WPS Office and Microsoft Office programs often take about 20 seconds for a test. This leads to the problem of low test efficiency of the traditional fuzz testing method, which cannot meet the performance requirements of software using COM technology.

[0024] In summary, the existing fuzz testing method has the problems of long development cycle, high development difficulty and low test efficiency. SUMMARY

[0025] The purpose of the present application is to provide a method and device for fuzz testing software using COM technology to solve the problems of long development cycle, high development difficulty and low test efficiency of the current fuzz testing software using COM technology.

[0026] In a first aspect, the present application provides a method for fuzz testing software using COM technology, comprising:

[0027] (1) starting the tested software using the COM interface of the software to be fuzz tested;

[0028] (2) attaching to the tested software using a debugger;

[0029] (3) generating a test case according to the test set mutation;

[0030] (4) inputting the test case using the COM interface of the software to be fuzz tested;

[0031] (5) waiting for the COM interface of the software to be fuzz tested to return, and detecting whether the software is abnormal, if abnormal, collecting the abnormal information, and terminating the process of the tested software, and executing step (1); otherwise, executing step (6);

[0032] (6) collecting the execution information of the tested software, and if the tested software has a new execution state, adding the test case to the test set.

[0033] In the step (1), the COM interface is used to start the software, including using the Dispatch and DispatchEx functions of the client class of the win32com library of Python to start the software to be tested, wherein the client class of the win32com library is a standard library of Python used to interact with the COM component, the Dispatch function can dynamically create an instance of the COM object according to the specified program identifier or class identifier, and can call the interface exposed by the object, and the DispatchEx function is similar to the Dispatch function, and is used to create and connect to the COM object, and meanwhile provides more flexible object initialization options, allowing to specify additional parameters when creating the COM object, so as to control the creation behavior of the object.

[0034] In the step (2), the debugger refers to software having the functions of reading and writing the memory of the software process to be tested and detecting whether the software to be tested has an abnormal function,

[0035] In the step (3), the test sample is generated according to the variation set, including randomly or sequentially taking a test sample from the test set, randomly modifying or strategically modifying the test sample, the strategic modification including modifying a certain field of the file or deleting a specific area of the file according to the structure of the file,

[0036] In the step (4), the test sample is input through the COM interface, including obtaining the COM object instance of the program by calling the function of the client class in the win32com library of Python, and then modifying the COM object attribute or calling the interface function provided by the COM object to input the test sample,

[0037] In the step (5), the abnormal state of the program includes the crash of the software process to be tested and the long-time non-ending of the COM interface function of the software to be tested,

[0038] In the step (6), the execution information of the software to be tested includes the coverage rate of the program and the execution time of the program, and the new execution state of the software to be tested includes the change of the coverage rate of the program.

[0039] In the second aspect, the application provides a device for using the COM technology software for fuzzy testing, as shown in the accompanying drawings, which comprises: Figure 2

[0040] ​(1)server module: generate mutation test samples at the beginning of the test, and provide a transmission channel to the client module; accept the results of the client module's submission of this round of test samples after the test is completed, judge whether there is an exception or whether there is a new execution state according to the submitted results, and make corresponding processing, in addition, provide a display function to allow users to view the running state of the fuzz testing program, such as the number of test rounds, test time, average test time per round, etc.

[0041] (2) client module: this module is divided into two sub-modules, Tracer sub-module and instrumentation sub-module. The Tracer sub-module is mainly responsible for starting the software under test and the instrumentation sub-module in the first round of testing; restarting the software under test and the instrumentation sub-module when the software under test is abnormal; pulling the test samples of this round from the server module at the beginning of the test and inputting them into the software under test, while notifying the instrumentation sub-module to start collecting the software under test running information; after the COM interface returns after inputting the samples, notifying the instrumentation sub-module to end the software under test running information collection, and uploading the software under test running information to the server module. The instrumentation sub-module is mainly responsible for collecting the software under test running information and judging whether the software under test is abnormal.

[0042] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for fuzzy testing using COM technology software when executing the program.

[0043] A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions are executed by a processor to implement the method for fuzzy testing using COM technology software.

[0044] Compared with the prior art, the advantages of the present application are as follows:

[0045] (1) The method uses the COM interface of the software under test to realize the functions of starting the software under test and parsing the test samples, thereby replacing the step of writing the Harness program in the traditional fuzzy testing method, and using the win32com library of Python to write a few lines of code to realize the same function as the Harness program, the example code is shown in the detailed description, which greatly reduces the development cycle and difficulty of the fuzzy testing program.

[0046] (2) In the traditional fuzzy testing, the Harness program is started every round of testing, and the dynamic link library of the software under test is loaded to call the binary interface of the software under test to parse the test samples. The method only needs to restart the program when the first round of testing and the program is abnormal, which greatly reduces the time of each round of testing.

[0047] Take the Excel software of WPS Office as an example, the test environment is as follows:

[0048] The test environment is configured as follows:

[0049] Windows 11 operating system

[0050] Intel i7-13700K CPU

[0051] 32G DDR4 memory

[0052] Using the traditional fuzz testing method (2) each round of test needs 20s, in which the initialization time of the tested software when opening reaches 7-10 seconds. Using the method, the average test time of each round is 5s, and the test efficiency is increased by 400%.

[0053] Because the traditional fuzz testing method (1) is difficult to write Harness program, it cannot be compared. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 For the steps of writing Harness program in traditional fuzz testing

[0055] Figure 2 For the method flowchart of the provided fuzz testing using COM technology software of the present application;

[0056] Figure 3 For the method of the provided fuzz testing using COM technology software of the present application, the method is implemented by fuzz

[0057] The Excel in WPS Office is taken as an example to implement the architecture diagram. DETAILED DESCRIPTION

[0058] The core of the present application is to provide a method, device and equipment for fuzz testing using COM technology software, which calls the COM interface of the tested program, without constructing complex parameters and repeatedly starting the tested software, thereby reducing the complexity and development difficulty of the fuzz testing program and improving the efficiency of the fuzz testing.

[0059] In order to make the personnel in the technical field better understand the present application scheme, the present application will be further described in detail below in combination with the drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the ordinary skilled in the art without creative labor belong to the scope of protection of the present application.

[0060] The present application will be introduced below by taking the fuzz testing of Excel in WPS Office as an example, for referenceFigure 3 The architecture of the specific implementation is as follows:

[0061] The server module is developed based on the Python-based django web framework. The server module implements three interfaces:

[0062] (1) The get_summary / interface is used to show the user the fuzzing test running state information, such as the test round, test time, average test time per round, etc.

[0063] (2) The tracer / get_sample_path / interface is used for the client to obtain the information of the test sample of the current round. The client module can transmit the test sample from the server module through ftp according to the sample information.

[0064] (3) The tracer / submit_res / interface is used for the client module to upload the execution information of the test sample of the current round.

[0065] The server module also provides an ftp service for the client to download the test sample and upload the program running state of the current round.

[0066] The Tracer submodule in the client module is developed based on Python and uses the Python win32com library to start the software under test and input the test sample to the software under test. The example Python code for starting the software under test and inputting the test sample to the software under test is as follows:

[0067] from time import sleep

[0068] import win32com.client

[0069] # Create a KET.Application application instance

[0070] wps_excel = win32com.client.Dispatch("KET.Application")

[0071] # Set to display UI

[0072] wps_excel.Visible = True

[0073] # Open the file

[0074] workbook = wps_excel.Workbooks.Open(r"path\to\1.xls")

[0075] # Show for 5 seconds and close the workbook

[0076] sleep(5)

[0077] workbook.Close()

[0078] # Exit the KET.Application application instance

[0079] wps_excel.Quit() Run the above code to start excel and open the excel file in excel, show for 5 seconds and exit.

[0080] The plug-in sub-module is modified based on the open source software mesos, and the mesos software is a software that uses breakpoint plug-in technology to collect program information.

[0081] The specific implementation steps are as follows:

[0082] (1) The Tracer sub-module starts the software under test using the COM interface of the software under test.

[0083] (2) The Tracer sub-module starts the plug-in sub-module.

[0084] (3) The plug-in sub-module is attached to the program under test and prepares for the collection of program under test running information.

[0085] (4) The client module calls the tracer / get_sample_path / interface to obtain the sample path.

[0086] (5) The Tracer sub-module notifies the plug-in sub-module to collect coverage.

[0087] (6) The Tracer sub-module opens the sample using the COM interface.

[0088] (7) The Tracer sub-module notifies the plug-in sub-module to stop collecting coverage.

[0089] (8) The Tracer sub-module calls the COM interface to close the sample.

[0090] (9) The client module submits the program under test running information path.

[0091] (10) The server module analyzes the program under test running information of this round and performs the next round of testing.

[0092] The schemes provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples; the above description of the examples is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description of the specification should not be understood as the limitation of the present application.

Claims

1. A method of fuzz testing software using COM technology, characterized by, The COM interface for starting the software under test and accepting input is implemented by the fuzz testing software, and the method comprises the following steps: (1) starting the software under test using the COM interface of the fuzz testing software; (2) attaching a debugger to the software under test; (3) generating a test sample according to the mutation of the test set; (4) inputting the test sample using the COM interface of the fuzz testing software; (5) waiting for the COM interface of the fuzz testing software to return a result, detecting whether the software is abnormal, if abnormal, collecting abnormal information, and terminating the process of the software under test, and executing step (1); otherwise, executing step (6); (6) collecting execution information of the software under test, and if a new execution state of the software under test occurs, adding a modified test sample to the test set.

2. The method of claim 1, wherein the COM technology software is used for fuzz testing. Step (1) uses the COM interface to start the software, including using the Dispatch and DispatchEx functions of the client class of the Python win32com library to start the software under test.

3. The method of claim 1, wherein the COM technology software is used for fuzz testing. The debugger in step (2) refers to software that has the functions of reading and writing the memory of the process of the software under test and detecting whether the software under test has an abnormality.

4. The method of claim 1, wherein the COM technology software is used for fuzz testing. In step (3), the test sample is generated according to the mutation set, including randomly or sequentially taking a test sample from the test set, randomly modifying or strategically modifying the test sample, and the strategic modification includes modifying a certain field of a file or deleting a certain specific area of the file according to the structure of the file.

5. The method of claim 1, wherein the COM technology software is used for fuzz testing. In step (4), the test sample is input through the COM interface, including obtaining the COM object instance of the program by calling the functions of the client class in the Python win32com library, and modifying the COM object properties or calling the interface functions provided by the COM object to input the test sample.

6. The method of claim 1, wherein the COM technology software is used for fuzz testing. In step (5), the abnormal state of the program includes the crash of the process of the software under test and the long time of not ending of the COM interface function of the software under test.

7. The method of claim 1, wherein the COM technology software is used for fuzz testing. In step (6), the execution information of the software under test includes the coverage rate of the program execution and the execution time of the program, and the new execution state of the software under test includes the change of the coverage rate of the program execution.

8. An apparatus for fuzz testing software using COM technology, characterized by: The device for implementing the method of claim 1-7 uses COM technology software, and the device comprises (1) a server module: generating a mutation test sample at the beginning of the test, and providing a transmission channel to the client module; after the test is completed, the client module submits the results of the test sample of this round, according to the submitted results, it is judged whether there is an abnormality or a new execution state, and corresponding processing is performed, and a display function is provided to enable the user to view the running state of the fuzz testing program, including the number of test rounds, the test time, the average test time per round, (2) client module: this module is divided into two sub-modules, Tracer sub-module and instrumentation sub-module. The Tracer sub-module is mainly responsible for starting the software under test and the instrumentation sub-module in the first round of testing; restarting the software under test and the instrumentation sub-module when the software under test is abnormal; pulling the sample of the current round of testing from the server module at the beginning of the test and inputting into the software under test, while notifying the instrumentation sub-module to start collecting the software under test running information; after the COM interface returns after inputting the sample, notifying the instrumentation sub-module to end the software under test running information collection, and uploading the software under test running information to the server module. The instrumentation sub-module is mainly responsible for collecting the software under test running information and judging whether the software under test is abnormal.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: The processor executes the program to implement the method for fuzzy testing software using COM technology according to any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon computer instructions, characterized in that: The computer instructions are executed by the processor to implement the method for fuzzy testing software using COM technology according to any one of claims 1 to 7.

Images