Data testing method and device and electronic equipment
By extracting and matching information on the data to be tested in the graphical Web API, target test cases are generated, and the problem of poor accuracy of data testing in existing fuzzy testing technology is solved, and the accuracy and security of test results are improved.
Patent Information
- Application Number
- CN202410992007.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
Existing fuzz testing techniques lack understanding and analysis of data context in data testing of graphical Web APIs, resulting in poor accuracy of test results.
By extracting and processing the test data, obtaining error information and target error parameters, matching and updating data, generating target test cases, and data testing is improved through the fuzzy tester.
Strengthen the understanding and analysis of the context of testing data, improve the accuracy of data test results, and enhance the security detection capabilities of graphical Web API.
Smart Images

Figure CN118940266A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to a data testing method and device, and electronic equipment. Background Art
[0002] Common graphical network application programming interfaces (Web APIs) usually use graphics rendering in World Wide Web (Web) browsers and 3D and 2D graphics applications written in JavaScript for graphics. Since graphical Web APIs usually run on the Internet, they need to process data from different sources and unknown sources, which increases the risk of security vulnerabilities and attacks on graphical Web APIs.
[0003] Since graphical Web APIs allow potentially malicious remote users to access the native graphics stack, Chrome and Firefox both have lists of inaccessible Application Programming Interfaces (APIs) for the Graphics Processing Unit (GPU) model, such as APIs that call an error in the local software stack or use certain parameters. This solution for each error is not suitable for cross-user space scenarios. In addition, if it is software with millions of lines of code in kernel space, the potential attack surface will be very large.
[0004] Therefore, fuzz testing technology can be used to help developers discover and fix security vulnerabilities in graphical Web API applications and enhance their security. Fuzz testing technology is a security testing technology that detects potential vulnerabilities and errors by inputting random, invalid or abnormal data into software applications. In graphical Web API applications, fuzz testing can be used to discover potential memory leaks, type errors, buffer overflows, integer overflows, and other security vulnerabilities. By inputting different types of random data into graphical Web API applications, you can test how graphical Web API applications handle errors and help developers discover and fix possible security vulnerabilities.
[0005] However, the fuzz testing technology in the prior art usually only randomizes and mutates the input data, lacks the understanding and analysis of the data context, resulting in poor accuracy of data testing results. Summary of the invention
[0006] The present disclosure provides a data testing method and device, and an electronic device, the main purpose of which is to solve the problem that the fuzzy testing technology in the prior art usually only randomizes and mutates the input data, lacks understanding and analysis of the data context, and leads to poor accuracy of the data testing results.
[0007] According to a first aspect of the present disclosure, a data testing method is provided, comprising:
[0008] Performing information extraction processing on the data to be tested to obtain first error information, and performing parameter extraction processing based on the first error information and the data to be tested to obtain target error parameters in the data to be tested;
[0009] performing data matching processing on the data to be tested according to the first error information to obtain initial interface data corresponding to the first error information, and performing data updating processing on the initial interface data to obtain target interface data;
[0010] Inputting the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case;
[0011] The data to be tested is processed by data testing through the target test case to obtain a data testing result.
[0012] Optionally, performing information extraction processing on the test data to obtain the first error information includes:
[0013] Performing data compilation processing on the data to be tested to obtain a target data file, and determining each program entry point of the data to be tested according to the target data file; wherein the data to be tested contains a plurality of program data, and the program entry point is an initial function of each program data in the data to be tested;
[0014] According to the program entry point and the target data file, a data construction process is performed by a preset data analysis algorithm to obtain control flow graph data, and data extraction process is performed according to the control flow graph data to obtain work list data;
[0015] Performing data search processing on the work list data to obtain call statement data; wherein each call statement in the call statement data carries an error identifier;
[0016] The calling statement data is traversed, and information extraction processing is performed on the calling statement data to obtain the first error information.
[0017] Optionally, performing parameter extraction processing according to the first error information and the data to be tested to obtain a target error parameter in the data to be tested includes:
[0018] Performing parameter symbolization processing on the data to be tested to obtain symbolized data to be tested;
[0019] Performing case generation processing through the preset fuzz tester to obtain a random test case, and performing data test processing on the symbolized data to be tested through the random test case to obtain second error information; wherein the random test case is a test case generated by inputting random data into the preset fuzz tester;
[0020] Perform information extraction processing according to the first error information and the second error information to obtain target error data;
[0021] The parameter dependency relationship between each parameter in the data to be tested and the first error information and the second error information is determined according to the parameter path conditions in the target error data, and data inference processing is performed according to the parameter dependency relationship and the target error data to obtain the target error parameter.
[0022] Optionally, performing data matching processing on the data to be tested according to the first error information to obtain initial interface data corresponding to the first error information includes:
[0023] Performing data analysis on the first error information to obtain a first variable, and performing data analysis on the test data to obtain a second variable;
[0024] Determine an error path condition of the first error information and interface call information of the data to be tested by a preset dynamic data flow algorithm, and determine an association relationship between the first error information and the data to be tested according to the error path condition and the interface call information;
[0025] Establishing a mapping relationship between the first variable and the second variable according to the association relationship;
[0026] Based on the mapping relationship, the interface data associated with the second variable in the data to be tested is used as the initial interface data corresponding to the first error information.
[0027] Optionally, performing data update processing on the initial interface data to obtain target interface data includes:
[0028] Input the target interface data into the preset fuzzy tester, analyze the dependency relationship between the target interface data and the preset precondition, and obtain invalid dependencies; wherein the invalid dependencies are interface data in the target interface data that has no dependency relationship with the preset precondition;
[0029] According to the data type and data characteristics of the invalid dependency, matching processing is performed from a preset strategy database to obtain a target repair strategy, wherein the invalid dependency includes the data type and data characteristics of the invalid dependency, and the preset strategy database includes multiple repair strategies;
[0030] The invalid dependency is subjected to data repair processing by using the target repair strategy to obtain the target interface data.
[0031] Optionally, the step of inputting the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case includes:
[0032] Extracting the error type, error structure and error content of the first error information;
[0033] According to the error type, error structure and error content, matching processing is performed from a preset variation mode database to obtain a target variation mode, wherein the preset variation mode database includes multiple parameter variation modes;
[0034] Performing parameter variation processing on the target error parameter by the target variation method to obtain a mutated target error parameter;
[0035] A case generation process is performed according to the target error parameter and the target interface data to obtain the target test case.
[0036] Optionally, performing data test processing on the data to be tested through the target test case to obtain a data test result includes:
[0037] The process of performing data testing processing on the data to be tested through the target test case in parallel according to the preset processing unit to obtain the data testing result.
[0038] Optionally, the step of inputting the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case includes:
[0039] According to the preset processing unit, the target error parameters and the target interface data are input into the preset fuzzy tester in parallel for case generation processing to obtain the target test case.
[0040] According to a second aspect of the present disclosure, a data testing device is provided, comprising:
[0041] A first extraction unit, used for performing information extraction processing on the test data to obtain first error information;
[0042] A second extraction unit, configured to perform parameter extraction processing according to the first error information and the data to be tested, to obtain a target error parameter in the data to be tested;
[0043] a matching unit, configured to perform data matching processing on the data to be tested according to the first error information, and obtain initial interface data corresponding to the first error information;
[0044] An updating unit, used for performing data updating processing on the initial interface data to obtain target interface data;
[0045] A generating unit, used for inputting the target error parameter and the target interface data into a preset fuzzy tester for case generation processing to obtain a target test case;
[0046] The testing unit is used to perform data testing on the data to be tested through the target test case to obtain a data testing result.
[0047] Optionally, the first extraction unit includes:
[0048] A compiling module, used for compiling the data to be tested to obtain a target data file, and determining each program entry point of the data to be tested according to the target data file; wherein the data to be tested contains a plurality of program data, and the program entry point is an initial function of each program data in the data to be tested;
[0049] A construction module, used to perform data construction processing according to the program entry point and the target data file through a preset data analysis algorithm to obtain control flow graph data;
[0050] An extraction module, used for performing data extraction processing according to the control flow graph data to obtain work list data;
[0051] A search module, used for performing data search processing on the work list data to obtain call statement data; wherein each call statement in the call statement data carries an error identifier;
[0052] The extraction module is also used to traverse the call statement data and perform information extraction processing on the call statement data to obtain the first error information.
[0053] Optionally, the second extraction unit includes:
[0054] A processing module, used for performing parameter symbolization processing on the data to be tested to obtain symbolized data to be tested;
[0055] The processing module is also used to perform case generation processing through the preset fuzz tester to obtain a random test case, and perform data test processing on the symbolized data to be tested through the random test case to obtain second error information; wherein the random test case is a test case generated by inputting random data into the preset fuzz tester;
[0056] An extraction module, used for performing information extraction processing according to the first error information and the second error information to obtain target error data;
[0057] A determination module, configured to determine a parameter dependency relationship between each parameter in the test data and the first error information and the second error information according to a parameter path condition in the target error data;
[0058] The processing module is further used to perform data inference processing according to the parameter dependency and the target error data to obtain the target error parameter.
[0059] Optionally, the matching unit includes:
[0060] An analysis module, configured to perform data analysis on the first error information to obtain a first variable, and perform data analysis on the test data to obtain a second variable;
[0061] A determination module, used to determine an error path condition of the first error information and interface call information of the data to be tested by a preset dynamic data flow algorithm, and determine an association relationship between the first error information and the data to be tested according to the error path condition and the interface call information;
[0062] An establishing module, used for establishing a mapping relationship between the first variable and the second variable according to the association relationship;
[0063] The determination module is further configured to, based on the mapping relationship, use the interface data associated with the second variable in the data to be tested as the initial interface data corresponding to the first error information.
[0064] Optionally, the updating unit includes:
[0065] An analysis module, used for inputting the target interface data into the preset fuzzy tester, analyzing the dependency relationship between the target interface data and the preset precondition, and obtaining an invalid dependency item; wherein the invalid dependency item is interface data in the target interface data that has no dependency relationship with the preset precondition;
[0066] A matching module, configured to perform matching processing from a preset strategy database according to the data type and data characteristics of the invalid dependency to obtain a target repair strategy, wherein the invalid dependency includes the data type and data characteristics of the invalid dependency, and the preset strategy database includes multiple repair strategies;
[0067] A repair module is used to perform data repair processing on the invalid dependency through the target repair strategy to obtain the target interface data.
[0068] Optionally, the generating unit includes:
[0069] An extraction module, used to extract the error type, error structure and error content of the first error information;
[0070] A matching module, used for performing matching processing from a preset variation mode database according to the error type, error structure and error content to obtain a target variation mode, wherein the preset variation mode database includes a plurality of parameter variation modes;
[0071] A mutation module, used for performing parameter mutation processing on the target error parameter by the target mutation method to obtain the mutated target error parameter;
[0072] A generation module is used to perform case generation processing according to the target error parameters and the target interface data to obtain the target test case.
[0073] Optionally, the testing unit is also used to perform data testing processing on the data to be tested through the target test case in parallel according to a preset processing unit to obtain a data testing result.
[0074] Optionally, the generating unit is also used to input the target error parameters and the target interface data into a preset fuzzy tester in parallel with the preset processing unit to perform case generation processing to obtain the target test case.
[0075] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0076] at least one processor; and
[0077] a memory communicatively connected to the at least one processor; wherein,
[0078] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.
[0079] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.
[0080] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.
[0081] The data testing method, device, and electronic device provided by the present disclosure perform information extraction processing on the data to be tested to obtain a first error message, and perform parameter extraction processing based on the first error message and the data to be tested to obtain a target error parameter in the data to be tested; perform data matching processing from the data to be tested based on the first error message to obtain initial interface data corresponding to the first error message, and perform data update processing on the initial interface data to obtain target interface data; input the target error parameter and the target interface data into a preset fuzzy tester for case generation processing to obtain a target test case; perform data testing processing on the data to be tested through the target test case to obtain a data test result. Compared with the related art, the embodiment of the present disclosure generates a target test case by obtaining the target error parameter related to the error message of the data to be tested, and the target interface data in the data to be tested corresponding to the error message, and performs data testing processing on the data to be tested through the generated target test case, which can enhance the understanding and analysis of the context of the data to be tested and improve the accuracy of the data test results.
[0082] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0083] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0084] Figure 1 A flowchart of a data testing method provided by an embodiment of the present disclosure;
[0085] Figure 2 A schematic diagram of a system architecture for data testing provided by an embodiment of the present disclosure;
[0086] Figure 3 A schematic diagram of a process for obtaining first error information provided by an embodiment of the present disclosure;
[0087] Figure 4A schematic diagram of another process for obtaining first error information provided by an embodiment of the present disclosure;
[0088] Figure 5 A schematic diagram of a process for obtaining a target error parameter provided by an embodiment of the present disclosure;
[0089] Figure 6 A schematic diagram of another process for obtaining target error parameters provided by an embodiment of the present disclosure;
[0090] Figure 7 A schematic diagram of a process for obtaining initial interface data provided by an embodiment of the present disclosure;
[0091] Figure 8 A schematic diagram of another process of obtaining initial interface data provided by an embodiment of the present disclosure;
[0092] Fig. 9 A schematic diagram of an update process of an initial data interface provided by an embodiment of the present disclosure;
[0093] Fig.10 A schematic diagram of the structure of a data testing device provided by an embodiment of the present disclosure;
[0094] Fig.11 A schematic diagram of the structure of another data testing device provided by an embodiment of the present disclosure;
[0095] Fig.12 A schematic block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0096] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0097] The data testing method, device, and electronic device of the embodiments of the present disclosure are described below with reference to the accompanying drawings.
[0098] Figure 1 A flowchart of a data testing method provided in an embodiment of the present disclosure.
[0099] like Figure 1 As shown, the method comprises the following steps:
[0100] Step 101 : performing information extraction processing on the data to be tested to obtain first error information, and performing parameter extraction processing based on the first error information and the data to be tested to obtain target error parameters in the data to be tested.
[0101] In the embodiment of the present disclosure, the data to be tested is function data in the software source code to be tested, for example, a graphical Web API function in the Chrome browser source code, etc. Specifically, the embodiment of the present disclosure does not limit the data to be tested.
[0102] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, the subsequent test data is described using the graphical Web API function in the Chrome browser source code as an example.
[0103] The graphical Web API (data to be tested) implements internal logic by C++ methods, but uses JavaScript language to call methods. The internal C++ methods are called internal functions. The Chrome browser uses SynthesizeGLError to issue an error message. Therefore, SynthesizeGLError can be called from the entry point of the graphical Web API internal function to enumerate all error information of the graphical Web API. In order to search for all call statements of SynthesizeGLError, you can use but are not limited to: a work list-based algorithm to traverse all code paths in the control flow graph from the entry point of the internal function of the graphical Web API, obtain all call statements, and then extract the first error information from the call statement. The first error information includes: error type, error content, error distribution, etc. Specifically, the content of the first error information is not limited in the embodiment of the present disclosure.
[0104] In order to further determine the relationship between the data to be tested and the first error message, if the error message is triggered by the parameters of the data to be tested, the variation is concentrated on the parameters that trigger the error, that is, it is necessary to analyze the parameters of the corresponding API (interface of the data to be tested) call in the graphical Web API (data to be tested) to determine the target error parameters (parameters that trigger the error) corresponding to the first error message in the data to be tested. The target error parameters refer to: the key input parameters that cause the error. The target error parameters include but are not limited to: functions, input parameters of the graphical Web API, properties or fields of an object, etc.
[0105] Regarding the parameter extraction and processing of the first error message and the data to be tested, the following methods may also be used but are not limited to: using reverse tag analysis to infer the target error parameters, that is, searching for the parameter path conditions of the internal functions of the data to be tested, finding the error location of the calling statement, and obtaining the target error parameters corresponding to the first error message.
[0106] Step 102: performing data matching processing on the data to be tested according to the first error information to obtain initial interface data corresponding to the first error information, and performing data update processing on the initial interface data to obtain target interface data.
[0107] In the embodiment of the present disclosure, the initial interface data is a dependent API set corresponding to the first error message in the data to be tested (graphical Web API), that is, a set of a series of APIs corresponding to the first error message, and the API is an application programming interface (Application Programming Interface), that is, the interface data in the data to be tested (the API in the graphical Web API function).
[0108] During the fuzz testing phase, when a given input is changed, if the interface data in the input triggers certain error messages, that is, the input initial interface data does not meet the application's prerequisites, the fuzz tester will try to repair the invalid dependencies on the dependent API set. Therefore, the target interface data is the dependent API set after the dependent API set (initial interface data) is repaired, and the data update processing of the initial interface data represents the repair processing of the initial interface data.
[0109] Step 103, input the target error parameter and the target interface data into a preset fuzzy tester for case generation processing to obtain a target test case.
[0110] In an embodiment of the present disclosure, during the test case generation process, the preset fuzz tester can generate various types of new error information to guide testing and repair work. The preset fuzz tester collects new error information as feedback and adds a new JavaScript API to the Chrome browser. The preset fuzz tester will first retrieve the previously executed API error information (first error information), and use multiple browsers to execute in the fuzz testing phase. The first error information-guided mutation is first implemented in the Chrome browser (compilation is performed based on the target error parameters guided by the first error information), and then the mutated target error parameters are used to generate target test cases to test other browsers. The target test cases can be applicable to multiple browsers.
[0111] Step 104: Perform data testing on the data to be tested through the target test case to obtain a data testing result.
[0112] In the embodiments of the present disclosure, the target test case performs data testing processing on the test data, which generally refers to designing a series of data inputs and expected output results (target test cases) for specific test objectives or requirements during the software testing process. By running the target test cases, it can be tested whether the software can correctly handle various expected and unexpected data to ensure the quality and stability of the software.
[0113] In data testing and processing, the target test cases are as follows: Data coverage: ensure that the test cases can cover each functional module of the software and various possible input data; Boundary conditions: check whether the software can correctly handle boundary data, such as maximum value, minimum value, critical value, etc.; Abnormal value processing: verify whether the software can correctly handle abnormal or erroneous data, including incorrect input data format, data type mismatch, etc.; Data persistence: test whether the data persistence function of the software is normal after processing the data, such as database addition, deletion, modification and query operations; Performance testing: evaluate the performance of the software when processing large amounts of data, such as response time, resource consumption, etc.
[0114] When conducting data testing, various testing methods and tools are usually used together with target test cases to conduct data testing, such as black box testing, white box testing, automated testing, etc., to ensure that the software can achieve the expected performance and functional requirements in actual operation.
[0115] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, the present disclosure provides a schematic diagram of a system architecture of a data test, such as Figure 2 As shown, the graphical Web API represents the data to be tested, the error information is the first error information, the target parameter is the target error parameter, the dependent API set is the target interface data, and the fuzzifier is the preset fuzz tester.
[0116] The data testing method provided by the present disclosure performs information extraction processing on the data to be tested to obtain a first error message, and performs parameter extraction processing based on the first error message and the data to be tested to obtain a target error parameter in the data to be tested; performs data matching processing from the data to be tested based on the first error message to obtain the initial interface data corresponding to the first error message, and performs data update processing on the initial interface data to obtain target interface data; inputs the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case; performs data testing processing on the data to be tested through the target test case to obtain a data test result. Compared with the related art, the embodiment of the present disclosure generates a target test case by obtaining the target error parameter related to the error message of the data to be tested, and the target interface data in the data to be tested corresponding to the error message, and performs data testing processing on the data to be tested through the generated target test case, which can enhance the understanding and analysis of the context of the data to be tested and improve the accuracy of the data test results.
[0117] In one possible implementation of the embodiment of the present disclosure, as a refinement of the above step 101, regarding performing information extraction processing on the test data to obtain the first error information, the embodiment of the present disclosure provides a flow chart of obtaining the first error information, such as Figure 3 As shown, including:
[0118] Step 301, compile the data to be tested to obtain a target data file, and determine each program entry point of the data to be tested based on the target data file; wherein the data to be tested contains multiple program data, and the program entry point is the initial function of each program data in the data to be tested.
[0119] In the embodiments of the present disclosure, the data to be tested is function data existing in the source code of the software. Therefore, data compilation processing of the data to be tested can also be achieved by compiling the source code of the software. Directly compiling the source code of the software reduces the steps of extracting the data to be tested and can improve the efficiency of obtaining the first error information.
[0120] Taking the graphical Web API function in the source code of the Chrome browser as an example, the process of compiling and processing the test data can be described in the following ways, but not limited to: downloading the Chrome source code, compiling and building it, and generating executable files and library files (target data files).
[0121] When determining the program entry point based on the executable file and the library file (target data file), the program entry point is the first function or statement executed from the beginning of the program. The process of determining the program entry point can be implemented in the following ways but is not limited to: analyzing the program startup process based on the executable file and the library file (target data file), or determining it by searching the main function or initialization function in the program code.
[0122] Step 302, according to the program entry point and the target data file, data construction processing is performed through a preset data analysis algorithm to obtain control flow graph data, and data extraction processing is performed based on the control flow graph data to obtain work list data.
[0123] In the embodiments of the present disclosure, the preset data analysis algorithm is a custom-set algorithm, such as a static analysis algorithm, etc. Tools for executing static analysis algorithms include, for example, LLVM compiler infrastructure (Low-Level Virtual Machine Compiler Infrastructure, LLVM), GNU compiler collection (GNU Compiler Collection, GCC), etc. Specifically, the embodiments of the present disclosure do not limit the preset data analysis algorithm.
[0124] The control flow graph data refers to the control flow graph of the graphical Web API function (data to be tested). The control flow graph is a control flow relationship graph between statements in a program, which can be automatically generated by static analysis tools such as LLVM, GCC, etc. The nodes in the control flow graph represent the basic blocks in the program, and the edges represent the control flow relationship between the basic blocks. The control flow graph of the graphical Web API function in the Chrome source code is generated.
[0125] Generating the work list data refers to: adding the entry point of the graphical Web API function in the control flow graph to the work list to obtain the work list data.
[0126] Step 303, performing data search processing on the work list data to obtain call statement data; wherein each call statement in the call statement data carries an error identifier.
[0127] In the embodiment of the present disclosure, the acquisition of the call data can be implemented in the following manner, but not limited to: processing the nodes in the work list, for each node in the work list, performing the following steps: visiting the node and finding whether it contains a call statement of the SynthesizeGLError function; if the node contains a call statement of the SynthesizeGLError function, adding the call statement to the call statement list; traversing the successor nodes of the node, and adding the successor nodes that have not been visited to the work list. In this way, the call statement data can be obtained.
[0128] The issuance of the first error message depends on SynthesizeGLError, and the first error message can be found by tracking SynthesizeGLError.
[0129] Step 304, traverse the call statement data, and perform information extraction processing on the call statement data to obtain the first error information.
[0130] In the disclosed embodiment, the first error message can be intuitively determined by traversing all call statements, and the first error message can be obtained by searching and counting all error messages of the graphical Web API. Specifically, the process of extracting information from the call statement data may include but is not limited to: using regular expressions, string matching and other methods to extract and classify error information and type information, and counting the number, frequency and distribution of different types of error information to obtain the first error message.
[0131] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, the embodiment of the present disclosure provides another flowchart for obtaining the first error information, such as Figure 4 As shown, the test data is described by taking the graphical WebAPI function in the Chrome browser source code as an example.
[0132] In one possible implementation of the embodiment of the present disclosure, since it is necessary to focus the mutation operation in the process of generating the target test case on the parameter that triggers the error, rather than randomly mutating the entire input, in order to improve the coverage of the target test case, it is necessary to obtain the target error parameter (the parameter that triggers the error) in the data to be tested. Regarding the acquisition of the target error parameter, the embodiment of the present disclosure provides a schematic diagram of the process of obtaining the target error parameter, as shown in FIG. Figure 5 As shown, including:
[0133] Step 501: perform parameter symbolization processing on the data to be tested to obtain symbolized data to be tested.
[0134] In the disclosed embodiment, in order to further determine the relationship between the data to be tested and the first error message, if the error message is triggered by the parameters of the data to be tested, the mutation is concentrated on the parameters that trigger the error. Since the total number of error messages of the software to be tested is very large, for example, there are more than three thousand error messages in the Chrome browser. The method of manually analyzing the first error message to infer the target error parameter is too cumbersome and not scalable. It requires a lot of effort to understand the internal state machine of the graphical Web API specification (existing function specification), resulting in the problem of low efficiency in obtaining the target error parameter.
[0135] At the same time, natural language can be used to analyze the first error message when obtaining the target error parameters. However, since there is no data set for training, manually marking the first error message as a training data set still requires a lot of effort. Therefore, in order to improve the efficiency of obtaining the target error parameters, it is necessary to symbolize the parameters of the data to be tested, and use reverse labeling analysis to infer the parameters, search the parameter path conditions of the internal functions of the graphical Web API, find the error location of the call statement, and determine the target error parameters.
[0136] Among them, the method of symbolizing the parameters of the data to be tested includes but is not limited to: symbolic execution technology, etc. Symbolic execution technology is a static analysis technology that can track the symbolic representation of variable values during program execution, rather than specific numerical values. Symbolic execution technology can create a symbolic execution path during program execution, and track each branch and condition in the program through the symbolic execution path, as well as the symbolic representation of the symbolic variables used in these branches and conditions. Therefore, symbolic execution technology can better handle changes in variable values, especially when the control flow and data flow of the program are very complex.
[0137] Step 502, use case generation processing is performed through the preset fuzz tester to obtain a random test case, and data testing processing is performed on the symbolized data to be tested through the random test case to obtain a second error message; wherein, the random test case is a test case generated by inputting random data into the preset fuzz tester.
[0138] In the disclosed embodiment, a preset fuzz tester is used to generate a large number of random test cases for testing graphical Web API functions (data to be tested). In the test of the graphical Web API, the error information (second error information) of the graphical Web API can be further extracted. For example, the getError() method of the WebGL API RenderingContext object can obtain the latest error code. If the return value is not gl.NO_ERROR, it indicates that an error has occurred and the second error information can be obtained. By performing parameter extraction processing on the first error information and the second error information, accurate target error parameters can be obtained.
[0139] Specifically, the data test processing of the symbolized data to be tested through the random test cases can be carried out in the following ways but are not limited to: selecting a suitable preset fuzz tester, such as AFL (American FuzzyLop), Radamsa or Peach Fuzzer, etc., the preset fuzz tester can automatically generate a large amount of random input data; customizing a fuzz test script to pass the input parameters and URL of the graphical Web API (data to be tested) function to the preset fuzz tester to automatically generate multiple random test cases; using the automatically generated random test cases, by calling the graphical Web API function and checking the return results, to determine possible problems, such as: function crash, return error or return incorrect data, etc., to obtain the second error information.
[0140] Step 503: perform information extraction processing according to the first error information and the second error information to obtain target error data.
[0141] In the embodiment of the present disclosure, the target error data is the relevant information of the call statement extracted from the error information, including but not limited to: error code, error information, stack information, error location of the call statement, parameter path conditions, parameter dependencies, parameter values that may cause errors, etc. Specifically, the embodiment of the present disclosure does not limit the content of the target error data.
[0142] In general, it is necessary to analyze the context in the first error message and the second error message, such as the function call stack, error stack, etc., to find the API or function call related to the error, and further analyze these API or function signatures to determine the input parameters that may cause the error. For example, the function signature is foo(int a, float b, string c). According to the error message, it may be parameter b or c that caused the problem.
[0143] The acquisition of target error data can be achieved by, but not limited to, the following methods: Extract error information: First, extract error code, error message, and stack trace information from the error information. This information can help locate the location and cause of the problem; Locate the error location: By analyzing the stack trace information, you can find out the file, line number, and other information of the specific call statement that caused the error. In this way, you can locate the exact location of the problem; Analyze the call parameters: After determining the error location, analyze the input parameters of the call statement. Check the parameter type, value range, dependency, and other information to find out the parameter value that may cause the error; Determine the cause of the error: Combine the error information, error location, and parameter analysis to try to determine the cause of the error, for example, it may be that the parameter value is illegal, the dependency is wrong, or there is a problem with the business logic itself. Among them, the above four steps are closely related: the error information extracted in step 1 provides a basis for the subsequent steps; the error location located in step 2 indicates the specific location of the problem; the parameter information analyzed in step 3 helps to determine the cause of the error; the error cause obtained in step 4. Through the above four steps, you can get the target error data containing multiple data.
[0144] Step 504, determining the parameter dependency relationship between each parameter in the test data and the first error information and the second error information according to the parameter path conditions in the target error data, and performing data inference processing according to the parameter dependency relationship and the target error data to obtain the target error parameter.
[0145] In the embodiments of the present disclosure, the process of obtaining the target error parameters can be implemented in the following manner but is not limited to: creating a parameter graph based on the parameter path conditions, describing the parameter dependency relationship between all possible parameter values and the first error message and the second error message based on the parameter graph, using the information extracted from the parameter graph and the error message (target error data) to reversely infer the missing or incorrect parameter values in the calling statement, and thus obtaining the target error parameters.
[0146] Among them, reverse analysis of the parameter graph: combined with the error information, analyze which parameter values in the parameter graph may have caused the error. For example, based on the error message, it can be reversely inferred that a certain parameter value is illegal, and then determine the specific parameter value that caused the error.
[0147] It should be noted that a large number of test cases are generated using a fuzz tester, and then these test cases are sent as input to the graphical Web API function for testing. The random test cases generated by the preset fuzz tester are not accurate enough. According to the results of the reverse tag analysis, that is, the target error parameters, more effective test cases are generated to further test the graphical Web API function. The test cases generated in this way are close to the actual input, which improves the effect of fuzz testing.
[0148] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, the embodiment of the present disclosure provides another schematic diagram of the process of obtaining target error parameters, such as Figure 6 As shown, the test data is described by taking the graphical WebAPI function in the Chrome browser source code as an example, and the possible parameter values and dependencies are analyzed from the error information to obtain the target error data, and the missing or wrong parameter values are reversely inferred to obtain the target error parameters.
[0149] This embodiment pre-statistically analyzes the first error information of all graphical Wab APIs, then uses reverse tag analysis to infer target error parameters, searches for parameter path conditions of internal functions of the graphical Wab API, and finds the error location of the call statement, so that the fuzz tester can make full use of the path graph for traversal, reduce missed reports, and improve the coverage of test cases. More importantly, the understanding and analysis of data context makes the test results meaningful and interpretable, thereby discovering real vulnerabilities and errors.
[0150] In one possible implementation of the disclosed embodiment, in order to allow the preset fuzz tester to fully utilize the dependent API set (target interface data), it is necessary to obtain the dependent API set of the graphical Web API (data to be tested) in advance. There are some technical difficulties in inferring the dependent API set of the graphical Web API. Some variables of the graphical Web API are updated by executing the APIs they depend on. For example, the first error message is a call statement with "no valid shader program is in use" nested in an if statement. The condition is to check the internal variable currnet_program_, which is updated by the internal function of useProgram. Therefore, the first error message needs to find the dependent API set. The process of obtaining the dependent API set needs to consider two issues:
[0151] Problem 1: Analysis: Internal variables and APIs (interfaces for data to be tested) may be updated. Since the program may change the value of internal variables or use different APIs for calls in certain circumstances, these possible changes need to be considered during the tracking process.
[0152] In order to solve the first problem, a method of dynamic data flow analysis is provided. Dynamic data flow analysis is a dynamic analysis technology that can track data flow during program execution to better identify API calls. Dynamic data flow analysis can track the data flow in a program, as well as the source and destination of the data flow. Therefore, dynamic data flow analysis can identify API calls in a program and determine the parameters and return values of these calls.
[0153] Symbolic execution technology and dynamic data flow analysis are used to track variable values and API calls during program execution. Symbolic execution can track the symbolic representation of variable values rather than specific numerical values during program execution, which can better handle changes in variable values. Dynamic data flow analysis can track data flow during program execution to better identify API calls.
[0154] Question 2: Use string analysis and static analysis to analyze the internal variables that the first error message depends on. The string analysis method can identify the error variables used in the first error message and match the error variables with the internal variables in the program. Usually, the first error message will contain some variable names or variable values (error variables). The variable names or variable values (error variables) can be extracted and compared with the internal variables in the program. If a matching internal variable is found, the API that the error variable depends on can be determined.
[0155] Therefore, it is necessary to use static analysis methods to analyze the structure and behavior of the program without executing the program, identify the internal variables and APIs in the program, and determine the dependency relationship between the internal variables and the error variables. Then, the internal variables on which the first error message depends can be analyzed, and based on the API corresponding to the internal variables on which the first error message depends, the API on which the error variable depends can be determined.
[0156] In summary, when obtaining initial interface data (depending on the API set), the embodiment of the present disclosure provides a schematic diagram of the process of obtaining initial interface data, such as Figure 7 As shown, including:
[0157] Step 701: Perform data analysis on the first error information to obtain a first variable, and perform data analysis on the test data to obtain a second variable.
[0158] In the disclosed embodiment, a string analysis is first performed on the first error message to identify the variable names used therein to obtain a first variable. The first variable refers to various dynamic elements that constitute the first error message, such as error code, error cause, error location and other information. The value of the first variable will eventually be filled into the first error message.
[0159] Then, the program code of the graphical Web API (test data) is analyzed by static analysis technology to identify the program variable (second variable) matching the first variable in the first error message, where the second variable refers to various variables used inside the program of the test data, such as function parameters, local variables, global variables, etc. The second variable will be assigned, read, and modified during program execution.
[0160] Step 702: determine the error path condition of the first error information and the interface call information of the data to be tested by a preset dynamic data flow algorithm, and determine the association relationship between the first error information and the data to be tested based on the error path condition and the interface call information.
[0161] In the disclosed embodiment, the path conditions when the first error message is issued are mainly analyzed by dynamic data flow, and the data flow and API call in the program execution process in the test data are tracked, as well as the relationship between the first error message and the test data. Regarding the implementation process of the disclosed embodiment, the following methods can be adopted but not limited to: Collect dynamic data in the program execution process: Use dynamic analysis tools (such as Profiler, Tracing, etc.) to record function calls, parameter transfers, variable value changes and other information in the program execution process. These information constitute the dynamic data flow of the program; Analyze the relationship between the data flow and the error message: According to the error message, locate the key call point that causes the error, trace back the data flow of this key call point, and analyze the parameter value, return value and the value of the relevant variable during the call; Extract the path conditions: By analyzing the data flow, extract the path conditions that cause the error, including the specific value of each variable / parameter when the error is triggered; Establish a data flow model: Integrate the dynamic data flow information obtained by analysis into a comprehensive data flow model, which describes the data dependency relationship between various components in the program execution process. The association relationship between the first error information and the data to be tested can be obtained, and the association relationship includes but is not limited to: the key call point that causes the error and its context information (parameter value, return value, etc.); the specific path conditions when the error is triggered, including the values of related variables / parameters; a comprehensive program execution process data flow model, etc.
[0162] Step 703: Establish a mapping relationship between the first variable and the second variable according to the association relationship.
[0163] In the disclosed embodiment, the mapping relationship refers to the corresponding relationship between the first variable of the first error message and certain variables used inside the program (the second variable of the test data). That is, certain dynamic elements in the first error message are actually taken from certain variables inside the program of the test data.
[0164] Step 704: Based on the mapping relationship, the interface data associated with the second variable in the data to be tested is used as the initial interface data corresponding to the first error information.
[0165] In the embodiment of the present disclosure, based on the mapping relationship between the above-mentioned first variable and the second variable, the variables in the test data on which the first error message depends are determined, and based on the API on which the variables in the test data depend, the API set (initial interface data) on which the first error message depends is determined.
[0166] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, the embodiment of the present disclosure provides another schematic diagram of the process of obtaining initial interface data, such as Figure 8 As shown, the error message is the first error message, the program is the program content in the data to be tested, and the API on which the error message depends is the initial interface data.
[0167] In one possible implementation of the embodiment of the present disclosure, during the fuzz testing phase, when a given input is changed, if the interface data in the input triggers some error messages, that is, the input initial interface data does not meet the preconditions of the application, the fuzz tester will try to repair the invalid dependencies of the dependent API set, that is, it is necessary to perform data update processing on the initial interface data to obtain the target interface data. Regarding the update processing of the initial interface data, the embodiment of the present disclosure provides a schematic diagram of the update process of the initial data interface, such as Fig. 9 As shown, including:
[0168] Step 901, input the target interface data into the preset fuzzy tester, analyze the dependency relationship between the target interface data and the preset preconditions, and obtain invalid dependencies; wherein the invalid dependencies are interface data in the target interface data that have no dependency relationship with the preset preconditions.
[0169] In the embodiments of the present disclosure, the preset preconditions are a series of custom set conditions, which refer to conditions that must be met before calling a function or method. For example, before calling the "open file" function, it is necessary to ensure that the file path exists and has read and write permissions. The preset preconditions reflect the internal dependencies of the program and are the basis for ensuring the correct operation of the program.
[0170] The determination of the preset preconditions can be achieved in the following ways, but not limited to: by statically analyzing the program code, the preconditions of each function / method can be identified, for example, by analyzing the input parameter definition of the function, the exception handling logic, etc., the preconditions can be derived. You can also refer to API documents, design documents, etc. to obtain more precondition information.
[0171] Step 902, performing matching processing from a preset strategy database according to the data type and data characteristics of the invalid dependency to obtain a target repair strategy, wherein the invalid dependency includes the data type and data characteristics of the invalid dependency, and the preset strategy database includes multiple repair strategies.
[0172] In an embodiment of the present disclosure, a suitable repair strategy is selected as a target repair strategy based on the type of invalid dependencies and the characteristics of the application. Regarding the repair strategy, for example: a strategy of using an alternative API to replace invalid dependencies, a strategy of using patches to repair errors in the application, etc.
[0173] Specifically, use alternative APIs to replace invalid dependencies: If you find that an API has invalid dependencies, you can consider looking for an alternative API with equivalent functionality but more reasonable dependencies. For example, if an API requires that the input file must exist, but the actual input file does not exist, you can try using another API that supports creating files. This replacement method can directly eliminate invalid dependencies, but you must ensure that the functionality and performance of the alternative API meet your needs.
[0174] Fix errors in the application: If the invalid dependency is caused by an error in the application itself, the program needs to be fixed. This may include modifying the logic for generating input data, adjusting the way preconditions are checked, etc. In this way, the invalid dependency at the root can be eliminated and the robustness of the program can be improved.
[0175] Add fault tolerance mechanisms: Even if we cannot completely eliminate invalid dependencies, we can consider adding some fault tolerance mechanisms. For example, we can add stricter parameter verification before API calls, and give friendly error prompts when invalid input is found instead of throwing exceptions directly. This approach can avoid the problems caused by invalid dependencies to a certain extent and improve the user experience.
[0176] Step 903: Perform data repair processing on the invalid dependency using the target repair strategy to obtain the target interface data.
[0177] In an embodiment of the present disclosure, a repair operation is applied according to the selected repair strategy. For example, if an alternative API is chosen to repair an invalid dependency, the invalid API can be replaced with the initial dependency API set and other APIs in the (initial interface data) to meet the application's prerequisites.
[0178] By fixing invalid dependencies in the initial set of dependent APIs, effective test cases can be better generated and more application errors and vulnerabilities can be discovered. At the same time, the repair strategy can be adjusted according to specific circumstances to adapt to different applications and testing needs.
[0179] The disclosed embodiment first statistically analyzes the first error messages of all graphical Wab APIs, and then obtains the parameter list (initial interface data) of the API on which the first error message depends according to the graphical Wab API (data function to be tested and the path condition when the first error message is issued. When the precondition is not met before calling the API, the dependent API set is used to repair the invalid dependency, thereby ensuring the accuracy of the interface dependency relationship, enhancing the accuracy of the test case, and reducing the situation of false positives.
[0180] In one possible implementation of the embodiment of the present disclosure, during the test case generation process, the preset fuzz tester can generate various types of new error information to guide testing and repair work. The preset fuzz tester collects the new error information as feedback and adds a new JavaScript to the Chrome browser. API, the preset fuzz tester will first retrieve the API error information (first error information) executed before, and use multiple browsers to execute in the fuzz testing stage. First, the first error information-guided mutation is implemented in the Chrome browser (compilation is performed based on the first error information-guided target error parameters), and then the mutated target error parameters are used to generate target test cases. Since the mutation rules of different types of first error information are different, it is necessary to match the mutation rules of different types of first error information to perform accurate mutation. Therefore, in the process of generating target test cases, it is also possible to adopt but not limited to the following methods: extract the error type, error structure and error content of the first error information; match and process from the preset mutation method database according to the error type, error structure and error content to obtain the target mutation method, wherein the preset mutation method database contains multiple parameter mutation methods; perform parameter mutation processing on the target error parameter through the target mutation method to obtain the mutated target error parameter; perform case generation processing according to the target error parameter and the target interface data to obtain the target test case.
[0181] In the disclosed embodiments, the mutation rules for each type of first error information may be slightly different. First, determine the type of the first error information, such as syntax error, parameter error, status error, memory error, etc., and then analyze the structure and content of the first error information, such as error code, error information, stack information, etc. Select a mutation method based on the structure and content of the first error information, such as: randomly mutating the target error parameter, modifying the function call stack corresponding to the target error parameter, etc., mutate the target error parameter accordingly according to the selected mutation method, and generate a mutated target error parameter. The mutated target error parameter should still comply with the definition of the graphical Web API specification, otherwise it may result in false positives or negatives. The following are different mutation rules:
[0182] (1) Syntax Error: Randomly generate API call statements that do not comply with the graphical Web API syntax specifications, such as missing required parameters, using incorrect data types, etc. The generated syntax error information includes the corresponding error code and error message.
[0183] (2) Parameter error: Randomly mutate the various parameters of the graphical Web API, for example, set the parameter value to a value outside the valid range, set the parameter type to an incorrect type, etc. The generated parameter error information should include the corresponding error code and error information, as well as the relevant parameter value and type information.
[0184] (3) Status error: Simulates incorrect status when calling the graphical Web API, such as calling related API functions when the buffer object is not bound, performing depth testing when the depth test is not enabled, etc. The generated status error information should include the corresponding error code and error information, as well as related status information.
[0185] (4) Memory error: Simulates memory errors during graphical Web API calls, such as attempting to access a buffer object without allocating memory, attempting to write data beyond the range of a buffer object, etc. The generated memory error information should include the corresponding error code and error information, as well as related memory information.
[0186] In summary of the above embodiments, first, according to the error information variation rules based on the graphical Web API summarized in the above embodiments of the present disclosure, randomly generated input data is used as a test case, including a graphical Web API function call sequence, parameters, and textures, etc. For example: when constructing the input, a random function call sequence is generated, including randomly selecting functions and parameters; for each function call, a random parameter value and parameter type are used; textures and texture parameters are randomly generated, including texture width, height, pixel format, etc.
[0187] Second, during fuzz testing, when an error message is triggered, the mutation will focus on the target parameter of the API that issued the error message. The mutation of the identified target parameter is generally random. In order to optimize the mutation of the identified target parameter, the embodiment of the present disclosure proposes two optimizations:
[0188] a. Mutation of enumeration type parameters: For enumeration type parameters, due to their small value space, the chance of randomly generating the same value is high, which easily leads to meaningless mutations. To avoid this, the present invention generates a value different from the current value to increase the diversity and effectiveness of the mutation.
[0189] b. Integrity check of integer parameters: For some integer parameters, they are used as indexes to access the graphical API objects in the memory buffer. If the integrity check of these parameters fails, it may cause the error message to be too large or too small. In order to avoid this, the present invention will perform special processing on these parameters when generating test cases to ensure that their values are within a reasonable range and will not cause the error message to be too large or too small.
[0190] At the same time, when performing mutations in the target test case generation process, compared with traditional seed-based fuzz testing, the embodiment of the present disclosure uses the error messages generated when the program is running to infer the parameters that cause the error, and further fuzz testing is performed on these parameters. When the program is running, many error messages are generated, such as "invalid parameters", "null pointer reference", etc. The fuzz testing tool proposed in the embodiment of the present disclosure collects these error messages and analyzes them to determine the parameters that cause the error, and then generates new test cases for these parameters, and continues to perform fuzz testing until more errors are found or the number of test cases reaches a preset upper limit. This method can find errors faster, and the generated test cases are more representative.
[0191] In one implementable manner of the disclosed embodiment, during the fuzz testing process, the execution of the application is a very time-consuming operation because each target test case needs to be executed and the result analyzed. Therefore, with regard to performing data testing processing on the data to be tested through the target test case to obtain the data test result, it can also be implemented in but not limited to the following manner: performing data testing processing on the data to be tested through the target test case in parallel according to a preset processing unit to obtain the process of data testing result.
[0192] Similarly, when the target error parameters and the target interface data are input into the preset fuzz tester for case generation processing to obtain the target test case, it can also be implemented but not limited to the following method: according to the preset processing unit, the target error parameters and the target interface data are input into the preset fuzz tester for case generation processing in parallel to obtain the target test case.
[0193] In the embodiments of the present disclosure, there are mainly two aspects:
[0194] (1) Use GPU to accelerate the test case generation process. During fuzz testing, generating test cases is a very time-consuming operation because the inputs need to be randomly mutated and combined. Using GPU can process multiple test cases at the same time, thereby speeding up the test case generation process.
[0195] (2) Use GPU to accelerate the execution of the application. During fuzz testing, the execution of the application is a very time-consuming operation because each test case needs to be executed and the results analyzed. Using GPU can execute multiple test cases at the same time, and the parallel computing power of GPU can be used to accelerate the execution process.
[0196] Parallel generation of test cases: Input data and random seeds are transferred to the GPU and implemented through the Unified Computing Device Architecture (CUDA) parallel computing unit to generate test cases in parallel.
[0197] Convert test case execution into parallel computing tasks on the GPU: Use the CUDA programming model to convert test case execution into parallel computing tasks on the GPU. When executing test cases, each test case is assigned to a different GPU thread for execution, and multiple computing units of the GPU are used to execute multiple test cases in parallel, further improving test efficiency.
[0198] Ensure the accuracy of test results: During the test case execution process, ensure that the execution results of each test case on the GPU are consistent with those on the CPU. For example, perform multiple verification technologies, such as memory verification and semaphore verification, to ensure the correctness of the test results.
[0199] Fuzz testing requires a lot of computing resources and time, especially when testing complex graphical Web API applications. This may cause the testing process to become slow and inefficient, thus affecting the work efficiency of developers and the accuracy of test results. The testing tool proposed in the present invention uses the parallel computing power of GPU to accelerate the fuzz testing process, which can greatly improve the testing efficiency.
[0200] In summary, the embodiments of the present disclosure can achieve the following effects:
[0201] 1. The embodiment of the present disclosure generates a target test case by acquiring target error parameters related to the error information of the data to be tested, and the target interface data in the data to be tested corresponding to the error information, and performs data testing on the data to be tested through the generated target test case, thereby enhancing the understanding and analysis of the context of the data to be tested and improving the accuracy of the data testing results.
[0202] 2. The disclosed embodiment reduces the error rate by automating the fuzz testing process, making full use of target parameters and dependent API sets, and generating a large amount of semi-random reliable data, which is then input into the graphical Web API implementation for testing.
[0203] 3. The target test cases generated by the embodiments of the present disclosure can cover most of the APIs and functions implemented by the graphical Web API, including shaders, textures, buffers, frame buffers, multisampling, depth testing, etc. This can improve the coverage and accuracy of the test and discover more potential problems.
[0204] 4. The embodiment of the present disclosure can perform test settings and parameter adjustments according to one's own needs to obtain the best test effect and results. The test tool proposed by the present invention uses the parallel computing power of the GPU to accelerate the fuzzy test process, which can greatly improve the test efficiency.
[0205] 5. The test tool proposed in the embodiment of the present disclosure can run on multiple operating systems, including Linux, Windows, and macOS, etc. This enables it to adapt to different test environments and requirements, and facilitates users to test on multiple platforms.
[0206] Corresponding to the above data testing method, the present invention also provides a data testing device. Since the device embodiment of the present invention corresponds to the above method embodiment, details not disclosed in the device embodiment can be referred to the above method embodiment, and will not be repeated in the present invention.
[0207] Fig.10 A schematic diagram of a data testing device provided in an embodiment of the present disclosure is shown in FIG. Fig.10 As shown, including:
[0208] A first extraction unit 11 is used to perform information extraction processing on the test data to obtain first error information;
[0209] A second extraction unit 12 is used to perform parameter extraction processing according to the first error information and the data to be tested to obtain a target error parameter in the data to be tested;
[0210] A matching unit 13, configured to perform data matching processing on the data to be tested according to the first error information, and obtain initial interface data corresponding to the first error information;
[0211] An updating unit 14, configured to perform data updating processing on the initial interface data to obtain target interface data;
[0212] A generating unit 15, configured to input the target error parameter and the target interface data into a preset fuzzy tester for case generation processing to obtain a target test case;
[0213] The testing unit 16 is used to perform data testing processing on the data to be tested through the target test case to obtain a data testing result.
[0214] The data testing device provided by the present disclosure performs information extraction processing on the data to be tested to obtain the first error message, and performs parameter extraction processing based on the first error message and the data to be tested to obtain the target error parameter in the data to be tested; performs data matching processing from the data to be tested based on the first error message to obtain the initial interface data corresponding to the first error message, and performs data update processing on the initial interface data to obtain the target interface data; inputs the target error parameter and the target interface data into a preset fuzzy tester for case generation processing to obtain a target test case; performs data testing processing on the data to be tested through the target test case to obtain a data test result. Compared with the related art, the embodiment of the present disclosure generates a target test case by obtaining the target error parameter related to the error message of the data to be tested, and the target interface data in the data to be tested corresponding to the error message, and performs data testing processing on the data to be tested through the generated target test case, which can enhance the understanding and analysis of the context of the data to be tested and improve the accuracy of the data test results.
[0215] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.11 As shown, the first extraction unit 11 includes:
[0216] The compiling module 111 is used to compile the data to be tested to obtain a target data file, and determine each program entry point of the data to be tested according to the target data file; wherein the data to be tested contains multiple program data, and the program entry point is the initial function of each program data in the data to be tested;
[0217] A construction module 112 is used to perform data construction processing according to the program entry point and the target data file through a preset data analysis algorithm to obtain control flow graph data;
[0218] An extraction module 113, used for performing data extraction processing according to the control flow graph data to obtain work list data;
[0219] A search module 114 is used to perform data search processing on the work list data to obtain call statement data; wherein each call statement in the call statement data carries an error identifier;
[0220] The extraction module 113 is further used to traverse the call statement data and perform information extraction processing on the call statement data to obtain the first error information.
[0221] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.11 As shown, the second extraction unit 12 includes:
[0222] The processing module 121 is used to perform parameter symbolization processing on the data to be tested to obtain symbolized data to be tested;
[0223] The processing module 121 is also used to perform case generation processing through the preset fuzz tester to obtain a random test case, and perform data test processing on the symbolized data to be tested through the random test case to obtain second error information; wherein the random test case is a test case generated by inputting random data into the preset fuzz tester;
[0224] An extraction module 122, configured to perform information extraction processing according to the first error information and the second error information to obtain target error data;
[0225] A determination module 123, configured to determine a parameter dependency relationship between each parameter in the test data and the first error information and the second error information according to a parameter path condition in the target error data;
[0226] The processing module 121 is further used to perform data inference processing according to the parameter dependency and the target error data to obtain the target error parameter.
[0227] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.11 As shown, the matching unit 13 includes:
[0228] The analysis module 131 is used to perform data analysis on the first error information to obtain a first variable, and to perform data analysis on the test data to obtain a second variable;
[0229] A determination module 132, configured to determine an error path condition of the first error information and interface call information of the data to be tested by a preset dynamic data flow algorithm, and determine an association relationship between the first error information and the data to be tested according to the error path condition and the interface call information;
[0230] An establishing module 133, configured to establish a mapping relationship between the first variable and the second variable according to the association relationship;
[0231] The determination module 132 is further configured to, based on the mapping relationship, use the interface data associated with the second variable in the data to be tested as the initial interface data corresponding to the first error message.
[0232] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.11 As shown, the updating unit 14 includes:
[0233] The analysis module 141 is used to input the target interface data into the preset fuzzy tester, analyze the dependency relationship between the target interface data and the preset precondition, and obtain invalid dependencies; wherein the invalid dependencies are interface data in the target interface data that has no dependency relationship with the preset precondition;
[0234] A matching module 142, configured to perform matching processing from a preset strategy database according to the data type and data characteristics of the invalid dependency to obtain a target repair strategy, wherein the invalid dependency includes the data type and data characteristics of the invalid dependency, and the preset strategy database includes multiple repair strategies;
[0235] The repair module 143 is used to perform data repair processing on the invalid dependency through the target repair strategy to obtain the target interface data.
[0236] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.11 As shown, the generating unit 15 includes:
[0237] An extraction module 151 is used to extract the error type, error structure and error content of the first error information;
[0238] A matching module 152 is used to perform matching processing from a preset variation mode database according to the error type, error structure and error content to obtain a target variation mode, wherein the preset variation mode database includes multiple parameter variation modes;
[0239] A mutation module 153 is used to perform parameter mutation processing on the target error parameter by using the target mutation method to obtain a mutated target error parameter;
[0240] The generating module 154 is used to perform a case generation process according to the target error parameter and the target interface data to obtain the target test case.
[0241] Furthermore, in a possible implementation of the embodiment of the present disclosure, the test unit 16 is also used to perform data test processing on the data to be tested through the target test case in parallel according to a preset processing unit to obtain a data test result.
[0242] Furthermore, in a possible implementation of the embodiment of the present disclosure, the generation unit 15 is also used to input the target error parameters and the target interface data into a preset fuzzy tester in parallel according to the preset processing unit for case generation processing to obtain the target test case.
[0243] It should be noted that the above explanation of the method embodiment is also applicable to the device of the embodiment of the present disclosure, and the principle is the same, which is no longer limited in the embodiment of the present disclosure.
[0244] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0245] Fig.12 A schematic block diagram of an example electronic device 1200 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0246] like Fig.12 As shown, the device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1202 or a computer program loaded from a storage unit 1208 to a RAM (Random Access Memory) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An I / O (Input / Output) interface 1205 is also connected to the bus 1204.
[0247] A number of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0248] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as a method for data testing. For example, in some embodiments, the method for data testing may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to execute the aforementioned data testing method in any other appropriate manner (eg, by means of firmware).
[0249] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0250] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0251] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory) or a flash memory, an optical fiber, a CD-ROM (Compact Dis sc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0252] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0253] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.
[0254] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0255] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.
[0256] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0257] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A data testing method, characterized in that: include: Performing information extraction processing on the data to be tested to obtain first error information, and performing parameter extraction processing based on the first error information and the data to be tested to obtain target error parameters in the data to be tested; performing data matching processing on the data to be tested according to the first error information to obtain initial interface data corresponding to the first error information, and performing data updating processing on the initial interface data to obtain target interface data; Inputting the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case; The data to be tested is processed by data testing through the target test case to obtain a data testing result.
2. The method according to claim 1, characterized in that The information extraction process is performed on the test data to obtain the first error message, which includes: Performing data compilation processing on the data to be tested to obtain a target data file, and determining each program entry point of the data to be tested according to the target data file; wherein the data to be tested contains a plurality of program data, and the program entry point is an initial function of each program data in the data to be tested; According to the program entry point and the target data file, a data construction process is performed by a preset data analysis algorithm to obtain control flow graph data, and data extraction process is performed according to the control flow graph data to obtain work list data; Performing data search processing on the work list data to obtain call statement data; wherein each call statement in the call statement data carries an error identifier; The calling statement data is traversed, and information extraction processing is performed on the calling statement data to obtain the first error information.
3. The method according to claim 1, characterized in that The performing parameter extraction processing according to the first error information and the data to be tested to obtain the target error parameter in the data to be tested includes: Performing parameter symbolization processing on the data to be tested to obtain symbolized data to be tested; Performing case generation processing through the preset fuzz tester to obtain a random test case, and performing data test processing on the symbolized data to be tested through the random test case to obtain second error information; wherein the random test case is a test case generated by inputting random data into the preset fuzz tester; Perform information extraction processing according to the first error information and the second error information to obtain target error data; The parameter dependency relationship between each parameter in the data to be tested and the first error information and the second error information is determined according to the parameter path conditions in the target error data, and data inference processing is performed according to the parameter dependency relationship and the target error data to obtain the target error parameter.
4. The method according to claim 1, characterized in that: The performing data matching processing from the data to be tested according to the first error information to obtain the initial interface data corresponding to the first error information includes: Performing data analysis on the first error information to obtain a first variable, and performing data analysis on the test data to obtain a second variable; Determine an error path condition of the first error information and interface call information of the data to be tested by a preset dynamic data flow algorithm, and determine an association relationship between the first error information and the data to be tested according to the error path condition and the interface call information; Establishing a mapping relationship between the first variable and the second variable according to the association relationship; Based on the mapping relationship, the interface data associated with the second variable in the data to be tested is used as the initial interface data corresponding to the first error information.
5. The method according to claim 1, characterized in that The performing data updating processing on the initial interface data to obtain the target interface data comprises: Input the target interface data into the preset fuzzy tester, analyze the dependency relationship between the target interface data and the preset precondition, and obtain invalid dependencies; wherein the invalid dependencies are interface data in the target interface data that has no dependency relationship with the preset precondition; According to the data type and data characteristics of the invalid dependency, matching processing is performed from a preset strategy database to obtain a target repair strategy, wherein the invalid dependency includes the data type and data characteristics of the invalid dependency, and the preset strategy database includes multiple repair strategies; The invalid dependency is subjected to data repair processing by using the target repair strategy to obtain the target interface data.
6. The method according to claim 1, characterized in that The step of inputting the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case comprises: Extracting the error type, error structure and error content of the first error information; According to the error type, error structure and error content, matching processing is performed from a preset variation mode database to obtain a target variation mode, wherein the preset variation mode database includes multiple parameter variation modes; Performing parameter variation processing on the target error parameter by the target variation method to obtain a mutated target error parameter; A case generation process is performed according to the target error parameter and the target interface data to obtain the target test case.
7. The method according to claim 1, characterized in that The performing data testing processing on the data to be tested by using the target test case to obtain the data testing result comprises: The process of performing data testing processing on the data to be tested through the target test case in parallel according to the preset processing unit to obtain the data testing result.
8. The method according to claim 7, characterized in that The step of inputting the target error parameter and the target interface data into a preset fuzz tester for case generation processing to obtain a target test case comprises: According to the preset processing unit, the target error parameters and the target interface data are input into the preset fuzzy tester in parallel for case generation processing to obtain the target test case.
9. A data testing device, characterized in that: include: A first extraction unit, used for performing information extraction processing on the test data to obtain first error information; A second extraction unit, configured to perform parameter extraction processing according to the first error information and the data to be tested, to obtain a target error parameter in the data to be tested; a matching unit, configured to perform data matching processing on the data to be tested according to the first error information, and obtain initial interface data corresponding to the first error information; An updating unit, used for performing data updating processing on the initial interface data to obtain target interface data; A generating unit, used for inputting the target error parameter and the target interface data into a preset fuzzy tester for case generation processing to obtain a target test case; The testing unit is used to perform data testing on the data to be tested through the target test case to obtain a data testing result.
10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.
Citation Information
Cited By
Method and device for simplifying database case based on error reporting information and electronic equipment
CN119226178A
