Test method and system based on artificial intelligence

Abstract

The invention provides a test method and system based on artificial intelligence. According to the method, the position of a given source code is set as a first target; the source code is compiled for the first time to obtain a function call graph and a control flow graph, the function call graph is used for representing the relation between functions, and the control flow graph is used for representing the relation between basic blocks; obtaining distance values between all the basic blocks and the first target according to the first target, the function call graph and the control flow graph; compiling the source code for the second time so as to carry out instrumentation on the basic block, and taking the distance value as a distance used during instrumentation; when the coverage rate of the test is smaller than a first threshold value, a first variation strategy is used for testing the source code, and when the coverage rate is larger than or equal to the first threshold value, a second variation strategy is used for testing the source code; the first variation strategy comprises the use of a decision model based on deep learning; the second variation strategy comprises the step of selecting the seed test case with the maximum use energy value.

Description

Technical Field

[0001] This application relates to the field of test network security technology, and particularly to a test method and system based on artificial intelligence. Background Art

[0002] With the development of the Internet, security monitoring and risk management have become increasingly important. Especially in preventing network attacks, more and more challenges are faced. How to achieve network operation and maintenance security has become a difficult problem that needs to be solved urgently.

[0003] The invention patent application No. CN202410641473.3 discloses a network security intelligent evaluation system based on artificial intelligence, including an artificial intelligence data set collection module, an Ethernet protocol analysis module, an artificial intelligence security detection module, a response module, and an output module. The artificial intelligence data set collection module is used to collect artificial intelligence data sets, including image data sets, text data sets, audio data sets, video data sets, numerical data sets, and geospatial data sets. The Ethernet protocol analysis module analyzes the Ethernet protocol for security detection of artificial intelligence, including an Ethernet type field analysis module and an IP type field analysis module. The artificial intelligence security detection module detects the security of artificial intelligence to determine whether the artificial intelligence data is secure, including vulnerability scanning, anomaly detection, malicious code detection, data filtering and cleaning, model monitoring and updating, model auditing and verification, access control and permission management, and encryption and data protection. The response module receives the data detected by the Ethernet protocol analysis module and the artificial intelligence security detection module and responds according to the situation, including a data receiving module and a data sorting module. The output module is used to output the feedback obtained by the response module for the operator to view. However, this network security system is relatively complex.

[0004] Therefore, it is necessary to provide a test method and system based on artificial intelligence to solve the above problems. Summary of the Invention

[0005] This application provides a test method and system based on artificial intelligence. By real-time monitoring the test coverage rate, when the coverage rates are different, the first mutation strategy and the second mutation strategy are respectively used, thereby greatly improving the test efficiency.

[0006] In a first aspect, this application provides a test method based on artificial intelligence, and the method includes:

[0007] Set the position of the given source code as the first target;

[0008] Perform the first compilation on the source code to obtain a function call graph and a control flow graph. The function call graph is used to represent the relationships between functions, and the control flow graph is used to represent the relationships between basic blocks;

[0009] Obtain the distance values between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph;

[0010] Perform the second compilation on the source code to instrument the basic blocks, and use the distance value as the distance used during instrumentation;

[0011] Monitor the test coverage in real time. When the coverage is less than the first threshold, use the first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use the second mutation strategy to test the source code; the first mutation strategy includes using a decision model based on deep learning; the second mutation strategy includes selecting the seed test case with the maximum energy value.

[0012] In a possible design, generate an attribute control flow graph based on the control flow graph, obtain the statistical features and external features of the instructions from the basic blocks, so as to embed the basic blocks into the vector space. The statistical features include string constants, numeric constants, the number of transfer instructions, the number of call instructions, the number of instructions, and the number of arithmetic instructions. The external features include the number of connected nodes and the number of child nodes, and use an adjacency matrix to record the node relationships of the external features.

[0013] In a possible design, the distance value is stored in shared memory, and the shared memory is mapped to the address space of the process through a function for access.

[0014] In a possible design, the decision model based on deep learning is used to predict the attack probability of the function and assign static vulnerability scores to the basic blocks in the vulnerable functions. Statistically sum up the static vulnerability scores of all basic blocks on the execution path, and select the seed test case with the highest total score for testing;

[0015] The static vulnerability score of the basic block is calculated by the following formula:

[0016]

[0017] Among them, Score(j) represents the static vulnerability score of the basic block; c represents a coefficient; Cover(j) represents the proportion of the number of branches triggered by the j-th test case in the total number of branches of the source code, that is, the coverage rate; Pred(j) represents the proportion of the number of test cases that trigger new paths among the test cases generated by mutating the j-th test case in the total number of mutations of the j-th test case; Freq(j) represents the number of times the new path triggered by the j-th test case is executed by all test cases in the queue.

[0018] In a possible design, the energy value of the seed test case is calculated according to the distance value, and the smaller the distance value, the greater the energy value assigned to the seed test case;

[0019] When the time for the seed test case to stay at the current position for testing is greater than the second threshold, the energy value of the seed test case is reduced.

[0020] In a possible design, the energy value of the seed test case is calculated according to the distance value and the constraints, and the seed test cases are arranged according to the weight of meeting the constraints. The more constraints the seed test case passes and the closer it is to the constraints, the greater the weight;

[0021] The constraints are composed of variables automatically generated by the operations used to capture the location of the first target. The variables include reference addresses, array sizes, and current positions, and the constraints are sorted according to a preset priority;

[0022] The variables form data conditions, and the data conditions are a boolean expression.

[0023] In a possible design, when there are multiple data conditions, the distance of the multiple data conditions is calculated by the following formula:

[0024]

[0025] Among them, represents the distance of the multiple data conditions, c data represents the maximum distance of a single data condition, N unsat represents the number of unmet data conditions, Q represents the data conditions, ρ represents the subscript of the first unmet data condition, represents the distance of the n-th basic block under Q.

[0026] In a second aspect, the present application provides an artificial intelligence-based test system, including:

[0027] A first target setting module, which is used to set the position of the given source code as the first target;

[0028] A first compilation module, which is used to perform a first compilation on the source code to obtain a function call graph and a control flow graph. The function call graph is used to represent the relationships between functions, and the control flow graph is used to represent the relationships between basic blocks;

[0029] A distance value acquisition module, which is used to obtain the distance values between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph;

[0030] A second compilation module, which is used to perform a second compilation on the source code to perform instrumentation on the basic blocks, and use the distance value as the distance used during instrumentation;

[0031] A test module, which is used to monitor the test coverage in real time. When the coverage rate is less than the first threshold, use a first mutation strategy to test the source code. When the coverage rate is greater than or equal to the first threshold, use a second mutation strategy to test the source code; the first mutation strategy includes using a decision model based on deep learning; the second mutation strategy includes selecting the seed test case with the maximum energy value.

[0032] In a third aspect, the present application provides an electronic device, including:

[0033] A processor; and,

[0034] A memory, which is used to store the executable instructions of the processor;

[0035] Wherein, the processor is configured to execute any possible method described in the first aspect by executing the executable instructions.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement any possible method described in the first aspect.

[0037] The testing method and system based on artificial intelligence provided by this application set the position of the given source code as the first target; perform the first compilation on the source code to obtain a function call graph and a control flow graph, where the function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks; obtain the distance values between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph; perform the second compilation on the source code to instrument the basic blocks, and use the distance value as the distance used during instrumentation; monitor the test coverage in real time. When the coverage is less than the first threshold, use the first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use the second mutation strategy to test the source code; the first mutation strategy includes using a decision model based on deep learning; the second mutation strategy includes selecting the seed test case with the maximum energy value. By combining the first mutation strategy and the second mutation strategy, the test efficiency is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0039] Figure 1 is a schematic flowchart of a testing method based on artificial intelligence according to an exemplary embodiment of the present application;

[0040] Figure 2 is a schematic flowchart of a testing method based on artificial intelligence according to another exemplary embodiment of the present application;

[0041] Figure 3 is a schematic structural diagram of a testing system based on artificial intelligence according to an exemplary embodiment of the present application;

[0042] Figure 4 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.

[0043] Through the above drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0045] Figure 1 is a schematic flowchart of an artificial intelligence-based testing method according to an exemplary embodiment of the present application. As Figure 1 shown, the method provided in this embodiment includes:

[0046] Step S101: Set the position of the given source code as the first target;

[0047] Step S102: Perform a first compilation on the source code to obtain a function call graph and a control flow graph. The function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks;

[0048] Step S103: Obtain the distance values between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph;

[0049] Step S104: Perform a second compilation on the source code to instrument the basic blocks, and use the distance value as the distance used during instrumentation;

[0050] Step S105: Monitor the test coverage in real time. When the coverage is less than the first threshold, use the first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use the second mutation strategy to test the source code; the first mutation strategy includes using a decision model based on deep learning; the second mutation strategy includes selecting the seed test case with the maximum energy value.

[0051] Specifically, perform a first compilation on the source code to obtain a function call graph and a control flow graph. The function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks. The function call graph is a graphical representation method that can show the call relationships between different modules and functions, indicating the overall structure and logic of the source code. The control flow graph is an abstract data structure used in compilers, indicating all the paths that will be traversed during the execution of the source code, and representing the possible flow directions of all basic blocks within a process in the form of a graph.

[0052] The source code is compiled for the second time to instrument the basic blocks. Instrumenting the basic blocks is to insert the distance value into the basic blocks, and this step is to calculate the energy value of the seed test cases based on the distance value. By monitoring the test coverage in real time, when the coverage is less than the first threshold, the first mutation testing strategy is used to test the source code, and when the coverage is greater than or equal to the first threshold, the second mutation strategy is used to test the source code. Different mutation strategies are used to test the source code according to the coverage situation, thus greatly improving the test efficiency.

[0053] In a specific implementation, an attribute control flow graph is generated based on the control flow graph, and the statistical features and external features of the instructions are obtained from the basic blocks, so as to embed the basic blocks into the vector space. The statistical features include string constants, numeric constants, the number of transfer instructions, the number of call instructions, the number of instructions, and the number of arithmetic instructions. The external features include the number of connected nodes and the number of child nodes. An adjacency matrix is used to record the node relationships of the external features. In a specific implementation, the distance value is stored in the shared memory, and the shared memory is mapped to the address space of the process through a function for access.

[0054] In a specific implementation, the deep learning-based decision model is used to predict the attack probability of the function and assign a static vulnerability score to the basic blocks in the vulnerable functions. The sum of the static vulnerability scores of all basic blocks on the execution path is calculated, and the seed test case with the highest score sum is selected for testing;

[0055] The static vulnerability score of the basic block is calculated by the following formula:

[0056]

[0057] Among them, Score(j) represents the static vulnerability score of the basic block; c represents a coefficient; Cover(j) represents the proportion of the branches triggered by the j-th test case in the total number of branches of the source code, that is, the coverage rate; Pred(j) represents the proportion of the number of test cases that trigger new paths among the test cases generated by mutating the j-th test case in the total number of mutations of the j-th test case; Freq(j) represents the number of times the new paths triggered by the j-th test case are executed by all test cases in the queue.

[0058] In a specific implementation, the energy value of the seed test case is calculated based on the distance value. The smaller the distance value, the greater the energy value assigned to the seed test case;

[0059] When the time for the seed test case to stay at the current position for testing is greater than the second threshold, the energy value of the seed test case is reduced.

[0060] Specifically, when the seed test case stays at the current position for too long, reduce the energy value of the seed test case.

[0061] In a specific implementation, the energy value of the seed test case is calculated according to the distance value and constraints, and the seed test cases are arranged according to the weights that satisfy the constraints. The more constraints the seed test case passes and the closer it is to the constraints, the greater the weight;

[0062] The constraints consist of variables automatically generated by the operations used to capture the location of the first target. The variables include reference addresses, array sizes, and the current position, and the constraints are sorted according to a preset priority;

[0063] The variables form data conditions, and the data conditions are a Boolean expression.

[0064] In a specific implementation, when there are multiple data conditions, the distance of the multiple data conditions is calculated by the following formula:

[0065]

[0066] Where, represents the distance of the multiple data conditions, c data represents the maximum distance of a single data condition, N unsat represents the number of data conditions that are not satisfied, Q represents the data conditions, ρ represents the subscript of the first data condition that is not satisfied, represents the distance of the nth basic block under Q.

[0067] Figure 2 is a schematic flowchart of a method for continuous testing of third-party components in a supply chain system according to another exemplary embodiment of the present application. As Figure 2 shown, this embodiment provides a method for continuous testing of third-party components in a supply chain system, including:

[0068] Step S201: Set the position of the given source code as the first target;

[0069] Step S202: Compile the source code for the first time to obtain a function call graph and a control flow graph. The function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks;

[0070] Step S203: Obtain the distance value between all basic blocks and the first target according to the first target, function call graph, and control flow graph;

[0071] Step S204: Perform a second compilation on the source code to instrument the basic blocks, and use the distance value as the distance used during instrumentation.

[0072] Step S205: Monitor the test coverage in real time. When the coverage is less than the first threshold, use the first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use the second mutation strategy to test the source code. The first mutation strategy includes using a decision model based on deep learning. The second mutation strategy includes selecting the seed test case with the maximum energy value.

[0073] Step S206: Generate an attribute control flow graph based on the control flow graph, and obtain the statistical features and external features of the instructions from the basic blocks, so as to embed the basic blocks into the vector space. The statistical features include string constants, numeric constants, the number of transfer instructions, the number of call instructions, the number of instructions, and the number of arithmetic instructions. The external features include the number of connected nodes and the number of child nodes. Use an adjacency matrix to record the node relationships of the external features.

[0074] Figure 3 It is a schematic structural diagram of a continuous testing system for third-party components in a supply chain system according to an exemplary embodiment of the present application. As Figure 3 shown, the continuous testing system for third-party components in a supply chain system provided in this embodiment includes:

[0075] A first target setting module 31, which is used to set the position of the given source code as the first target;

[0076] A first compilation module 32, which is used to perform a first compilation on the source code to obtain a function call graph and a control flow graph. The function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks;

[0077] A distance value obtaining module 33, which is used to obtain the distance values between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph;

[0078] A second compilation module 34, which is used to perform a second compilation on the source code to instrument the basic blocks, and use the distance value as the distance used during instrumentation;

[0079] A testing module 35, which is used to monitor the test coverage in real time. When the coverage is less than the first threshold, use the first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use the second mutation strategy to test the source code. The first mutation strategy includes using a decision model based on deep learning. The second mutation strategy includes selecting the seed test case with the maximum energy value.

[0080] Figure 4 This is a schematic structural diagram of an electronic device shown according to an exemplary embodiment of the present application. As Figure 4 shown, an electronic device 400 provided in this embodiment includes: a processor 401 and a memory 402; wherein:

[0081] The memory 402 is used to store a computer program, and this memory can also be a flash (flash memory).

[0082] The processor 401 is used to execute the execution instructions stored in the memory to implement each step in the above method. Specifically, reference can be made to the relevant descriptions in the foregoing method embodiments.

[0083] Optionally, the memory 402 can be either independent or integrated with the processor 401.

[0084] When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include:

[0085] A bus 403 for connecting the memory 402 and the processor 401.

[0086] This embodiment also provides a readable storage medium, in which a computer program is stored. When at least one processor of the electronic device executes this computer program, the electronic device executes the methods provided by the above various embodiments.

[0087] This embodiment also provides a program product, which includes a computer program, and this computer program is stored in a readable storage medium. At least one processor of the electronic device can read this computer program from the readable storage medium, and the execution of this computer program by at least one processor enables the electronic device to implement the methods provided by the above various embodiments.

[0088] The testing method and system based on artificial intelligence provided by the embodiments of the present application set the position of the given source code as the first target; perform a first compilation on the source code to obtain a function call graph and a control flow graph, where the function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks; obtain the distance values between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph; perform a second compilation on the source code to instrument the basic blocks, and use the distance value as the distance used during instrumentation; monitor the test coverage in real time. When the coverage is less than the first threshold, use the first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use the second mutation strategy to test the source code; the first mutation strategy includes using a decision-making model based on deep learning; the second mutation strategy includes selecting the seed test case with the maximum energy value. By combining the first mutation strategy and the second mutation strategy, the testing efficiency is greatly improved.

[0089] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0090] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A testing method based on artificial intelligence, characterized in that: The method comprises: Set the given source location as the first target; Compiling the source code for the first time to obtain a function call graph and a control flow graph, wherein the function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks; Obtaining distance values ​​between all basic blocks and the first target according to the first target, a function call graph, and a control flow graph; Compiling the source code for a second time to insert a stub into the basic block, and using the distance value as the distance used during the insertion; Monitor the test coverage in real time. When the coverage is less than a first threshold, use a first mutation strategy to test the source code. When the coverage is greater than or equal to the first threshold, use a second mutation strategy to test the source code. The first mutation strategy includes using a decision model based on deep learning. The second mutation strategy includes selecting a seed test case with the largest energy value.

2. The artificial intelligence-based testing method according to claim 1, characterized in that: An attribute control flow graph is generated based on the control flow graph, and statistical features and external features of instructions are obtained from the basic block, so as to embed the basic block into a vector space, wherein the statistical features include string constants, numerical constants, the number of transfer instructions, the number of call instructions, the number of instructions, and the number of arithmetic instructions; the external features include the number of connected nodes and the number of child nodes, and the node relationships of the external features are recorded using an adjacency matrix.

3. The artificial intelligence-based testing method according to claim 1, characterized in that: The distance value is stored in a shared memory, and the shared memory is mapped to the address space of the process through a function specification for access.

4. The artificial intelligence-based testing method according to claim 1, characterized in that: The deep learning-based decision model is used to predict the attack probability of the function and assign a static vulnerability score to the basic block in the vulnerable function, calculate the sum of the static vulnerability scores of all basic blocks on the execution path, and select the seed test case with the highest score sum for testing; The static vulnerability score of the basic block is calculated by the following formula: Among them, Score(j) represents the static vulnerability score of the basic block; c represents the coefficient; Cover(j) represents the ratio of the number of branches triggered by the j-th test case to the total number of branches in the source code, that is, the coverage rate; Pred(j) represents the ratio of the number of test cases that trigger new paths in the test cases generated by the j-th test case mutation to the total number of mutations of the j-th test case; Freq(j) represents the number of times the new path triggered by the j-th test case is executed by all test cases in the queue.

5. The artificial intelligence-based testing method according to claim 1, characterized in that: The energy value of the seed test case is calculated according to the distance value, and the smaller the distance value is, the greater the energy value assigned to the seed test case is; When the time that the seed test case stays at the current position for testing is greater than a second threshold, the energy value of the seed test case is reduced.

6. The artificial intelligence-based testing method according to claim 1, characterized in that: The energy value of the seed test case is calculated according to the distance value and the constraint, and the seed test case is arranged according to the weight of satisfying the constraint. The more constraints the seed test case passes and the closer it is to the constraint, the greater the weight; The constraints are composed of variables automatically generated by capturing the operations used at the location of the first target, the variables including the reference address, the array size and the current location, and the constraints are sorted according to a preset priority; The variables constitute a data condition, which is a Boolean expression.

7. According to the artificial intelligence-based testing method of claim 6, when there are multiple data conditions, the distances of the multiple data conditions are calculated by the following formula: in, represents the distance of the multiple data conditions, c data Indicates the maximum distance of a single data condition, N unsat represents the number of unsatisfied data conditions, Q represents the data condition, ρ represents the subscript of the first unsatisfied data condition, represents the distance of the nth basic block under Q.

8. A testing system based on artificial intelligence, characterized in that: The system comprises: A first target setting module, which is used to set the location of a given source code as a first target; A first compiling module, which is used to compile the source code for the first time to obtain a function call graph and a control flow graph, wherein the function call graph is used to represent the relationship between functions, and the control flow graph is used to represent the relationship between basic blocks; A distance value acquisition module, which is used to obtain the distance values ​​between all basic blocks and the first target according to the first target, the function call graph, and the control flow graph; A second compiling module, which is used to compile the source code for a second time to insert the basic block, and use the distance value as the distance used in the insertion; A test module is used to monitor the test coverage in real time. When the coverage is less than a first threshold, the source code is tested using a first mutation strategy. When the coverage is greater than or equal to the first threshold, the source code is tested using a second mutation strategy. The first mutation strategy includes using a decision model based on deep learning. The second mutation strategy includes selecting a seed test case with the largest energy value.

9. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images