Symbolic execution method based on machine learning techniques

Abstract

The application discloses a symbolic execution method based on machine learning technology, which uses a machine learning technology, a weighted K nearest neighbor algorithm (KNN), to learn the relationship between program path constraints and their solvability, and uses the same to predict the solvability of path constraints that exceed a certain solving time threshold, thereby guiding the symbolic execution engine whether to continue the current solving process, fill the sample library with path constraints generated during the symbolic execution process and have real solving results, and assign data in the sample library according to the similarity between programs and update the weight according to the prediction error. The application extracts features on a file in the SMT-LIB format, is platform-independent, and can reduce a large number of invalid constraint solving processes in the symbolic execution process, and improve the practicability of the symbolic execution when applied to large programs.

Description

TECHNICAL FIELD

[0001] The application is designed in the field of symbolic execution method in automated software testing, and particularly designs a symbolic execution framework based on machine learning technology. BACKGROUND

[0002] As a program analysis technique for automated software testing, symbolic execution collects path conditions and converts them into constraints in the process of traversing the program, and provides them to the constraint solver to obtain feasible solutions. However, when applying symbolic execution to large and complex programs, there is still a low scalability and practicability situation. The main reason for the above situation is that the constraint solving is too time-consuming. Constraint solving, as the key driver of symbolic execution, is also its main performance bottleneck, but it is actually a very difficult problem to calculate. The constraint solver used by the mainstream symbolic execution engine is usually based on Satisfiability Modulo Theory (SMT), that is, SMT solver, such as Z3 and CVC4. They convert the path constraints collected in the symbolic execution process into SMT formulas, and then check their satisfiability through the SMT solver. Only the path constraints that are satisfiable trigger the execution of the input value of the input value. In real-world programs, there are usually a large number of nonlinear structures, function calls, and various challenging theories. Therefore, it is gradually difficult to realize the traversal of challenging programs in a short time and generate the input value of the triggering path. The existing symbolic execution tools terminate the path constraints of the solving time process by setting a time threshold, and abandon the solving process that exceeds the time threshold, resulting in a large number of invalid solving processes. SUMMARY

[0003] In view of the deficiencies of the prior art, the application provides a symbolic execution method based on machine learning technology, which uses a machine learning algorithm, K-nearest neighbor algorithm, to classify the solvability of path constraints and preferentially solve constraints with a classification result of solvable. The specific technical solutions are as follows:

[0004] The symbolic execution method based on machine learning technology comprises the following steps:

[0005] Step 1: Construct a path constraint feature data set containing categories and assign weights to the data; the specific steps are as follows:

[0006] Step 1.1: Use a symbolic execution tool to test m programs (P1, P2,..., P m ) within its own set time threshold T, record the SMT-LIB format file corresponding to the path constraint whose solving time exceeds the time threshold t (t < T), and according to the solving result, divide it into two categories, C = {c1, c2};

[0007] Step 1.2: Define N key features and extract features from the SMT-LIB format file obtained in Step 1.1. Map each feature value to the range [0,1] to form a feature vector x = R that can represent the path constraint characteristics. N The sample library D = {d1, d2, ..., d} is obtained. n}, where n represents the total number of samples;

[0008] Step 1.3: Assign weights W(w1, w2, ..., w) to each sample in the sample library D obtained in Step 1.2 based on the similarity between the test program and the historical programs (programs that have already been tested). n ).

[0009] Step 2: Weighted KNN uses the sample library D obtained in Step 1 to learn the relationship between path constraints and their solvability, and constructs a path constraint classifier;

[0010] Step 3: During the symbolic execution of the program under test, if the current path constraint can be successfully solved within a certain small time threshold t, the same as the traditional symbolic execution process is followed, and the input value that triggers the current path is output; if the solution process is not completed, the path constraint classifier in step 2 is triggered, and the process proceeds to step 4.

[0011] Step 4: Extract features from the SMT-LIB file corresponding to the current path constraint to obtain feature vectors, and use weighted KNN to classify their solvability. If the classification result is solvable, continue to send it to the constraint solver for solving; if it is unsolvable, abandon the current solving process, and you can choose to process all paths and then solve again.

[0012] Step 5: Add the path constraints with real solution results verified in Step 4 to the sample library of weighted KNN, expand the dataset and update the sample weights according to the prediction loss, then go to Step 3 to explore the next program path until the entire program traversal is completed.

[0013] Furthermore, the solvability mentioned in step 1.1 refers to whether the path constraint can be constrained to complete the solution within the time threshold and output the specific input value that triggers the current path. Solvability means that the SMT solver returns "sat", while unsolvability means that the SMT solver returns "unknown" or "unsat".

[0014] Furthermore, the specific steps for calculating the similarity between the test program and the historical program in step 1.3 are as follows:

[0015] Step 1.3.1: Take the test program and the historical program as text respectively, embed them by using the text embedding tool Word2Vec to obtain E(e, e1, e2,..., ei), and calculate the similarity S(s1, s2,..., si) between P and P (i = 1, 2,..., m) by using the cosine similarity formula, which is used as the reference for assigning initial weights to sample data, and the same weight is assigned to samples from the same program. The cosine similarity calculation formula is as follows: m i m Step 1.3.2: Finally, in order to make the weight sum equal to 1, the weights of all samples obtained in step 1.3.1 are normalized by using the following formula:

[0016] Where i represents the number of historical programs;

[0017] Step 1.3.2: Finally, in order to make the weight sum equal to 1, the weights of all samples obtained in step 1.3.1 are normalized by using the following formula:

[0018] Where w i represents the weight corresponding to the i-th sample;

[0019] Further, the specific steps of using weighted KNN to classify the constraint solvability in step 4 include, for example, the distance between the sample d i in the sample library and pc is:

[0020]

[0021] Step 4.2: Sort dist in step 4.1 and select the k samples closest to pc, and the k nearest neighbors and their corresponding categories form a set

[0022] Step 4.3: Calculate the probability PR that pc belongs to each category based on the weight W and the distance dist, and the calculation formula is as follows Where is an indicator function, as shown in the following formula: Where represents the category to which the i-th sample k in D belongs;

[0023] Step 4.4: Finally, pc is classified into the category corresponding to the maximum PR, and there are two categories, solvable and unsolvable.

[0024] Compared with the closest prior art, the beneficial effects of the present application are as follows:

[0025] ​​1.The application realizes a path constraint solvability classifier based on a weighted KNN algorithm, and embeds it into a symbolic execution framework, in the process of symbolic execution, the solvability of path constraints exceeding a certain solving time threshold is predicted, if it is solvable, it is sent to the constraint solver for normal solving, if the classification result is unsolvable, the current solving process is temporarily abandoned, and the solving after all paths are processed can be selected, which can reduce a large number of invalid solving processes, improve the practicability and efficiency of symbolic execution applied to large programs.

[0026] 2.The application assigns weights to samples in the weighted KNN sample library according to the similarity between the tested program and the historical program, and updates the weights based on the prediction loss, and adds the path constraints belonging to the current program generated in the symbolic execution process to the sample library, which is beneficial to improve the accuracy of path constraint solvability classification. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a flowchart of the symbolic execution method based on machine learning technology of the embodiment.

[0028] Figure 2 is a flowchart of the path constraint solvability classification of the embodiment. DETAILED DESCRIPTION

[0029] The application will be further described in detail below according to the drawings and preferred embodiments, and the specific embodiments described herein are only used to explain the application, but do not limit the protection scope of the application.

[0030] As shown in Figure 1 , the symbolic execution method based on machine learning technology comprises the following steps:

[0031] Step 1: Construct a path constraint feature data set containing categories and assign weights to the data; the specific steps are as follows:

[0032] Step 1.1: use a symbolic execution tool to test m programs (P1, P2,..., P m ) within the time threshold T set by itself, record the path constraints corresponding to the SMT-LIB format file whose solving time exceeds the time threshold t (t

[0033] Further, the solvability refers to whether the path constraint can be solved by the constraint solver within the time threshold and output the specific input value triggering the current path, solvable means that the SMT solver returns sat, and unsolvable means that the SMT solver returns both unknown and unsat.

[0034] Step 1.2: Define N key features and extract features from the SMT-LIB format file obtained in Step 1.1. Map each feature value to the range [0,1] to form a feature vector x = R that can represent the path constraint characteristics. N The sample library D = {d1, d2, ..., d} is obtained. n}, where n represents the total number of samples;

[0035] Step 1.3: Assign weights W(w1, w2, ..., w) to each sample in the sample library D obtained in Step 1.2 based on the similarity between the test program and the historical programs (programs that have already been tested). n ).

[0036] The specific steps for calculating the similarity between the test program and the historical program are as follows:

[0037] Step 1.3.1: Treat the test program and the historical program as text, and embed them using the text embedding tool Word2Vec to obtain E(e,e1,e2,...,e m And use the cosine similarity formula to calculate P and P i The similarity S(s1,s2,...,s) between (i = 1, 2, ..., m) m This serves as the benchmark for assigning initial weights to sample data, ensuring that samples from the same program are assigned the same weights. The cosine similarity calculation formula is shown below:

[0038] Where i represents the number of the historical procedure;

[0039] Step 1.3.2: Finally, to ensure the sum of the weights equals 1, normalize the weights of all samples obtained in Step 1.3.1 using the following formula:

[0040] Where w i This represents the weight corresponding to the i-th sample;

[0041] Step 2: Weighted KNN uses the sample library D obtained in Step 1 to learn the relationship between path constraints and their solvability, and constructs a path constraint classifier;

[0042] Step 3: During the symbolic execution of the program under test, if the current path constraint can be successfully solved within a certain small time threshold t, the same as the traditional symbolic execution process is followed, and the input value that triggers the current path is output; if the solution process is not completed, the path constraint classifier in step 2 is triggered, and the process proceeds to step 4.

[0043] Step 4: Extract features from the SMT-LIB file corresponding to the current path constraint to obtain a feature vector, and use weighted KNN to classify the solvability thereof, if the classification result is solvable, continue to send it into the constraint solver for solving, if it is unsolvable, abandon the current solving process, and all paths can be selected to be processed again for solving after being processed;

[0044] The specific steps of using weighted KNN to classify the solvability of the constraint include, for example, the distance between the sample d i in the sample library and pc is:

[0045]

[0046] Step 4.2: Sort dist in step 4.1 and select the k samples closest to pc, and the k nearest neighbors and their corresponding categories form a set

[0047] Step 4.3: Calculate the probability PR that pc belongs to each category based on the weight W and the distance dist, and the calculation formula is as follows wherein is an indicator function, as shown in the following formula: wherein represents the category to which the i-th sample k in D belongs;

[0048] Step 4.4: Finally, pc is classified into the category corresponding to the maximum PR, and there are two categories, solvable and unsolvable.

[0049] Step 5: Add the path constraint with the true solving result verified in step 4 to the sample library of weighted KNN, expand the data set and update the sample weight according to the prediction loss, and then go to step 3 to explore the next program path, until the entire program is traversed

[0050] The above method of the embodiment, in the process of software testing of the program, the solvability of the path constraint is judged based on the machine learning technology, the solvability of the constraint model can be quickly predicted, and the path constraint with the prediction result of unsolvable is abandoned, unnecessary constraint solving process is avoided, and the efficiency of the symbolic execution is improved.

[0051] Embodiment 1:

[0052] The following will first list the use of symbolic execution tool angr as the basic embedded path constraint classifier, test 10 programs from GNU Coreutils 9.0, and use angr to perform symbolic execution as the original solving time comparison. Then the overall test results of the symbolic execution engine will be explained, and the present application will be further described in detail.

[0053] The SMT constraint solver in angr is set to Z3, the same time threshold T is 300 seconds, and the time threshold t in the present application is set to 2 seconds. The solvability of the path constraint is only predicted when the solving time is longer than 2 seconds. If the result is solvable, it is solved again within 300 seconds.

[0054] The experimental results are shown in Table 1. Finally, compared with the symbolic execution tool angr without integrating the path constraint solvability classification module, the present application reduces the time overhead by 20.7% to 67.9% for different programs. In contrast, the present application has better performance when processing large programs, and the path constraint solvability classifier proposed in this paper can be combined with most symbolic execution engines as long as they use the SMT-LIB specification.

[0055] Table 1. Symbolic execution time comparison

[0056]

[0057] Finally, it should be noted that: the above only describes the preferred embodiments of the present application, and does not limit the present application in any form. Although the present application has been described in detail with reference to the foregoing examples, it can still be modified slightly by those skilled in the art according to the technical solutions recorded in the foregoing examples, or some technical features can be replaced with equivalent ones. Any modification, equivalent change and modification made within the spirit and principles of the present application shall fall within the scope of the technical solutions of the present application.

Claims

1. A symbolic execution method based on machine learning technology, characterized in that, The method includes the following steps: Step 1: Construct a path constraint feature dataset containing categories and assign weights to the data; the specific steps are as follows: Step 1.1: Utilize the symbolic execution tool at its self-defined time threshold. Internal testing A program The record shows that the solution time exceeds the time threshold. The path constraints are represented by the corresponding SMT-LIB format files, and the results are used to classify them into two categories: solvable and unsolvable. ; Step 1.2: Definition The key features are identified and features are extracted from the SMT-LIB format file obtained in step 1.

1. Each feature value is then mapped to... Within a certain range, feature vectors that can represent the characteristics of path constraints are formed. , obtain the sample library ,in Represents the total number of samples; Step 1.3: Based on the similarity between the test program and the historical programs, compile the sample library obtained in Step 1.

2. Each sample in the sample is assigned a weight. ; Step 2: Weighted KNN using the sample library obtained in Step 1 Learn the relationship between path constraints and their solvability, and construct a path constraint classifier; Step 3: During the symbolic execution of the program to be tested, if the current path constraint can be met within a certain time threshold... If the solution is successfully obtained, the process is the same as the traditional symbolic execution process, and the input value that triggers the current path is output; if the solution process is not completed, the path constraint classifier in step 2 is triggered, and the process proceeds to step 4. Step 4: Extract features from the SMT-LIB file corresponding to the current path constraint to obtain feature vectors, and use weighted KNN to classify their solvability. If the classification result is solvable, continue to send it to the constraint solver for solving; if it is unsolvable, abandon the current solving process, process all paths and solve again. The specific steps for classifying constraint solvability using weighted KNN in step 4 include: Step 4.1: First, calculate the current path constraints using Euclidean distance. Distance between each data point in the sample database , and samples in the sample library The distance is: ； Step 4.2: Regarding the steps in Step 4.1 Sort and select the ones that are far away. Recent Each sample will have its nearest neighbor. A set consisting of individual samples and their corresponding categories ; Step 4.3: Based on weights and distance Calculate Probability of belonging to each class The calculation formula is as follows: ,in The indicator function is shown in the following formula: ,in express The Middle Sample The category to which it belongs; Step 4.4: Finally, Return to the largest There are two categories: solvable and unsolvable. Step 5: Add the path constraints with real solution results verified in Step 4 to the sample library of weighted KNN, expand the dataset and update the sample weights according to the prediction loss, then go to Step 3 to explore the next program path until the entire program traversal is completed.

2. The symbolic execution method based on machine learning technology according to claim 1, characterized in that, The solvability mentioned in step 1.1 refers to whether the path constraint can be constrained to complete the solution within the time threshold and output the specific input value that triggers the current path. Solvability means that the SMT solver returns... In this case, "unsolvable" means the SMT solver returns an error. and Two scenarios.

3. The symbolic execution method based on machine learning technology according to claim 1, characterized in that, The specific steps for calculating the similarity between the test program and the historical program in step 1.3 are as follows: Step 1.3.1: Treat the test program and the historical program as text respectively, and use a text embedding tool. Embed it to obtain And calculate using the cosine similarity formula. and Similarity between This serves as the benchmark for assigning initial weights to sample data, ensuring that samples from the same program are assigned the same weights. The cosine similarity calculation formula is shown below: ,in Indicates the number of the historical procedure; Step 1.3.2: Finally, to ensure the sum of the weights equals 1, normalize the weights of all samples obtained in Step 1.3.1 using the following formula: ,in Representing the The weights corresponding to each sample.

4. The symbolic execution method based on machine learning technology according to claim 1, characterized in that, The specific weight update strategy in step 5 is as follows: if the predicted result is consistent with the actual solution result, then the weight is increased accordingly. Weights of samples of the same type Otherwise, reduce ,in Setting it to 0.2 adds path constraints with actual solution results to the sample library. And assign weights to them Finally, the weights of all samples are normalized.

Images