Program symbolic execution test method, device and equipment

Abstract

The invention relates to the field of automatic testing, and provides a program symbolic execution testing method, device and equipment, and the method comprises the steps: obtaining a to-be-tested program, carrying out the symbolic execution testing of the to-be-tested program through a plurality of trained prediction models, and obtaining a plurality of test results; combining the test results of each prediction model to obtain a final test result; the trained prediction model is obtained by training according to a training set, and the training set is a program set. According to the method and the device, the defect that a single heuristic method in the prior art is difficult to comprehensively cover a program path is overcome, the efficient and comprehensive symbolic execution test on the to-be-tested program is realized, and the test coverage rate is improved.

Description

Technical Field

[0001] The present invention relates to the field of automated testing technology, and in particular to a program symbolic execution testing method, device and equipment. Background Art

[0002] Symbolic execution is an automated software testing technology that discovers errors and vulnerabilities in a program by simulating all possible paths during program execution. However, as the complexity of the program increases, the number of paths that need to be explored during symbolic execution grows exponentially, leading to the so-called "path explosion" problem, which greatly limits the efficiency and feasibility of symbolic execution in practical applications. Therefore, how to improve the efficiency of symbolic execution so that it can explore more program paths under limited resources has become an important research direction in the field of software engineering.

[0003] In order to solve the path explosion problem faced by symbolic execution, many scholars have proposed a variety of heuristic methods. Although these methods have alleviated the path explosion problem to a certain extent, the path selection of these methods often relies on a single metric attribute, such as fitness value, subpath count, constraint search depth, etc., which causes symbolic execution exploration to be biased towards the program part with higher metric attributes, while ignoring other program areas that may be equally important. Therefore, it is difficult for existing methods to fully cover all possible paths of the program, limiting the further improvement of code coverage. In addition, most of these methods rely on manually designed heuristic rules, lack the ability to automatically learn and optimize, and are difficult to adapt to complex and changeable program characteristics. Summary of the invention

[0004] The present invention provides a program symbolic execution testing method, device and equipment, which solves the defect that a single heuristic method in the prior art is difficult to fully cover the program path, realizes efficient and comprehensive symbolic execution testing of the program to be tested, and improves the test coverage.

[0005] The present invention provides a program symbolic execution test method, comprising the following steps: Obtain a program to be tested, and perform symbolic execution testing on the program to be tested using multiple trained prediction models to obtain multiple test results; The test results of each prediction model are combined to obtain the final test result; The trained prediction model is obtained by training based on a training set, and the training set is a program set.

[0006] According to a program symbolic execution testing method provided by the present invention, a prediction model is trained according to a training set to obtain a trained prediction model, which specifically includes: obtaining an unlabeled training set program set, and determining a labeled initial supervision data set according to the unlabeled training set program set; training a regression model based on the initial supervision data set to obtain an initial prediction model; performing update training based on active learning on the initial prediction model to obtain a final prediction model; using the final prediction model to run symbolic execution in a labeled training set program set to generate new test data, and iteratively training the regression model again according to the new test data, and obtaining a corresponding number of final prediction models according to the number of iterative training times.

[0007] According to a program symbolic execution testing method provided by the present invention, the step of obtaining an unlabeled training set program set and determining a labeled initial supervised data set based on the unlabeled training set program set specifically includes: obtaining an unlabeled training set program set, selecting an initial number of programs from the unlabeled training set program set for symbolic execution, and generating test data; obtaining multiple symbolic states in the test data, and determining a reward value for each symbolic state as a label; and labeling multiple symbolic states in the test data generated after symbolic execution of an initial number of programs according to the labels, and determining a labeled initial supervised data set.

[0008] According to a program symbolic execution test method provided by the present invention, the initial prediction model is updated and trained based on active learning to obtain a final prediction model, which specifically includes: based on a preset quality assessment standard, firstly selecting a program from a set of unlabeled training set programs for symbolic execution, and then using an active learning algorithm to select program state samples that are higher than a preset quality standard from the generated test data for labeling to obtain symbolic state samples; adding the symbolic state samples to the initial supervised data set, and updating and training the initial prediction model according to the updated supervised data set to obtain an updated prediction model; obtaining the prediction error of the updated prediction model for the symbolic state reward value, and judging whether to continue to update and train the prediction model based on active learning according to the prediction error and the current number of iterations; if the prediction error is less than a preset error threshold and the update training based on active learning reaches the number of iterations, then generating a final prediction model.

[0009] According to a program symbolic execution testing method provided by the present invention, based on a preset quality assessment standard, a program is first selected from an unlabeled training set program set for symbolic execution, and then an active learning algorithm is used to select program state samples with a quality higher than the preset quality standard from the generated test data for labeling. Specifically, the method includes: obtaining program state samples in the test data generated after symbolic execution of each candidate program in the unlabeled training set program set as candidate program state samples, and determining the uncertainty and representativeness of the candidate program state samples; determining the amount of information of each candidate program state sample based on a combined value of the uncertainty and representativeness of each candidate program state sample; clustering the candidate program state samples with an information amount higher than a preset information amount threshold to obtain multiple clusters; and selecting cluster centers of multiple clusters as program state samples with a quality higher than the preset standard for automatic labeling.

[0010] According to a program symbolic execution testing method provided by the present invention, the uncertainty and representativeness of each candidate program state sample in the unlabeled training set program set are obtained, specifically including: taking the mean square error between the predicted value and the true value of the unlabeled program state sample by the prediction model as the uncertainty measurement of the sample; obtaining the first entropy of the candidate program state sample, and the second entropy of other program state samples in the unlabeled training set program set except the candidate program state sample, and taking the difference between the first entropy and the second entropy as the representativeness of the candidate program state sample.

[0011] According to a program symbolic execution testing method provided by the present invention, after generating test data, the method further includes: extracting multidimensional features of symbolic states from the generated test data, and generating feature vectors corresponding to each symbolic state based on the multidimensional features; the multidimensional features include static features and dynamic features, the static features include bag-of-words representation of path constraints, and the dynamic features include state change information during the symbolic execution process.

[0012] The present invention also provides a program symbolic execution test device, comprising the following modules: A testing module is used to obtain a program to be tested, and to perform symbolic execution testing on the program to be tested through multiple trained prediction models to obtain multiple test results; A result merging module is used to merge the test results of each prediction model to obtain the final test result; The trained prediction model is obtained by training based on a training set, and the training set is a program set.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the program symbolic execution test methods described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the program symbolic execution test method described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the program symbolic execution testing methods described above.

[0016] The present invention provides a program symbolic execution testing method, device and equipment, which include the following beneficial effects: by using multiple trained prediction models to perform symbolic execution testing on the program to be tested, the program can be comprehensively and carefully evaluated from multiple angles, and using multiple models for testing can reduce the risks brought by a single model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the program symbolic execution analysis process based on active learning provided by the present invention.

[0019] Figure 2 It is a flowchart of the program symbolic execution testing method provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the program symbolic execution test device provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Symbolic execution technology aims to explore program paths in a symbolic input manner, thereby effectively improving code coverage. However, with the exponential growth of the number of symbolic states, symbolic execution often faces the problem of path explosion.

[0024] In order to solve the path explosion problem faced by symbolic execution technology and improve the efficiency of symbolic execution, many scholars have proposed heuristic methods for selecting execution paths that are most likely to lead to new discoveries.

[0025] Xie et al. proposed a heuristic method called Fitnex, which uses the fitness value of the program path state during symbolic execution to measure how close a discovered feasible path is to a specific test goal (for example, covering branches that have not yet been covered). The fitness value is calculated through a fitness function. The fitness value is used to guide the search strategy of path exploration to achieve high code coverage.

[0026] Li et al. systematically approximate full-path information to guide symbolic execution path exploration using a specific type of path spectrum, the length-n subpath program spectrum. The authors use the frequency distribution of exploring subpaths of length n to prioritize less explored program parts to improve test coverage and bug detection.

[0027] Păsăreanu et al. proposed the Symbolic Pathfinder (SPF), which combines symbolic execution with model checking and constraint solving, and uses constraint search depth metric properties to guide symbolic execution path exploration to automatically generate test cases and perform error detection.

[0028] He et al. proposed a learning-based strategy that estimates the contribution of each program path state in the symbolic execution process to achieve the maximum coverage target within the time budget, selects the most promising program path state for symbolic execution, and generates test cases with higher code line coverage while alleviating the path explosion problem.

[0029] Nicola et al. proposed an intelligent classification system to solve the path explosion problem: using supervised machine learning techniques to replicate the expertise of human experts to discover vulnerable paths. The method monitors execution traces in vulnerable programs and extracts relevant features - register and memory accesses, function complexity, system calls - to guide symbolic exploration. The model is trained to learn the patterns of vulnerable paths from the extracted features, and the model predictions are used to discover vulnerable execution paths in new programs.

[0030] Zhang Xiaosong et al. proposed a dynamic symbolic execution method to alleviate path explosion. This method can eliminate the redundant constraints generated by loops during program execution by mapping and acquiring the constraints generated by symbolic execution on the static decision graph of the program, thereby eliminating the path explosion problem caused by loops in dynamic symbolic execution.

[0031] Wang Lixin et al. proposed a learnable cooperative concolic testing technology (LCCT). LCCT designed a dynamic symbolic executor to use a reinforcement learning algorithm to learn the optimal path search strategy. The search strategy will be continuously adjusted according to the reward during the entire test process. LCCT takes the currently observable program execution tree as the state, and defines all the current negatable branch conditions as actions. It also designs a self-attention-based LSTM combined with a fully connected layer to fit the state behavior value function, namely Q(s, a). The generation time of the test case and the weighted sum of the number of newly covered branches after the execution of the test case are used as rewards. The DDQN algorithm is used to learn the state action value function, and the improved ε-greedy strategy is used to select actions. The executor optimizes the state action value prediction function based on the reward to guide the path exploration of symbolic execution.

[0032] In summary, although the above methods can alleviate the path explosion problem to a certain extent, due to the lack of effective state prediction capabilities, the state selection basis is too limited, which makes the symbolic execution exploration biased towards the program part of the metric attribute and unable to explore other parts of the program. If a dynamic state selection prediction strategy can be provided based on the comprehensive measurement attributes, then the symbolic execution exploration will most likely cover these parts, the code coverage will be improved, and more security violations may be detected.

[0033] In response to the path explosion problem faced by symbolic execution and the limitations of existing heuristic path exploration methods, this paper proposes a new data-driven method to learn a state selection strategy that can effectively explore the input program using symbolic execution tools. The key idea is to use a machine learning regression model to estimate a reward value for each symbolic state (referred to as state), that is, the total code coverage of the symbolic state per unit time. During symbolic execution, the state with the highest reward value is selected for exploration, so that the final generated test achieves a higher coverage rate.

[0034] The estimation of the symbolic state reward value is a regression problem, so the present invention transforms the state selection problem into a regression problem. Active learning strategically selects samples that need to be manually labeled from the unlabeled program sample set by designing sophisticated query rules, thereby achieving the goal of minimizing the number of labeled samples required for the machine learning task goal, which helps to establish an accurate regression model. In the existing pool-based active learning scenario, the mainstream selection strategies include uncertainty-based methods, representativeness-based methods, and committee-based methods. In addition, many scholars have mixed two different selection strategies, considering the uncertainty and representativeness of the query samples at the same time, and trying to find a balance between the two selection strategies. Because a single uncertainty-based selection strategy is usually prone to sampling bias: that is, the currently selected sample is not representative in the distribution of the unlabeled data pool. On the other hand, considering only a single representativeness-based selection strategy may lead to an increase in the labeling cost: that is, a considerable number of samples with low information content may be selected. In summary, although research related to active learning has been developed quite extensively, how to design a suitable query rule is still an unresolved issue. The present invention hopes to design a suitable query strategy to reduce the redundancy of samples while selecting samples with high uncertainty and representativeness to obtain better regression performance and build an accurate regression model.

[0035] In view of the above symbolic execution and active learning sample query problems, the present invention proposes a symbolic execution program path exploration strategy MS-ALS (Multi-strategy active learning search) based on multi-strategy active learning. MS-ALS first integrates multiple heuristic methods and introduces a machine learning model to learn the symbolic state of the program path in the training set, which is used to predict the reward value of the symbolic state to achieve the purpose of selecting the optimal state. In order to obtain an accurate prediction model, the present invention proposes an active learning method of multiple query strategies. First, uncertainty sampling is used to predict the reward value of all states in the state pool to be tested, and the inaccuracy of state prediction is used as the uncertainty of the state. Then, the state in the state pool to be tested is representatively sampled using a Gaussian process, and a high-information sample is obtained by combining the high uncertainty and high representativeness of the state. For the sample set with high information content, the kernel k-means clustering algorithm is used for diversity sampling to obtain the final sample and automatically annotate it. The final sample has high uncertainty and representativeness, and low redundancy. The final sample is fed back to the reward value prediction model of the symbolic state to guide the update of the prediction model, and finally, whether to enter the next round of testing is determined based on the performance of the prediction model and the quality of its test generation.

[0036] In the training set generation phase, a variety of heuristic methods are combined, and a machine learning model is introduced to predict the reward value of the symbolic state, so that it learns the symbolic state of the program path in the training set to achieve the purpose of selecting the optimal state. When using the prediction model for testing, each test uses a pool-based active learning method to select states with high uncertainty and representativeness and low redundancy for annotation, guiding the model to update in the direction of more accurate predictions, so that the symbolic execution tool can explore the input program more efficiently. When selecting a state, a reward value is predicted for each state, and each time the state with the highest predicted reward value is prioritized for further path exploration, aiming to cover more code in less time.

[0037] Combine the following Figure 1-Figure 4 Embodiments of the present invention are described in detail.

[0038] Before applying the trained prediction model, it needs to be trained based on the training set.

[0039] According to a program symbolic execution testing method provided by the present invention, a prediction model is trained according to a training set to obtain a trained prediction model, which specifically includes: obtaining an unlabeled training set program set, and determining a labeled initial supervision data set according to the unlabeled training set program set; training a regression model based on the initial supervision data set to obtain an initial prediction model; performing update training based on active learning on the initial prediction model to obtain a final prediction model; using the final prediction model to run symbolic execution in a labeled training set program set to generate new test data, and iteratively training the regression model again according to the new test data, and obtaining a corresponding number of final prediction models according to the number of iterative training times.

[0040] Specifically, Figure 1 The figure shows a schematic diagram of the program symbol execution analysis process based on active learning of the present invention, which mainly includes: (1) Initial sampling stage: Initialize the training set and the labeled set, and select some programs to run symbolic execution to generate test data.

[0041] (2) Feature extraction stage: Extract state features from the test data and calculate reward values ​​to construct the initial supervised dataset.

[0042] (3) Initial model training phase: Train the regression model on the initial supervised dataset to obtain the initial prediction model.

[0043] (4) State prediction stage: Select a program for symbolic execution and apply an active learning algorithm to select high-quality state samples for annotation.

[0044] (5) Model update training phase: Use the newly labeled data to update the training set and retrain the model to improve prediction accuracy.

[0045] (6) Model evaluation phase. Evaluate the model prediction performance. If the conditions are met, generate the final model. Otherwise, continue iterative training. Condition: The prediction error MSE of the state reward value is less than the preset error threshold e1, and the number of iterations It reaches the maximum number of active learning iterations N. AL When , the final prediction model is generated; if the above conditions are not met, it jumps back to the state prediction stage for the next round of training.

[0046] (7) Model iteration stage: Use the final model to generate more data to train the iterative model, and combine multiple prediction models for comprehensive testing.

[0047] According to a program symbolic execution test method provided by the present invention, an unlabeled training set program set is obtained, and an initial labeled supervised data set is determined based on the unlabeled training set program set, specifically comprising: obtaining an unlabeled training set program set, selecting an initial number of programs from the unlabeled training set program set for symbolic execution, and generating test data; obtaining multiple symbolic states in the test data, and determining a reward value for each symbolic state as a label; and labeling multiple symbolic states in the test data generated after symbolic execution of the initial number of programs according to the labels, and determining the initial labeled supervised data set.

[0048] According to a program symbolic execution test method provided by the present invention, after generating test data, multidimensional features of the symbolic state are extracted from the generated test data, and a feature vector corresponding to each symbolic state is generated based on the multidimensional features; the multidimensional features include static features and dynamic features, the static features include bag-of-words representation of path constraints, and the dynamic features include state change information during the symbolic execution process.

[0049] Specifically, in the initial sampling stage, the unlabeled training set program set N is obtained. U , initialize it to be equal to the full set E; set the labeled training set program set to N L and initialized to an empty set Ø, where N U +N L =E; Generate a supervised dataset D based on the labeled training set and initialize it to an empty set Ø. U Select N0 programs from the training set to run symbolic execution to generate test data R, and add N0 programs to the labeled training set program set N L In N U Delete these N0 programs.

[0050] Feature extraction stage: For each symbol state in the test data R, a set of n-dimensional feature vectors is extracted, and the reward value of each state is calculated using formula (1), thereby constructing the initial supervised data set D, and building a learning strategy based on D.

[0051] Specifically, the present invention uses 47 features to describe the properties of the state during symbolic execution, and these features are listed in Table 1. Among these features, there are 32 static features and 15 dynamic features. The static features describe the state properties that can be extracted without executing the program, and the dynamic features require the execution of the program and are extracted during symbolic execution. Among them, 32 static features are represented by the Constraints feature in the table, which is a 32-dimensional vector represented by a bag of words containing state path constraints. The count of each node type in the tree is obtained by traversing the path constraints represented by the expression tree. Dynamic features 1-13 are used to describe the changes in the state during symbolic execution, among which features 9-13 are based on existing heuristic methods and use the measurement attributes of these states as features, so that the present method can learn the advantages of these heuristic methods.

[0052] Table 1

[0053] Initial model training phase: Train the regression model on the initial supervised dataset D to obtain the initial prediction model.

[0054] Specifically, the present invention uses a machine learning regression model to predict the reward value of each symbolic state, i.e., code coverage, to guide the symbolic execution tool to explore the input program more effectively. The state selection problem is transformed into a regression problem, and the symbolic state with a high reward value is selected through a prediction model to improve the quality of test case generation.

[0055] First, the existing heuristic method is used to perform symbolic execution on the N0 training set program state samples, and the 47-dimensional features of the symbolic state are extracted from the generated test data, thereby obtaining the feature vector representation F corresponding to each symbolic state. For each symbolic state, the present invention uses formula (1) to calculate a reward value as a label:

[0056] Among them, testsUnder(state) represents the test set generated by state state, statesUnder(state) represents the state set generated by state state, Coverage(t) represents the code coverage of test t in the test set, and stateTime(st) represents the total time spent on state st, including execution time, constraint solving time, etc. Reward means the code coverage of the state per unit time. This strategy selects the state with the highest Reward value each time during symbolic execution.

[0057] Formula (1) is used to calculate a reward value as a label for each symbol state, and an initial supervised data set with a reward value label is obtained. Then, a feedforward neural network FNN is used to construct a prediction model learning strategy. FNN transmits input data between layers, processes the input data through weight coefficients and activation functions, and finally obtains output, thereby realizing the mapping from one vector space to another vector space. FNN is selected as a prediction model because it has been proven to be an efficient regression model and has been widely used in various prediction fields. In addition, in the active learning stage, the active learning method based on multiple strategies proposed in the present invention is used to selectively select high-quality program samples from the unlabeled state program sample pool (i.e., the unlabeled training set program set) to add to the supervised training set, and retrain according to the new supervised training set to update the model and improve the accuracy of regression model prediction. Algorithm 1 describes the overall process of model training in this scheme. Lines 4 to 9 of Algorithm 1 shown in Table 2 describe the training process of the initial model, lines 10 to 19 describe the active learning process of the training set and model update, and lines 21 to 27 describe the iterative process of the learning strategy, and finally four prediction model learning strategies are obtained.

[0058] Table 2

[0059] According to a program symbolic execution test method provided by the present invention, an initial prediction model is updated and trained based on active learning to obtain a final prediction model, which specifically includes: based on a preset quality assessment standard, firstly selecting a program from an unlabeled training set program set for symbolic execution, and then using an active learning algorithm to select program state samples that are higher than a preset quality standard from the generated test data for labeling to obtain symbolic state samples; adding the symbolic state samples to an initial supervised data set, and updating and training the initial prediction model based on an updated supervised data set to obtain an updated prediction model; obtaining a prediction error of the updated prediction model for the symbolic state reward value, and judging whether to continue to perform update training on the prediction model based on active learning based on the prediction error and the current number of iterations; if the prediction error is less than a preset error threshold and the update training based on active learning reaches the number of iterations, then generating a final prediction model.

[0060] Specifically, in the state prediction stage. U Each time, one program is selected for symbolic execution, and an active learning algorithm is applied to select high-quality state samples, which are added to the supervised dataset D for updating the prediction model. L middle.

[0061] Model update training phase: Train the regression model on the updated supervised dataset D to obtain an updated model.

[0062] Model evaluation phase. When the model's prediction error for the state reward value is less than the preset error threshold e1, and the maximum number of active learning iterations N is reached AL When , the final prediction model is generated; if the above conditions are not met, it jumps back to the state prediction stage for the next round of training.

[0063] Model iteration stage. After obtaining the final prediction model, use the final prediction model in N L Symbolic execution is run again in the training set program to generate new test data. The regression model is iteratively trained again using the supervised data set obtained after processing the new test data to obtain a new final prediction model. This iterative process is repeated three times to obtain four final prediction models, i.e., four prediction model learning strategies. U The remaining programs are tested.

[0064] According to a program symbolic execution testing method provided by the present invention, based on a preset quality assessment standard, a program is first selected from an unlabeled training set program set for symbolic execution, and then an active learning algorithm is used to select state samples that are higher than the preset quality standard from the generated test data for labeling, specifically including: obtaining the uncertainty and representativeness of each candidate program state sample in the unlabeled training set program set; determining the amount of information of each candidate program state sample based on a combined value of the uncertainty and representativeness of each candidate program state sample; clustering the candidate program state samples whose information amount is higher than a preset information amount threshold to obtain multiple clusters; and selecting the cluster centers of multiple clusters as program state samples that are higher than the preset quality standard for automatic labeling.

[0065] According to a program symbolic execution testing method provided by the present invention, the uncertainty and representativeness of each candidate program state sample in an unlabeled training set program set are obtained, specifically including: taking the mean square error between the predicted value and the true value of the unlabeled program state sample by the prediction model as the uncertainty measure of the sample; obtaining the first entropy of the candidate program state sample, and the second entropy of other program state samples in the unlabeled training set program set except the candidate program state sample, and taking the difference between the first entropy and the second entropy as the representativeness of the candidate program state sample.

[0066] Specifically, the present invention proposes an active learning method that combines three sampling strategies, taking into account the uncertainty, representativeness, and diversity of query samples, and selectively selects high-quality samples from the unlabeled state sample pool for training set and model update operations to improve the accuracy of regression model prediction. That is, multi-strategy active learning (MS-ALS): combining uncertainty sampling and representative sampling, through Gaussian process and kernel k-means clustering algorithm, selects samples with high uncertainty and high representativeness, and at the same time, small redundancy, to build an accurate regression model.

[0067] The method consists of four key components. The active learning method of the present invention is shown in Algorithm 2 in Table 3. First, the uncertainty and representativeness of the sample are considered simultaneously to obtain a sample set S with high information content. Then, the sample set S is clustered and redundant samples are filtered to obtain a sample set with high information content and low redundancy for labeling. The following is a specific introduction to the active learning method of the present invention.

[0068] 1. Uncertainty of unlabeled samples (i.e., programs in the unlabeled training set). The purpose of uncertainty sampling is to select the most uncertain samples for labeling. The present invention uses the mean square error between the predicted value and the true value of the prediction model for unlabeled samples as the uncertainty measure of the sample, that is, for each unlabeled sample , and its uncertainty is calculated according to formula (2):

[0069] 2. Representativeness of unlabeled samples. As mentioned above, uncertainty sampling may encounter the problem of selecting abnormal samples. In order to prevent the selection of these samples, representative sampling is an effective solution. The representativeness of a sample can be evaluated based on the number of samples similar to it. Therefore, a sample with high representativeness is unlikely to be an abnormal sample. A Gaussian process can be used to measure the representativeness information between the current sample and the remaining unlabeled sample set. The present invention divides the candidate samples into The definition of representativeness metric is as follows:

[0070] in Indicates from Delete After the unlabeled sample set, and Respectively and the entropy of the remaining unlabeled sample set.

[0071] A Gaussian process is a joint distribution over a set of random variables, and the marginal distribution over any finite subset is a multivariate Gaussian distribution. The entropy term of formula (5) is calculated using a Gaussian process. For the problem of the present invention, each sample is associated with a random variable, and the present invention then uses a symmetric kernel function To generate the covariance matrix,

[0072] The covariance matrix In fact, it is a kernel matrix defined on all unlabeled examples, assuming , according to the properties of multivariate Gaussian distribution, we can know:

[0073] The entropy term has a closed solution, then

[0074] Using formulas (7) and (8), the representative definition of the sample can finally be obtained as shown in the following formula (9):

[0075] 3. The amount of information of unlabeled samples. Considering the uncertainty metric and representative metric defined above, the present invention combines the two metrics to select unlabeled samples with both high uncertainty and high representativeness for labeling. The present invention uses the combined value of the two metrics as the measure of the amount of information of the unlabeled samples. The higher the combined value, the higher the amount of information of the corresponding sample. The specific calculation method is as follows:

[0076] in It is a trade-off control parameter between the two metrics. The present invention selects The sample with the value of α is marked. The present invention sets α to 1.

[0077] 4. Diversity of unlabeled samples. As mentioned above, the sample set with high information content may contain samples in the same cluster. In order to avoid selecting redundant samples, the present invention uses the kernel k-means clustering algorithm to cluster the sample set with high information content and obtain k clusters. Then select the cluster centers of k clusters Marking can effectively ensure that the information content of the labeled samples is high and the redundancy is small. The present invention sets the k size to be the same as the batch size batchsize.

[0078] Table 3

[0079] By integrating the MS-ALS strategy into the symbolic execution tool KLEE, automatic labeling of active learning query samples is achieved, reducing the cost of manual labeling.

[0080] Figure 2 This is one of the flowcharts of the program symbol execution test method provided by the present invention. Figure 1 As shown, the method comprises the following steps: S210, obtaining a program to be tested, and performing symbolic execution testing on the program to be tested using a plurality of trained prediction models to obtain a plurality of test results. The trained prediction models are obtained by training based on a training set, and the training set is a set of programs.

[0081] S220: Combine the test results of each prediction model to obtain a final test result.

[0082] Specifically, when applying four prediction models for symbolic execution testing, the time budget is evenly allocated to each model. For example, when the time budget is 1 hour, each prediction model will be allocated 15 minutes of running time, and finally the test results will be merged to obtain the final result.

[0083] The present invention achieves the following technical effects: 1) The state selection problem is transformed into a regression problem, and a symbolic execution path exploration strategy based on multi-strategy active learning is proposed. By building an accurate regression model and selecting symbolic states with high reward values ​​to guide symbolic execution, the goal of generating high-quality test cases is achieved.

[0084] 2) A prediction model update learning framework guided by optimal states based on multiple learning strategies is constructed, which combines the advantages of multiple heuristic search strategies and evaluates the program symbolic states from multiple angles.

[0085] 3) Through the reward value calculation method of symbolic state, the symbolic execution tool KLEE is used to realize the automatic labeling of active learning query samples, reducing the cost of manual labeling.

[0086] 4) Compared with existing heuristic methods, MS-ALS can achieve higher code coverage and discover more security violations. Moreover, the test cases generated by MS-ALS are of higher quality. When the test cases generated by MS-ALS are used as the initial seeds for fuzz testing, the fuzz testing tool AFL (American Fuzzy Lop) can discover more program paths.

[0087] The program symbolic execution testing device provided by the present invention is described below. The program symbolic execution testing device described below and the program symbolic execution testing method described above can be referenced to each other.

[0088] like Figure 3 The present invention provides a program symbol execution test device, comprising: A testing module 310 is used to obtain a program to be tested, and to perform symbolic execution testing on the program to be tested using multiple trained prediction models to obtain multiple test results; A result merging module 320 is used to merge the test results of each prediction model to obtain a final test result; The trained prediction model is obtained by training based on the training set, and the training set is a program set.

[0089] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the program symbol execution test method, which includes: obtaining a program to be tested, performing symbol execution tests on the program to be tested through multiple trained prediction models, and obtaining multiple test results; merging the test results of each prediction model to obtain a final test result; the trained prediction model is obtained by training according to a training set, and the training set is a program set.

[0090] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0091] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the program symbolic execution test method provided by the above methods, which includes: obtaining a program to be tested, performing symbolic execution tests on the program to be tested through multiple trained prediction models to obtain multiple test results; merging the test results of each prediction model to obtain a final test result; the trained prediction model is obtained by training based on a training set, and the training set is a program set.

[0092] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the program symbolic execution test method provided by the above-mentioned methods. The method includes: obtaining a program to be tested, performing symbolic execution tests on the program to be tested through multiple trained prediction models to obtain multiple test results; merging the test results of each prediction model to obtain a final test result; the trained prediction model is obtained by training based on a training set, and the training set is a program set.

[0093] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0094] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A program symbolic execution testing method, characterized in that: include: Obtain a program to be tested, and perform symbolic execution testing on the program to be tested using multiple trained prediction models to obtain multiple test results; The test results of each prediction model are combined to obtain the final test result; The trained prediction model is obtained by training based on a training set, and the training set is a program set.

2. The program symbolic execution testing method according to claim 1, characterized in that: The prediction model is trained according to the training set to obtain a trained prediction model, which specifically includes: Obtaining an unlabeled training set program set, and determining a labeled initial supervision data set according to the unlabeled training set program set; Training a regression model based on the initial supervised data set to obtain an initial prediction model; Performing update training based on active learning on the initial prediction model to obtain a final prediction model; The final prediction model is used to run symbolic execution in the labeled training set program set to generate new test data, and the regression model is iteratively trained again according to the new test data, and the corresponding number of final prediction models are obtained according to the number of iterative training times.

3. The program symbolic execution testing method according to claim 2, characterized in that: The step of obtaining an unlabeled training set program set and determining a labeled initial supervision data set according to the unlabeled training set program set specifically includes: Obtaining an unlabeled training set program set, selecting an initial number of programs from the unlabeled training set program set for symbolic execution, and generating test data; Acquire multiple symbol states in the test data, and determine a reward value of each symbol state as a label; A plurality of symbolic states in test data generated after an initial number of program symbols are executed are annotated according to the labels to determine an initial supervised data set with labels.

4. The program symbolic execution testing method according to claim 2, characterized in that: The updating and training of the initial prediction model based on active learning to obtain a final prediction model specifically includes: Based on the preset quality assessment standard, programs are first selected from the unlabeled training set program set for symbolic execution, and then the active learning algorithm is used to select program state samples with higher quality than the preset standard from the generated test data for labeling to obtain symbolic state samples; Adding the symbol state samples to the initial supervision data set, and updating and training the initial prediction model according to the updated supervision data set to obtain an updated prediction model; Obtaining the prediction error of the updated prediction model for the symbol state reward value, and determining whether to continue to perform update training based on active learning on the prediction model according to the prediction error and the current number of iterations; If the prediction error is less than a preset error threshold and the update training based on active learning reaches the number of iterations, a final prediction model is generated.

5. The program symbolic execution testing method according to claim 4, characterized in that: Based on the preset quality assessment standard, firstly, a program is selected from the unlabeled training set program set for symbolic execution, and then an active learning algorithm is used to select program state samples with higher quality than the preset standard from the generated test data for labeling, specifically including: Obtaining program state samples in test data generated after symbolic execution of each candidate program in the unlabeled training set program set as candidate program state samples, and determining uncertainty and representativeness of the candidate program state samples; Determine the information content of each candidate program state sample according to the combined value of uncertainty and representativeness of each candidate program state sample; Clustering candidate program state samples whose information amount is higher than a preset information amount threshold to obtain multiple clusters; The cluster centers of multiple clusters are selected as program status samples that are higher than a preset quality standard for automatic annotation.

6. The program symbolic execution testing method according to claim 5, characterized in that: Obtaining the uncertainty and representativeness of each candidate program state sample in the unlabeled training set program set, specifically including: The mean square error between the predicted value and the true value of the unlabeled program state sample by the prediction model is used as the uncertainty measure of the sample; A first entropy of the candidate program state sample and a second entropy of other program state samples in the unlabeled training set program set except the candidate program state sample are obtained, and a difference between the first entropy and the second entropy is used as the representativeness of the candidate program state sample.

7. The program symbolic execution testing method according to claim 3, characterized in that: After generating the test data, the method further includes: Extracting multidimensional features of symbol states from the generated test data, and generating feature vectors corresponding to each symbol state according to the multidimensional features; The multi-dimensional features include static features and dynamic features, the static features include bag-of-words representation of path constraints, and the dynamic features include state change information during symbolic execution.

8. A program symbolic execution testing device, characterized in that: include: A testing module is used to obtain a program to be tested, and to perform symbolic execution testing on the program to be tested through multiple trained prediction models to obtain multiple test results; A result merging module is used to merge the test results of each prediction model to obtain the final test result; The trained prediction model is obtained by training based on a training set, and the training set is a program set.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the program symbolic execution testing method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the program symbolic execution testing method according to any one of claims 1 to 7 is implemented.

Images