A path coverage test method based on agent-assisted variable multi-objective optimization

Abstract

The application discloses a path coverage test method based on agent-assisted variable multi-objective optimization, aiming at improving the effectiveness and efficiency of test case generation. The specific steps are as follows: (1) a variable multi-objective optimization model of the test case generation problem of the parallel program path coverage is established and updated, which is used for guiding the generation of the test case with high effectiveness; (2) after a certain number of iterations of solving the variable multi-objective optimization model based on an intelligent optimization algorithm, each difficult-to-cover target sub-path is determined, and a sample set is formed, laying a foundation for improving the test efficiency of the difficult-to-cover target sub-path; (3) based on the formed sample set, the agent model corresponding to each difficult-to-cover target sub-path is managed, and the superior evolution individual is selected, which is used for actually executing the parallel program, and the execution times and costs of the program are greatly reduced.

Description

Technical Field

[0001] This patent belongs to the field of software testing, specifically involving a path coverage testing method based on agent-assisted variable multi-objective optimization, which is used to improve the effectiveness and efficiency of test case generation. Background Technology

[0002] The target path of a parallel program consists of target sub-paths within each process. Parallel programs are widely used to efficiently solve many complex problems in national economic development, social progress, and national defense. When testing a parallel program, the testing criteria employed are crucial, as they not only guide the generation of test cases but also evaluate their sufficiency. To date, various testing criteria have been proposed, among which path coverage is a commonly used structural coverage criterion with strong defect detection capabilities.

[0003] Intelligent optimization algorithms are widely used in parallel program path coverage testing and have been proven to effectively generate test cases covering the target path of a parallel program. However, existing methods do not consider the issue of inconsistent coverage difficulty among target sub-paths within a parallel program's target path. Difficult-to-cover target sub-paths limit the testing efficiency of the entire target path because the test case generation process for all target sub-paths must finish simultaneously, but the coverage difficulty varies between different target sub-paths. The testing efficiency of easily covered and difficult-to-cover target sub-paths differs; difficult-to-cover target sub-paths typically require high testing costs, thus limiting the coverage testing of the entire target path. Improving the testing efficiency of those difficult-to-cover target sub-paths would inevitably reduce the testing cost of the entire target path.

[0004] To address the issue of difficult-to-cover target sub-paths limiting the testing efficiency of the entire target path, this invention proposes a path coverage testing method based on surrogate-assisted variable multi-objective optimization. This method improves the efficiency and effectiveness of test case generation by establishing and updating a variable multi-objective optimization model for parallel program path coverage test case generation, determining difficult-to-cover target sub-paths and formulating sample set formation strategies, and managing the surrogate model and designing a superior evolutionary individual selection method. Summary of the Invention

[0005] This invention first establishes and updates a variable multi-objective optimization model for the problem of generating test cases for parallel program path coverage, guiding the generation of test cases with high efficiency. Then, during the solution process of the variable multi-objective optimization model based on intelligent optimization algorithms, each difficult-to-cover target sub-path is identified and a sample set is formed, laying the foundation for the training of the surrogate model. Finally, based on the formed sample set, the surrogate model corresponding to each difficult-to-cover target sub-path is managed, and superior evolved individuals are selected for actual execution of the parallel program, thereby reducing the number of program executions and costs, and efficiently generating test cases.

[0006] The technical problem to be solved by this invention is to overcome the two problems in existing methods: the efficiency and effectiveness of test case generation for the entire target path are limited by the difficulty in covering target sub-paths, and the high cost of testing for difficult-to-cover target sub-paths. This invention provides a path coverage testing method based on proxy-assisted variable multi-objective optimization, which is used to efficiently generate test cases that cover the target path of parallel programs.

[0007] The technical solution of this invention: A path coverage testing method based on agent-assisted variable multi-objective optimization is proposed, characterized by the following steps:

[0008] Step 1: Establishment and Update of the Variable Multi-Objective Optimization Model

[0009] Based on sub-path similarity, the problem of generating test cases covering sub-paths is transformed into a maximum optimization problem, and a target optimization model oriented towards the target sub-path is established. For all target sub-paths in a target path, the target optimization models of all target sub-paths are merged together to establish a variable multi-objective optimization model. After evolving the population for a certain number of generations using a non-dominated sorting genetic algorithm, the target optimization functions of the covered target sub-paths are deleted from the established multi-objective optimization model, and the variable multi-objective optimization model is updated.

[0010] Step 2: Determining the difficult-to-cover target sub-paths and forming the sample set

[0011] The program executes for each evolved individual within the population, calculates the multi-objective function value for the target path, and stores the multi-objective function value for each evolved individual and the target path. It then determines whether test cases covering the target path have been generated. If test cases are not successfully generated, the non-dominated solution set obtained by the non-dominated sorting genetic algorithm is updated. A solution from the non-dominated solution set is randomly selected as the optimal evolved individual, and the target sub-path not covered by the optimal evolved individual is identified as the difficult-to-cover target sub-path. The relevant variable values ​​of the difficult-to-cover target sub-path are analyzed and combined with the objective function value to form a sample set.

[0012] Step 3: Management of the surrogate model and selection of superior evolutionary individuals

[0013] The management of the surrogate model is divided into three stages: training, application, and updating. Based on each formed sample set, a surrogate model is trained to estimate the objective function value of each evolved individual in the population. The evolved individual with the highest objective function estimate and the evolved individual with the largest sum of all objective function estimates are selected as the superior evolved individuals in the population. Based on the selected superior evolved individuals, the parallel program is executed to obtain the actual objective function value. If the superior evolved individual covers the objective path of the parallel program, the superior evolved individual is saved as a test case, and the algorithm stops. If the superior evolved individual does not cover the objective path of the parallel program, the superior evolved individual and the actual objective function value are added to the sample set to update the surrogate model, and the process jumps to step one.

[0014] Compared with the prior art, the beneficial effects of the present invention are:

[0015] 1. A variable multi-objective optimization model for effectively guiding the generation of test cases covering parallel program paths was established and updated;

[0016] 2. Strategies were developed for accurately identifying difficult-to-cover target sub-paths and forming sufficient sample sets;

[0017] 3. A high-efficiency management and low-cost selection method for superior evolutionary individuals were designed using the agent model. Attached Figure Description

[0018] Figure 1 Pseudocode for determining the difficult-to-cover target sub-path and forming the corresponding sample set

[0019] Figure 2 Pseudocode for agent model management and selection of superior evolutionary individuals Detailed Implementation

[0020] The inventors implemented the method proposed in this invention by developing techniques such as variable multi-objective optimization model establishment and updating, determination of difficult-to-cover target sub-paths, sample set formation, surrogate model management, and selection of superior evolutionary individuals, and combined them with... Figures 1 to 2 The present invention will be described in detail below. The specific implementation steps of the method proposed in this invention are as follows:

[0021] Step 1: Establishment and Update of the Variable Multi-Objective Optimization Model

[0022] To establish the target optimization model, the following design details are given. From the sub-path similarity formula, it can be seen that the similarity sim(p) *i ,p i The larger (X) is, the more subpaths p that the program traverses when inputting X. i (X) The closer to the target subpath p *i When sim(p) *i ,p i When (X))=1, pi (X) is p *i X is the coverage of p *i The expected test cases. Therefore, the convertible process π i The problem of generating test cases for inner subpath coverage is a maximum optimization problem, and an objective optimization model is established as shown in formula (1):

[0023]

[0024] In equation (1), Optimize the function for the objective. Indicates the influence of process π i inner p *i The relevant program input variables for coverage, also known as Decision variables. Changes in the value of the internal variable will affect π i p i (X) and p *i Similarity between them. Indicates with p *i Input variables with irrelevant coverage. When only X is changed... When the value of the variable is included, the parallel program traverses π based on X. i inner p i (X) will not change, therefore Changes in the value of the internal variable will not affect π i p i (X) and p *i Similarity between them. and They are respectively and Cartesian product over a range of values.

[0025] Regarding the establishment and updating of the variable multi-objective optimization model, the following working details are given: On the one hand, all objective optimization models are merged together to establish a variable multi-objective optimization model, as shown in formula (2); on the other hand, the currently uncovered objective sub-paths are determined, and the objective optimization functions corresponding to the covered objective sub-paths are deleted from the established multi-objective optimization model, and the variable multi-objective optimization model is updated.

[0026]

[0027] In equation (2), and These correspond to process π respectively i π j and π k The objective function is called the objective function value. F(X) is a multi-objective optimization function used to represent the multi-objective function value. and They are and The decision variables are Cartesian products belonging to their respective value intervals. and All are contained in the program input X. The values ​​of i, j, and k can be any process ID within a parallel program, but they cannot be equal.

[0028] Step 2: Determining the difficult-to-cover target sub-paths and forming the sample set

[0029] Before determining the difficult-to-cover target sub-path and forming the sample set, the type of intelligent optimization algorithm used in this invention needs to be selected. Considering that the Non-Dominated Sorting Genetic Algorithm (NSGA-II) is one of the most widely used and effective methods for solving multi-objective optimization problems, and has been successfully applied in software testing, proving to outperform algorithms such as multi-objective particle swarm optimization in solving multi-objective optimization problems in software testing, this invention chooses NSGA-II to solve the established variable multi-objective optimization model, and based on... Figure 1 The pseudocode identifies the difficult-to-cover target sub-paths and forms a sample set.

[0030] To determine each difficult-to-cover target sub-path, firstly, a certain number of evolutionary individuals are generated based on NSGA-II and used to execute parallel programs. Then, based on the established variable multi-objective optimization model, the multi-objective function value of each evolutionary individual is calculated, and it is determined whether test cases covering the current target path are generated. Next, if test cases covering the current target path are generated, the next target path is tested; otherwise, the non-dominated solution set is updated. Finally, if the current evolutionary generation equals the generation threshold EvoThr, the optimal evolutionary individual is selected from the non-dominated solution set, and the target sub-paths not covered by the optimal evolutionary individual are determined and identified as the difficult-to-cover target sub-path set, denoted as […]. Indicates that it is located in process π r The r-th target sub-path.

[0031] The following implementation process is given for forming the sample set corresponding to the difficult-to-cover target sub-paths: First, determine each difficult-to-cover target sub-path. The process π r And extract the data from each generated evolutionary individual that are related to π. r Related program input variable values Use this as the input to the sample; then, from the multi-objective function values ​​of this evolutionary individual, determine whether it is related to... Corresponding objective function value As the output of the sample; finally, combine the sample set inputs and outputs corresponding to all generated evolutionary individuals to form a... The corresponding sample set is denoted as Let the resulting set of samples be denoted as

[0032] Step 3: Management of the surrogate model and selection of superior evolutionary individuals

[0033] Before managing the surrogate model and selecting superior evolutionary individuals, the type of surrogate model to be used needs to be determined. Radial basis function neural networks (RBFNs) have been widely used to estimate the objective function values ​​of evolutionary individuals, and have been successfully applied to test case generation tasks for parallel program path coverage, verifying that RBFNs can accurately estimate the objective function values ​​of evolutionary individuals. Therefore, this invention adopts RBFN as the surrogate model type, and is based on... Figure 2 The pseudocode describes the management of the surrogate model and the selection of superior evolutionary individuals. RBFN management mainly consists of three operational phases: training, application, and updating. Regarding the training of the surrogate model, it utilizes each difficult-to-cover target sub-path... Corresponding sample set Train an agent model, denoted as against The application extracts the relationship between each evolutionary individual and process π. r Related variable values And run as input Estimation and The corresponding objective function value is denoted as

[0034] After obtaining the objective function estimate for each difficult-to-cover target sub-path, superior evolved individuals need to be selected. Considering that the evolved individual EIHSOFE with the highest objective function estimate is most likely to traverse the corresponding difficult-to-cover target sub-path, and the evolved individual EILSASOFEs with the largest sum of all objective function estimates is most likely to traverse all difficult-to-cover target sub-paths, EIHSOFE and EILSASOFEs are selected as superior evolved individuals in the current population for actually executing the parallel program and determining whether test cases for traversing difficult-to-cover target sub-paths have been generated. If all objective function values ​​are currently 1, the corresponding superior and optimal evolved individuals are saved to Ts, and the algorithm is stopped.

[0035] If one or more objective function values ​​are not 1, then EIHSOFE and EILSASOFEs are corresponding to Relevant variable values As sample input, the actual objective function value after the program is executed. As a sample output, it is added to the sample set. After the population has evolved through UpdThr generations, the agent model is updated based on the supplementary sample set, and then the process jumps to step one.

[0036] Next, we will use a parallel program, DepSolver, as an example to verify the effectiveness and efficiency of the method proposed in this invention.

[0037] The parallel program DepSolver has 3-dimensional input variables with an input range of [0, 200] and 6 processes. The NSGA-II population size is 50, the maximum number of generations MaxGen is 1000, and the crossover and mutation probabilities are 0.75 and 0.1, respectively; the generation threshold EvoThr is set to 560; and the surrogate model update threshold UpdThr is set to 10.

[0038] The proposed method was compared with the existing method EvoGen, and three batches of experiments were conducted under the same experimental environment. Each batch ran both the proposed method and EvoGen 20 times. The success rate and time consumption for generating test cases covering the DepSolver target path were statistically analyzed. The success rate represents effectiveness, calculated as the ratio of the number of runs that successfully generated test cases covering the parallel program target path to the total number of runs. The time consumption represents efficiency, calculated as the time to generate test cases covering the parallel program target path, or the time to reach the maximum number of evolution generations without generating test cases. The experimental results are shown in the table below.

[0039]

[0040] As shown in the table, the method proposed in this invention can efficiently generate test cases that cover the target path of parallel programs.

Claims

1. A path coverage testing method based on agent-assisted variable multi-objective optimization, used to improve the effectiveness and efficiency of test case generation, characterized in that... The following steps are required: Step 1: Establishment and Update of the Variable Multi-Objective Optimization Model Based on the similarity between the sub-path and the target sub-path, the problem of generating sub-path coverage test cases is transformed into a maximum optimization problem, and a target optimization model oriented towards the target sub-path is established. For all target sub-paths in a target path, the target optimization models of all target sub-paths are merged together to establish a variable multi-objective optimization model. After a certain number of generations of evolution of the population based on a non-dominated sorting genetic algorithm, the target optimization functions of the covered target sub-paths are deleted from the established multi-objective optimization model, and the variable multi-objective optimization model is updated. Step 2: Determining the difficult-to-cover target sub-paths and forming the sample set The program executes for each evolved individual within the population, calculates the multi-objective function value for the target path, and stores the multi-objective function values ​​for each evolved individual and the target path. It then determines whether test cases covering the target path have been generated. If test cases are not successfully generated, the non-dominated solution set obtained by the non-dominated sorting genetic algorithm is updated. A solution from the non-dominated solution set is randomly selected as the optimal evolved individual, and the target sub-path not covered by the optimal evolved individual is identified as the difficult-to-cover target sub-path. Finally, the program input variable values ​​for the difficult-to-cover target sub-path are analyzed. And together with the objective function values, they form a sample set; Step 3: Management of the surrogate model and selection of superior evolutionary individuals The management of the agent model is divided into three stages: training, application, and updating. Based on each formed sample set, a surrogate model is trained to estimate the objective function value of each evolved individual in the population; the evolved individual with the highest objective function estimate and the evolved individual with the largest sum of all objective function estimates are selected as the superior evolved individuals in the population; based on the selected superior evolved individuals, a parallel program is executed to obtain the actual objective function value; the superior evolved individuals and the actual objective function value are added to the sample set to update the surrogate model.

Images