A path coverage test data generation method and system based on reinforcement learning selection strategy

Abstract

The application discloses a path coverage test data generation method and system based on a reinforcement learning selection strategy, and relates to the technical field of software testing. The method solves the problems of long calculation time, low generation efficiency, unstable exploration process and redundant test data in the existing test data generation of large-scale path coverage or complex programs. The method comprises the following steps: acquiring all executable paths and numbers of a program to be tested; defining the executable paths as agent states and data selection as agent actions, and all states forming a whole state space; generating multiple groups of data according to an initial state obtained by an agent from initial data, and generating new data when the whole state space is not covered; selecting the data by a reinforcement learning selection strategy combining Q learning and a greedy method, defining an action reward function for the path coverage, and retaining the data with the maximum reward value as the new test data; and continuously updating until the test data covers the whole state space. The application is applied to the field of software product testing.

Description

Technical Field

[0001] This invention relates to the field of computer software testing technology, and in particular to a method for generating path coverage test data based on a reinforcement learning selection strategy. Background Technology

[0002] Software testing plays a crucial role in the current software development process and is an important method for ensuring software quality. It involves conducting extensive testing to uncover potential defects and errors in the software. Furthermore, current statistics in the software industry show that software testing costs are very high, often accounting for half of the entire project development cost. The core of software testing lies in generating effective test data to meet the prescribed test adequacy criteria. Therefore, how to efficiently generate test data has significant practical and economic value.

[0003] Coverage-oriented test data generation methods, which cover the test requirements that the software under test must meet, are classic test data generation methods. Path coverage, as one of the most basic sufficiency criteria in coverage-oriented methods, requires selecting enough test data to ensure that each reachable path of the program under test is executed at least once. The problem can be described as: for a given target path of a program, it is necessary to find or generate test data in the program's input space that can cover the target path.

[0004] While traditional optimization algorithms can effectively address test data generation problems, they are limited by search capabilities and update speeds, making them prone to premature convergence. Furthermore, their lack of parallelism leads to excessive computation time and low efficiency in generating test data for large-scale path coverage or complex programs. Reinforcement learning, as a method that utilizes the interaction between an agent and its environment to learn and maximize rewards or achieve specific goals, can automate and streamline the test data generation process for complex programs, thus overcoming the shortcomings of traditional optimization methods in testing complex programs.

[0005] Reinforcement learning methods, by transforming the data generation process into an interactive learning process with the environment, can avoid premature convergence in search algorithms. Simultaneously, the agent, through autonomous exploration of the solution space, fully utilizes computational resources, improving the efficiency of test data generation. However, existing reinforcement learning algorithms still face some challenges in test data generation research involving path coverage, including unstable exploration processes and redundant test cases. Summary of the Invention

[0006] This invention addresses the problems of excessive computation time, low data generation efficiency, unstable exploration process, and redundant test cases in existing large-scale path coverage or complex program test data generation methods. It proposes a path coverage test data generation method based on a reinforcement learning selection strategy. The specific solution includes:

[0007] A method for generating path coverage test data based on a reinforcement learning selection strategy, the method comprising:

[0008] S1: Obtain all executable paths and the number of executable paths for the program under test;

[0009] S2: Define the executable path as an agent state, define the data selection of each round of updates as an action, and all the agent states constitute a state space;

[0010] S3: Generate initial data using a random method, and use an agent to transform the initial data to obtain an initial state set;

[0011] S4: When the initial state is included in the state space, it means that the initial state set has not completely covered the executable path of the program under test. Update the initial data and generate multiple sets of updated data.

[0012] S5: Select the updated data and confirm the new generation of data;

[0013] S6: Assign different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation data, retain the data with the largest reward value as the newly generated test data, and continuously iterate and update the data until the generated data covers the state space.

[0014] Furthermore, a preferred method is proposed, wherein the calculation method for all executable paths and the number of executable paths in step S1 is as follows:

[0015] Executable path = All theoretical paths - Unreachable paths;

[0016] Total number of theoretical paths = number of sibling nodes + node * [number of branch nodes * (number of branch child nodes) + number of sibling nodes];

[0017] Substitute the value 1 into all nodes to get the number of paths;

[0018] Number of executable paths = Total number of theoretical paths - Number of unreachable paths.

[0019] Furthermore, a preferred method is proposed, in which the initial data is transformed using an intelligent agent to obtain an initial state set, specifically as follows:

[0020] Obtain the initial data t0 and the currently covered executable path p0;

[0021] Based on the currently covered executable path, an initial state X0 and an initialization action value Q0 are obtained.

[0022] Furthermore, a preferred method is proposed, wherein the method of updating the initial data in step S4 includes mutation and perturbation.

[0023] Furthermore, a preferred approach is proposed, wherein step S5 includes: selecting updated data through a reinforcement learning selection strategy that combines Q-learning with a greedy method to determine the next generation of data.

[0024] Furthermore, a preferred approach is proposed, wherein step S6 assigns different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation data, specifically as follows:

[0025] If the new generation of data discovers an executable path not covered by the original data or the previous generation of data, a positive reward value is assigned; otherwise, no reward value is assigned, and the data with the largest reward value is retained as the newly generated test data.

[0026] Furthermore, a preferred method is proposed, wherein the iterative data update in step S6 includes:

[0027] newQ X,A =(1-α)Q X,A +α(R X,A +γ*maxQ'(x',a')),

[0028] Among them, newQ X,A Let Q be the new Q value, and α be the agent's learning efficiency. X,A R is the current Q value. X,A The reward for the action performed is γ, which is a discount factor, and maxQ'(x',a') is the maximum action reward obtained in the new state.

[0029] Based on the same inventive concept, this invention also proposes a path coverage test data generation system based on a reinforcement learning selection strategy, the method comprising:

[0030] The path acquisition module is used to obtain all executable paths and the number of executable paths of the program under test;

[0031] The definition module is used to define the executable path as the agent state, define the data selection of each round of updates as the action, and all the agent states constitute the state space.

[0032] The initial state set acquisition unit is used to generate initial data according to a random method, and then use an agent to transform the initial data to obtain the initial state set.

[0033] The update unit is used to update the initial data and generate multiple sets of updated data when the initial state is included in the state space, indicating that the initial state set fails to completely cover the executable path of the program under test.

[0034] The selection unit is used to select updated data and determine the next generation of data.

[0035] The reward module is used to assign different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation of data. The data with the highest reward value is retained as newly generated test data, and the data is continuously iterated and updated until the generated data covers the state space.

[0036] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium for storing a computer program that executes a path coverage test data generation method based on reinforcement learning selection strategy as described in any of the above claims.

[0037] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a path coverage test data generation method based on a reinforcement learning selection strategy as described in any one of the preceding claims.

[0038] The advantages of this invention are:

[0039] This invention solves the problems of excessive computation time and low data generation efficiency in the existing test data generation for large-scale path coverage or complex programs, as well as the problems of unstable exploration process and redundant test cases.

[0040] This invention provides a path coverage test data generation method based on a reinforcement learning selection strategy. It generates test cases using reinforcement learning, defining the executable path of the program under test as a state and defining the data selection in each round as an action. This definition method significantly reduces the generation of invalid data and lowers the possibility of generating redundant test data. Furthermore, it designs a greedy selection strategy based on environmental feedback Q-value learning, using deterministic action selection to improve the stability and efficiency of agent learning. This avoids the instability problems that may occur during random exploration, providing guidance for reinforcement learning strategies and avoiding the large amount of redundancy caused when using reinforcement learning methods to generate data. This improves the quality and efficiency of generated test data, and has significant practical and economic value.

[0041] This invention is applied to the field of software product testing. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the path coverage test data generation method based on reinforcement learning selection strategy described in Implementation Method 1.

[0043] Figure 2 This is a schematic diagram illustrating the definition of the state and action procedures of the intelligent agent as described in Implementation Method 1, wherein... Figure 2 (a) is the source code. Figure 2 (b) is the control flow graph;

[0044] Figure 3 This is a schematic diagram illustrating the data update and strategy selection process described in Implementation Method Six;

[0045] Figure 4 This is a graph showing the change in coverage of each algorithm over time in the benchmark program described in Implementation Method Eleven. Figure 4 (a) is a schematic diagram showing the relationship between the running time and coverage of Program 1. Figure 4 (b) is a schematic diagram showing the relationship between the running time and coverage of Program 2;

[0046] Figure 5 This is a graph showing the change in coverage of each algorithm over time in the industrial process described in Implementation Method Eleven. Figure 5 (a) shows the relationship between the running time and coverage of program 3. Figure 5 (b) shows the relationship between the running time and coverage of program 4;

[0047] Figure 6 This is a comparison chart of the running time of the method described in Implementation Method 11 and the Q-learning algorithm under different exploration rates;

[0048] Figure 7 This is a schematic diagram illustrating the stability analysis of algorithms such as RLSA and GPSGA described in Implementation Method Eleven. Figure 7 (a) is a schematic diagram of the stability analysis using RLSA, PSO, TOGA, and GPSGA algorithms in benchmark program 1. Figure 7 (b) is a schematic diagram of the stability analysis of RLSA, GA, PSO and other algorithms used in benchmark program 2. Figure 7 (c) is a schematic diagram of the stability analysis of RLSA, TOGA, and GPSGA algorithms used in Industrial Program 3. Figure 7 (d) is a schematic diagram of the stability analysis of RLSA, TOGA, and GPSGA algorithms used in Industrial Program 4. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0050] Implementation Method 1, see [link] Figure 1 and Figure 2 This embodiment describes a method for generating path coverage test data based on a reinforcement learning selection strategy. The method includes:

[0051] S1: Obtain all executable paths and the number of executable paths for the program under test;

[0052] S2: Define the executable path as an agent state, define the data selection of each round of updates as an action, and all the agent states constitute a state space;

[0053] S3: Generate initial data using a random method, and use an agent to transform the initial data to obtain an initial state set;

[0054] S4: When the initial state is included in the state space, it means that the initial state set has not completely covered the executable path of the program under test. Update the initial data and generate multiple sets of updated data.

[0055] S5: Select the updated data and confirm the new generation of data;

[0056] S6: Assign different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation data, retain the data with the largest reward value as the newly generated test data, and continuously iterate and update the data until the generated data covers the state space.

[0057] This implementation utilizes a reinforcement learning algorithm, enabling the agent to optimize test data generation through iterative processes, achieving faster full coverage of executable paths. The reinforcement learning algorithm adjusts its test data generation strategy based on actual conditions, making it more adaptable to different program structures and complexities, thus improving test case coverage. By rewarding the agent for actions that effectively cover new paths, the agent reduces unnecessary test cases, improves test data generation efficiency, and avoids redundancy. The iterative process of the reinforcement learning algorithm helps stabilize the test data generation exploration process, reducing instability caused by randomness.

[0058] In this implementation, the executable paths of the program under test are defined as the state space of the agent, with each path representing a state. This state space represents all executable paths of the program. The data selection for each round of updates is defined as an action. The agent explores the state space, selecting appropriate actions to update the test data to achieve better path coverage. Different reward values ​​are assigned to different actions of the agent and the agent's path coverage based on the generated test data. This reward mechanism is the core of the reinforcement learning algorithm optimization, guiding the agent to choose actions more conducive to path coverage. By continuously iterating and updating the data, the efficiency and coverage of test data generation are gradually improved through the agent's learning process, until the generated data covers the entire state space. Guided by the reinforcement learning algorithm, test data is continuously generated, ensuring that all executable paths of the program are covered, thereby improving the comprehensiveness and accuracy of the test. Through the agent's learning and optimization process, the direction of test data updates can be selected more effectively, reducing computation time and improving the efficiency of test data generation. By introducing the incentive function of reinforcement learning, the agent can better cope with complex program structures and different testing requirements, improving the stability and reliability of test data generation.

[0059] Implementation Method Two: This implementation method further defines the path coverage test data generation method based on reinforcement learning selection strategy described in Implementation Method One. The calculation method for all executable paths and the number of executable paths in step S1 is as follows:

[0060] Executable path = All theoretical paths - Unreachable paths;

[0061] Total number of theoretical paths = number of sibling nodes + node * [number of branch nodes * (number of branch child nodes) + number of sibling nodes];

[0062] Substitute the value 1 into all nodes to get the number of paths;

[0063] Number of executable paths = Total number of theoretical paths - Number of unreachable paths.

[0064] This implementation, by more strictly limiting the executable path calculation method, can more accurately determine the number of executable paths in the program, improving the accuracy of path coverage test data generation. By considering unreachable paths, paths that cannot be reached in actual execution can be excluded, thus generating test data more specifically and reducing unnecessary algorithm exploration. Considering combinations of nodes, branch nodes, and sibling nodes, a more comprehensive estimate of the total number of theoretical paths in the program is obtained, providing more comprehensive information for subsequent path coverage test data generation.

[0065] A more detailed calculation method was designed to reasonably estimate the total number of theoretical paths in the program through combinations of nodes, branch nodes, and sibling nodes. This calculation method helps to gain a more comprehensive understanding of the program structure and provides a better reference for test data generation. By subtracting the number of unreachable paths when calculating executable paths, the agent can more accurately determine which paths are unreachable in actual execution. This helps to eliminate paths that are unnecessary to cover in testing, improving the efficiency of test data generation. Through a more detailed and accurate calculation method, the purpose of this implementation is to improve the accuracy of estimating the number of executable paths in the program, making subsequent test data generation more precise. Considering unreachable paths helps to eliminate those paths that are unreachable in actual execution, thereby reducing unnecessary test cases and improving testing efficiency.

[0066] Implementation Method 3: This implementation method further defines the path coverage test data generation method based on reinforcement learning selection strategy described in Implementation Method 1. Specifically, the method involves using an agent to transform the initial data to obtain an initial state set.

[0067] Obtain the initial data t0 and the currently covered executable path p0;

[0068] Based on the currently covered executable path, an initial state X0 and an initialization action value Q0 are obtained.

[0069] This implementation utilizes agent perception to obtain currently covered executable paths, providing targeted initial states and action values, thereby accelerating the convergence process of reinforcement learning. By leveraging already covered path information, repeatedly exploring already covered sections can be avoided, thus saving computational resources and improving the efficiency of test data generation.

[0070] By perceiving (transforming initial data) through agent perception, executable paths already covered by the current test data are obtained. These paths can serve as heuristics to guide the generation of subsequent test data. Based on the covered paths, a set of initial states is determined. These states can serve as the starting point for test data generation, and an initial action value is assigned to each initial state to guide the agent's behavior during the reinforcement learning process. By providing initial states and action values ​​based on the currently covered paths, the reinforcement learning agent can converge to effective test data more quickly, thereby improving the efficiency of test data generation. Utilizing information from already covered paths avoids generating duplicate test data, thus optimizing the testing process and reducing resource waste. The effective selection of initial states and action values ​​can guide the agent to explore more targeted paths that are not yet covered, thereby improving path coverage.

[0071] Implementation Method 4: This implementation method further defines the path coverage test data generation method based on reinforcement learning selection strategy described in Implementation Method 1. The method of updating the initial data in step S4 includes mutation and perturbation.

[0072] In practical applications, updating the previous generation of data also includes mutation and perturbation.

[0073] The update method in this embodiment, which introduces mutations and perturbations, allows for greater variation in the generation of test data, thereby increasing the diversity of the test data and improving its adaptability to different scenarios and conditions. Mutations and perturbations help introduce new behaviors and decision choices, enabling the agent to more comprehensively cover possible paths, including those with lower probability but still significant importance. Introducing mutations and perturbations makes the agent more robust, capable of coping with changes and uncertainties in the test environment, thus generating more robust and reliable test data.

[0074] Implementation Method 5: This implementation method further defines the path coverage test data generation method based on reinforcement learning selection strategy described in Implementation Method 4. Step S5 includes: selecting updated data through a reinforcement learning selection strategy combining Q-learning and a greedy method to determine the next generation of data.

[0075] Specifically, a greedy selection algorithm replaces the probabilistic selection in the original exploration process. The greedy algorithm makes the reinforcement learning agent more inclined to choose the action with the highest current Q-value. The accumulated Q-value guides the agent's decision-making, enabling the algorithm to converge faster and obtain a set of high-quality test data. The combination of the greedy algorithm and Q-learning reduces many unnecessary attempts in the probabilistic exploration process, more specifically selecting uncovered executable paths. This makes the generated test data more likely to cover unexplored executable paths, thereby improving the quality of the test data and increasing the algorithm's operating efficiency.

[0076] This implementation utilizes a greedy selection strategy based on Q-score learning, enabling the agent to choose its next action more intelligently, thereby improving the efficiency of generating path coverage test data. Generating multiple sets of updated data increases diversity, facilitating a more comprehensive exploration of the test space and enhancing the breadth of path coverage. Through Q-score learning based on environmental feedback, the agent can adapt to changing test environments through continuous learning, improving the adaptability of generated test data.

[0077] An agent learns Q-values ​​through interaction with its environment; these Q-values ​​reflect the expected rewards of taking different actions in different states. Through Q-value learning, the agent can better understand the dynamic characteristics of the testing environment. Utilizing a greedy selection strategy based on Q-value learning, the agent chooses the action with the highest Q-value at each step to maximize expected rewards. Such a strategy enables the faster finding of optimized path coverage strategies during reinforcement learning.

[0078] This implementation utilizes a greedy selection strategy based on Q-score learning to more intelligently select actions during test data generation, thereby covering test paths more quickly. Generating multiple sets of updated data helps the agent more comprehensively cover the test space, increasing the breadth of path coverage and thus enhancing the comprehensiveness and robustness of the test. Q-score learning based on environmental feedback enables the agent to adapt to changes in the test environment, maintaining the validity of the generated test data.

[0079] Implementation Method Six, see below Figure 3 This embodiment further defines the path coverage test data generation method based on reinforcement learning selection strategy described in Embodiment 4. Step S6 assigns different reward values ​​to different actions of the agent and the agent's path coverage based on the new generation data, and retains the data with the highest reward value as the newly generated test data. Specifically:

[0080] If the updated data discovers an executable path not covered by the initial data or the previous generation data, a positive reward value is assigned; otherwise, no reward value is assigned, and the data with the largest reward value is retained as the newly generated test data.

[0081] This implementation assigns positive rewards to data that discovers new execution paths, encouraging the agent to explore previously uncovered paths more frequently, thereby improving path coverage and ensuring more comprehensive test coverage. This implementation adjusts the reward value based on the update status of the test data, allowing the agent to adaptively learn which actions are effective, thus generating more representative and effective test data. Replacing the data with the highest reward value with newly generated test data helps reduce redundant test data, optimizes the test set, and makes testing more efficient.

[0082] Different reward values ​​are assigned based on whether the updated data discovers new executable paths. If a new path is found, a positive reward is given; otherwise, no reward is given. This reward mechanism improves path coverage by incentivizing the agent to explore new executable paths. Replacing the data with the highest reward value with newly generated test data helps continuously optimize the test dataset, ensuring that each update is based on the most representative and effective data. This implementation aims to optimize the path coverage test data generation method based on reinforcement learning selection strategies through a reward value-based mechanism, enabling it to more effectively generate test data with high coverage and representativeness. By adaptively adjusting reward values ​​and optimizing the test dataset, this implementation aims to improve the efficiency and quality of test data generation, ensuring that the agent achieves effective test coverage under various conditions.

[0083] Implementation Method Seven: This implementation method further defines the path coverage test data generation method based on reinforcement learning selection strategy described in Implementation Method Four. The iterative data update in step S6 includes:

[0084] newQ X,A =(1-α)Q X,A +α(R X,A +γ*maxQ'(x',a')),

[0085] Among them, newQ X,A Let Q be the new Q value, and α be the agent's learning efficiency. X,A R is the current Q value. X,A The reward for the action performed is γ, which is a discount factor, and maxQ'(x',a') is the maximum action reward obtained in the new state.

[0086] This implementation introduces a learning efficiency parameter to adjust the agent's learning speed, making learning more efficient and facilitating rapid adaptation to the agent's dynamic changes. Introducing a discount factor takes into account the impact of long-term rewards, allowing the agent to better weigh future returns and make more comprehensive path selections, rather than solely focusing on short-term rewards. Considering both the current value and the maximum action reward obtainable in the new state makes the data update process more comprehensive, relying not only on the reward of the current action but also considering the maximum possible future reward.

[0087] In this implementation, the learning efficiency parameter is used to adjust the agent's learning speed. Higher learning efficiency allows the agent to adapt to new information more quickly, but may lead to instability; lower learning efficiency makes the learning process smoother, but may be too slow. A discount factor is used to consider the impact of long-term rewards. By introducing a discount factor, the agent is better able to balance current rewards and potential future rewards, enabling it to choose paths more intelligently. Combining the current value, the reward for the action performed, the discount factor, and the maximum reward for the action obtained in the new state, the update process for the new value is more comprehensive. This comprehensive consideration allows the agent to more fully evaluate the value of each action, thereby choosing paths more intelligently.

[0088] Implementation Method 8: A path coverage test data generation system based on reinforcement learning selection strategy described in this implementation method includes:

[0089] The path acquisition module is used to obtain all executable paths and the number of executable paths of the program under test;

[0090] The definition module is used to define the executable path as the agent state, define the data selection of each round of updates as the action, and all the agent states constitute the state space.

[0091] The initial state set acquisition unit is used to generate initial data according to a random method, and then use an agent to transform the initial data to obtain the initial state set.

[0092] The update unit is used to update the initial data and generate multiple sets of updated data when the initial state is included in the state space, indicating that the initial state set fails to completely cover the executable path of the program under test.

[0093] The selection unit is used to select updated data and determine the next generation of data.

[0094] The reward module is used to assign different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation of data. The data with the highest reward value is retained as newly generated test data, and the data is continuously iterated and updated until the generated data covers the state space.

[0095] Implementation Method Nine: A computer-readable storage medium according to this implementation method, the computer-readable storage medium being used to store a computer program, the computer program executing a path coverage test data generation method based on reinforcement learning selection strategy as described in any one of Implementation Methods One to Seven.

[0096] Implementation Method 10: A computer device according to this implementation method includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a path coverage test data generation method based on a reinforcement learning selection strategy according to any one of Implementation Methods 1 to 7.

[0097] Implementation Method 11, see below Figure 4 , Figure 5 , Figure 6 and Figure 7 This embodiment describes a specific example of the path coverage test data generation method based on reinforcement learning selection strategy described in Embodiment 1. It also serves to explain Embodiments 2 through 7. Specifically:

[0098] This implementation proposes a Reinforcement Learning Selection Algorithm (RLSA), integrating reinforcement learning concepts into the generation of test data oriented towards path coverage. First, the agent's state and actions are defined: the executable paths of the program under test are defined as states, and the data selection in each round of updates is defined as actions. This definition significantly reduces the generation of invalid data and lowers the possibility of generating redundant test data. Next, a greedy strategy is combined with the Q-learning method to design a greedy selection strategy based on environmental feedback Q-value learning. Deterministic action selection improves the stability and efficiency of the agent's learning, thus avoiding the instability that may occur during random exploration. Finally, a reward function guides the agent's decision-making, enabling the agent to quickly generate data covering all executable paths, thereby improving the algorithm's efficiency.

[0099] The specific implementation approach is as follows: First, the number of all executable paths is obtained through a control flow graph, and each executable path is defined as a state of the agent. Then, initial input data is generated using a random method, and the agent obtains the initial state through perception. When the obtained state is not in the state space X, the initial input data is updated through mutation, random perturbation, etc., and multiple sets of updated data are generated to increase the probability of selecting uncovered executable paths. Next, the reward function value of the updated data is set. Among these newly generated data, if any data discovers an execution path not covered by the original data, the agent assigns a positive reward value to this set of data; otherwise, no reward value is given. Finally, the agent replaces the data with the highest reward value with the next generation of test data, and continues to iterate until the agent's state covers the entire state space, that is, covers all executable paths of the program under test.

[0100] The pseudocode for the algorithm is shown in Algorithm 1 below:

[0101]

[0102]

[0103] In this embodiment, the control flow graph is obtained as a directed graph G representing the program execution flow.<V,W,s,e> This also corresponds to the overall learning environment of the agent in reinforcement learning. For example... Figure 2 As shown, nodes represent basic blocks of the program, and edges represent jump relationships between basic blocks. Let V = {v1, v2, ..., v...} n Let W be the set of nodes in a directed graph, where W = {w1, w2, ..., w...} n Let} be the edge set of a directed graph, and w jk = <v j ,v k >, s and e are the program's entry and exit points, respectively. Figure 2 For example, we have V = {s, 1, 2, 3, 4, 5, 6, 7, e}, w s1 =<s,1> .

[0104] In this embodiment, the calculation of the execution path and its number is as follows:

[0105] The execution path is a path from the program's entry point to its exit point, represented by a sequence of nodes, such as... Figure 2 middle"<s,1,3,7,e> "It is an execution path consisting of a sequence of nodes. The number of executable paths is calculated using the number of nodes, sibling nodes, and branch nodes." Figure 2 For example, the calculation method is as follows: Number of paths = sibling nodes + node * [branch node * (branch child node + branch child node) + sibling node], M = ② + ③ * [④ * (⑤ + ⑥) + ⑦]. Substituting all nodes into the value 1, the number of paths is 4.

[0106] The state space in this embodiment is specifically as follows:

[0107] For the state space X = {x0,...,x...} i ,...}, where each state point x i These are descriptions of a single executable path in the program under test, and the state space is the set of all executable paths. Figure 2 For example, its state space is X = {<s,1,2,e> ,<s,1,3,4,5,e> 1,<s,1,3,4,6,e> ,<s,1,3,7,e>}

[0108] In this embodiment, the action space is specifically as follows:

[0109] For the action space A = {a0, a1, a2, ..., a...} i ,...}, each action a i Both depict the act of selecting data after a new round of updates. Figure 2 Taking the program as an example, the initial input data is t. After a new round of update operations on the input data t, the generated data are t1, t2, t3, ..., and there is an action a0 =<t,t1> a1 =<t,t2> .For example Figure 2 In the example, if the input data is (a,b,c), the data generated after the update are (a1,b1,c1), (a2,b2,c2),..., and there are actions a0=<(a,b,c),(a1,b1,c1)>, a1=<(a,b,c),(a2,b2,c2)>...

[0110] The specific reinforcement learning selection strategy in this embodiment is as follows:

[0111] In terms of the specific behavior of the entire strategy, firstly, by analyzing the control flow graph structure of the program under test, the number of all executable paths N and the corresponding complete state space X of the program under test are calculated. After randomly generating initial data t0, the executable paths currently covered by the initial data t0 are obtained through the agent's perception of environmental information, resulting in one or a set of initial states X0 and an initialized Q0 value. Next, by comparing X0 with X, if X0 = X, it means that the initial data t0 has completely covered the executable paths of the program under test, and t0 is the set of test cases that meets the conditions; if... This indicates that the initial data t0 did not fully cover the executable path of the program under test.

[0112] The specific implementation of the reinforcement learning selection strategy during the update process: In the specific update process, the reinforcement learning selection strategy uses a combination of Q-learning and a greedy method to guide the agent to further explore and learn. By performing operations such as mutation and perturbation on the initial data t0, the updated dataset explored by the agent is denoted as {t}. 11 ,t 12 , L,t 1n According to the definition of the agent's action space A in the reinforcement learning selection strategy, the set of actions available for the agent to choose from is A. 0,1 ={a 0,11 : <t0,t 11 >,a 0,12 : <t0,t 12 >,L,a 0,1n : <t0,t 1n The agent then uses this updated dataset to perceive its state and, based on the definition of the reward function R, assigns a different reward value R to each action in the action set. iBy greedily selecting the data that maximizes the reward, i.e., t0→t 1i |R 1i =maxR, and simultaneously update the state X0→X1. While updating the data and state, a reinforcement learning selection strategy is used to update the initial Q value, which continuously guides the agent's decision-making. The update formula is as follows:

[0113] newQ X,A =(1-α)Q X,A +α(R X,A +γ*maxQ'(x',a'))

[0114] Among them, newQ X,A Let Q be the new Q value, and α be the agent's learning efficiency. X,A R is the current Q value. X,A The reward for the action is γ, where γ is the discount factor, and maxQ'(x',a') is the maximum possible reward for the action in the new state. Finally, the updated data t is used. 1i As the next generation of test data, the above process is repeated, continuously exploring downwards and iterating until data covering all executable paths is generated and output.

[0115] In this implementation, a reward function is used to guide the agent's decision-making, specifically as follows:

[0116] The reward function R is related to the objective to be achieved in the test data. For test data path coverage, the reward function influences the agent's decision-making. Therefore, different reward values ​​need to be assigned to different actions of the agent. In each round of updated data, if the data can cover the new executable path, then the agent is given a positive reward; otherwise, there is no reward. This is specifically expressed by the following formula:

[0117]

[0118] by Figure 2 For example, if the updated data (a1,b1,c1) contains an executable path that was not included in the initial data (a,b,c), then the updated data (a1,b1,c1) receives a reward of R=1.

[0119] To verify the effectiveness of the method provided in this embodiment, this embodiment also conducts analysis and experiments on the efficiency of test data generation and coverage advantages, as well as experiments on the effect of the greedy exploration strategy in the algorithm.

[0120] The basic information of the test programs used and their related experimental parameters are shown in Tables 1 and 2 below. Among them, Triangle classification (Program 1) and Password strength (Program 2) are commonly used benchmark programs in traditional software testing. Their program logic is simple and can adapt to various testing environments to meet testing under different conditions. Text sentiment (Program 3) and Decision-making Support (Program 4) are industrial programs widely used in real life, with certain real-world application scenarios, which can ensure the performance and reliability of the proposed test strategies in practical applications.

[0121] Table 1

[0122]

[0123] Table 2

[0124]

[0125] Table 3 shows the time and coverage performance of each algorithm on the benchmark program.

[0126] The method proposed in this embodiment is abbreviated as RLSA. The algorithms compared include: random method (RA), genetic algorithm (GA) in traditional optimization algorithms, improved particle swarm optimization (PSO) proposed in 2021, hyperheuristic algorithm (TOGA) proposed in 2022, and multipath coverage GPSGA algorithm based on best point set proposed in 2022.

[0127] Table 3

[0128]

[0129] For the coverage of each algorithm over time in the benchmark program, please refer to [link / reference]. Figure 4 . Figure 4 (a) shows the relationship between the running time and coverage of program 1, where the RLSA strategy of this patent can achieve 100% coverage as early as possible; Figure 4 (b) shows the relationship between the running time and coverage of program 2. The RLSA strategy can also reach 100% coverage the earliest. Compared with other strategies, the RLSA strategy can reach 100% coverage, and at the same time, it has the shortest running time and higher efficiency.

[0130] The algorithm's time and coverage performance in industrial applications:

[0131] Table 4 shows a comparison of the running time, number of iterations, and average coverage after multiple runs of the four algorithms in two industrial test programs.

[0132] Table 4

[0133]

[0134] As shown in Table 4, in Program 3, the RLSA strategy has a shorter runtime than PSO, GA, and RA, further demonstrating the applicability of reinforcement learning selection strategies in intelligent programs. RLSA also exhibits better performance in terms of iteration count, indicating that reinforcement learning selection strategies have good convergence speed in data generation. Regarding coverage, the RLSA algorithm can still successfully find test data covering all executable paths, exceeding GA and PSO. In Program 4, the runtime of GA, PSO, and RLSA is similar, but all are significantly shorter than the RA method. RLSA still has an advantage in iteration count. In terms of coverage, RLSA achieves 100% coverage, while other algorithms have lower coverage. This is because, when facing complex programs, RLSA uses a reward function to continuously guide the agent to autonomously learn and explore unknown states, greatly increasing the agent's exploration range and search efficiency in complex programs, thereby achieving better path coverage.

[0135] Algorithm coverage analysis in industrial applications:

[0136] Figure 5 This indicates the coverage of each algorithm over time in the industrial program. Figure 5 (a) shows the relationship between the running time and coverage of each algorithm in Program 3, where only RLSA achieves 100% coverage; Figure 5 (b) shows the relationship between the running time and coverage of each algorithm in Program 4. Again, only RLSA can achieve 100% coverage. Compared to the other algorithms, only the RLSA strategy can achieve 100% coverage, indicating that the RLSA strategy proposed in this implementation is superior to other methods and achieves 100% coverage.

[0137] Time complexity analysis:

[0138] In this method, the computational cost of updating the Q-value table each time is related to the number of states and actions. Specifically, for a program with n executable paths, the number of states is n. Let m be the number of actions selected in each iteration, where m does not exceed the maximum mutation number set by the algorithm. Each time the Q-value table is updated, the rewards for all possible actions in all states need to be calculated, and the reward for each action requires m calculations. Since state updates are required for each training iteration, the total computational cost is O(N(nm)), where N represents the number of algorithm iterations.

[0139] Efficiency improvement effect experiment:

[0140] Table 5 compares the running time of reinforcement learning selection strategies and other reinforcement learning Q-learning algorithms using different exploration rates after running four test programs.

[0141] Table 5

[0142]

[0143] As shown in Table 5, for the test data generation problem oriented towards path coverage, since the executable paths of the program are determined, the exploration rate ε of the reinforcement learning algorithm and the program's code execution time are positively correlated. The greedy strategy proposed in this embodiment achieves the highest exploration rate, enabling the reinforcement learning selection strategy to achieve the shortest execution time. This is because, due to the determination of the executable paths, in this embodiment, the executable paths are used as the basis for action selection. Each action update involves selecting an uncovered executable path. Each time the agent explores forward, it prioritizes unexplored executable paths, thereby significantly reducing the execution time.

[0144] Figure 6 The display shows a comparison of the running time of the reinforcement learning Q-learning algorithm and the method described in this embodiment in the test program at different exploration rates.

[0145] Stability analysis:

[0146] See Figure 7 Further analysis of the stability of the RLSA strategy was conducted, comparing its stability with algorithms such as TOGA and GPSGA. Furthermore, since this implementation achieves 100% executable path coverage with a maximum of 50 iterations, only algorithms with 50 or fewer iterations from the aforementioned experimental analysis were selected for stability comparison.

[0147] It should be understood that, in the embodiments of this application, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0148] Memory may include read-only memory, flash memory, and random access memory, and provides instructions and data to the processor. Some or all of the memory may also include non-volatile random access memory.

[0149] It should be understood that if the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0150] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of the modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0153] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating path coverage test data based on a reinforcement learning selection strategy, characterized in that, The method includes: S1: Obtain all executable paths and the number of executable paths for the program under test; S2: Define the executable path as an agent state, define the data selection of each round of updates as an action, and all the agent states constitute a state space; S3: Generate initial data using a random method, and use an agent to transform the initial data to obtain an initial state set; S4: When the initial state is included in the state space, it means that the initial state set has not completely covered the executable path of the program under test. Update the initial data and generate multiple sets of updated data. S5: Select the updated data and confirm the new generation of data; S6: Assign different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation data, retain the data with the largest reward value as the newly generated test data, and continuously iterate and update the data until the generated data covers the state space.

2. The method for generating path coverage test data based on a reinforcement learning selection strategy according to claim 1, characterized in that, The calculation method for all executable paths and the number of executable paths in step S1 is as follows: Executable path = All theoretical paths - Unreachable paths; Total number of theoretical paths = number of sibling nodes + node * [number of branch nodes * (number of branch child nodes) + number of sibling nodes]; Substitute the value 1 into all nodes to get the number of paths; Number of executable paths = Total number of theoretical paths - Number of unreachable paths.

3. The method for generating path coverage test data based on a reinforcement learning selection strategy according to claim 1, characterized in that, The process of using an intelligent agent to transform the initial data to obtain an initial state set specifically involves: Obtain initial data Currently covered executable path p 0; A set of initial states is obtained based on the currently covered executable paths according to the initial data. And an initialization action value .

4. The method for generating path coverage test data based on a reinforcement learning selection strategy according to claim 1, characterized in that, The methods for updating the initial data in step S4 include mutation and perturbation.

5. The method for generating path coverage test data based on a reinforcement learning selection strategy according to claim 1, characterized in that, Step S5 includes: selecting updated data through a reinforcement learning selection strategy that combines Q-learning with a greedy method to determine the next generation of data.

6. The method for generating path coverage test data based on a reinforcement learning selection strategy according to claim 4, characterized in that, Step S6, which assigns different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation data, specifically involves: If the new generation of data discovers an executable path not covered by the original data or the previous generation of data, a positive reward value is assigned; otherwise, no reward value is assigned, and the data with the largest reward value is retained as the newly generated test data.

7. The method for generating path coverage test data based on a reinforcement learning selection strategy according to claim 4, characterized in that, The iterative data updates in step S6 include: ， in, For the new value, To improve the learning efficiency of intelligent agents, For the present value, Rewards for the actions performed. As a discount factor, The maximum action reward obtained in the new state.

8. A path coverage test data generation system based on reinforcement learning selection strategy, characterized in that, The system includes: The path acquisition module is used to obtain all executable paths and the number of executable paths of the program under test; The definition module is used to define the executable path as the agent state, define the data selection of each round of updates as the action, and all the agent states constitute the state space. The initial state set acquisition unit is used to generate initial data according to a random method, and then use an agent to transform the initial data to obtain the initial state set. The update unit is used to update the initial data and generate multiple sets of updated data when the initial state is included in the state space, indicating that the initial state set fails to completely cover the executable path of the program under test. The selection unit is used to select updated data and determine the next generation of data. The reward module is used to assign different reward values ​​to different actions of the agent and the agent's coverage path based on the new generation of data. The data with the highest reward value is retained as newly generated test data, and the data is continuously iterated and updated until the generated data covers the state space.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that executes a path coverage test data generation method based on a reinforcement learning selection strategy as described in any one of claims 1-7.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a path coverage test data generation method based on a reinforcement learning selection strategy as described in any one of claims 1-7.

Images