An Adaptive Test Case Generation Method and System Based on Reinforcement Learning

Abstract

This invention discloses an adaptive test case generation method and system based on reinforcement learning, belonging to the field of intelligent testing system technology. It aims to solve the problems of poor adaptability, low quality, difficulty in dynamic optimization, and insufficient integration with the development process in test case generation. The method acquires heterogeneous data from multiple sources, such as requirement documents and source code, and extracts semantic, structural, and historical pattern features through graph neural networks and data mining, fusing them to generate an environment state vector. An actor-critic framework is adopted to output test action decision vectors. After executing test cases, rewards are calculated through a multi-objective composite reward function, and agent parameters are updated using a priority experience replay mechanism. Rapid initialization of the new system is achieved through pre-trained model parameter fusion, and a redundant test case set is constructed and prioritized through clustering. This invention improves the relevance and efficiency of test cases, enhances system adaptability, and can be widely applied to software automated testing scenarios.

Description

Technical Field

[0001] This application relates to the field of intelligent testing technology, and in particular to an adaptive test case generation method and system based on reinforcement learning. Background Technology

[0002] Software testing is a crucial step in ensuring software quality and system reliability. As software systems become increasingly larger and more complex, traditional testing methods relying on manual writing and execution are no longer sufficient to meet the demands for efficient and comprehensive testing. Especially in agile development models like continuous integration / continuous delivery, rapid iteration places higher demands on the automation and intelligence of testing. Currently, while automated testing technologies are widely used, the quality and efficiency of test case generation remain major bottlenecks restricting testing effectiveness. How to dynamically generate test cases that adapt to system changes and effectively reveal deep-seated defects has become a critical problem that urgently needs to be solved in the field of software engineering.

[0003] Currently, test case generation primarily relies on rule-based, random, or search-based software testing techniques. Rule-based testing requires pre-defined and comprehensive test specifications, making it difficult to adapt to complex and ever-changing business logic. While random testing is simple and easy to implement, it suffers from low test case effectiveness and insufficient exploration efficiency. Search-based testing generates test cases by optimizing fitness functions, but these fitness functions often rely on manual design and are difficult to dynamically adjust and optimize based on feedback during testing. Furthermore, existing methods mostly treat test generation in isolation, failing to deeply integrate software requirement semantics, code structure characteristics, and historical test data. This results in generated test cases lacking understanding of the business scenario and targeting of potential risk areas, making it difficult to guarantee the sufficiency and effectiveness of test coverage.

[0004] In recent years, reinforcement learning, as a type of machine learning method that optimizes sequential decisions through interaction with the environment, has achieved significant success in fields such as games and robot control. Its core advantage lies in its ability to learn through trial and error, gradually mastering the optimal strategy for achieving goals in a specific environment without requiring a large amount of pre-labeled data. This characteristic provides a new approach to solving adaptive test case generation: modeling the test generation process as a reinforcement learning problem, enabling the agent to automatically adjust its testing strategy based on feedback from test execution. However, how to effectively encode the multi-dimensional information of the software system into the environment state of reinforcement learning, how to design a reasonable action space and reward function to guide the agent to generate high-quality test cases, and how to achieve seamless integration with existing development and testing processes remain challenges that have not been fully addressed in current technologies. Summary of the Invention

[0005] The purpose of this invention is to solve the core problems of low efficiency, poor adaptability, and insufficient intelligence in traditional software testing test case generation. This application provides an adaptive test case generation method and system based on reinforcement learning.

[0006] To achieve the above objectives, this application provides an adaptive test case generation method and system based on reinforcement learning, which adopts the following technical solution:

[0007] Firstly, this application discloses an adaptive test case generation method based on reinforcement learning, including:

[0008] Acquire multi-source heterogeneous input data of the target software system, including requirements documents, design documents, source code, and historical test and defect data; perform semantic parsing on the requirements documents and design documents to extract semantic features, perform structural analysis on the source code to extract structural features, and perform pattern mining on the historical test and defect data to extract historical pattern features; fuse the semantic features, structural features, and historical pattern features to generate an environmental state vector representing the current test context and the system under test;

[0009] The environmental state vector is input into a pre-trained reinforcement learning agent; the reinforcement learning agent outputs a test action decision vector based on an internal policy network, and the test action decision vector is used to define the policy parameters for generating test cases;

[0010] Based on the strategy parameters defined by the test action decision vector, the corresponding test generator is invoked to generate or select test cases, which are then executed in the target software system or simulation test environment. Execution process data is collected to obtain test feedback results, including coverage metrics, defect discovery status, and execution time.

[0011] Based on the test feedback results, the immediate reward value corresponding to the test action is calculated using a preset reward function; the state, action, reward, and next state data generated by the interaction are stored as a transition sample in the experience replay buffer; based on the transition sample sampled in the experience replay buffer, the policy gradient is calculated and the policy network parameters in the reinforcement learning agent are updated to achieve adaptive optimization of the test-generated policy.

[0012] Preferably, the method for constructing the environment state vector includes:

[0013] The pre-trained natural language processing model is used to encode and understand the requirements document and design document, identify the functional entities, operations and logical constraints between entities, and construct a structured requirements semantic graph.

[0014] Static analysis is performed on the source code to generate an abstract syntax tree, a control flow graph, and a program dependency graph; a graph neural network is then used to encode the abstract syntax tree, control flow graph, and program dependency graph to obtain code structure feature vectors that represent the code structure and semantics.

[0015] The global semantic vector obtained by aggregating the demand semantic graph through graph pooling, the code structure feature vector, and the historical pattern feature vector composed of the historical pattern features are concatenated, and then fused and dimensionality reduced through a fully connected layer to generate the final environment state vector.

[0016] Preferably, when constructing the demand semantic graph, a graph attention network is used to process the identified functional entities and logical constraint relationships to generate a context-aware feature vector for each entity node in the demand semantic graph.

[0017] Preferably, the reinforcement learning agent adopts an actor-critic framework, and the test action decision vector includes at least the following dimensions:

[0018] The input data generation pattern dimension indicates whether a random generation, equivalence class partitioning generation, or search-based adaptive generation strategy is used.

[0019] The execution path orientation dimension is used to define the weight distribution for exploring different types of code paths during test execution;

[0020] The assertion strength dimension controls the level of complexity of the content verified by the generated test assertions.

[0021] Preferably, the reward function is a multi-objective composite function, whose output immediate reward value is positively correlated with the improvement of test coverage and the severity of the discovered defects, negatively correlated with the test execution time, and includes a reward item for the diversity of test cases.

[0022] Preferably, when testing the adaptive optimization of the generation strategy, a priority experience replay mechanism is adopted to calculate a priority score for each transfer sample stored in the experience replay buffer. The priority score is dynamically determined based on the absolute value of the immediate reward of the transfer sample or the temporal difference error generated by the reinforcement learning agent. During sampling training, non-uniform sampling is performed based on the priority score.

[0023] Preferably, before being applied to a new target software system for the first time, a model initialization step is also included:

[0024] Extract the project metadata features of the current target software system;

[0025] The project metadata features are matched with a pre-stored project feature library to find several reference projects with the highest similarity.

[0026] Obtain pre-trained reinforcement learning agent models corresponding to the plurality of reference projects, wherein each pre-trained reinforcement learning agent model contains a policy network;

[0027] The policy network parameters of the several pre-trained reinforcement learning agent models are fused together to serve as the initial policy network parameters of the reinforcement learning agent in the current new system.

[0028] Preferred options also include:

[0029] When the policy network parameters of the reinforcement learning agent converge or meet the preset conditions after multiple rounds of testing and adaptive optimization of the generated policy, the current environmental state vector is input into the optimized reinforcement learning agent, and the reinforcement learning agent outputs the final test action decision vector.

[0030] Based on the final test action decision vector, several test cases are generated; cluster analysis based on execution path or input space is performed on the several test cases, and one or more representative test cases are selected from each cluster to form a redundant and optimized test case set covering diverse scenarios.

[0031] Based on the strategic value implied by the final test action decision vector used to generate each representative test case, or the average reward value obtained in historical iterations, execution priorities are assigned to the test cases in the optimized test case set.

[0032] Preferably, the method is integrated with a continuous integration / continuous deployment system; when a change in the source code of the target software system is detected, an environment state vector is constructed, a new environment state vector is generated based on the changed system state, and the optimized reinforcement learning agent is used to generate a priority-based optimized test case set for this code change based on the final test action decision vector, and the test case set is injected into the CI / CD pipeline for execution.

[0033] Secondly, this application discloses an adaptive test case generation system based on reinforcement learning, applied to the adaptive test case generation method based on reinforcement learning as described in the first aspect, comprising:

[0034] An environment state vector construction module is used to acquire multi-source heterogeneous input data of the target software system. The multi-source heterogeneous input data includes requirement documents, design documents, source code, and historical test and defect data. The module performs semantic parsing on the requirement documents and design documents to extract semantic features, performs structural analysis on the source code to extract structural features, and performs pattern mining on the historical test and defect data to extract historical pattern features. The module then fuses the semantic features, structural features, and historical pattern features to generate an environment state vector that represents the current test context and the state of the system under test.

[0035] The reinforcement learning decision module is used to input the environment state vector into a pre-trained reinforcement learning agent; the reinforcement learning agent outputs a test action decision vector based on an internal policy network, and the test action decision vector is used to define the policy parameters for generating test cases;

[0036] The test execution and feedback module is used to call the corresponding test generator to generate or select test cases according to the strategy parameters defined by the test action decision vector, execute the test cases in the target software system or simulation test environment, collect execution process data, and obtain test feedback results including coverage indicators, defect discovery status and execution time.

[0037] The model optimization and update module is used to calculate the immediate reward value corresponding to the test action based on the test feedback result through a preset reward function; store the state, action, reward and next state data generated by the interaction as a transition sample in the experience replay buffer; calculate the policy gradient and update the policy network parameters in the reinforcement learning agent based on the transition sample sampled in the experience replay buffer, so as to adaptively optimize the test-generated policy.

[0038] The test case set construction and tagging module is used to assign execution priorities to test cases in the optimized test case set based on the strategic value implied by the final test action decision vector on which each representative test case was generated, or the average reward value obtained in historical iterations.

[0039] Compared with existing technologies, this invention provides an adaptive test case generation method and system based on reinforcement learning, which has the following beneficial effects:

[0040] 1. By deeply integrating reinforcement learning agents with multi-source heterogeneous data, dynamic self-optimization of test case generation is achieved. The system can automatically adjust testing strategies based on real-time test feedback. Compared with traditional methods that rely on fixed rules or manually written test cases, it can more accurately explore high-risk code paths and generate more targeted and diverse test cases, thereby significantly improving the depth and breadth of defect detection and enhancing the reliability and stability of software products from the source.

[0041] 2. This solution constructs a fully automated closed loop of "generation-execution-feedback-optimization". On the one hand, the automated generation of test cases reduces a lot of repetitive and tedious manual labor; on the other hand, through priority experience replay, clustering to remove redundancy and priority marking, it can output a concise and efficient set of optimized test cases, so that test resources can be concentrated on the most effective test activities, shorten the test cycle and speed up software delivery.

[0042] 3. Cross-project migration initialization mechanism and seamless integration with CI / CD pipelines. The former allows new projects to reuse training results from existing projects, lowering the threshold and training cost of model cold start, and enabling the technology to quickly adapt to different software projects. The latter embeds intelligent testing capabilities into the mainstream workflow of modern software development, realizing automated and intelligent regression testing of code changes. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the steps of an adaptive test case generation method based on reinforcement learning, according to an embodiment of this application. Figure 2 This is a flowchart illustrating the steps of constructing an environment state vector in an adaptive test case generation method based on reinforcement learning, according to an embodiment of this application. Figure 3 This is a flowchart of the model initialization steps in an adaptive test case generation method based on reinforcement learning, according to an embodiment of this application. Detailed Implementation

[0044] The following is in conjunction with the appendix Figure 1-3 This application will be described in further detail.

[0045] This application discloses an adaptive test case generation method and system based on reinforcement learning.

[0046] Firstly, referring to Figure 1 This application discloses an adaptive test case generation method based on reinforcement learning, comprising:

[0047] S1. Acquire multi-source heterogeneous input data of the target software system, including requirements documents, design documents, source code, and historical test and defect data; perform semantic parsing on the requirements and design documents to extract semantic features, perform structural analysis on the source code (using static code analysis tools such as abstract syntax tree parsers or control flow graph generators) to extract structural features, and perform pattern mining on the historical test and defect data to extract historical pattern features; fuse semantic features, structural features, and historical pattern features to generate an environment state vector representing the current test context and the system under test.

[0048] The purpose of this fusion process is to integrate information from different sources and of different types into a unified numerical representation. For example, the extracted semantic features, structural features, and historical pattern features can be assigned weights and then weighted summation or weighted averaging can be performed to highlight the importance of the three features.

[0049] The pre-trained natural language processing model refers to a deep learning model that has been trained on a large-scale text corpus and is selected from pre-trained language models such as BERT, RoBERTa, or ALBERT. It is used to semantically encode requirement documents and design documents, identify functional entities and logical constraints, and understand and generate human language. Its role is to automatically extract meaningful information from unstructured text data.

[0050] For the acquisition of multi-source heterogeneous data, a combination of multi-channel collection and standardized preprocessing is used. Specifically, requirement documents and design documents are pulled from the Git version control system; local code directories and Git repositories are scanned using static analysis tools such as JavaParser, the source code is read and formatted, and the JIRA defect tracking system and TestRail test management tool are connected to collect historical test cases, execution results and defect reports through API interfaces. At the same time, offline data in CSV format is imported for completion, duplicate and invalid records are cleaned and the field format is unified, and finally a structured multi-source heterogeneous dataset covering requirement documents, design documents, source code, historical test and defect data is formed.

[0051] Specifically, refer to Figure 2 The methods for constructing environment state vectors include:

[0052] S11. Semantic Feature Extraction: Encode and understand the requirements and design documents using a pre-trained Natural Language Processing (NLP) model, identify the functional entities, operations, and logical constraints between entities, and construct a structured requirements semantic graph.

[0053] The encoding and understanding mentioned above refer to the process of converting natural language text into a numerical representation that the model can process and extracting semantic information from it. Its role is to transform unstructured text into computable features, laying the foundation for subsequent analysis.

[0054] The constructed demand semantic graph is as follows: ,

[0055] in, A collection of functional entity nodes. It is a set of logical constraint relationships between entities.

[0056] When constructing the demand semantic graph, a graph attention network (GAT) is used to process the identified functional entities and logical constraints, generating a context-aware feature vector for each entity node in the demand semantic graph. The global semantic vector is obtained by aggregation through graph pooling operations. The calculation formula is as follows:

[0057]

[0058] in, The graph pooling function can be average pooling, max pooling, or attention pooling.

[0059] The context-aware feature vector is a high-dimensional vector containing the entity node's own attributes and the logical relationship information between it and its directly associated nodes. It is calculated through the multi-head attention mechanism of the graph attention network (GAT).

[0060] S12. Structural Feature Extraction: Perform static analysis on the source code to generate an Abstract Syntax Tree (AST), a Control Flow Graph (CFG), and a Program Dependency Graph (PDG).

[0061] Static analysis refers to the technique of analyzing source code to discover potential problems or extract program structure information without actually executing the program. Its advantage lies in obtaining the internal structure and behavioral characteristics of the program without running it, which is highly efficient and can discover problems that are difficult to trigger at runtime.

[0062] By using graph neural networks (GNNs) to encode abstract syntax trees, control flow graphs, and program dependency graphs, code structure feature vectors representing code structure and semantics are obtained. ,Right now:

[0063]

[0064] Where d is the dimension of the structural feature vector.

[0065] S13. Historical Pattern Feature Extraction: Perform pattern mining on historical data such as historical test cases and defect reports to extract historical pattern features such as modules with high defect incidence, high-frequency test paths, and defect type distribution, and construct historical pattern feature vectors.

[0066] S14. Feature Fusion: Integrating the global semantic vector Code structure feature vector and historical pattern feature vectors The data is then concatenated and fused through a fully connected layer to reduce dimensionality, generating the final environment state vector. The formula is as follows:

[0067]

[0068] Where W is the weight matrix of the fully connected layer, b is the bias term, ReLU is the activation function, and [;] represents the global semantic vector. Code structure feature vector and historical pattern feature vectors splicing operation.

[0069] S2, convert the environment state vector The input is fed into a pre-trained reinforcement learning agent; the reinforcement learning agent, based on its internal policy network, outputs a test action decision vector. The test action decision vector is used to define the strategy parameters for generating test cases;

[0070] Among them, the reinforcement learning agent adopts the actor-critic framework;

[0071] The Actor network is a policy network used to output action decisions, while the Critic network is a value network used to evaluate the value of the current state-action.

[0072] The "pre-trained reinforcement learning agent" refers to an Actor-Critic framework agent that completes policy learning and parameter convergence in advance based on a large amount of historical data and test tasks of similar software testing scenarios. Its function is to enable it to adapt to general testing scenarios, without having to train from scratch for a single new system. It can quickly adapt to the target system through subsequent parameter fine-tuning, thereby shortening the convergence cycle.

[0073] The specific steps involved in the pre-training process of the reinforcement learning agent, based on standardized datasets and simulation testing environments for similar software testing scenarios, are as follows:

[0074] S21. Construction of pre-training dataset: Collect full-volume multi-source heterogeneous data from multiple similar reference projects. The multi-source heterogeneous data includes the requirements documents, design documents, source code, historical test cases, defect reports and test execution logs of each project. After cleaning and standardization, construct a general pre-training dataset covering different business scenarios and system complexities. At the same time, label the test coverage, defect discovery status and execution efficiency of each sample as the supervision basis for pre-training.

[0075] The standardization mentioned here refers to unifying field formats and feature dimensions.

[0076] S22. Pre-training environment setup: Build a general simulation test environment adapted to similar scenarios, simulate the operating logic, interface calls and exception feedback mechanism of the software system, and support the automated execution of test cases and the collection of feedback data.

[0077] Simultaneously, a standardized environmental state representation method is defined, consistent with the environmental state vector structure in step S1 above, to ensure compatibility between pre-training and subsequent fine-tuning, as well as an action space (fixed input data generation mode, execution path guidance, assertion construction strength, and unified value range and semantics).

[0078] S23. Multi-round iterative pre-training: The pre-training dataset is input into the established simulation environment to initialize the Actor-Critic framework agent, i.e., initial parameters are randomly assigned, and a closed-loop process of "state input - action decision - test case generation and execution - feedback collection - reward calculation - parameter update" is executed. In each iteration, the initialized Actor-Critic framework agent outputs test actions based on the current state, generates test cases and executes them in the simulation environment, collects feedback such as coverage improvement, defect severity level, and execution time, calculates the reward value through a multi-objective composite reward function, and optimizes the Actor-Critic network parameters by combining a priority replay mechanism.

[0079] S24. Pre-training convergence determination and model saving: When the change in policy network parameters of the reinforcement learning agent in multiple consecutive iterations is less than a threshold, such as 0.00001, and the test coverage on the validation set of similar project data that has not participated in the training is stable at more than 80%, and the defect detection rate reaches 1.2 times the benchmark, then the pre-training can be determined to be converged. The parameters of the initial Actor-Critic framework agent policy network (Actor) and value network (Critic) at this time are solidified and stored in the project feature library as the pre-trained model of the corresponding reference project for subsequent new system initialization.

[0080] S25. Incremental pre-training optimization: Regularly incorporate new data from similar reference projects and test tasks, incrementally train the saved pre-trained models, update network parameters, continuously optimize the general adaptability and test performance of the agent, and cover more subdivided scenarios and boundary cases.

[0081] After the above pre-training steps are completed, the pre-trained agent has a general testing strategy for similar scenarios. When applied to a new target system, by matching project features and fusing pre-training parameters from multiple reference projects, an initial agent adapted to the new system can be obtained. Then, by combining the specific data of the target system and the testing requirements to fine-tune the parameters, accurate test cases can be generated quickly.

[0082] The test action decision vector includes at least the following three dimensions:

[0083] (1) Input data generation pattern dimension : Used to indicate whether a random generation, equivalence class partitioning generation, or search-based adaptive generation strategy is used; the value range is [value range missing]. These correspond to three generation modes;

[0084] (2) Execution path-oriented dimension Used to define the weight distribution for exploring different types of code paths during test execution. ,in Let the exploration weight of the i-th type of code path satisfy the following condition: k is the number of code path categories;

[0085] (3) Assert the strength dimension of the construction : Used to control the complexity level of the content verified by the generated test assertions, with a value range of . ,in, A larger value indicates a higher complexity in assertion verification.

[0086] S3, Based on the test action decision vector The defined strategy parameters call the corresponding test generator to generate or select test cases, execute the test cases in the target software system or simulation test environment, and collect execution process data to obtain test feedback results including coverage metrics, defect discovery status, and execution time. It can execute test cases in a simulated environment, determine defect discovery through log analysis and assertion checks, and record the simulation execution time;

[0087] The test generator can be a template-based generator that selects a predefined test case template based on the decision vector and fills in the corresponding parameters.

[0088] The coverage metrics include statement coverage, branch coverage, and path coverage; defect discovery includes the number of defects and defect severity level, where defect severity level can be divided into fatal, severe, moderate, and minor; execution time is the execution time of a single test case or the total execution time of a test batch.

[0089] S4. Based on the test feedback results The instant reward value corresponding to this test action is calculated using a preset reward function. The state s, action a, and reward generated by this interaction and next state data As a transfer sample (s, a, , Store in the experience replay buffer Based on experience replay buffer The sampled transition samples are used to calculate the policy gradient and update the policy network parameters in the reinforcement learning agent to achieve adaptive optimization of the test-generated policy.

[0090] Specifically, the reward function is a multi-objective composite function. Its output immediate reward value is positively correlated with the improvement of test coverage and the severity of the discovered defects, negatively correlated with the test execution time, and includes a reward item for the diversity of test cases.

[0091] For example, a positive reward is given for each defect discovered, and a positive reward is given for each percentage point increase in test coverage. However, a negative reward is given for each second increase in test execution time. This immediate reward value directly reflects the effectiveness of the test case generation and execution.

[0092] The specific form of the reward function is as follows:

[0093]

[0094] in: This represents the coverage improvement compared to the previous test, with a value ranging from [0,1]. ], This refers to the coverage rate from the previous round of testing.

[0095] Where m is the number of defects found in this test. Score the severity level of the i-th defect (fatal = 4, serious = 3, moderate = 2, minor = 1);

[0096] T represents the standardized execution time of this test, through... Standardized to the range [0,1], where t is the actual execution time. , These represent the minimum and maximum execution times in historical tests, respectively.

[0097] The diversity measure of the generated test case set U is calculated using Jaccard distance, with a value range of [0,1].

[0098] Let be the weighting coefficient, satisfying =1, which can be dynamically adjusted according to actual testing needs.

[0099] When testing the adaptive optimization of the generation strategy, a Prioritized Experience Replay (PER) mechanism is used, with each experience replay buffer being used to store the results. A priority score p is calculated for each transition sample. The priority score p is dynamically determined based on the absolute value of the immediate reward of the transition sample or the temporal difference error (TD error) generated by the reinforcement learning agent. The formula for determining the priority using the TD error is as follows:

[0100]

[0101] in, = For TD error, The state value output by the Critic network. As a discount factor, To minimize the value, avoid a priority of 0.

[0102] During sampling training, non-uniform sampling is performed based on priority scores, and the sampling probability is positively correlated with the priority scores.

[0103] Furthermore, refer to Figure 3 Before being applied to a new target software system for the first time, a model initialization step is also included:

[0104] S01. Extract the project metadata features of the current target software system. The project metadata features include project size, technology stack, business domain, number of modules, etc.

[0105] The project metadata features include, but are not limited to, quantitative and categorical features such as the number of lines of code, development language and framework, application scenarios, and the number of core functional modules.

[0106] S02. Match the project metadata features with the pre-stored project feature library, use cosine similarity to calculate the similarity between the current project and each reference project in the feature library, and find the k reference projects with the highest similarity (k≥1).

[0107] S03. Obtain pre-trained reinforcement learning agent models corresponding to k reference projects, wherein each pre-trained reinforcement learning agent model contains a policy network.

[0108] S04. The policy network parameters of the k pre-trained reinforcement learning agent models are fused together to form the initial policy network parameters of the reinforcement learning agent in the new system. The parameter fusion adopts a weighted average method, with the weight being the similarity between the current project and the reference project. The formula is as follows:

[0109]

[0110] in, These are the initial policy network parameters. The similarity weight of the i-th reference item ( =1), These are the pre-trained policy network parameters corresponding to the i-th reference project.

[0111] Among them, the pre-trained reinforcement learning agent model (including policy network parameters) After being bound to the metadata features of the corresponding reference project, the feature vector is stored in the project feature library in the form of "project feature vector-model parameter" key-value pairs. The project feature library adopts distributed database storage and supports fast matching queries based on cosine similarity.

[0112] S5. After the policy network parameters of the reinforcement learning agent converge or meet preset conditions through adaptive optimization of the generated policy after multiple rounds of testing, the current environment state vector is input into the optimized reinforcement learning agent. The reinforcement learning agent outputs the final test action decision vector. Based on the final test action decision vector, several test cases are generated. Cluster analysis based on execution path or input space is performed on the test cases. The K-means clustering algorithm is used to divide the test cases into n clusters, where n is determined according to the number and diversity requirements of the test cases. One or more representative test cases are selected from each cluster, such as the test cases corresponding to the cluster center, to form a redundant optimized test case set covering diverse scenarios. Based on the policy value implied by the final test action decision vector used to generate each representative test case, or the average reward value obtained in historical iterations, execution priorities are assigned to the test cases in the optimized test case set. The priority score P is calculated as follows:

[0113]

[0114] Where T represents the historical iteration round, Let be the reward value corresponding to test case u in the t-th iteration.

[0115] S6. Integration with CI / CD systems: This method is further integrated with continuous integration / continuous deployment systems. When changes to the source code of the target software system are detected, the environment state construction step is automatically triggered. A new environment state vector is generated based on the changed system state. Using an optimized reinforcement learning agent, based on the final test action decision vector, a priority-based optimized test case set for this code change is generated and injected into the CI / CD pipeline for execution.

[0116] Secondly, this application also discloses an adaptive test case generation system based on reinforcement learning, applied to the adaptive test case generation method based on reinforcement learning as described in the first aspect, comprising:

[0117] The environment state vector construction module is used to acquire multi-source heterogeneous input data of the target software system. The multi-source heterogeneous input data includes requirements documents, design documents, source code, and historical test and defect data. Semantic parsing is performed on requirements documents and design documents to extract semantic features, structural analysis is performed on source code to extract structural features, and pattern mining is performed on historical test and defect data to extract historical pattern features. Semantic features, structural features, and historical pattern features are fused to generate an environment state vector that represents the current test context and the state of the system under test.

[0118] The reinforcement learning decision module is used to input the environment state vector into the pre-trained reinforcement learning agent; the reinforcement learning agent outputs test action decision vectors based on the internal policy network, and the test action decision vectors are used to define the policy parameters for generating test cases;

[0119] The test execution and feedback module is used to call the corresponding test generator to generate or select test cases according to the strategy parameters defined by the test action decision vector, execute the test cases in the target software system or simulation test environment, collect execution process data, and obtain test feedback results including coverage indicators, defect discovery status and execution time.

[0120] The model optimization and update module is used to calculate the immediate reward value corresponding to the test action based on the test feedback result through a preset reward function; store the state, action, reward and next state data generated by the interaction as a transition sample in the experience replay buffer; calculate the policy gradient and update the policy network parameters in the reinforcement learning agent based on the transition sample sampled in the experience replay buffer, so as to adaptively optimize the test-generated policy.

[0121] The test case set construction and tagging module is used to assign execution priorities to test cases in the optimized test case set based on the strategic value implied by the final test action decision vector used to generate each representative test case, or the average reward value obtained in historical iterations.

[0122] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. An adaptive test case generation method based on reinforcement learning, characterized in that, include: Acquire multi-source heterogeneous input data of the target software system, including requirements documents, design documents, source code, and historical test and defect data; Semantic parsing is performed on the requirements document and design document to extract semantic features, structural analysis is performed on the source code to extract structural features, and pattern mining is performed on the historical test and defect data to extract historical pattern features. The semantic features, structural features, and historical pattern features are fused to generate an environmental state vector representing the current test context and the system under test. The environmental state vector is input into a pre-trained reinforcement learning agent; The reinforcement learning agent outputs test action decision vectors based on an internal policy network. These test action decision vectors are used to define the policy parameters for generating test cases. Based on the strategy parameters defined by the test action decision vector, the corresponding test generator is invoked to generate or select test cases, which are then executed in the target software system or simulation test environment. Execution process data is collected to obtain test feedback results, including coverage metrics, defect discovery status, and execution time. Based on the test feedback results, the immediate reward value corresponding to the test action is calculated using a preset reward function; the state, action, reward and next state data generated by this interaction are stored as a transition sample in the experience replay buffer. Based on the transition samples sampled from the experience replay buffer, the policy gradient is calculated and the policy network parameters in the reinforcement learning agent are updated to adaptively optimize the test-generated policy.

2. The adaptive test case generation method based on reinforcement learning according to claim 1, characterized in that, The method for constructing the environment state vector includes: The pre-trained natural language processing model is used to encode and understand the requirements document and design document, identify the functional entities, operations and logical constraints between entities, and construct a structured requirements semantic graph. Static analysis is performed on the source code to generate an abstract syntax tree, a control flow graph, and a program dependency graph; a graph neural network is then used to encode the abstract syntax tree, control flow graph, and program dependency graph to obtain code structure feature vectors that represent the code structure and semantics. The global semantic vector obtained by aggregating the demand semantic graph through graph pooling, the code structure feature vector, and the historical pattern feature vector composed of the historical pattern features are concatenated, and then fused and dimensionality reduced through a fully connected layer to generate the final environment state vector.

3. The adaptive test case generation method based on reinforcement learning according to claim 2, characterized in that: When constructing the demand semantic graph, a graph attention network is used to process the identified functional entities and logical constraint relationships to generate a context-aware feature vector for each entity node in the demand semantic graph.

4. The adaptive test case generation method based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning agent employs an actor-critic framework, and the test action decision vector includes at least the following dimensions: The input data generation pattern dimension indicates whether a random generation, equivalence class partitioning generation, or search-based adaptive generation strategy is used. The execution path orientation dimension is used to define the weight distribution for exploring different types of code paths during test execution; The assertion strength dimension controls the level of complexity of the content verified by the generated test assertions.

5. The adaptive test case generation method based on reinforcement learning according to claim 1, characterized in that: The reward function is a multi-objective composite function. Its output immediate reward value is positively correlated with the improvement of test coverage and the severity of the discovered defects, negatively correlated with the test execution time, and includes a reward item for the diversity of test cases.

6. The adaptive test case generation method based on reinforcement learning according to claim 1, characterized in that: When testing the adaptive optimization of the generation strategy, a priority experience replay mechanism is adopted. A priority score is calculated for each transfer sample stored in the experience replay buffer. The priority score is dynamically determined based on the absolute value of the immediate reward of the transfer sample or the temporal difference error generated by the reinforcement learning agent. During sampling training, non-uniform sampling is performed based on the priority score.

7. The adaptive test case generation method based on reinforcement learning according to claim 1, characterized in that, Before being applied to a new target software system for the first time, a model initialization step is also included: Extract the project metadata features of the current target software system; The project metadata features are matched with a pre-stored project feature library to find several reference projects with the highest similarity. Obtain pre-trained reinforcement learning agent models corresponding to the plurality of reference projects, wherein each pre-trained reinforcement learning agent model contains a policy network; The policy network parameters of the several pre-trained reinforcement learning agent models are fused together to serve as the initial policy network parameters of the reinforcement learning agent in the current new system.

8. The adaptive test case generation method based on reinforcement learning according to claim 1, characterized in that, After the policy network parameters of the reinforcement learning agent have converged or met preset conditions through multiple rounds of testing and adaptive optimization of the policy, the following steps are also included: When the policy network parameters of the reinforcement learning agent converge or meet the preset conditions after multiple rounds of testing and adaptive optimization of the generated policy, the current environmental state vector is input into the optimized reinforcement learning agent, and the reinforcement learning agent outputs the final test action decision vector. Based on the final test action decision vector, several test cases are generated; cluster analysis based on execution path or input space is performed on the several test cases, and one or more representative test cases are selected from each cluster to form a redundant and optimized test case set covering diverse scenarios. Based on the strategic value implied by the final test action decision vector used to generate each representative test case, or the average reward value obtained in historical iterations, execution priorities are assigned to the test cases in the optimized test case set.

9. The adaptive test case generation method based on reinforcement learning according to claim 8, characterized in that, The method integrates with continuous integration / continuous deployment systems, specifically including: When a change in the source code of the target software system is detected, an environment state vector is constructed, a new environment state vector is generated based on the changed system state, and the optimized reinforcement learning agent is used to generate a priority set of optimized test cases for this code change based on the final test action decision vector, and then injected into the CI / CD pipeline for execution.

10. An adaptive test case generation system based on reinforcement learning, applied to the adaptive test case generation method based on reinforcement learning as described in any one of claims 1-9, characterized in that, include: The environment state vector construction module is used to acquire multi-source heterogeneous input data of the target software system. The multi-source heterogeneous input data includes requirements documents, design documents, source code, and historical test and defect data. Semantic parsing is performed on the requirements document and design document to extract semantic features, structural analysis is performed on the source code to extract structural features, and pattern mining is performed on the historical test and defect data to extract historical pattern features. The semantic features, structural features, and historical pattern features are fused to generate an environment state vector that represents the current test context and the state of the system under test. A reinforcement learning decision module is used to input the environmental state vector into a pre-trained reinforcement learning agent; The reinforcement learning agent outputs test action decision vectors based on an internal policy network. These test action decision vectors are used to define the policy parameters for generating test cases. The test execution and feedback module is used to call the corresponding test generator to generate or select test cases according to the strategy parameters defined by the test action decision vector, execute the test cases in the target software system or simulation test environment, collect execution process data, and obtain test feedback results including coverage indicators, defect discovery status and execution time. The model optimization and update module is used to calculate the instant reward value corresponding to the current test action based on the test feedback results and a preset reward function. The state, action, reward, and next state data generated in this interaction are stored as a transition sample in the experience replay buffer. Based on the transition sample sampled in the experience replay buffer, the policy gradient is calculated and the policy network parameters in the reinforcement learning agent are updated to adaptively optimize the test-generated policy. The test case set construction and labeling module is used to generate test cases based on the optimized test action decision vector after the policy network parameters of the reinforcement learning agent converge or meet the preset conditions, obtain an optimized test case set by removing redundancy through cluster analysis, and assign execution priority to the test cases in the optimized test case set according to the policy value or historical average reward value.

Images