Test case adaptive evolution method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a test case adaptive evolution method, electronic equipment and a storage medium. The method comprises the steps of determining a new version code and an old version code in response to a target code request; based on the new version code and the old version code, determining a code difference and a difference influence report corresponding to the code difference; testing is carried out based on the new version code and an existing test case, a target invalid case is determined, and the target invalid case is the existing test case which is invalid in execution and related to the difference influence report; a case repair language model is adopted to conduct semantic repair on the target invalid case, a target repair case is determined, and the case repair language model is a large language model used for conducting semantic repair on the test case. According to the method, the updating / maintaining cost of the test case can be remarkably reduced, the repairing efficiency is high, and errors are not prone to occurring.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of software testing technology, and in particular to a test case adaptive evolution method, electronic device, and storage medium. Background Art

[0002] In the field of software testing, especially in continuous integration environments, automated testing has become a key means of improving software quality and development efficiency. However, automated testing methods still face many challenges. During version iterations, frequent code changes (such as interface adjustments, parameter modifications, and logic refactoring) often cause a large number of existing test cases to become invalid, requiring manual identification, repair, and updates. This is not only time-consuming and labor-intensive, but also prone to errors, increasing the cost and complexity of test maintenance. Summary of the Invention

[0003] The embodiments of the present application provide a test case adaptive evolution method, electronic device and storage medium, aiming to improve the problems of high maintenance cost and low efficiency of test cases during version iteration.

[0004] A test case adaptive evolution method, comprising: In response to the object code request, determining a new version code and an old version code; Determining, based on the new version code and the old version code, a code difference and a difference impact report corresponding to the code difference; Performing tests based on the new version code and existing test cases to determine target failure cases, where the target failure cases are existing test cases that fail to execute and are related to the difference impact report; A use case repair language model is used to perform semantic repair on the target failure case to determine a target repair case. The use case repair language model is a large language model used to perform semantic repair on the test case.

[0005] In one embodiment, determining a code difference and a difference impact report corresponding to the code difference based on the new version code and the old version code includes: Determining code differences based on the new version code and the old version code, and determining a code context and a project knowledge base corresponding to the code differences; Using an impact analysis language model to perform semantic understanding on the code differences, the code context, and the project knowledge base, and determine code change items, where the code change items include at least one of an interface change, a parameter change, a logic change, and a dependency change; Determining an affected code function based on the code change item, where the affected code function is a code function affected by the code change item; Determining a difference impact report corresponding to the code difference based on the code difference and the affected code function; The impact analysis language model is a large language model used to perform impact analysis on code differences.

[0006] In one embodiment, the testing based on the new version code and existing test cases to determine target failure cases includes: Performing tests based on the new version code and existing test cases to determine original failure cases, where the original failure cases are existing test cases that fail to execute; Performing feature extraction on the original failure case to determine a failure feature corresponding to the original failure case; Based on the failure characteristics corresponding to the original failure case and the difference impact report corresponding to the code difference, a target failure case is determined, where the target failure case is the original failure case whose failure characteristics match the difference impact report.

[0007] In one embodiment, the using of a use case repair language model to semantically repair the target failure use case and determine the target repair use case includes: Performing semantic repair on the target failure use case using a use case repair language model to determine a semantic repair item, where the semantic repair item includes at least one of a request parameter, an assertion statement, and a business process; Determining an original repair use case based on the target failure use case and the semantic repair item; The original repair use case is verified to determine a target repair use case, where the target repair use case is the original repair use case that has passed verification.

[0008] In one embodiment, after determining the code difference and the difference impact report corresponding to the code difference based on the new version code and the old version code, the method further includes: Based on the code differences, determining a code context and a project knowledge base corresponding to the code differences; Using a use case generation language model to process the code differences, the code context, and the project knowledge base to determine an original generation use case; The original generation use case is verified to determine a target generation use case, where the target generation use case is the original generation use case that has passed the verification.

[0009] In one embodiment, the method further comprises: Intelligently optimizing a first test case using an optimization agent to obtain a second test case that meets preset optimization conditions, wherein the optimization agent is an agent for implementing intelligent optimization, and the first test case is a target repair case, a target generation case, or an existing test case; A decision-making agent is used to make intelligent decisions on all the second test cases to determine target test cases that meet preset decision conditions. The decision-making agent is an agent used to implement intelligent decision-making.

[0010] In one embodiment, the intelligent optimization of the first test case by using an optimization agent to obtain a second test case that meets a preset optimization condition includes: Using multiple optimization agents to perform single verification on the first test case respectively, and determining a first verification result corresponding to the first test case, wherein the first verification result is determined based on the multiple single verification results, and the single verification result is a result of single verification of the first test case by a single optimization agent; Using multiple optimized agents to challenge and verify the first test case, and determining a second verification result corresponding to the first test case, wherein the second verification result is determined based on the multiple challenge verification results, and the challenge verification result is the result of two of the optimized agents challenging and verifying the first test case respectively; When the first verification result and the second verification result corresponding to the first test case meet a preset optimization condition, the first test case is determined as the second test case.

[0011] In one embodiment, the step of using a decision agent to perform intelligent decision-making on all the second test cases to determine a target test case that meets a preset decision condition includes: Using a decision agent to comprehensively evaluate all of the second test cases to determine decision evaluation values ​​corresponding to all of the second test cases; Based on the decision evaluation values ​​corresponding to all the second test cases, multiple target test cases are determined, and the decision evaluation values ​​corresponding to the multiple target test cases meet the preset decision conditions.

[0012] An electronic device includes a processor and a memory, wherein: Memory for storing computer programs; The processor is used to execute the program stored in the memory to implement the above-mentioned test case adaptive evolution method.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned test case adaptive evolution method.

[0014] In the above-mentioned test case adaptive evolution method, electronic device and storage medium, the code differences and corresponding difference impact reports in the version iteration process are first determined based on the new version code and the old version code; then, testing is performed based on the new version code and the existing test cases to determine the target failure cases related to the difference impact report, so as to quickly and accurately locate the target failure cases caused by version iteration; the target failure cases are intelligently repaired by the use case repair language model, which can greatly reduce the workload of manual update and maintenance of test cases due to version iteration, and can significantly reduce the update / maintenance cost of test cases, with high repair efficiency and less error-prone. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a test case adaptive evolution method provided by an embodiment of the present application; Figure 2 This is another flow chart of the test case adaptive evolution method provided by one embodiment of the present application; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the technical problems, technical solutions and beneficial effects solved by this application more clearly understood, this application is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0017] Embodiments of the present application provide a method for adaptive test case evolution. This method is applicable to an adaptive evolution system that operates in a continuous integration environment (hereinafter referred to as a "CI environment"). The adaptive evolution system is installed on an electronic device and utilizes a large language model and intelligent agents to automatically perceive, repair, generate, and optimize test cases. The entire test case adaptive evolution method begins with a user-triggered target code request and ultimately results in an optimized test case library. The electronic device has data processing capabilities and may be, but is not limited to, a computer.

[0018] The following is an explanation of the relevant terms involved in this application: The new version code is the latest version code. The old version code is the version code built before the new version code and serves as a reference for the new version code. It can be the version code that was most recently updated before the new version code. The historical version code is the version code before the new version code, which can include the old version code and all previous version codes.

[0019] A test case is a collection of specific test scenarios, steps, inputs, and expected outputs designed to verify that a function or feature meets expectations. Existing test cases are those created before the target code is requested and are stored in a test case library.

[0020] Original failure cases are existing test cases with execution failures. Related failure cases are original failure cases that successfully match the details of a code difference. Target failure cases are original failure cases whose failure characteristics match the difference impact report. Specifically, target failure cases are existing test cases with execution failures and related to the difference impact report. These can be understood as existing test cases with execution failures caused by code differences.

[0021] The original repair case is an unverified test case generated by the repair language model for the target failure case. The target repair case is the original repair case that has passed verification.

[0022] The original generated test case is an unverified test case generated by the use case generation language model based on code differences and related information. The target generated test case is the verified original generated test case.

[0023] The first test case is the test case that requires optimization. This first test case can be a target repair case, a target generation case, or an existing test case. The second test case is the first test case that meets the preset optimization conditions. The target test case is the test case that meets the preset decision conditions. Preset optimization conditions are pre-set optimization conditions that primarily filter test cases from the perspective of improving the overall performance of the test case library (such as quality, coverage, and efficiency). Preset decision conditions are pre-set decision conditions that specifically select which test cases to test the new version of the code.

[0024] The impact analysis language model is a large language model used to analyze the impact of code differences. The use case repair language model is a large language model used to perform semantic repairs on test cases. The use case generation language model is a large language model used to generate test cases.

[0025] In this example, the impact analysis language model, use case remediation language model, and use case generation language model can be trained using the same large language model or different large language models. The large language model can be, but is not limited to, general-purpose models such as sequence-to-sequence (Seq2Seq), the GPT series, CodeLlama, or specialized models such as Transformer, TestGPT-7B, and Codex. During large language model training, the large language model must understand test case writing standards, common test patterns (such as the common structure of unit and integration tests), and how to adjust or generate test cases based on code changes.

[0026] The training data for the aforementioned large language model training process primarily comes from the following sources: First, a large number of open source projects and their corresponding test case libraries. This data provides a rich mapping between test cases and the code being tested, as well as the structure and writing patterns of the test cases themselves. Second, the code change history and test case evolution history. By analyzing the code change history and test case modification records, we can learn how test cases typically need to be adjusted when specific types of code changes occur. For example, we collect a large amount of data such as old training code, new training code, training failure cases, training repair cases, training generation cases, and training analysis reports. Third, we use datasets specifically constructed for test case generation and repair tasks, such as those containing examples from various programming languages, test frameworks, and common test scenarios. Old training code refers to the old code used for model training; new training code refers to the new code used for model training. Training failure cases refer to the new code used for model training, which are the result of iterations of the old training version. Training failure cases refer to failed cases used for model training. Training repair cases are test cases generated during the model training phase. They serve as reference test cases for evaluating whether the output of training failure cases after repair processing by the large language model meets preset requirements. Training generation cases are test cases generated during the model training phase. They serve as reference test cases for evaluating whether the output of training failure cases after generation processing by the large language model meets preset requirements. Training analysis reports are analysis reports generated during the model training phase. They serve as reference analysis reports for evaluating whether the output of impact analysis by the large language model meets preset requirements.

[0027] The impact analysis language model training process requires obtaining new training code and old training code, determining the difference between the new training code and the old training code based on the new training code and the old training code, determining the training code difference and the code context and project knowledge base corresponding to the training code difference, and obtaining a training analysis report generated during the training process; forming first training data based on the training code difference, code context, project knowledge base and training analysis report, inputting the first training data into the large language model for training, and determining a prediction analysis report based on the training code difference, code context and project knowledge base output; performing loss calculation based on the prediction analysis report and the training analysis report, and determining a first target loss; updating the model parameters corresponding to the large language model based on the first target loss until the first target loss meets the preset convergence condition (for example, the first target loss is less than the first loss threshold), and determining the large language model after the model parameters are updated as the impact analysis language model.

[0028] The use case repair language model training process requires forming second training data based on the old training code, new training code, training failure use cases, and training repair use cases, inputting the second training data into the large language model for training, and determining the predicted repair use cases output based on the old training code, new training code, and training failure use cases; performing loss calculation based on the predicted repair use cases and the training repair use cases to determine the second target loss; updating the model parameters corresponding to the large language model based on the second target loss until the second target loss meets the preset convergence condition (for example, the second target loss is less than the second loss threshold), and determining the large language model after the model parameters are updated as the use case repair language model.

[0029] During the use case generation language model training process, third training data is determined based on the new training code, target metrics, and training-generated use cases. This third training data is then fed into the large language model for training, and prediction-generated use cases based on the new training code that meet the target metrics are determined. Loss is calculated based on the prediction-generated use cases and the training-generated use cases to determine a third target loss. The model parameters of the large language model are updated based on the third target loss until the third target loss meets a preset convergence condition (e.g., the third target loss is less than a third loss threshold). The large language model with updated model parameters is then determined as the use case generation language model. The target metric is used to evaluate whether the generated test cases meet specific requirements. This target metric may include, but is not limited to, test case coverage or defect detection capabilities. In other words, reinforcement learning methods can be introduced during training, using target metrics such as test case coverage and defect detection capabilities as reward signals to further optimize the model generation performance.

[0030] Furthermore, the model leverages structured information such as the code's abstract syntax tree (AST) to enhance its understanding of code semantics. By training on large-scale, diverse test-related data, the large model will be able to possess powerful test case repair and generation capabilities, significantly improving the automation and efficiency of testing activities in continuous integration.

[0031] The embodiment of the present application provides a test case adaptive evolution method, which is described by taking the method applied to an adaptive evolution system (hereinafter referred to as the "system") as an example. Figure 1 As shown in FIG, the test case adaptive evolution method includes: S1: In response to a target code request, determining a new version code and an old version code; S2: Based on the new version code and the old version code, determine the code differences and the difference impact report corresponding to the code differences; S3: Test the new version of the code and existing test cases to identify target failure cases. Target failure cases are existing test cases that fail and are related to the difference impact report. S4: Use the use case repair language model to perform semantic repair on the target failure case and determine the target repair case. The use case repair language model is a large language model used to perform semantic repair on the test case.

[0032] The target code request is a request that triggers the execution of the adaptive evolution of the test case.

[0033] As an example, in step S1, the system may receive a target code request, and in response to the target code request, may obtain a new version code and an old version code corresponding to the target code request. The target code request here may be a code submission request or a code merge request. In this example, when a developer submits a target code request to a code repository (such as Git), a continuous integration server (such as Jenkins, GitLab CI) will be triggered, and the system will automatically obtain the new version code corresponding to the target code request, query the code repository based on the project identifier of the new version code, and determine its corresponding old version code. The project identifier here is an identifier used to uniquely indicate the projects to which different versions of code belong. In this example, the new version code and the old version code carry the same project identifier, but carry different version identifiers, which are used to uniquely identify different versions.

[0034] Code differences refer to the differences between the new version and the old version, including but not limited to fixed, added, or deleted files, functions, classes, interfaces, and parameters. A difference impact report is used to reflect the code functionality and other information affected by the code differences.

[0035] As an example, in step S2, after determining the new version code and the old version code, the system can use a code comparison tool to compare the new version code and the old version code to determine the code differences between the two at the code level. The code comparison tool here includes but is not limited to a Diff tool based on an abstract syntax tree AST. Then, the system calls a pre-set impact analysis tool to analyze the code differences and content related to the code differences, so as to comprehensively analyze the scope of impact of the code differences on the changes in the code functions, and form a corresponding difference impact report based on the scope of the change impact. The scope of change impact here can be understood as the code functions affected by the code differences, and the difference impact report is a report formed based on the code functions affected by the code differences, which can include not only the affected code functions, but also deeper analysis results.

[0036] As an example, in step S3, the system performs tests based on the new version code and existing test cases in the test case library. Specifically, in the CI environment, the new version code and all existing test cases in the test case library are tested to determine the existing test cases that have failed to execute. Then, based on all the existing test cases that have failed to execute and the difference impact report, the reasons for the failure of the existing test cases are analyzed. When the reason for the failure to execute is related to the code difference, the existing test case that has failed to execute is the target failure case. On the contrary, when the reason for the failure to execute is not related to the code difference, it is determined that the existing test case that has failed to execute is not the target failure case.

[0037] As an example, in step S4, after determining the target failure use case that is caused by a code difference, the system can use a pre-trained use case repair language model to perform semantic repair on the target failure use case. The model understands the intent of the target failure use case and the semantics of the code difference, and other deep semantic content. Based on this deep semantic content, the system adjusts the request parameters, assertion statements, and business processes in the target failure use case to determine the target repair use case, so that the target repair use case can re-execute the code function that is valid and verify the new version of the code. This target repair use case is mainly used to resolve execution failure issues caused by code differences.

[0038] Existing technologies for handling failed test cases often rely on simple keyword-driven updates and recording recycling. This example uses a use case repair language model to adaptively repair the target failed test case, identifying the target repair case. This allows for subsequent adaptive optimization based on the target repair case, achieving automated and intelligent results. In this example, after determining the target repair case, the target repair case can be updated to the test case library, enabling adaptive updates to the test case library.

[0039] In this embodiment, the code differences and corresponding difference impact reports during the version iteration process are first determined based on the new version code and the old version code; then, testing is performed based on the new version code and the existing test cases to determine the target failure use cases related to the difference impact report, so as to quickly and accurately locate the target failure use cases caused by version iteration; the target failure use cases are intelligently repaired by the use case repair language model, which can greatly reduce the workload of manual update and maintenance of test cases due to version iteration, and can significantly reduce the update / maintenance cost of test cases, with high repair efficiency and less error-prone.

[0040] In one embodiment, after step S2, that is, after determining the code difference and the difference impact report corresponding to the code difference based on the new version code and the old version code, the method further includes: S5: Use the use case generation language model to generate use cases for code differences and determine the target generation use cases. The use case generation language model is a large language model used to generate test cases.

[0041] As an example, after step S2, that is, after performing a difference analysis based on the new version code and the old version code and determining the code differences, a pre-trained use case generation language model can be used to process input data such as the code differences, and based on its output results, a target generation use case can be determined. In this example, the use case generation language model can be used to process the code differences in the code differences that represent changes in the new version code relative to the old version code (such as the newly added code paths, modified logical branches, and newly added code functions in the new version code) to determine the target generation use case, so that the target generation use case covers the new version code and ensures the completeness of the new version code test. The target generation use case is mainly used to solve the test coverage problem of the new version code. In this example, after determining the target generation use case, the target generation use case can be updated to the test case library to achieve adaptive updating of the test case library.

[0042] In one embodiment, the method further comprises: S6: Using an optimization agent to intelligently optimize the first test case to obtain a second test case that meets preset optimization conditions, where the optimization agent is an agent used to implement intelligent optimization, and the first test case is a target repair case, a target generation case, or an existing test case; S7: Using a decision-making agent to make intelligent decisions on all second test cases, and determine target test cases that meet preset decision conditions. The decision-making agent is an agent used to implement intelligent decision-making.

[0043] An agent is a model based on a large model capable of proactive thinking and action. For example, the system activates agents with different roles (for example, some agents are responsible for generating variant test cases, some for executing test cases and collecting test results, some for evaluating test case effectiveness and coverage, and some for raising challenges and raising questions).

[0044] As an example, in step S6, after the system identifies a target repair case, target generation case, or existing test case as the first test case, it can use multiple pre-set optimization agents to intelligently optimize all first test cases. Specifically, multiple optimization agents are used to comprehensively evaluate the first test case to determine whether it meets the preset optimization conditions, and then the second test case of the first test case that meets the preset optimization conditions is selected. In this example, each optimization agent is built based on the large model, giving it stronger understanding and reasoning capabilities. The use of multiple optimization agents to intelligently optimize the first test case fully utilizes the advantages of the large model in natural language understanding, code generation, and logical reasoning, thereby improving the intelligence level of swarm intelligent optimization.

[0045] As an example, in step S7, the system may use multiple pre-set decision-making agents to make intelligent decisions on the second test cases to analyze the effectiveness, robustness or other performance of the second test cases, and determine all second test cases whose performance meets the preset decision conditions as target test cases.

[0046] In this embodiment, an optimization agent is used to perform group intelligent optimization on the first test case. Since the optimization agent is built based on a large model, it has strong understanding and reasoning capabilities and can quickly and accurately screen out the second test case. Then, a decision-making agent is used to make an intelligent decision on the second test case. Since the decision-making agent is built based on a large model, it also has strong understanding and reasoning capabilities. As an evaluation tool and auxiliary decision-making tool, it can more accurately determine the validity of the first test case, generate challenge test cases, identify redundancy, etc., so that it fully utilizes the advantages of the large model in natural language understanding, code generation and logical reasoning, and improves the intelligence level of the group intelligent optimization process.

[0047] In one embodiment, after step S7, after using the decision agent to perform intelligent decision making on all second test cases and determining a target test case that meets the preset decision condition, the method further includes: S8: Perform tests based on the target test cases and the new version code to obtain the test results of the test cases.

[0048] In this embodiment, testing is performed based on target test cases and new version codes, test case test results are obtained, and an end-to-end continuous integration test case self-evolution closed loop is constructed, that is, a complete closed-loop adaptive evolution system is constructed from code change perception to test case repair and generation, to swarm intelligent optimization and intelligent decision-making of test cases, and finally to test execution and feedback. This adaptive evolution system can be embedded in the existing CI / CD pipeline to realize the automated and continuous self-evolution of the test case library, significantly reducing the need for manual intervention and being able to quickly adapt to the rapid iterative development rhythm.

[0049] Furthermore, after step S8—that is, after testing the target test cases and the new code version and obtaining the test results—the knowledge base needs to be updated and the model needs to be iterated based on the test results. While this step isn't required for the system's immediate operation, it is crucial for its long-term performance and adaptability. By continuously recording data from the testing process (code changes, use case evolution, execution results, and agent interactions) and using this data to update the knowledge base and fine-tune the large model, the system can continuously learn and evolve, better adapting to project changes and improving the accuracy and efficiency of future version difference analysis, use case repair and generation, and swarm intelligence optimization. This effectively empowers the system with the ability to continuously learn and self-improve.

[0050] As an example, the system can submit the new version of code and the target test cases to a test execution engine (such as Selenium or JUnit). These are run in the target environment (e.g., CI) to obtain the test case results. These test case results (including pass / fail status, coverage reports, and execution logs) are collected and fed back to the developer and the system itself. These test case results can be used to evaluate the effectiveness of this self-evolution process.

[0051] Furthermore, the system can regularly fine-tune its large models (including the impact analysis language model, the use case remediation language model, and the use case generation language model) based on collected use case test results and feedback, improving their accuracy and efficiency in detecting version differences, remediating use cases, and generating use cases. Simultaneously, the agents in swarm intelligence optimization can learn and adjust based on historical data to better suit the testing needs of specific projects.

[0052] The current code development process lacks effective technical means to accurately and comprehensively identify the differences between different versions of code, especially the lack of in-depth understanding of interface contracts, parameter semantics, and potential cross-module impacts. Testers usually need to spend a lot of time analyzing the scope of the change impact, which is inefficient and prone to errors. To overcome the above problems, there is an urgent need for intelligent perception and analysis of code differences.

[0053] In one embodiment, step S2, i.e., determining code differences and a difference impact report corresponding to the code differences based on the new version code and the old version code, includes: S21: Based on the new version code and the old version code, determine the code differences, and determine the code context and project knowledge base corresponding to the code differences; S22: Using an impact analysis language model to perform semantic understanding on code differences, code context, and a project knowledge base, and determining code change items, where the code change items include at least one of an interface change, a parameter change, a logic change, and a dependency change; S23: Determine, based on the code change item, an affected code function, where the affected code function is a code function affected by the code change item; S24: Based on the code difference and the affected code function, determine a difference impact report corresponding to the code difference.

[0054] The impact analysis language model is a large language model used to perform impact analysis on code differences.

[0055] In this example, the pre-trained impact analysis language model inherently possesses powerful semantic understanding capabilities. When using this model for impact analysis, combining code differences and code context with the project knowledge base (such as historical defect records, API specifications, and requirements documents) significantly improves the accuracy and relevance of the difference analysis. For example, understanding the historical defect records of a particular interface and identifying its historically error-prone functional points can help the large model more accurately assess the risks posed by code differences.

[0056] The code change item is used to represent the specific changes in the code difference.

[0057] As an example, in step S21, the system may use a code comparison tool to compare the new version code and the old version code to determine the code differences between the two at the code level. Then, based on the code differences, the code context corresponding to the code differences is determined from the new version code and the old version code. The code context here includes but is not limited to the class where the changed function is located, other functions called, and interface definitions, etc. The system can also query the system database based on the project identifier corresponding to the new version code to determine the project knowledge base corresponding to the project identifier. The project knowledge base includes but is not limited to historical defect records, API specifications, and requirement documents, etc., which are related to the development of a certain project code.

[0058] As an example, in step S22, the system can use the impact analysis language model to conduct a deep semantic understanding of the code differences, code context and project knowledge base, and identify the code change items corresponding to the code differences. The code change items include at least one of interface changes, parameter changes, logic changes and dependency changes. Among them, interface changes are used to reflect changes related to interfaces in code differences, including but not limited to interface additions, deletions, URL or method modifications. Parameter changes are used to reflect changes related to parameters in code differences, including but not limited to the addition, deletion and modification of parameters, changes in parameter types, constraints, default values, etc. Logic changes are used to reflect changes related to internal logic in code differences, including but not limited to function internal algorithm adjustments, conditional branch changes, and loop logic modifications. Dependency changes are used to reflect changes related to dependencies in code differences, including but not limited to the introduction of new external services and library version upgrades.

[0059] As an example, in step S23, after identifying at least one code change item among an interface change, a parameter change, a logic change, and a dependency change, the system may further perform an impact analysis based on the code change item and the code functions reflected in the project knowledge base, determine the code functions affected by the code change item, and identify them as the affected code functions corresponding to the code difference. In this example, the affected code functions are the code functions that may be affected by changes in the at least one code change item corresponding to the code difference, i.e., the code functions potentially affected by the code change item.

[0060] As an example, in step S24, after determining the code difference and its corresponding affected code function, the system can process the code difference and its corresponding affected code function based on the pre-set report generation rules to determine the difference impact report corresponding to the code difference. In this example, the system can compile the code difference, code context, code modules and function points corresponding to the affected code function, and the code modules and function points according to a preset format to form a difference impact report, so that based on the difference impact report, the affected code function corresponding to the code difference can be quickly and accurately determined, so that in the subsequent adaptive evolution operation process such as repair, generation and optimization based on the affected test case, the test case repair, generation and optimization operations can be completed based on the affected code function to ensure that the code functions that may be affected can be effectively tested.

[0061] Furthermore, the system can also query a pre-set function case mapping table based on the affected code function. The function case mapping table is used to record different code functions and their corresponding existing test cases, and determine the existing test cases corresponding to the affected code functions in the function case mapping table as affected test cases. In other words, after determining the affected code functions that may be affected by the code difference, the system needs to further evaluate which of the existing test cases will become invalid or need to be adjusted, and then determine the existing test cases that may be affected and determine them as affected test cases. The system can also process the affected code functions and affected test cases based on pre-set report generation rules to determine the difference impact report corresponding to the code difference. In this example, the difference impact report can organize the code differences, code context, code modules and function points corresponding to the affected code functions, and the existing test cases corresponding to the code modules and function points according to a preset format to form a difference impact report, so that subsequent adaptive evolution operations such as repair, generation and optimization can be performed based on the affected test cases to ensure the efficiency of the adaptive evolution operation.

[0062] In this example, the selection of a large language model and the determination of a fine-tuning strategy influence the accuracy of impact analysis of code differences. Specifically, the choice of a basic large model (such as the GPT series, CodeLlama, or other general-purpose or specialized models), the fine-tuning strategy (such as full-parameter fine-tuning or efficient fine-tuning methods like LoRA), and the quality and size of the fine-tuning dataset (including code changes, impact analysis, and API documentation) directly impact the accuracy of the difference analysis.

[0063] During fine-tuning of the impact analysis language model, adaptive adjustments are required to the code context window size, change granularity, and impact confidence threshold to ensure accurate impact analysis of code differences. The code context window size refers to the impact analysis language model's analysis of code differences. This consideration must be made to the scope of the code context corresponding to the code differences. Filtering may fail to capture cross-file dependencies, while an overly large window may increase computational overhead and introduce noise. The impact confidence threshold is the confidence score the impact analysis language model uses to determine whether a code change impacts a specific code function or its corresponding existing test cases. Analysis results below this impact confidence threshold require manual review or more conservative processing. Change granularity refers to the level of difference analysis, such as function, class, and file levels. Different granularities affect the accuracy and efficiency of the analysis.

[0064] In this embodiment, a pre-trained impact analysis language model is used to conduct a deep semantic understanding of code differences, code context, and project knowledge base to determine at least one code change item among interface changes, parameter changes, logic changes, and dependency changes; then, based on the above code change items, the code functions that may be affected by the code differences are determined, and they are determined as affecting code functions. Based on the code differences and their corresponding affecting code functions, a difference impact report is formed, so that adaptive evolution operations such as repairing, generating, and optimizing existing test cases can be implemented based on the difference impact report to ensure the efficiency of the adaptive evolution operation.

[0065] In one embodiment, step S3, i.e., performing testing based on the new version code and existing test cases to determine target failure cases, includes: S31: Testing is performed based on the new version code and existing test cases to determine the original failed test cases, where the original failed test cases are the existing test cases that failed to execute; S32: Extract features of the original failure case to determine the failure features corresponding to the original failure case; S33: Based on the failure characteristics corresponding to the original failure case and the difference impact report corresponding to the code difference, determine the target failure case, where the target failure case is the original failure case whose failure characteristics match the difference impact report.

[0066] As an example, in step S31, the system performs testing based on the new version of the code and all existing test cases in the test case library. Specifically, in a CI environment, the system tests the new version of the code and all existing test cases in the test case library, identifies existing test cases that have failed to execute, and identifies them as original failed test cases. That is, in an IC environment, the system can perform testing based on the new version of the code and existing test cases (including pre-defined specific test scenarios, steps, inputs, and expected outputs), and determine the test results. The test results can be either valid or failed. If the test result of any existing test case is failure, the corresponding existing test case is identified as the original failed test case.

[0067] As an example, in step S32, after determining the original failure use case, the system may perform feature extraction on the original failure use case. Specifically, the failure feature corresponding to the original failure use case may be determined based on pre-set feature extraction rules. In this example, the failure feature of the original failure use case includes at least one of the failure context, associated historical changes, and tested code references. The failure context here refers to the code module and its context related to the execution failure in the original failure use case, including but not limited to the error type of the failure (such as assertion failure, runtime error), error message stack, input parameters and environment status at the time of failure. Associated historical changes refer to the code change items associated with the original failure use case in the historical version code, which are used to indicate which code change items the original failure use case is associated with in the historical version code. Tested code references refer to the code content directly or indirectly referenced in the original failure use case, including but not limited to identifiers of code modules, functions, and interfaces.

[0068] As an example, in step S33, the system may match the failure characteristics corresponding to the original failure use case with the difference impact report corresponding to the code difference, and analyze whether the failure characteristics of the original failure use case match the change impact range recorded in the difference impact report. If the failure characteristics match the change impact range, the original failure use case is determined to be the target failure use case; if the failure characteristics do not match the change impact range, the original failure use case is determined not to be the target failure use case.

[0069] In this embodiment, testing is first performed based on the new version code and all existing test cases, and the existing test cases with execution failures are determined as original failure cases; feature extraction is performed on the original failure cases, and the failure features that cause execution failures in the original failure cases are quickly and accurately located; matching processing is performed based on the failure features and the difference impact report, so that the data processing volume of the matching processing process is less and more targeted, and the target failure cases caused by code differences can be quickly and accurately located.

[0070] In one embodiment, step S33, i.e., determining a target failure case based on the failure characteristics corresponding to the original failure case and the difference impact report corresponding to the code difference, includes: S331: Match the original failure use case with the details of the code difference to determine the relevant failure use case, where the relevant failure use case is the original failure use case that successfully matches the details of the code difference. S332: Perform semantic matching on the failure characteristics of the relevant failure use cases and the difference impact reports corresponding to the code differences to determine the semantic matching degree, and determine the relevant failure use cases with a semantic matching degree greater than a preset matching degree as target failure use cases.

[0071] The preset match degree is a pre-set threshold used to evaluate whether the semantic match degree meets a certain threshold. For example, the preset match degree can be set to 0.7 or another constant. The semantic match degree is the degree of semantic matching determined by semantically matching the failure characteristics of the relevant failure cases with the difference impact report of the code difference.

[0072] As an example, in step S331, the system can interpret the original failure use case, extract detailed information such as the code location, function name, and file path corresponding to the tested code reference from the original failure use case, and match it with the corresponding detailed information in the code difference. Specifically, the matching can be performed based on a rule matching algorithm. If the two match successfully, the successfully matched original failure use case will be determined as the relevant failure use case.

[0073] As an example, in step S332, the system may perform a semantic match between the failure signature of the relevant failure case and the difference impact report corresponding to the code difference based on a natural language processing algorithm (e.g., NLP), and determine the semantic match between the two. This semantic match can represent the degree of match between the failure signature and all code changes in the code difference. This match can be a matching score based on different matching criteria (semantic similarity and / or code reference matching). This semantic match is then compared with a preset match. If the semantic match is greater than the preset match, it can be determined that the failure signature has a high correlation with the code difference, indicating a high probability of an execution failure caused by the code difference. Therefore, the relevant failure case can be identified as the target failure case.

[0074] In this embodiment, detailed information matching is first performed on the original failure cases and code differences, and relevant failure cases that may cause execution failures due to code differences are preliminarily screened out from all original failure cases. A deeper semantic matching is then performed based on the failure characteristics of the relevant failure cases and the difference impact reports of the code differences to determine the target failure cases with successful semantic matching. Since the relevant failure cases are the original failure cases that have been preliminarily screened out, the number of relevant failure cases is smaller than the number of original failure cases. Compared with directly matching the failure characteristics of the original failure cases with the difference impact reports, this method can save data processing and help improve the processing efficiency of semantic matching. The failure characteristics are features related to execution failures extracted from the test cases. The semantic matching is performed based on the failure characteristics of the relevant failure cases and the difference impact reports of the code differences, making the semantic matching process more targeted and helping to improve processing efficiency and accuracy.

[0075] In one embodiment, step S4, i.e., using the use case repair language model to semantically repair the target failure use case and determine the target repair use case, includes: S41: Perform semantic repair on the target failure use case using a use case repair language model, and determine a semantic repair item, where the semantic repair item includes at least one of a request parameter, an assertion statement, and a business process; S42: Determine the original repair use case based on the target failure use case and the semantic repair item; S43: Verify the original repair use case and determine the target repair use case, which is the original repair use case that has passed verification.

[0076] Among them, the semantic repair item is used to represent the specific repair content of this semantic repair.

[0077] As an example, in step S41, the system may use a use case repair language model to semantically repair the target failure case. Specifically, the system processes the failure characteristics of the target failure case (including at least one of the failure context, associated historical changes, and references to the tested code) and determines a semantic repair item corresponding to the target failure case. The semantic repair item includes at least one of a request parameter, an assertion statement, and a business process. For example, if, based on the failure characteristics of the target failure case, the request parameters of the target failure case are determined to have changed (i.e., the request parameters in the target failure case do not match the interface contract of the new version of the code), the use case repair language model will adjust the corresponding request parameters to match the interface contract of the new version of the code and determine the adjusted request parameters as the repaired request parameters. If, based on the failure characteristics of the target failure case, the structure or semantics of the interface return value of the target failure case are determined to have changed (i.e., the interface return value of the target failure case does not match the assertion statement of the new version of the code), the use case repair language model will adaptively update the assertion statement of the target failure case to match the assertion statement of the new version of the code and conform to the expected output or state of the new version of the code, and determine the adjusted assertion statement as the repaired assertion statement. Based on the failure characteristics of the target failure use case, the business process of the target failure use case is determined to have changed. The use case repair language model will then adaptively adjust the test steps or test data in the target failure use case to adapt to the business process in the new version of the code. The business process will be repaired based on the adjusted test steps or test data.

[0078] As an example, in step S42, after determining at least one semantic repair item corresponding to the target failure case, the system may fuse the unrepaired content (i.e., content other than the failure feature that does not require repair) in the target failure case with the at least one semantic repair item to determine one or more original repair cases. These original repair cases can be understood as new test cases formed by fusing the unrepaired content in the target failure case with the semantic repair item.

[0079] As an example, in step S43, after determining the original repair use case, the system can verify the original repair use case based on the preset verification rules to determine the verification result. When the verification result is that the verification fails, the process of updating and determining the original repair use case is repeated; when the verification result is that the verification passes, the original repair use case can be determined as the target repair use case. In this example, the original repair use case is verified, including but not limited to compilation checks and syntax checks, etc. The verification process can be automatically verified based on preset verification rules, which helps to ensure verification efficiency. Alternatively, for some original repair use cases that have been determined through complex changes, the large model is allowed to interact with humans, such as providing repair suggestions, which are confirmed or modified by humans to determine the final target repair model.

[0080] In this example, the test case adaptive evolution method also includes: when the verification result corresponding to the original repair use case is verification failure, the number of failures is updated; when the number of failures is less than the preset number, the use case repair language model is repeatedly used to perform semantic repair on the target failure use case to determine the semantic repair items; when the number of failures is not less than the preset number, the target failure use case is marked as a test case requiring manual intervention, so that subsequent manual intervention and adjustment can be performed, or a simpler rollback can be adopted, or a rollback strategy can be executed on the target failure use case.

[0081] In this embodiment, the use case repair language model is used to intelligently repair the target failed use case, which can significantly reduce the workload of manually updating and maintaining test cases due to version iterations, and improve the efficiency of test case updates and maintenance. Compared with the traditional automated testing process, even small changes in the UI or API may cause a large number of test cases to fail, requiring manual investigation, updating, and maintenance one by one, which is a time-consuming and error-prone method. In this example, the use case repair language model is used to intelligently repair the target failed use case, which can significantly reduce the update / maintenance cost of the test case, has high repair efficiency, and is less prone to errors. In addition, the original repair use case directly generated by the use case repair language model needs to be verified to ensure the validity of the target repair use case finally obtained.

[0082] In one embodiment, after step S2, that is, after determining the code difference and the difference impact report corresponding to the code difference based on the new version code and the old version code, the method further includes: S51: Based on the code differences, determine the code context and project knowledge base corresponding to the code differences; S52: Use the use case generation language model to process the code differences, code context and project knowledge base to determine the original generated use case; S53: Verify the original generated use case and determine the target generated use case, which is the original generated use case that has passed the verification.

[0083] Among them, steps S51-S53 are a specific implementation of step S5.

[0084] As an example, in step S51, after determining the code differences between the new version code and the old version code, the system may determine, based on the code differences, the code context corresponding to the code differences from the new and old versions of the code. The code context herein includes, but is not limited to, the class in which the changed function resides, other functions called, and interface definitions. The system may also query the system database based on the project identifier corresponding to the new version code to determine the project knowledge base corresponding to the project identifier. The project knowledge base includes, but is not limited to, historical defect records, API specifications, requirements documents, and other knowledge bases related to the development of a particular project code.

[0085] As an example, in step S52, the system can use the use case generation language model to process the code changes contained in the new version of the code (e.g., new code paths and modified logic branches in the new code), combined with the code context and project knowledge base, in the code differences. Based on the processing results, the system determines the original generated use case. The original generated use case can be understood as a test case directly generated by the use case generation language model after deep semantic learning of the code differences, code context, and project knowledge base, covering the new code paths, modified logic branches, new boundary conditions, and new exception scenarios.

[0086] As an example, in step S53, after determining the original generated use case, the system can verify the original generated use case based on pre-set verification rules and determine the verification result; when the verification result is verification failure, a prompt message is output so that the user can adjust and update the original generated use case based on the prompt message and repeat the verification; when the verification result is verification success, the original generated use case is determined as the target generated use case.

[0087] In this embodiment, the powerful learning ability of the use case generation language model is utilized to conduct deep semantic learning on code differences, code context, project knowledge base, etc., to generate more targeted and comprehensive original generation use cases to ensure that the new version of the code can be fully tested; the original repair use cases are then verified to ensure the effectiveness of the target repair use cases finally obtained.

[0088] In one embodiment, step S6, i.e., using an optimization agent to intelligently optimize the first test case to obtain a second test case that meets a preset optimization condition, includes: S61: Using multiple optimization agents to perform single verification on the first test case respectively, and determining a first verification result corresponding to the first test case, where the first verification result is determined based on the multiple single verification results, and the single verification result is a result of single verification of the first test case by a single optimization agent; S62: Using multiple optimized agents to challenge and verify the first test case, and determining a second verification result corresponding to the first test case, where the second verification result is determined based on the multiple challenge verification results, and the challenge verification result is a result of the two optimized agents challenging and verifying the first test case respectively; S63: When the first verification result and the second verification result corresponding to the first test case meet a preset optimization condition, the first test case is determined as the second test case.

[0089] As an example, in step S61, the system constructs multiple optimization agents based on a large model, each of which has the ability to understand test requirements, analyze code, generate test cases, and evaluate test cases. It can distribute target repair cases, target generation cases, or existing test cases as first test cases to multiple optimization agents, so that multiple optimization agents can perform single verification on the first test case respectively, and determine the single verification result output by each optimization agent respectively; then the single verification results of the multiple optimization agents are fused to determine the first verification result corresponding to each first test case. Single verification refers to the process in which each optimization agent verifies the first test case separately based on the built-in verification rules, without the need for interactive verification with other optimization agents. In this example, each optimization agent can perform single verification on the effectiveness, code coverage, redundancy and other performance of the first test case to determine its corresponding single verification result.

[0090] As an example, in step S62, the system can control any two optimization agents to challenge and verify the same first test case, so that any two optimization agents can challenge and verify each other through preset rules or learned strategies. Specifically, any optimization agent is first used to challenge the first test case to determine the test case after the challenge, and then another optimization agent is used to try to "crack" the test case after the challenge to find the scenarios or defects that the test case after the challenge fails to cover, so as to determine the corresponding challenge verification result. Alternatively, when any optimization agent is first used to question the validity or other performance of the first test case, the code module corresponding to the validity or other performance in the first test case can be adaptively adjusted to determine the test case after the challenge, and then another optimization agent is used to try to "crack" the test case after the challenge, so as to determine the challenge verification result corresponding to the first test case. In this example, the challenge verification between any two optimization agents can be performed through simulation execution, symbolic execution, logical reasoning, etc. By having any two optimization agents challenge the first test case, the first test case can be analyzed and verified more deeply, for example, its code coverage, comparison results with other similar test cases, reference code specifications, etc., to ensure that the challenge verification results can effectively reflect the overall performance of the first test case.

[0091] As an example, in step S63, the system may process the first verification result and the second verification result corresponding to the first test case based on a preset consensus mechanism to determine a comprehensive evaluation result of the first test case, and select the first test case whose comprehensive evaluation result satisfies the preset optimization conditions. For example, the first verification result may be voted, weighted averaged, or a arbitrator agent may be introduced to perform a comprehensive evaluation to determine the comprehensive verification result of the first test case. The comprehensive verification result integrates the results of a single verification of the first test case by multiple optimization agents, as well as the results of mutual challenge and verification of the first test case by any two optimization agents, so that it can more effectively reflect the overall performance of the first test case.

[0092] In this embodiment, multiple optimization agents are used to intelligently optimize the first test case. By defining the roles of different optimization agents (such as generator, executor, evaluator, and challenger) and establishing rules for collaboration and competition among these agents, dynamic optimization of all first test cases is achieved. This multi-agent collaborative optimization mechanism surpasses traditional single-objective optimization algorithms, enabling comprehensive evaluation and optimization of first test cases across multiple dimensions (such as coverage, efficiency, and robustness). This approach more closely resembles the collaborative decision-making process of human test experts, ensuring efficient test case generation and testing, and making it more widely applicable in test case optimization scenarios.

[0093] In this example, multiple optimizing agents are used to perform a single verification of the first test case, and two optimizing agents challenge and verify each other. This process is the core mechanism for building continuous test case optimization. Through the collaboration and competition of multiple optimizing agents, the quality, coverage, and efficiency of the test case library are continuously improved. Without this optimization process, the system may simply generate and repair cases, but fail to achieve the overall evolution of the test case library. Furthermore, more complex agent behavior models can be introduced into the swarm intelligence optimization process. For example, agents can not only evaluate and challenge cases, but also actively learn new testing strategies or patterns from the knowledge base, or predict which types of test cases are more likely to discover defects based on historical data. This can guide the generation and screening of test cases, further improving the system's intelligence level, adaptability, and ultimately the user experience, enabling it to more effectively cope with complex and changing software development scenarios.

[0094] In one embodiment, the optimization agent includes a challenge agent and a verification agent, wherein the challenge agent is an agent for implementing use case challenge processing, and the verification agent is an agent for implementing use case verification; Step S62, i.e., using multiple optimized agents to perform challenge verification on the first test case and determining a second verification result corresponding to the first test case, includes: S621: Using a challenge agent to perform challenge processing on the first test case, and determining a variant test case corresponding to the first test case; S622: Using a verification agent to perform verification processing on the variant test case corresponding to the first test case, and determining a challenge verification result of the first test case; S623: Determine a second verification result corresponding to the first test case based on the challenge verification result corresponding to the first test case.

[0095] When the system constructs multiple optimization agents, the multiple optimization agents can be divided into N challenge agents and M verification agents. The challenge agent is mainly used to challenge the first test case to determine its corresponding variant test case. The verification agent is mainly used to verify the variant test case to determine its corresponding challenge verification result.

[0096] As an example, in step S621, the system may use N challenge agents to challenge the first test case respectively to determine N variant test cases corresponding to the first test case. The variant test case can be a test case generated by adjusting the first test case based on the built-in mutation rules of each challenge agent. The variant test case includes a challenged test case and a questioned test case. The challenged test case is a test case generated based on the first test case and contains marginal or abnormal test inputs. The challenged test case is mainly used to ensure the robustness of the first test case. The questioned test case is a test case formed by adjusting the assertion statement of the first test case, that is, when the challenge agent analyzes the first test case to evaluate whether its assertion statement is too loose or strict, a test case is formed by adaptive adjustment. The questioned test case is mainly used to ensure the validity of the first test case.

[0097] As an example, in step S622, the system may use M verification agents to analyze and verify the N variant test cases corresponding to the same first test case. Specifically, it may use code coverage, comparative analysis with other similar test cases, etc. to evaluate whether scenarios or defects that are not covered by the variant test case are found, thereby determining the N*M challenge verification results corresponding to the first test case.

[0098] As an example, in step S623, after the system has verified the N*M challenge results corresponding to the first test case, it may process the N*M challenge verification results based on a pre-set consensus mechanism to determine a second verification result corresponding to the first test case. For example, the N*M challenge verification results may be voted on, weighted averaged, or comprehensively evaluated by an arbitration agent to determine the second verification result corresponding to the first test case.

[0099] In this embodiment, a challenge agent is used to challenge the first test case to determine a variant test case, so as to ensure the robustness and effectiveness of the first test case based on the variant test case; then, a verification agent is used to verify the variant test case, and through collaboration and competition among multiple agents, a second verification result corresponding to the first test case is determined, so as to achieve dynamic optimization of the first test case based on the second verification result.

[0100] In one embodiment, step S7, i.e., using a decision agent to make intelligent decisions on all second test cases to determine target test cases that meet preset decision conditions, includes: S71: Using a decision agent to comprehensively evaluate all second test cases to determine decision evaluation values ​​corresponding to all second test cases; S72: Determine multiple target test cases based on the decision evaluation values ​​corresponding to all second test cases, and the decision evaluation values ​​corresponding to the multiple target test cases meet preset decision conditions.

[0101] Among them, the preset decision conditions are pre-set decision conditions, specifically the conditions used to select which test cases to test the new version of the code. Each decision condition includes a decision indicator and the indicator threshold corresponding to the decision indicator, including but not limited to priority, redundancy, code coverage, execution time, etc.

[0102] As an example, in step S71, the system may employ a decision-making agent to comprehensively evaluate each of the second test cases, thereby evaluating the measured values ​​of multiple decision indicators corresponding to each second test case from multiple dimensions. For example, when the decision indicators include priority, use case redundancy, code coverage, and execution time, the decision-making agent may need to analyze the measured value of the priority of each second test case, the use case redundancy between each second test case and other test cases, the code coverage between each second test case and other test cases, and the execution time of each second test case.

[0103] As an example, in step S72, after determining the decision evaluation values ​​corresponding to all second test cases, the system may conduct a comprehensive evaluation of the decision evaluation values ​​corresponding to all second test cases to determine multiple target test cases from all second test cases, and the decision evaluation values ​​corresponding to the multiple target test cases must meet the preset decision conditions. In other words, the system needs to determine the multiple second test cases that meet the preset decision conditions as multiple target test cases to ensure the overall quality of the target test cases finally determined. In this example, the decision agent can make intelligent decisions based on the decision evaluation values ​​of the second test cases in the analysis, and can generate new valid test cases, modify and improve existing test cases, merge test cases with similar functions to reduce redundancy, eliminate test cases that have been proven to be invalid or redundant, and prioritize the cases according to risk assessment, etc., to determine the target test cases that will eventually be used for testing.

[0104] In this embodiment, the decision-making agent conducts a comprehensive evaluation of the second test cases, and determines the multiple second test cases that meet the preset decision conditions as target test cases by analyzing whether the decision evaluation values ​​corresponding to the multiple second test cases meet the preset decision conditions. The preset decision conditions can be set independently according to user needs, and they can reflect effectiveness, robustness or other performance to ensure the overall quality of the target test cases.

[0105] In one embodiment, the decision evaluation value includes the use case redundancy between any two second test cases, the use case coverage corresponding to each second test case, and the use case execution time; Step S72, i.e., determining multiple target test cases based on the decision evaluation values ​​corresponding to all second test cases, wherein the decision evaluation values ​​corresponding to the multiple target test cases satisfy the preset decision conditions, includes: S721: Determine a plurality of test case combinations, where the test case combinations include a plurality of second test cases, and a case redundancy between any two second test cases is less than a preset redundancy.

[0106] S722: Determine a combined coverage and a combined execution time corresponding to each test case combination, where the combined coverage is determined based on the use case coverages corresponding to the plurality of second test cases, and the combined execution time is determined based on the use case execution times corresponding to the plurality of second test cases; S723: Determine multiple second test cases in the test case combination with the smallest combined execution time among all test case combinations whose combined coverage is greater than the preset coverage as multiple target test cases.

[0107] Among them, use case redundancy refers to the redundancy between two test cases. Preset redundancy is the threshold used to evaluate whether the redundancy between two test cases meets the redundancy standard. Use case coverage refers to the code coverage of each test case. Combined coverage is the code coverage determined based on all use case coverage corresponding to the same test case combination, which can be the union of multiple use case coverages. Preset coverage is the preset expected coverage. Use case execution time refers to the execution time of each test case. Combined execution time is the execution time determined based on all use case execution times corresponding to the same test case combination. When all test cases in the same test case combination are executed serially, the combined execution time is the sum of all use case execution times.

[0108] As an example, in step S721, when a decision-making intelligent agent is used to perform a comprehensive evaluation of all second test cases, it is necessary to determine the use case redundancy between any two second test cases; if the use case redundancy between any two second test cases is less than the preset redundancy, it means that the use case redundancy between any two second test cases is small, and their code functions are unrelated, and the two second test cases can be determined in the same test case combination; when the use case redundancy between any two second test cases is not less than the preset redundancy, it means that the use case redundancy between any two second test cases is large, and it is necessary to delete one of the second test cases, or merge the two second test cases so that only one test case is retained in the same test case combination.

[0109] In this example, the system can perform a similarity analysis on the code functions corresponding to any two second test cases to determine their functional similarity; when the functional similarity is greater than the preset similarity, the code functions of the two second test cases are determined to be relatively similar, and they can be classified into the same functional use case set; when the functional similarity is not greater than the preset similarity, the code functions of the two second test cases are determined to be dissimilar, and the two second test cases can be respectively classified into different functional use case sets; after determining multiple functional use case sets, since the code functions of any two functional use case sets are different, the case redundancy of the test cases in any two functional use case sets is small, therefore, one second test case can be selected from each of all functional use case sets, and the test case combination can be determined based on all the selected second test cases.

[0110] As an example, in step S722, after determining multiple test case combinations, the system may process the use case coverage corresponding to multiple second test cases in the test case combination to determine the combination coverage corresponding to the test case combination by taking the union of the multiple use case coverages; and process the use case execution time corresponding to multiple second test cases in the test case combination to determine the combination execution time.

[0111] As an example, in step S723, the system may compare the combined coverage corresponding to multiple test case combinations with the preset coverage, first screen out the test case combinations whose combined coverage is greater than the preset coverage, and determine them as the initially screened test case combinations; then compare the combined execution times corresponding to the initially screened test case combinations, and determine the test case combination with the shortest combined execution time as the finally screened test case combination; and determine the multiple second test cases corresponding to the finally screened test case combination as the target test cases.

[0112] In this embodiment, a comprehensive evaluation is performed based on the use case redundancy of any two second test cases, the use case coverage of each second test case, and the use case execution time, so that there are no two or more use cases that are redundant with each other in the multiple target test cases determined. Moreover, the combined coverage of the multiple target test cases meets expectations, and the combined execution time is minimized, which can ensure the comprehensiveness of the new version code test and improve the test efficiency.

[0113] In one embodiment, step S8, i.e., performing a test based on the target test case and the new version code to obtain the test result of the test case, includes: S81: Use multiple decision-making agents to analyze the code defects of the target test case and determine the execution risk of the target test case; S82: Based on the order of use case execution risk from high to low, test the target test case and the new version code in turn to obtain the use case test results.

[0114] As an example, after the system determines multiple target test cases based on existing test cases, target repair cases, and target generation cases, it further analyzes the execution risk of each target test case. This execution risk characterizes the risk level of defects in the target test case. Next, based on the order of execution risk from high to low, the system performs tests on the target test cases and the new version of the code, obtaining test results. By prioritizing the execution risk of the test cases, the system can quickly identify high-risk critical defects in the new version of the code, shortening the feedback cycle for critical defects and enabling developers to quickly correct them.

[0115] Figure 2 The following is a test case adaptive evolution method in an example, which specifically includes the following stages: The starting stage specifically includes: obtaining a target code request, which can be a code submission request or a code merge request.

[0116] The version difference perception stage specifically includes: determining the new version code and the old version code in response to the target code request; determining the code difference based on the new version code and the old version code; using the impact analysis language model to perform semantic-level impact analysis on the code difference, and determining a difference impact report based on its specific impact.

[0117] The adaptive test case evolution phase specifically involves identifying target failure cases and their failure characteristics, using a use case repair language model to semantically repair these failure cases and their failure characteristics, and determining target repair cases. Furthermore, a use case generation language model can be used to generate code differences. For newly added code paths, the large language model can be used to intelligently supplement new use cases to form target generation use cases.

[0118] Swarm intelligence optimization stage: Use multiple intelligent agents to evaluate and challenge the first test case (including but not limited to target repair case and target generation case), determine the second test case based on the mutual challenge and verification results, make intelligent decisions on the second test case, determine the target test case, optimize and integrate the test case set, and output the optimized and integrated test case set; test and execute all target test cases in the optimized and integrated test case set, collect their use case test results, and feedback the use case test results; update the knowledge base and iterate the model based on the feedback use case test results (i.e., update the impact analysis language model, use case repair language model, and use case generation language model).

[0119] The above test case adaptive evolution method has the following advantages: (1) Significantly reduce the maintenance cost of test cases: By using the use case repair language model to intelligently repair the target failed use cases, the workload of manually updating and maintaining test cases due to version iteration can be greatly reduced, thereby improving the efficiency of test case updates and maintenance. Compared with the traditional automated testing process, a small change in the UI or API may cause a large number of test cases to fail, requiring manual investigation, updating, and maintenance one by one, which is time-consuming and prone to errors. In this example, the use case repair language model is used to intelligently repair the target failed use cases, which can significantly reduce the update / maintenance cost of test cases, has high repair efficiency, and is less prone to errors. The present invention can automatically adapt to these changes through semantic-level repair, thereby saving a lot of manpower costs.

[0120] (2) Improving the quality and coverage of test cases: Leveraging the powerful generation capabilities of the large model and its understanding of code semantics, the system is able to generate more targeted and comprehensive target generation test cases, including those for boundary conditions, abnormal scenarios, and new code paths. The swarm intelligence optimization mechanism further screens and refines test cases through challenges and verification by multiple agents, eliminating invalid and redundant cases, thereby improving the quality and code coverage of the overall test case set.

[0121] (3) Improve testing efficiency and feedback speed: In a continuous integration environment, fast feedback is crucial. By automatically generating, fixing, and optimizing test cases, the system can quickly respond to code submissions and execute an efficient set of test cases. The swarm intelligence optimization mechanism can also prioritize test cases, giving priority to high-risk target test cases, thereby discovering critical defects faster and shortening the feedback cycle.

[0122] (4) Enhanced testing intelligence and adaptability: The system is no longer a tool that passively executes preset scripts. Instead, it is an intelligent testing partner with learning and evolutionary capabilities. It can understand the semantic impact of code changes, automatically adjust test strategies, and continuously optimize test assets. This adaptability makes the testing process more adaptable to the rapid iteration and complex and changing needs of modern software.

[0123] (5) Promote overall improvement of software quality: Through more efficient, comprehensive, and intelligent testing, this invention helps to discover and fix more defects in the early stages of development, reducing the probability of defects leaking into the production environment, thereby improving the quality and reliability of the final software product. At the same time, it frees testers from tedious and repetitive work, allowing them to focus more on higher-value testing activities such as exploratory testing and scenario testing.

[0124] (6) Improved collaboration between development and testing: Automated test case evolution and timely test feedback help developers more quickly understand the impact of their code changes on the system and quickly locate and fix problems, thereby promoting smoother collaboration between development and testing teams and accelerating the software delivery process.

[0125] The embodiment of the present application also provides an electronic device 30, such as Figure 3 As shown, it includes a memory 31 and a processor 32, wherein the memory 31 is used to store computer programs; the processor 32 is used to execute the programs stored in the memory 31 to implement the test case adaptive evolution method introduced in any embodiment of the present application.

[0126] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method A introduced in any embodiment of the present application is implemented.

[0127] In this application, a plurality refers to two or more.

[0128] The terms "first," "second," "third," "fourth," etc. (if any) in this application are used to distinguish similar objects and are not necessarily used to describe a particular sequential order.

[0129] The term "and / or" in this application simply describes an association between related objects, indicating that three possible relationships exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application generally indicates that the related objects are in an "or" relationship.

[0130] Unless otherwise specified, all steps of this application may be performed sequentially or randomly. For example, "the method includes steps A and B" means that the method may include steps A and B performed sequentially, or may include steps B and A performed sequentially. For example, "the method may also include step C" means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or steps A, C, and B, or steps C, A, and B, etc.

[0131] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A test case adaptive evolution method, characterized in that: include: In response to the object code request, determining a new version code and an old version code; Determining, based on the new version code and the old version code, a code difference and a difference impact report corresponding to the code difference; Performing tests based on the new version code and existing test cases to determine target failure cases, where the target failure cases are existing test cases that fail to execute and are related to the difference impact report; A use case repair language model is used to perform semantic repair on the target failure case to determine a target repair case. The use case repair language model is a large language model used to perform semantic repair on the test case.

2. The method according to claim 1, characterized in that The determining, based on the new version code and the old version code, a code difference and a difference impact report corresponding to the code difference, includes: Determining code differences based on the new version code and the old version code, and determining a code context and a project knowledge base corresponding to the code differences; Using an impact analysis language model to perform semantic understanding on the code differences, the code context, and the project knowledge base, and determine code change items, where the code change items include at least one of an interface change, a parameter change, a logic change, and a dependency change; Determining an affected code function based on the code change item, where the affected code function is a code function affected by the code change item; Determining a difference impact report corresponding to the code difference based on the code difference and the affected code function; The impact analysis language model is a large language model used to perform impact analysis on code differences.

3. The method according to claim 1, characterized in that The testing based on the new version code and existing test cases to determine target failure cases includes: Performing tests based on the new version code and existing test cases to determine original failure cases, where the original failure cases are existing test cases that fail to execute; Performing feature extraction on the original failure case to determine a failure feature corresponding to the original failure case; Based on the failure characteristics corresponding to the original failure case and the difference impact report corresponding to the code difference, a target failure case is determined, where the target failure case is the original failure case whose failure characteristics match the difference impact report.

4. The method according to claim 1, characterized in that The using the use case repair language model to perform semantic repair on the target failure use case and determine the target repair use case includes: Performing semantic repair on the target failure use case using a use case repair language model to determine a semantic repair item, where the semantic repair item includes at least one of a request parameter, an assertion statement, and a business process; Determining an original repair use case based on the target failure use case and the semantic repair item; The original repair use case is verified to determine a target repair use case, where the target repair use case is the original repair use case that has passed verification.

5. The method according to claim 1, characterized in that: After determining a code difference and a difference impact report corresponding to the code difference based on the new version code and the old version code, the method further includes: Based on the code differences, determining a code context and a project knowledge base corresponding to the code differences; Using a use case generation language model to process the code differences, the code context, and the project knowledge base to determine an original generation use case; The original generation use case is verified to determine a target generation use case, where the target generation use case is the original generation use case that has passed the verification.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Intelligently optimizing a first test case using an optimization agent to obtain a second test case that meets preset optimization conditions, wherein the optimization agent is an agent for implementing intelligent optimization, and the first test case is a target repair case, a target generation case, or an existing test case; A decision-making agent is used to make intelligent decisions on all the second test cases to determine target test cases that meet preset decision conditions. The decision-making agent is an agent used to implement intelligent decision-making.

7. The method according to claim 6, characterized in that The intelligent optimization of the first test case by using an optimization agent to obtain a second test case that meets a preset optimization condition includes: Using multiple optimization agents to perform single verification on the first test case respectively, and determining a first verification result corresponding to the first test case, wherein the first verification result is determined based on the multiple single verification results, and the single verification result is a result of single verification of the first test case by a single optimization agent; Using multiple optimized agents to challenge and verify the first test case, and determining a second verification result corresponding to the first test case, wherein the second verification result is determined based on the multiple challenge verification results, and the challenge verification result is the result of two of the optimized agents challenging and verifying the first test case respectively; When the first verification result and the second verification result corresponding to the first test case meet a preset optimization condition, the first test case is determined as the second test case.

8. The method according to claim 6, characterized in that The step of using a decision agent to make intelligent decisions on all the second test cases to determine target test cases that meet preset decision conditions includes: Using a decision agent to comprehensively evaluate all of the second test cases to determine decision evaluation values ​​corresponding to all of the second test cases; Based on the decision evaluation values ​​corresponding to all the second test cases, multiple target test cases are determined, and the decision evaluation values ​​corresponding to the multiple target test cases meet the preset decision conditions.

9. An electronic device, characterized in that: comprising a processor and a memory, wherein, Memory for storing computer programs; A processor is used to execute a program stored in a memory to implement the test case adaptive evolution method described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the test case adaptive evolution method according to any one of claims 1 to 8 is implemented.

Images