Software testing method and system based on large language model and recurrent reinforcement prompt

Abstract

The application relates to the field of software testing, and provides a software testing method and system based on a large language model and a cycle-enhanced prompt, which comprises the following steps: generating program test input according to input generation constraints, and generating a test program according to program generation constraints; obtaining a running result, calculating program credibility weight through the running result, and selecting a weighted main candidate output according to the program credibility weight; obtaining a reference program output from the running result, obtaining a verification result level of the reference program output, obtaining a deviation mode through the weighted main candidate output, obtaining a candidate defect cluster through the verification result level and the deviation mode, and calculating defect confidence of the candidate defect cluster; obtaining program coverage and exception times according to the running result, calculating input scores, calculating program evaluation parameters, and calculating program quality scores through the program evaluation parameters; and updating the program test input and the test program to obtain target test input, so that the application can be used for completing software testing.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a software testing method and system based on large language models and loop enhancement prompts. Background Technology

[0002] As software systems become increasingly larger and more complex, software testing has become a crucial means of ensuring software quality and improving system reliability. Test cases, as the core carrier of software testing activities, directly impact defect detection capabilities, test coverage, and testing costs. Traditional software testing methods primarily rely on manually writing test cases, random testing, rule-based test generation, search-based test generation, and symbolic execution-based test generation. While these methods have some applicability in specific scenarios, they often suffer from low generation efficiency, limited coverage, and difficulty in discovering deep-seated defects when dealing with software with large input spaces, complex program logic, hidden boundary conditions, or strong semantic constraints.

[0003] In recent years, large language models have demonstrated strong capabilities in code understanding, code generation, program analysis, and automated testing, making software testing methods based on large language models a research hotspot. These methods typically utilize large language models to understand program functional descriptions, code context, or existing test cases, and further generate test inputs, test scripts, or assertions, thereby reducing the cost of manually writing test cases and improving test automation. Compared to traditional methods, large language models possess stronger semantic modeling capabilities, extracting potential test information from natural language requirements, function signatures, program logic, and existing examples, providing a new technical path for generating test cases for complex programs.

[0004] In the field of program defect detection, some existing technologies attempt to incorporate large language models into the defect detection process. For example, they use large language models to generate test cases, generate reference implementations, or assist in analyzing program output results, aiming to improve defect detection capabilities. However, directly applying large language models to program defect detection tasks still faces several technical challenges. Test case generation still has a certain degree of probability, and its output quality is significantly affected by factors such as prompt word design, context organization, model sampling strategies, and historical generation results. One method is called differential prompting. This method generates program variants and batch inputs through a prompting large language model and performs differential testing to uncover potential errors in the program. However, the test cases generated by this method are mostly quite ordinary and difficult to uncover tricky defects in software or programs that often require very extreme test cases to find. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a software testing method and system based on a large language model and loop enhancement prompts, enabling the generation of high-quality test cases.

[0006] This invention provides a software testing method based on a large language model and loop enhancement prompts, comprising: S1: Establish input generation constraints and program generation constraints, generate program test inputs based on input generation constraints, and generate test programs based on program generation constraints; S2: Obtain the running results of the program test input on the test program, calculate the program credibility weight based on the running results, select the weighted main candidate output based on the program credibility weight, and calculate the output divergence degree; S3: Obtain the reference program output from the running results, get the verification result level of the reference program output, obtain the deviation mode by weighting the main candidate output, obtain the candidate defect cluster by the verification result level and the deviation mode, and calculate the defect confidence of the candidate defect cluster. S4: Based on the running results, obtain the program coverage and number of exceptions. Calculate the input score by outputting the divergence degree, program coverage, and number of exceptions. Calculate the program evaluation parameters and then calculate the program quality score using the program evaluation parameters. S5: Update the program test input and test program based on the defect confidence level, input score and program quality score to obtain the target test input, and use the target test input to complete the software test.

[0007] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S1, when establishing input generation constraints, structured generation constraints are determined, difference-inducing information is constructed, difference-inducing items are extracted from the difference-inducing information, and the input generation constraints are obtained through the difference-inducing items and structured generation constraints.

[0008] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S2, the input-output pass rate, program stability, and proportion of supporting primary candidate behavior of the test program are obtained from the running results, and the program credibility weight is calculated by the input-output pass rate, program stability, and proportion of supporting primary candidate behavior.

[0009] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S2, a weighted support strength is obtained according to the program credibility weight, and a weighted master candidate output is obtained based on the weighted support strength.

[0010] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S3, the output of the reference program is judged to obtain a verification result level including strong consistency verification, weak consistency verification and inconsistency verification, and the output of the reference program is compared with the weighted master candidate output to obtain the deviation mode.

[0011] According to the software testing method based on a large language model and recurring enhancement prompts provided by the present invention, in step S3, the support strength, reference program verification strength, repetitive triggering degree and overall program variant confidence of the candidate defect cluster are obtained, and the defect confidence is calculated based on the support strength, reference program verification strength, repetitive triggering degree and overall program variant confidence.

[0012] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S4, a coverage score is obtained based on the program coverage, a stability score is obtained based on the number of anomalies, a sensitivity score is obtained based on the output divergence, and the input score is obtained through the coverage score, stability score, and sensitivity score.

[0013] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S4, program evaluation parameters including program stability score, primary candidate consistency, reference support and basic sample pass rate are obtained, and program quality score is calculated through program evaluation parameters.

[0014] According to the software testing method based on a large language model and loop enhancement prompts provided by the present invention, in step S5, a high defect confidence threshold and a growth threshold are determined, and a high-confidence defect cluster is determined based on the defect confidence and the high defect confidence threshold. During the update process, when the growth number of the high-confidence defect cluster is lower than the growth threshold, the update of the program test input and the test program is stopped.

[0015] This invention also provides a software testing system based on a large language model and loop enhancement prompts, comprising: Input and program generation module: used to establish input generation constraints and program generation constraints, generate program test inputs based on input generation constraints, and generate test programs based on program generation constraints; Input divergence module: used to obtain the running results of the program test input on the test program, calculate the program credibility weight based on the running results, select the weighted main candidate output based on the program credibility weight, and calculate the output divergence. Defect confidence module: used to obtain the reference program output from the running results, obtain the verification result level of the reference program output, obtain the deviation mode by weighting the main candidate output, obtain the candidate defect cluster by the verification result level and the deviation mode, and calculate the defect confidence of the candidate defect cluster; The scoring module is used to obtain program coverage and number of exceptions based on the running results. It calculates the input score by outputting the divergence degree, program coverage, and number of exceptions, calculates program evaluation parameters, and calculates the program quality score based on the program evaluation parameters. Test generation module: Used to update the program test input and test program based on defect confidence, input score and program quality score to obtain target test input, and use the target test input to complete software testing.

[0016] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: This invention provides a software testing method and system based on a large language model and loop enhancement prompts. By dynamically constructing dual generation constraints for input and program, it achieves collaborative optimization of test cases and test programs, significantly improving the automation and accuracy of the software testing process. It calculates program confidence weights using the execution results and filters weighted master candidate outputs. Combined with output divergence metrics, it quantifies the effectiveness of test inputs, enabling precise location of behavioral deviations. Simultaneously, it integrates verification levels and deviation patterns to generate candidate defect clusters and calculate defect confidence, effectively reducing false positive rates. Furthermore, it comprehensively evaluates input quality based on program coverage, anomaly count, and output divergence, and calculates program quality scores through program evaluation parameters, providing a scientific basis for the dynamic allocation of test resources. Finally, iteratively updating the test inputs and test programs based on defect confidence, input scores, and program quality scores generates target test inputs, comprehensively improving test coverage sufficiency, defect detection capabilities, and result interpretability. This reduces manual intervention costs and enhances the stability and reliability of the testing process, providing strong support for efficient quality assurance of complex software systems.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the software testing method based on a large language model and loop enhancement prompts provided by this invention.

[0020] Figure 2 This is a schematic diagram of the structure of the software testing system based on a large language model and loop enhancement prompts provided by the present invention.

[0021] Figure label: 100. Input and program generation module; 200. Input divergence module; 300. Defect confidence module; 400. Scoring module; 500. Test generation module. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0023] In the description of the embodiments of the present invention, it should be noted that the terms "first", "second" and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0024] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0025] The following is combined Figures 1 to 2 Specific embodiments of the present invention are described below. Figure 1 This is a flowchart illustrating the software testing method based on a large language model and loop enhancement prompts provided by the present invention, including: S1: Establish input generation constraints and program generation constraints, generate program test inputs based on input generation constraints, and generate test programs based on program generation constraints; Furthermore, the purpose of this stage is to generate test inputs and test the program. Specifically, in step S1, when establishing input generation constraints, structured generation constraints are determined, difference-inducing information is constructed, difference-inducing terms are extracted from the difference-inducing information, and the input generation constraints are obtained through the difference-inducing terms and structured generation constraints.

[0026] The specific implementation method for the above steps in this embodiment is as follows: Here, the first step is to establish program generation constraints. Predefined problem descriptions, input / output formats, and example input / outputs are provided as prompts for the large language model to generate these constraints. The large language model then generates the constraints based on the predefined problem descriptions, input / output formats, example input / outputs, and predefined prompts. Additionally, sample programs need to be identified to provide reference for testing the program.

[0027] In addition, input generation constraints need to be established. This requires first determining the number of rows in the input, the field types in each row, and the constraint relationships between different fields. It also requires determining the complexity of the input constraints, the strength of field dependencies, the density of boundary conditions, and the specificity of the input structure. This generates risk value labels for marking boundary and extreme value regions, resulting in structured generation constraints. Next, a prompt template is constructed that can be used by the large language model. Besides containing a static task description, the prompt template also reserves several dynamically updatable enhancement slots for writing high-quality input examples, high-quality program variants, misjudged examples, undecidable examples, and descriptions of insufficiently covered regions in subsequent loops.

[0028] Subsequently, to ensure that the generated inputs cover boundary value inputs, extreme value inputs, abnormal combination inputs, and potentially highly divergent induced inputs, several different algorithmic approach hints, data structure selection hints, control flow organization hints, boundary handling strategy hints, and input style hints need to be defined as differential inducement information. One hint is randomly selected from these differential inducement information as a differential inducement term, and this term is added to the structured generation constraints to obtain the input generation constraints. Inputting the input generation constraints and hint templates into the large language model yields multiple program test inputs. Inputting the program generation constraints, example programs, and preset hint words into the large language model generates multiple test programs.

[0029] S2: Obtain the running results of the program test input on the test program, calculate the program credibility weight based on the running results, select the weighted main candidate output based on the program credibility weight, and calculate the output divergence degree; Furthermore, the objective of this stage is to select a weighted primary candidate output and calculate the program credibility weight and output divergence. Specifically, in step S2, the input-output pass rate, program stability, and proportion of primary candidate behavior supported by the test program are obtained from the running results, and the program credibility weight is calculated using the input-output pass rate, program stability, and proportion of primary candidate behavior supported by the test program.

[0030] In step S2, the weighted support strength is obtained based on the program credibility weight, and the weighted master candidate output is obtained based on the weighted support strength.

[0031] The specific implementation method for the above steps in this embodiment is as follows: First, the test input needs to be fed into the test program to obtain the results. The results include the program's output and its running status, which can be one of the following: normal termination, error, timeout, crash, empty output, or unresolved output. Next, the results with a normal termination status are selected. Test programs with the same output are grouped into the same candidate behavior class, resulting in a candidate behavior set. For each candidate behavior class, the average running stability and program distribution within it need to be analyzed to obtain the overall support strength of the candidate behavior class. Higher average running stability and a wider program distribution indicate higher overall support strength. The candidate behavior class with the highest overall support strength is defined as the primary candidate behavior.

[0032] Subsequently, for each test program in the candidate behavior set, its output after being input to the example input is calculated. The output of the example input after being input to the example program is also used as the example output. Based on the difference between the example output and the output after being input to the example input, the input-output pass rate of the i-th test program can be obtained. The smaller the difference, the higher the input / output pass rate. Furthermore, the program stability of the i-th test program needs to be determined based on its running status. The proportion of the i-th test program that supports the primary candidate behavior is determined based on the degree of overlap between the output and the output of the primary candidate behavior. The higher the degree of overlap, the higher the proportion of the primary candidate behavior supported. This allows us to calculate the i-th test program. Program credibility weight : in, The input and output pass-through weights are preset. The preset program stability weights, The weights are the preset proportions of the primary candidate behaviors, and the sum of these weights is 1.

[0033] Subsequently, for any candidate behavior class, the sum of the program credibility weights of the test programs in the class is used as the weighted support strength, and the program output corresponding to the candidate behavior class with the highest weighted support strength is used as the weighted main candidate output.

[0034] Finally, the output divergence of the test input x needs to be calculated. : in, The total number of all candidate behavior classes. This represents the total number of candidate behavior classes obtained when the input is a specific program test input x.

[0035] S3: Obtain the reference program output from the running results, get the verification result level of the reference program output, obtain the deviation mode by weighting the main candidate output, obtain the candidate defect cluster by the verification result level and the deviation mode, and calculate the defect confidence of the candidate defect cluster. Furthermore, the objective of this stage is to calculate the defect confidence of the candidate defect clusters. Specifically, in step S3, the output of the reference program is evaluated to obtain verification result levels including strong consistency verification, weak consistency verification, and inconsistent verification. The output of the reference program is compared with the weighted master candidate output to obtain the deviation pattern.

[0036] In step S3, the support strength, reference program verification strength, repetitive triggering degree, and overall program variant confidence of the candidate defect cluster are obtained, and the defect confidence is calculated based on the support strength, reference program verification strength, repetitive triggering degree, and overall program variant confidence.

[0037] The specific implementation method for the above steps in this embodiment is as follows: Here, the test program generated in step S1 also includes an example program. All program test inputs are input into the example program to obtain the reference program output. The results are then compared with the reference program output. If the reference program outputs are completely consistent, the verification result level is strong consistency verification. If most of the reference program outputs are consistent with each other, and there are no obvious conflicts between the outputs, the verification result level is weak consistency verification. If most of the reference program outputs are inconsistent with each other, or there are obvious conflicts between the outputs, the verification result level is inconsistent verification. The reference program output corresponding to the program test input x is then compared with the weighted master candidate output. If the two are not equal, it indicates that the example program deviates from the mainstream behavior of the test program at the program test input x, which is recorded as a defect deviation. Based on the defect deviation, the deviation mode can be obtained. The deviation mode can be one of numerical deviation, format deviation, empty output deviation, abnormal termination deviation, and logical result deviation. For defect deviations, the output divergence degree can also be recorded.

[0038] Subsequently, the program test inputs that deviated from the expected defect pattern are filtered and merged to obtain candidate defect clusters. The program test inputs in a candidate defect cluster must meet at least one of the following characteristics: their validation result level is strong consistency validation; the deviation pattern of the program test inputs in the candidate defect cluster is stable and consistent; similar program test inputs can also trigger defect deviations; and the result of the reference program output has a high similarity to the weighted master candidate output. Then, the candidate defect clusters can be obtained based on the weighted support strength of the weighted master candidate output. Support strength Based on the average program confidence weight of the test programs in the weighted master candidate output, the candidate defect clusters can be obtained. Overall credibility of program variants Candidate defect clusters can be obtained based on the number of defect deviations. Repeated triggering degree Furthermore, the greater the number of defect deviations, the higher the degree of repeated triggering. Candidate defect clusters can be obtained based on the verification result level. Reference procedure verification strength This allows us to calculate candidate defect clusters. Defect confidence level : in, To support the strength weighting of the advantage, To verify the strength weights for the reference procedure, Weights for the degree of repetition. This represents the overall credibility weight of the program variant, and the sum of the weights is 1.

[0039] S4: Based on the running results, obtain the program coverage and number of exceptions. Calculate the input score by outputting the divergence degree, program coverage, and number of exceptions. Calculate the program evaluation parameters and then calculate the program quality score using the program evaluation parameters. Furthermore, the objective of this stage is to calculate the input score and the program quality score. Specifically, in step S4, a coverage score is obtained based on the program coverage, a stability score is obtained based on the number of anomalies, and a sensitivity score is obtained based on the output divergence. The input score is then obtained using the coverage score, stability score, and sensitivity score.

[0040] In step S4, program evaluation parameters including program stability score, primary candidate consistency, reference support and basic sample pass rate are obtained, and program quality score is calculated using program evaluation parameters.

[0041] The specific implementation method for the above steps in this embodiment is as follows: First, based on the execution status in the results, we can obtain the number of times a given program test input entered an abnormal state when executed on each test program. This gives us the exception count. Abnormal states include runtime errors, timeouts, crashes, empty outputs, and unresolved outputs. Then, based on the average code call rate across all test programs when a given program test input is executed, we can obtain the program coverage. Finally, based on the program coverage, we can obtain the coverage score for program test input x. : in, The code call rate of input x in the test program P can be expressed as the code call rate, which can be branch coverage. This is a collection of test programs.

[0042] It can also calculate the stability score of the program's test input x. : in, To test the number of exceptions for input x in the program, The total number of times the program test input x is run. Furthermore, different values ​​of output divergence correspond to corresponding sensitivity scores; therefore, the sensitivity score of the program test input x can also be obtained based on the output divergence. This allows us to calculate the input score for the program's test input x. : in, The defect trigger contribution score is assigned to the program test input x. When the program test input x participates in the formation of a candidate defect cluster and the defect confidence of the candidate defect cluster is high, the score is higher. As a weight for stability scoring, As the weight for coverage scoring, For sensitivity score weights, The score weights for defect triggering are calculated, and the sum of the above weights is 1.

[0043] Then, the i-th test program is statistically analyzed. The stability score of the i-th test program is calculated when the program runs in an abnormal state under test input x. : in, Let N be the set of program test inputs, and N be the number of program test inputs. For the i-th test program The runtime status parameter under the program test input x is set to 1 if the runtime status is abnormal, and 0 otherwise. Subsequently, the value of the proportion of supported primary candidate behaviors obtained in step S2 can be mapped to the primary candidate consistency degree of the i-th test program. The input / output pass rate can be mapped to the base sample pass rate of the i-th test program. The overall support strength of the candidate behavior class to which the test program belongs can be mapped to the reference support of the i-th test program. This allows us to calculate the program quality score for the i-th test program. : in, Based on the degree weight of the basic sample, As a weight for stability scoring, Weighting for consistency with the primary candidate. The sum of the above weights is 1, which is used as a reference for support weights.

[0044] S5: Update the program test input and test program based on the defect confidence level, input score and program quality score to obtain the target test input, and use the target test input to complete the software test.

[0045] Furthermore, the objective of this stage is to update the program test input and test program to obtain the target test input. Specifically, in step S5, a high defect confidence threshold and a growth threshold are determined. Based on the defect confidence and the high defect confidence threshold, high-confidence defect clusters are determined. During the update process, when the growth number of high-confidence defect clusters falls below the growth threshold, the update of the program test input and test program is stopped.

[0046] The specific implementation method for the above steps in this embodiment is as follows: First, a high-confidence threshold needs to be determined, and candidate defect clusters with confidence levels higher than this threshold are designated as high-confidence defect clusters. During subsequent updates, if the number of high-confidence defect clusters increases while the proportion of anomalous states in the results remains unchanged or decreases, the current update strategy is considered successful; otherwise, adjustments are needed. During the update process, test programs with high quality scores, test inputs with high input scores, and high-confidence defect clusters are considered positive samples. Test inputs with high output divergence and test inputs with inconsistent or weakly consistent verification results are considered negative samples. Then, using positive samples as references and negative samples as objects to be avoided, efforts are made to improve the coverage of boundary values, extreme values, and anomalous combinations by the test inputs. Positive and negative samples are input into the large language model, and steps S1 to S4 are repeated to update the test inputs and test programs. Updates to the test inputs and test programs can be stopped when the increase in the number of high-confidence defect clusters falls below a preset growth threshold. Furthermore, updates can be stopped when the output divergence of the program test inputs tends to stabilize or when there are too many abnormal states in the running results. The program test input obtained from the last update can then be used as the target test input, allowing software testing to be completed using the target test input.

[0047] Here, the effectiveness of the software testing method based on large language models and loop enhancement prompts is also verified. For pre-written test programs, the target test input is used to detect errors in the subroutines of the test programs. The detection results of this method and other existing methods on the test programs are shown in Table 1: Table 1. Schematic diagram of the detection results of this method and other methods on the test program.

[0048] Differential Prompting and Differential Prompting + Same Reference Validation are both existing test input generation methods. During the detection process, an error report is generated for each subroutine error detected by the test input. The cumulative number of detected subroutines is the number of error reports, the cumulative detection rate is the ratio of the number of error reports to the total number of errors in all subroutines of the test program, and the reporting accuracy is the proportion of error reports that are verified as true among all error reports. Both the cumulative detection rate and the reporting accuracy are percentages. It can be seen that this invention can effectively generate high-quality target test input, which can then be used to detect errors in programs.

[0049] The software testing system based on large language models and loop enhancement prompts provided by this invention will be described below. The software testing system based on large language models and loop enhancement prompts described below can be referred to in correspondence with the software testing method based on large language models and loop enhancement prompts described above.

[0050] Figure 2 A schematic diagram of the structure of a software testing system based on a large language model and loop enhancement prompts is shown, such as... Figure 2 As shown, the software testing method based on a large language model and loop enhancement prompts as described above includes: Input and program generation module 100: used to establish input generation constraints and program generation constraints, generate program test inputs based on input generation constraints, and generate test programs based on program generation constraints; Input divergence module 200: used to obtain the running results of the program test input on the test program, calculate the program credibility weight based on the running results, select the weighted main candidate output based on the program credibility weight, and calculate the output divergence. Defect confidence module 300: It is used to obtain the reference program output from the running results, obtain the verification result level of the reference program output, obtain the deviation mode by weighting the main candidate output, obtain the candidate defect cluster by the verification result level and the deviation mode, and calculate the defect confidence of the candidate defect cluster. Scoring module 400: It is used to obtain program coverage and number of exceptions based on the running results, calculate the input score by outputting divergence degree, program coverage and number of exceptions, calculate program evaluation parameters, and calculate program quality score by program evaluation parameters; Test generation module 500: It is used to update the program test input and test program based on defect confidence, input score and program quality score to obtain target test input, and use the target test input to complete software testing.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A software testing method based on a large language model and loop enhancement prompts, characterized in that, include: S1: Establish input generation constraints and program generation constraints, generate program test inputs based on input generation constraints, and generate test programs based on program generation constraints; S2: Obtain the running results of the program test input on the test program, calculate the program credibility weight based on the running results, select the weighted main candidate output based on the program credibility weight, and calculate the output divergence degree; S3: Obtain the reference program output from the running results, obtain the verification result level of the reference program output, obtain the deviation mode by weighting the main candidate output, obtain the candidate defect cluster by the verification result level and the deviation mode, and calculate the defect confidence of the candidate defect cluster. S4: Based on the running results, obtain the program coverage and number of exceptions. Calculate the input score by outputting the divergence degree, program coverage, and number of exceptions. Calculate the program evaluation parameters and then calculate the program quality score using the program evaluation parameters. S5: Update the program test input and test program based on the defect confidence level, input score and program quality score to obtain the target test input, and use the target test input to complete the software test.

2. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S1, when establishing input generation constraints, structured generation constraints are determined, difference-inducing information is constructed, difference-inducing terms are extracted from the difference-inducing information, and the input generation constraints are obtained through the difference-inducing terms and structured generation constraints.

3. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S2, the input-output pass rate, program stability, and proportion of supporting primary candidate behavior of the test program are obtained from the running results. The program credibility weight is calculated using the input-output pass rate, program stability, and proportion of supporting primary candidate behavior.

4. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S2, the weighted support strength is obtained based on the program credibility weight, and the weighted master candidate output is obtained based on the weighted support strength.

5. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S3, the reference program output is judged to obtain the verification result level including strong consistency verification, weak consistency verification and inconsistency verification. The reference program output is compared with the weighted master candidate output to obtain the deviation mode.

6. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S3, the support strength, reference program verification strength, repetitive triggering degree, and overall program variant confidence of the candidate defect cluster are obtained, and the defect confidence is calculated based on the support strength, reference program verification strength, repetitive triggering degree, and overall program variant confidence.

7. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S4, a coverage score is obtained based on the program coverage, a stability score is obtained based on the number of anomalies, and a sensitivity score is obtained based on the output divergence. The input score is obtained through the coverage score, stability score, and sensitivity score.

8. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S4, program evaluation parameters including program stability score, primary candidate consistency, reference support and basic sample pass rate are obtained, and program quality score is calculated using program evaluation parameters.

9. The software testing method based on a large language model and loop enhancement prompts according to claim 1, characterized in that, In step S5, a high-confidence defect threshold and a growth threshold are determined. Based on the defect confidence and the high-confidence defect threshold, a high-confidence defect cluster is determined. During the update process, when the growth number of the high-confidence defect cluster is lower than the growth threshold, the update of the program test input and the test program is stopped.

10. A software testing system based on a large language model and loop enhancement prompts, used to execute the software testing method based on a large language model and loop enhancement prompts as described in any one of claims 1 to 9, characterized in that, include: Input and program generation module: used to establish input generation constraints and program generation constraints, generate program test inputs based on input generation constraints, and generate test programs based on program generation constraints; Input divergence module: used to obtain the running results of the program test input on the test program, calculate the program credibility weight based on the running results, select the weighted main candidate output based on the program credibility weight, and calculate the output divergence. Defect confidence module: used to obtain the reference program output from the running results, obtain the verification result level of the reference program output, obtain the deviation mode by weighting the main candidate output, obtain the candidate defect cluster by the verification result level and the deviation mode, and calculate the defect confidence of the candidate defect cluster; The scoring module is used to obtain program coverage and number of exceptions based on the running results. It calculates the input score by outputting the divergence degree, program coverage, and number of exceptions, calculates program evaluation parameters, and calculates the program quality score based on the program evaluation parameters. Test generation module: Used to update the program test input and test program based on defect confidence, input score and program quality score to obtain target test input, and use the target test input to complete software testing.

Images