A program comprehension and evolution method combining large language models and test-based formal verification

Abstract

The application discloses a program understanding and evolution method combining a large language model and test-based formal verification, and the program understanding comprises the following steps: submitting target program code to the large language model to generate a function scenario form of the target program code; verifying completeness and mutual exclusivity of the generated function scenario form; if the generated function scenario form does not have the completeness or the mutual exclusivity, generating counterexamples and feeding the generated counterexamples back to the large language model, and re-generating the function scenario form by the large language model until the function scenario form has the completeness and the mutual exclusivity; and if the generated function scenario form has the completeness and the mutual exclusivity, performing TBFV verification on all test constraint conditions and specified conditions in the generated function scenario form based on the target program code, and if the verification is passed, a function scenario form consistent with a function description of the target program code is obtained.

Description

Technical Field

[0001] This invention belongs to the field of computer science and technology, specifically a method for program understanding and evolution that combines a large language model with test-based formal verification. Technical Background

[0002] In the field of software engineering, program understanding and evolution are core components for ensuring the sustainability of a software system's lifecycle, directly impacting software maintenance efficiency, the quality of feature iterations, and system stability. As software scales and logical complexity increases, traditional methods of program understanding and evolution, relying on manual analysis, are no longer sufficient to meet the demands of efficient development. Automation technology has become a core direction for industry research and application.

[0003] Program understanding is the cognitive process by which developers comprehend the functionality, structure, and behavior of a software system. This process involves analyzing source code, documentation, and related artifacts to grasp the program's operating mechanisms, design intent, and the relationships between components. As a crucial element in tasks such as debugging, maintenance, enhancement, and collaboration, program understanding helps developers navigate complex codebases, identify potential problems, and implement precise modifications. Effective program understanding forms the foundation for successful software evolution.

[0004] Program evolution is a continuous process by which software adapts to changing requirements, fixes defects, and improves performance. Its core is to achieve precise functional iteration while maintaining the stability of the original system. Existing program evolution technologies face multiple challenges: on the one hand, rule-based code modification methods lack flexibility and struggle to handle complex functional changes; on the other hand, while large language model-driven code generation can quickly respond to requirements, the generated code is prone to deviating from the original program style and lacks automated verification mechanisms, leading to a mismatch between evolved program functionality and requirements, increasing subsequent maintenance costs. Furthermore, in existing technologies, program understanding and evolution are disconnected; the results generated in the understanding phase cannot directly provide precise support for evolution, further reducing iteration efficiency.

[0005] In terms of verification techniques, traditional formal verification, relying on loop invariants, has significant limitations when dealing with programs containing complex loop structures. While specification-based testing can verify program behavior by generating test cases, it cannot prove the absence of vulnerabilities, resulting in insufficient verification completeness. The emergence of Test-Based Formal Verification (TBFV) integrates specification-based testing with Hall logic, enabling program correctness verification without loop invariants, thus providing a new technological foundation for solving the aforementioned problems. However, TBFV can only verify the consistency between the program and existing specifications; it cannot directly support specification generation in program understanding and code adaptation in program evolution, and its technological potential has not yet been fully realized.

[0006] In recent years, large language models have demonstrated powerful capabilities in code generation and understanding tasks, providing new opportunities for the automation of program understanding and evolution. However, existing technologies have failed to effectively integrate the generative capabilities of large language models with the verification advantages of TBFV, resulting in a lack of accuracy guarantees for specification generation in program understanding and a lack of reliable support for code modification in program evolution. Therefore, constructing an integrated method combining large language models and TBFV to achieve automated and high-precision closed-loop program understanding and evolution has high application value and development prospects in the field of software engineering. Summary of the Invention

[0007] Purpose of the Invention: To address the problems of incomplete specification generation, insufficient reliability of code modifications, and reliance on loop invariants for verification in existing program understanding and evolution techniques, this invention proposes a program understanding and evolution method that combines large language models with test-based formal verification (TBFV). By deeply integrating the generation capabilities of large language models with the verification advantages of TBFV, an automated and high-precision program understanding and evolution method is achieved. It eliminates the need for manual intervention in loop invariant derivation, accurately generating complete and consistent Functional Scenario Forms (FSFs) to complete program understanding. Simultaneously, based on the modified FSF, it efficiently generates evolutionary code that meets requirements and retains the original style. TBFV closed-loop verification ensures functional correctness, improves software maintenance and iteration efficiency, and reduces development costs.

[0008] Technical Solution: In a first aspect, this invention proposes a program understanding method that combines a large language model with test-based formal verification, including:

[0009] Step 1: Submit the target program code to the large language model, and use prompt words to guide the large language model to generate the functional scenario form of the target program code. The functional scenario form includes: test constraints and specified conditions; the test constraints are constraints that restrict input variables, and the specified conditions are constraints that restrict at least one output variable.

[0010] Step 2: Verify the completeness and mutual exclusivity of the functional scenario forms generated by the large language model; if the generated functional scenario forms are incomplete or mutually exclusive, generate a counterexample and feed the generated counterexample back to the large language model, which will then regenerate the functional scenario forms of the target program code. Repeat Step 2 until the functional scenario forms generated by the large language model are complete and mutually exclusive; if the generated functional scenario forms are complete and mutually exclusive, proceed to Step 3.

[0011] Step 3: Based on the target program code, perform TBFV verification on all test constraints and specified conditions in the generated functional scenario form. If the verification passes, a functional scenario form consistent with the functional description of the target program code is obtained. If the verification fails, determine whether the number of functional scenario form generation rounds exceeds the preset upper limit. If it does not exceed the limit, feed back the TBFV verification counterexamples generated during the verification process to the large language model, and guide the large language model to regenerate the functional scenario form of the target program code with prompt words, and then execute Step 2. If it exceeds the limit, the program understanding process fails and ends.

[0012] Furthermore, in step 1, the functional scenario form of the target program code is a functional scenario decomposition, represented as:

[0013]

[0014] In the formula, This represents the test constraint i. Indicates the specified condition i, It is a tautology. It is a contradiction, for any These correspond to the completeness and mutual exclusion of functional scenarios, respectively.

[0015] Furthermore, the verification of the completeness and mutual exclusivity of the functional scenario forms generated by the large language model includes:

[0016] All test constraints generated from the functional scenario form of the large language model are connected by logical OR, inverted as a whole, and then solved using the SMT solver. If the SMT solver gives a counterexample, it means that the system is not complete; otherwise, it is complete.

[0017] All test constraints in the form of functional scenarios generated by the large language model are ANDed pairwise. The SMT solver is then used to solve each logical expression after the AND. If the SMT solver gives a counterexample, it means that the expressions are not mutually exclusive; otherwise, they are mutually exclusive.

[0018] Furthermore, the target program code for TBFV verification in step 3 is obtained through the following preprocessing:

[0019] Instrumentation of the target program code includes: printing out the branch judgment statements of the target program code and their runtime results;

[0020] The branching statements include conditional expressions in if, else, else if, for, while, return, and ternary operators.

[0021] Furthermore, in step 3, the TBFV verification of all test constraints and specified conditions in the generated functional scenario form based on the target program code includes:

[0022] Generate a verification test constraint. Test case t;

[0023] Use test case t to execute the target program code P and obtain the execution path. The execution path A sequence consisting only of assignment statements, branch statements, or their negations;

[0024] The execution path is derived using Hall's axioms. Preconditions ;

[0025] judge Is it a tautology? If so, it proves that all test cases triggering the same path satisfy the functional scenario. Then, update the test constraints to... And repeat this process until the updated test constraints are met. No longer satisfied; otherwise, indicating The process has been terminated due to a defect.

[0026] Secondly, this invention proposes a program evolution method that combines a large language model with test-based formal verification, including:

[0027] S100: Read the target program code, the original functional scenario form consistent with the target program code, and the Modified FSF corresponding to the target evolution instructions; the Modified FSF includes: test constraints and specified conditions; the test constraints are constraints that restrict input variables, and the specified conditions are constraints that restrict at least one output variable;

[0028] S200: Submit the target program code, the original functional scenario form, and the functional scenario form Modified FSF to the large language model, and guide the large language model to generate the evolved program code Modified Code by comparing the functional scenario form Modified FSF with the program evolution requirements contained in the original functional scenario form.

[0029] S300: Using TBFV, verify whether the Modified Code generated by the large language model meets the requirements described in the Modified FSF of the functional scenario. If it does not meet the requirements, determine whether the upper limit of program evolution has been reached. If the upper limit has been reached, terminate the program understanding process. If the upper limit has not been reached, feed back the counterexamples generated during the TBFV verification process to the large language model and execute S200 to regenerate the Modified Code. If the requirements are met, obtain program code that meets the evolution requirements.

[0030] Furthermore, the original functional scenario form that is consistent with the target program code is generated according to a program understanding method that combines a large language model with test-based formal verification.

[0031] Thirdly, the present invention provides an electronic device, the electronic device comprising:

[0032] At least one processor;

[0033] and a memory communicatively connected to the at least one processor;

[0034] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to execute a program understanding method that combines a large language model with test-based formal verification to generate a program or a program evolution method that combines a large language model with test-based formal verification.

[0035] Fourthly, the present invention proposes a computer-readable storage medium storing computer instructions for causing a processor to generate or evolve a program by combining a large language model with a test-based formal verification method when executed.

[0036] Fourthly, the present invention proposes a computer program product, which includes a computer program that, when executed by a processor, implements a program understanding method that combines a large language model with test-based formal verification to generate a program or a program evolution method that combines a large language model with test-based formal verification.

[0037] Beneficial effects: In the program understanding method, this invention performs the following operations on a given original program: First, it generates the Functional Scenario Form (FSF) of the program through a large language model; then, it generates representative test cases and verifies whether the original program's functionality is consistent with the FSF description through a test-based formal verification method (TBFV); if a discrepancy is detected between the original program's functionality and the FSF description during TBFV verification, a corresponding counterexample is given; after obtaining the counterexample, it is resubmitted to the large language model in conjunction with specific prompt words to regenerate the FSF; the TBFV verification and FSF generation are repeated continuously until the TBFV verification passes, resulting in an FSF that is consistent with the original program's functionality, thus completing the program understanding. In the program evolution method, this invention performs the following operations on a given program and its functionally consistent Functional Scenario Form (FSF): First, it receives the user's program evolution requirements, obtaining the updated functional scenario form (Modified FSF); then, combined with prompt words, it guides the large language model to compare the functional changes reflected in the original FSF with the Modified FSF, and generates the updated program Modified Code based on the original program code; the TBFV method is used to verify whether the updated code is consistent with the evolution requirements; if the TBFV verification finds inconsistencies, the specific counterexamples obtained are submitted to the large language model along with specific prompt words, causing it to regenerate the updated program; the TBFV verification and the generation of the updated program are repeated continuously until verification is passed, thus completing the evolution of the original program to be consistent with the user's requirements. The two modules can be used together to achieve a unified workflow for program understanding, formal verification, requirement change handling, code generation, and correctness verification.

[0038] This invention deeply integrates the ability to generate large language models with the verification advantages of test-based formal verification (TBFV), constructing an integrated closed-loop method for program understanding and evolution, which has significant advantages over existing technologies:

[0039] (1) Achieve fully automated program understanding: Without the need for manual writing of formal specifications or loop invariants, the program functions can be automatically generated and verified through the collaborative work of large language models and TBFV, which greatly reduces the technical threshold and time cost of program understanding;

[0040] (2) Avoid the bottleneck of loop invariant dependency: The verification strategy of combining functional scenario form (FSF) with TBFV completely bypasses the problem of manually providing loop invariants in traditional formal verification, so that programs containing complex loop structures can also be automatically verified.

[0041] (3) Ensure the completeness and accuracy of formal expression: Verify the mutual exclusivity and completeness of FSF through formal methods to ensure that the generated FSF can fully cover all input scenarios and avoid the common problems of incomplete formal description or description deviation in existing methods;

[0042] (4) Support for highly reliable program evolution: Using the verified FSF as the evolution blueprint, combined with TBFV for continuous verification of the generated code, the program evolution process is always under the correctness constraint, realizing safe and controllable functional changes;

[0043] (5) Construct an efficient iterative feedback mechanism: By using a multi-round dialogue mechanism driven by verification counterexamples, the system can learn from errors and quickly converge to the correct solution, significantly improving the success rate of tasks while controlling interaction costs;

[0044] (6) Forming a complete technical closed loop: It organically integrates program understanding, formal verification, requirement change, code generation and correctness verification into a unified workflow, providing an end-to-end automated solution for software maintenance and evolution, and has high engineering practical value. Attached Figure Description

[0045] Figure 1 This is a basic framework diagram of TBFV;

[0046] Figure 2 Flowchart for program understanding methods;

[0047] Figure 3 Here is a flowchart of the program evolution method;

[0048] Figure 4 A framework diagram is used to combine program understanding and program evolution. Detailed Implementation

[0049] Example 1:

[0050] Formal specifications with preconditions and postconditions are used to check the (partial) correctness of the program relative to the specification, that is, to verify whether the final state after normal program execution satisfies the postconditions when the initial state satisfies the preconditions. Specification-based testing generates test cases based on the preconditions and executes the program to verify whether the result meets the postconditions; while formal verification rigorously proves the program logic using mathematical tools such as Hall logic and inductive assertions, but requires manual derivation of necessary constraints such as loop invariants. It should be noted that specification-based testing cannot prove the absence of defects, and formal verification of programs with loop structures often requires manual derivation of loop invariants. Test-based formal verification (TBFV) strengthens specification-based testing by using Hall logic and theorem proofs to detect errors in the program. The "Functional Scenario Form" (FSF) specification used by the TBFV method eliminates the dependency on loop invariants, thus avoiding the bottleneck of manually writing loop invariants, reducing testing costs, improving the automation of formal verification, improving the efficiency of verification during program understanding, and enhancing reliability.

[0051] Similar to most formal verification techniques, the program correctness in this embodiment refers to partial correctness; that is, if the input satisfies the preconditions and the program terminates successfully, the output should satisfy the postconditions. Given the functional scenario of program S... Test-based formal verification (TBFV) process as follows Figure 1 The following is an explanation of what is shown.

[0052] F1: Generate a verification test condition Test case t.

[0053] F2: Obtain the execution path by executing program P using test cases. The path consists only of a sequence of assignment statements, branch statements, or their negations.

[0054] F3: Derived from the following Hall axiom Preconditions ,in, yes The formula is obtained by conjuncting the Boolean values ​​of all conditional statements and replacing intermediate variables with input variables. This is the formula after replacing all output variables with the expressions of input variables during the derivation process.

[0055] (Assignment)

[0056] (Branch-T)

[0057] (Branch-F)

[0058] In the Hall axioms above, the first axiom is the assignment axiom, where Q represents a postcondition, a predicate indicating the property that should be satisfied after the program is executed. x represents a variable in the program. E represents an expression in the program. The expression , indicates that the new predicate is obtained by replacing all freely occurring x in the postcondition Q with the expression E. The second and third axioms are the branch-true and branch-false axioms, respectively, where B is a Boolean expression representing the condition in a program's conditional statement such as if or while. Branch-T indicates that the Boolean expression represented by B is true, while Branch-F indicates that it is false.

[0059] F4: Judgment Is it a tautology? If so, it proves that all test cases triggering the same path satisfy the functional scenario, and the test conditions are then updated to... And repeat this process until the updated version is available. Unsatisfiable; otherwise, it indicates The process has been terminated due to a defect.

[0060] The following example will be used to introduce and explain in detail the problem solved by the embodiments of the present invention during the program understanding phase, and will also be used to explain the subsequent workflow design and details.

[0061] Example 1: Consider a simple program that takes an integer variable as input and outputs the absolute value of that variable. The following program is given as an implementation of this function:

[0062] public class Absolute {

[0063] public int calculate(int num) {

[0064] if (0 <= num)

[0065] return num;

[0066] else

[0067] return -num;

[0068] }

[0069] }

[0070] The program in Example 1 can be expressed in the form of a functional scenario as follows:

[0071] T1: num >= 0, D1: r_v = num;

[0072] T2: num < 0, D2: r_v = -num;

[0073] Note: r_v represents the return value of the program.

[0074] The goal of the program understanding phase of this invention is to transform the program code into a Functional Scenario Form (FSF). The definition of the Functional Scenario Form FSF is designed here as follows:

[0075] Functional scenario formal definition: a pair of test conditions (Constraints that only restrict input variables) and specified conditions (At least one output variable is required), denoted as .

[0076] The functional scenario form of program S is the disjunction of the functional scenario, represented as:

[0077]

[0078] in, It is a tautology. It is a contradiction, for any , which correspond to the completeness and mutual exclusion of FSF, respectively.

[0079] In this embodiment of the invention, the specific process is as follows: Figure 2 As shown, the following will describe step by step, based on Example 1, the workflow and specific implementation method of a program understanding method that combines a large language model with test-based formal verification proposed in this embodiment of the invention.

[0080] S1: Read the target program code.

[0081] S2: Submit the target program code to the large language model and guide it to generate a Functional Scenario Form (FSF). When guiding the large model to generate the FSF, it needs to be supplemented with some examples (2-4) for the large language model's learning. Each example contains a program and an FSF with a consistent functional description. The large language model used in this step does not require specialized training and tuning. In the practice of this invention, publicly available general-purpose large models can perform the FSF generation task quite well.

[0082] S3: Use the SMT solver to verify the completeness and mutual exclusivity of the FSF generated by the large language model. If it is not mutually exclusive or complete, a counterexample is obtained and fed back to the large language model to regenerate the FSF until the generated FSF has completeness and mutual exclusivity.

[0083] This invention uses the Z3 solver to complete the verification process. Specifically, it first reads the FSF returned after calling the large language model and verifies its completeness, that is, it checks all elements in the FSF. (Test constraints) use logical OR ( Connecting the dots, taking Example 1 as an example, yields... To verify that it is a tautology, it needs to be inverted globally before being submitted to the Z3 solver. If completeness is not satisfied, the Z3 solver will provide a counterexample. The mutual exclusion verification method is to... Pairwise AND, taking Example 1 as an example, we get... (Since there are only two pairs in Example 1) Therefore, only one logical expression needs to be solved for each pairwise combination. In cases where the logarithm is greater than 2, each such logical expression needs to be solved and verified separately before being submitted to the Z3 solver. If mutual exclusion is not satisfied, a counterexample will be returned.

[0084] S4: Obtain the solver's return content and determine whether the completeness and mutual exclusion verifications have passed. If it is a counterexample, it means that the currently generated FSF does not simultaneously satisfy completeness and mutual exclusion, i.e., the verification fails, and the FSF needs to be regenerated. Return to S2. Otherwise, the current FSF satisfies completeness and mutual exclusion, and enter the TBFV verification stage.

[0085] S5: To successfully perform TBFV verification, some preprocessing is required for the target program. Preprocessing includes formatting the program and instrumentation. The instrumentation involves printing out the branch judgment statements of the target program and their runtime results, specifically the conditional expressions in statements such as if, else, else if, for, while, return, and ternary operators.

[0086] S6: Check all instances in the current FSF one by one. To perform TBFV verification, taking Example 1 as an example, we will... 、( The preprocessed program is then input into the TBFV workflow. The specific workflow of TBFV has been described above. After TBFV verification is completed, the solution result of the SMT solver will be obtained. In this embodiment of the invention, the test-based formal verification method TBFV verifies the correctness of the program without the need for manually providing loop invariants by generating test cases that meet the test conditions, executing the program to obtain the specific path, and using Hall logic reasoning to verify whether the path meets the corresponding definition conditions.

[0087] S7a: Obtain the solver result and determine whether the TBFV verification is successful. If the solver returns a counterexample, the verification fails. After obtaining the counterexample, proceed to step S7b. Otherwise, the TBFV verification is successful, meaning that the program function is consistent with the current FSF description, the program understanding process is successful, the program understanding process ends, and the functional scenario form consistent with the target program functional description is obtained.

[0088] S7b: After obtaining a counterexample, determine whether the current FSF generation rounds exceed the preset upper limit. If the upper limit has not been reached (in this invention, the upper limit is set to 10 times), then the counterexample is fed back to the large language model, and the large language model is guided to regenerate the FSF with the prompt words, that is, return to S2. Otherwise, if the completeness and mutual exclusion verification still cannot be passed even after reaching the upper limit, the program understanding process fails and ends.

[0089] Example 2:

[0090] This invention proposes a program evolution method that combines a large language model with test-based formal verification. Program evolution refers to the continuous modification and adjustment of a software system to meet changing requirements, fix defects, improve performance, or add functionality. Efficient program understanding typically requires a combination of technical capabilities, domain knowledge, and auxiliary tools to ensure correctness. Among these, program specifications written in precise statements can be used to understand the functional behavior of a program.

[0091] Before detailing the process of program evolution, let's assume that the program evolution requirement for Example 1 is to return the opposite of the absolute value of the input parameter. This evolution requirement can be described using Modified FSF as follows:

[0092] Example 2: Evolve the program based on the FSF of Example 1. The Modified FSF content corresponding to the evolution target is:

[0093] T1: num >= 0, D1: r_v = -num;

[0094] T2: num < 0, D2: r_v = num;

[0095] Corresponding to this evolutionary requirement, the updated program code, based on the code in Example 1, is as follows:

[0096] Based on Example 1, the code is evolved according to the Modified FSF evolution requirements in Example 2.

[0097] public class Absolute {

[0098] public int calculate(int num) {

[0099] if (0 <= num)

[0100] return -num;

[0101] else

[0102] return num;

[0103] }

[0104] }

[0105] The process of program evolution module is as follows Figure 3 As shown, the workflow and specific implementation methods of program understanding will be introduced step by step based on Example 1 and the assumed evolution requirements.

[0106] P1: Read the target program code, the original FSF with the same function as the target program, and the modified functional scenario form (Modified FSF) corresponding to the target evolution instructions. Taking Example 1 and its evolution requirements as an example, the corresponding contents are the code content of Example 1, the FSF content in Example 1, and the Modified FSF content of Example 2. The Modified FSF conforms to the definition of functional scenario form.

[0107] P2: Provide a small number of examples, the original program code, the original FSF, and the Modified FSF to the large language model, and guide it to generate modified code ModifiedCode according to the program evolution requirements implied by the Modified FSF compared with the original FSF. During this process, it is necessary to mention in the prompts that when the large language model generates Modified Code, it must refer to the style of the original code and make as few changes as possible to meet the evolution requirements.

[0108] P3: Perform preprocessing on the Modified Code obtained from the large language model, similar to S5 in Example 1.

[0109] P4: Perform TBFV verification. The input to the TBFV workflow at this stage is the preprocessed ModifiedCode and Modified FSF. The purpose of this process is to verify whether the Modified Code generated by the large language model meets the requirements described in the Modified FSF. Taking Example 1 and its evolution requirements as an example, assuming the obtained Modified Code is as described in the code section of Example 2, this code, along with the Modified FSF content from Example 2, is delivered to the TBFV verification stage. After the TBFV verification process is completed, the solution result from the SMT solver is obtained.

[0110] P5a: Check the TBFV verification result. If the verification fails, i.e. the solver returns a counterexample, proceed to step P5b. Otherwise, the Modified Code that matches the Modified FSF description is obtained, the program evolution ends, and the updated program that meets the evolution requirements is obtained.

[0111] P5b: Determine whether the program has reached its maximum evolution cycle. If it has, terminate the program understanding process. Otherwise, feed back the obtained counterexamples to the large language model and return to step P2 to regenerate the Modified Code.

[0112] Example 3

[0113] Although the program understanding module in Example 1 and the program evolution module in Example 2 each perform different tasks, they achieve excellent functional integration centered on functional scenarios. Using them together organically integrates program understanding, formal verification, requirement change management, code generation, and correctness verification into a unified workflow, providing an end-to-end automated solution for software maintenance and evolution. Their combined framework is as follows: Figure 4 As shown.

[0114] L1: Select the target program.

[0115] L2: Use program understanding methods to understand the target program and derive the Functional Scenario Form (FSF) that is consistent with the functional description of the target program.

[0116] L3: Determine the program evolution requirements and express them as the updated functional scenario form Modified FSF.

[0117] L4: Use the program evolution method to perform program evolution, and obtain the target evolution code Modified Code, which is updated according to the evolution requirements based on the original target program and is consistent with the Modified FSF description.

[0118] L5: Obtain the target evolution procedure.

[0119] Among them, the functional scenario form generated by the program understanding method can be directly used as the input of the program evolution method.

Claims

1. A program understanding method that combines a large language model with test-based formal verification, characterized in that: include: Step 1: Submit the target program code to the large language model, and use prompt words to guide the large language model to generate the functional scenario form of the target program code. The functional scenario form includes: test constraints and specified conditions; the test constraints are constraints that restrict input variables, and the specified conditions are constraints that restrict at least one output variable. Step 2: Verify the completeness and mutual exclusivity of the functional scenario forms generated by the large language model; if the generated functional scenario forms are incomplete or mutually exclusive, generate a counterexample and feed the generated counterexample back to the large language model, which will then regenerate the functional scenario forms of the target program code. Repeat Step 2 until the functional scenario forms generated by the large language model are complete and mutually exclusive; if the generated functional scenario forms are complete and mutually exclusive, proceed to Step 3. Step 3: Based on the target program code, perform TBFV verification on all test constraints and specified conditions in the generated functional scenario form. If the verification passes, a functional scenario form consistent with the functional description of the target program code is obtained. If the verification fails, determine whether the number of functional scenario form generation rounds exceeds the preset upper limit. If it does not exceed the limit, feed back the TBFV verification counterexamples generated during the verification process to the large language model, and guide the large language model to regenerate the functional scenario form of the target program code with prompt words, and then execute Step 2. If it exceeds the limit, the program understanding process fails and ends.

2. The program understanding method combining a large language model and test-based formal verification as described in claim 1, characterized in that: In step 1, the functional scenario form of the target program code is the disjunction of the functional scenario, represented as: ； In the formula, This represents the test constraint i. Indicates the specified condition i, It is a tautology. It is a contradiction, for any These correspond to the completeness and mutual exclusion of functional scenarios, respectively.

3. The program understanding method combining a large language model and test-based formal verification according to claim 1, characterized in that: The verification of the completeness and mutual exclusivity of the functional scenario forms generated by the large language model includes: All test constraints generated from the functional scenario form of the large language model are connected by logical OR, inverted as a whole, and then solved using the SMT solver. If the SMT solver gives a counterexample, it means that the system is not complete; otherwise, it is complete. All test constraints in the form of functional scenarios generated by the large language model are ANDed pairwise. The SMT solver is then used to solve each logical expression after the AND. If the SMT solver gives a counterexample, it means that the expressions are not mutually exclusive; otherwise, they are mutually exclusive.

4. The program understanding method combining a large language model and test-based formal verification as described in claim 1, characterized in that: The target program code for TBFV verification in step 3 is obtained through the following preprocessing: Instrumentation of the target program code includes: printing out the branch judgment statements of the target program code and their runtime results; The branching statements include conditional expressions in if, else, else if, for, while, return, and ternary operators.

5. The program understanding method combining a large language model and test-based formal verification according to claim 1, characterized in that: In step 3, the TBFV verification is performed on all test constraints and specified conditions in the generated functional scenario form based on the target program code, including: Generate a verification test constraint. Test case t; Use test case t to execute the target program code P and obtain the execution path. The execution path A sequence consisting only of assignment statements, branch statements, or their negations; The execution path is derived using Hall's axioms. Preconditions ; judge Is it a tautology? If so, it proves that all test cases triggering the same path satisfy the functional scenario. Then, update the test constraints to... And repeat this process until the updated test constraints are met. No longer satisfied; otherwise, indicating The process has been terminated due to a defect.

6. A program evolution method combining a large language model with test-based formal verification, characterized in that: include: S100: Read the target program code, the original functional scenario form consistent with the target program code, and the Modified FSF corresponding to the target evolution instructions; the Modified FSF includes: test constraints and specified conditions; the test constraints are constraints that restrict input variables, and the specified conditions are constraints that restrict at least one output variable; S200: Submit the target program code, the original functional scenario form, and the functional scenario form Modified FSF to the large language model, and guide the large language model to generate the evolved program code Modified Code by comparing the functional scenario form Modified FSF with the program evolution requirements contained in the original functional scenario form. S300: Using TBFV, verify whether the Modified Code generated by the large language model meets the requirements described in the Modified FSF of the functional scenario. If it does not meet the requirements, determine whether the upper limit of program evolution has been reached. If the upper limit has been reached, terminate the program understanding process. If the upper limit has not been reached, feed back the counterexamples generated during the TBFV verification process to the large language model and execute S200 to regenerate the Modified Code. If the requirements are met, obtain program code that meets the evolution requirements.

7. The program evolution method combining a large language model and test-based formal verification as described in claim 6, characterized in that: The original functional scenario form, which is consistent with the target program code, is generated by a program understanding method that combines a large language model with test-based formal verification, as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the program understanding method combining a large language model and test-based formal verification as described in any one of claims 1-5 to generate a program or the program evolution method combining a large language model and test-based formal verification as described in claim 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the program understanding method combining a large language model and test-based formal verification as described in any one of claims 1-5, or the program evolution method combining a large language model and test-based formal verification as described in claim 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the program understanding method combining a large language model and test-based formal verification as described in any one of claims 1-5, or the program evolution method combining a large language model and test-based formal verification as described in claim 6.

Images