Test case generation method and device, electronic equipment, storage medium and system

Abstract

The application provides a test case generation method and device, electronic equipment, a computer readable storage medium and system. The method comprises: in response to a generation request, obtaining a test generation basis record of a program under test, and generating a test generation context; constructing a structured generation prompt record with block positioning information; generating and executing a candidate case batch to obtain an execution feedback record; generating a prompt reconstruction instruction based on a differential reach state, and reconstructing feedback guide content to generate return content again and encapsulate it as a test case. The application can enable test execution feedback to participate in the subsequent case generation process.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and more specifically, to a test case generation method, a test case generation device, an electronic device, a computer-readable storage medium, and a test case generation system. Background Technology

[0002] As software systems continue to expand in scale and continuous integration and continuous delivery processes become more widespread, the efficiency and coverage quality of software test case generation have a significant impact on the stability of software delivery. In complex business interfaces, regression testing, and automated testing platforms, test cases not only need to meet the interface call format, but also need to reach different execution paths in the program under test as much as possible, and adjust subsequent testing directions in a timely manner when execution results change, so as to improve the adaptability of the test case generation process to the running state of the program under test.

[0003] Existing test case generation solutions typically employ automatic generation methods based on fixed templates or static rules. This approach generally first reads the interface description file or an existing test script template, then generates candidate test scripts according to preset field filling rules. Subsequently, the candidate test scripts are run in the test execution environment, and a coverage report is output. The coverage report is usually saved as a test evaluation result for later review of the reached and unreached code areas in the tested program.

[0004] However, the above approach lacks a stable technical loop between test execution results and subsequent generation processes. Coverage reports typically remain at the level of result presentation or single evaluation, making it difficult to continuously participate in adjusting the direction of subsequent test case generation. Furthermore, interface format constraints and execution feedback guidance are easily mixed in the same generation input, resulting in a lack of clear block connections between the generation constraint content and the feedback guidance content. Consequently, in the subsequent generation process, candidate test cases are prone to repeatedly accessing existing paths or continuing to generate test cases with similar execution failure characteristics, making the test case generation process less adaptable to the actual execution feedback of the program under test. Summary of the Invention

[0005] This application provides a test case generation method, a test case generation device, an electronic device, a computer-readable storage medium, and a test case generation system to at least alleviate the aforementioned technical problems.

[0006] A test case generation method, comprising: In response to the generation request, the test generation basis record of the program under test is obtained to generate the test generation context; Based on the test generation context, a structured generation prompt record with block location information is constructed. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. The structured generated prompts are input into the text generation service to obtain a batch of candidate test cases, and the batch of candidate test cases is executed in the isolated execution environment to generate execution feedback records; Based on the differential reach status of the execution feedback record relative to the test generation basis record, a prompt reconstruction instruction is generated, and the feedback guidance content is reconstructed according to the block location information based on the prompt reconstruction instruction to obtain the reconstruction generation prompt record. The reconstructed prompt record is input into the text generation service again. The generated return content output by the text generation service is received, and the generated return content is encapsulated in an executable format to generate test cases.

[0007] This application also provides a test case generation apparatus, comprising: The context generation unit is used to respond to the generation request, obtain the test generation basis record of the program under test, and generate the test generation context. The prompt record construction unit is used to construct a structured generation prompt record with block positioning information based on the test generation context. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. The candidate execution unit is used to input the structured generated prompt record into the text generation service to obtain a batch of candidate test cases, and execute the batch of candidate test cases in the isolated execution environment to generate execution feedback records; The prompt reconstruction unit is used to generate prompt reconstruction instructions based on the differential reach status of the execution feedback record relative to the test generation basis record, and to reconstruct the feedback guidance content according to the block positioning information based on the prompt reconstruction instructions to obtain the reconstruction generation prompt record. The test case encapsulation unit is used to re-input the refactoring generated prompt record into the text generation service, receive the generated return content output by the text generation service, and encapsulate the generated return content into an executable format to generate test cases.

[0008] This application also provides an electronic device, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to read and execute the computer instructions to implement any of the above-described test case generation methods.

[0009] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement any of the above-described test case generation methods.

[0010] This application also provides a test case generation system, including a front-end device and a back-end cloud server, wherein the front-end device and the back-end cloud server are connected in communication. The front-end device is used to perform the following steps: receive a generation request for the program under test; send the generation request to the back-end cloud server; receive the test cases returned by the back-end cloud server; and display the test cases. The backend cloud server performs the following steps in response to the generation request: It obtains the test generation basis record of the program under test to generate a test generation context; based on the test generation context, it constructs a structured generation prompt record with block location information. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content; it generates a batch of candidate test cases based on the structured generation prompt record and executes the batch of candidate test cases in an isolated execution environment to generate an execution feedback record; it generates a prompt refactoring instruction based on the differential reach state of the execution feedback record relative to the test generation basis record, and refactors the feedback guidance content according to the block location information based on the prompt refactoring instruction to obtain a refactored generation prompt record; it generates generation return content based on the refactored generation prompt record and encapsulates the generation return content in an executable format to generate test cases; and it returns the test cases to the frontend device.

[0011] The technical advantages of the technical solution provided in this application are: To address the lack of a stable technical loop between test execution results and subsequent generation processes in existing test case generation schemes, this application obtains the test generation basis record of the program under test in response to the generation request to generate a test generation context. Based on the test generation context, a structured generation prompt record with block location information is constructed, providing a prompt carrier foundation for the subsequent generation process that can be located and reconstructed. Compared to the traditional approach of saving execution results as a single coverage report, the first generation state block in the structured generation prompt record carries generation constraint content, and the second generation state block carries feedback guidance content. This creates a partitioned carrier relationship between generation constraint content and feedback guidance content in the technical structure, thus providing clearer update boundaries for interface format constraints and execution feedback guidance during subsequent reconstruction, helping to reduce generation direction drift caused by the mixing of different types of prompt content.

[0012] To address the issue in existing solutions where coverage reports are difficult to incorporate into subsequent generation direction adjustments, this application inputs structured generation prompt records into a text generation service to obtain candidate test case batches. These candidate test case batches are then executed in an isolated execution environment to generate execution feedback records. Subsequently, based on the differential reach status of the execution feedback records relative to the test generation basis records, prompt refactoring instructions are generated. This allows the execution results of the candidate test case batches to be used as the basis for subsequent prompt content refactoring. Compared to methods that only output coverage statistics after execution, differential reach status establishes a comparison relationship between the execution feedback records and the test generation basis records, enabling both already reached and newly reached execution states to participate in subsequent generation direction adjustments. This improves the adaptability of the test case generation process to the actual execution state of the program under test.

[0013] To address the issue in existing solutions where candidate test cases easily repeat existing paths or perpetuate failed input characteristics, this application reconstructs the feedback guidance content based on block location information according to the prompt reconstruction instruction, thereby obtaining a reconstructed prompt record. Since the prompt reconstruction instruction originates from the differential reach state of the execution feedback record relative to the test generation basis record, the reconstructed prompt record can update the feedback guidance content while retaining the generation constraint content. Compared to fixed prompts or fixed templates, this application uses block location information to perform location-based reconstruction of the feedback guidance content, enabling the text generation service to generate return content based on the updated feedback guidance content upon regeneration. This reduces reliance on single static input in subsequent rounds of generation, providing better guidance capabilities for repeated reach and failure avoidance.

[0014] To address the issue that existing solutions struggle to directly integrate generated results into automated test execution processes, this application refactors the generation prompt records, re-enters the text generation service, receives the generated return content from the text generation service, and encapsulates the generated return content into an executable format to generate test cases. Compared to generation methods that only output natural language or loose script fragments, executable format encapsulation transforms the generated return content into test cases that conform to the interface of the program under test, thus improving the usability of test cases in automated testing systems or continuous integration processes. Furthermore, through a closed-loop processing approach of "generation—execution—feedback—refactoring—regeneration—encapsulation," this application enables continuous participation of execution feedback in the test case generation process without altering the internal parameters of the text generation service, thereby improving the continuous evolution capability of test case generation. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0016] Figure 1This is a flowchart illustrating a test case generation method according to an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of a test case generation device according to an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0019] Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of this application.

[0020] Figure 5 This is a schematic diagram of the structure of a test case generation system according to an embodiment of this application. Detailed Implementation

[0021] like Figure 1 As shown, this is an embodiment of a test case generation method according to this application, including: In response to the generation request, the test generation basis record of the program under test is obtained to generate the test generation context; Based on the test generation context, a structured generation prompt record with block location information is constructed. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. The structured generated prompt record is input into the text generation service to obtain a batch of candidate test cases, and the batch of candidate test cases is executed in the isolated execution environment to generate execution feedback records; Based on the differential reach status of the execution feedback record relative to the test generation basis record, a prompt reconstruction instruction is generated, and the feedback guidance content is reconstructed according to the block positioning information based on the prompt reconstruction instruction to obtain the reconstruction generation prompt record; The reconstructed prompt record is input into the text generation service again, the generated return content output by the text generation service is received, and the generated return content is encapsulated in an executable format to generate test cases.

[0022] Specifically, in this application, the generation request is the data entry point that triggers the test case generation process. This generation request can be initiated by an automated testing platform, a continuous integration platform, or a test task configuration interface. The generation request carries the program identifier, round identifier, and test task entry information of the program under test. The program identifier is used to locate the program under test for which test cases need to be generated, the round identifier is used to distinguish the current generation process from previous generation processes, and the test task entry information is used to locate the interface description text and coverage marker file corresponding to the program under test. After reading the generation request, the program under test to which the generation request points is first determined based on the program identifier, and then the test generation basis record of the program under test is read based on the test task entry information. The test generation basis record is not a simple test configuration text, but a structured record of the data of the program under test that needs to be understood and inherited by the text generation service under the current round identifier. The test generation basis record includes at least interface constraint descriptions and historical coverage baselines. The interface constraint descriptions are used to constrain the input format, output format, and execution preconditions of the test cases, and the historical coverage baselines are used to indicate the executable code locations reached in the previous round of test execution. By placing the interface constraint description and the historical coverage baseline in the same test generation basis record, this application enables the subsequent structured generation prompt record to simultaneously inherit the generation constraint content formed by the interface constraint description and the initial feedback guidance content formed by the historical coverage baseline, thus avoiding the test case generation process relying solely on the interface description text while ignoring the execution feedback represented by the historical coverage baseline.

[0023] When generating a test generation context, the test generation basis records are first processed for round assignment. Round assignment can be achieved using round identifier binding, which binds the interface constraint description, historical coverage baseline, and the program under test (DUT) to which the generation request points to the same round identifier. The round identifier can use an incrementing sequence number, timestamp encoding, or task pipeline encoding. The role of the round identifier is not to participate in numerical calculations, but rather to ensure that the test generation basis records, structured generation prompt records, execution feedback records, and test cases in each round of generation can be traced back to the same technical chain. After round assignment is completed, the test generation basis records carrying the round identifier are encapsulated into a test generation context. The test generation context is the direct input for subsequently building structured generation prompt records. The test generation context retains the reference relationships of the test generation basis records, allowing the structured generation prompt records to still be traced back to the corresponding interface constraint description and historical coverage baseline when subsequently refactored. Thus, the test generation context both receives generation requests and provides a stable data foundation for subsequent block-based prompt construction.

[0024] Optionally, in response to the generation request, the program under test to which the generation request is directed is determined; Read the interface description text corresponding to the program under test, extract the request field, return field, field format constraint and call precondition from the interface description text, and write the request field, the return field, the field format constraint and the call precondition into the interface constraint description; Read the coverage marker file generated by the coverage acquisition tool after the previous round of execution, extract code coverage markers from the coverage marker file, and record the code coverage markers to the historical coverage baseline according to the executable code location; The interface constraint description and the historical coverage baseline are written into the test generation basis record, and a round identifier is assigned to the test generation basis record to generate the test generation context with the round identifier and corresponding to the program under test.

[0025] Specifically, in this application, the interface description text can originate from the interface specification file, interface comment file, or interface description fragment configured in the test task corresponding to the program under test. The role of the interface description text is to provide parsable call boundaries for the subsequent test case generation process, rather than providing the natural language description itself. After reading the interface description text, it can be segmented and located according to the interface entry line, request field line, return field line, and call description line to obtain interface description segment records. The interface description segment records continue to participate in field extraction processing to extract request fields, return fields, field format constraints, and call preconditions from the interface description text. Request fields are used to represent the input fields that the test case needs to provide when calling the program under test; return fields are used to represent the output fields that the program under test can return after execution; field format constraints are used to represent the field type, field level, field mandatory status, and field value boundaries of request and return fields; and call preconditions are used to represent the login status, context token, initialization data, or dependent interface execution status that need to be satisfied before calling the interface. The aforementioned request fields, return fields, field format constraints, and preconditions are written into the interface constraint specification, enabling the interface constraint specification to constrain the test case format of the candidate test case batch in the subsequent first generated state block.

[0026] The coverage marker file is an execution trace file generated by the coverage acquisition tool after the previous execution. Coverage acquisition tools can be instrumented code coverage acquisition tools, such as those targeting bytecode, script interpretation, or runtime function calls. The coverage marker file does not record test case text, but rather the executable code locations reached by the previous batch of candidate test cases or the test cases generated in the previous round within the program under test. After reading the coverage marker file, code coverage markers can be extracted. Code coverage markers represent the executable code locations reached by execution in the program under test. These locations can be expressed by source file paths, function entry points, branch entry points, statement locations, or instrumentation numbers. To avoid inconsistencies in executable code location representations caused by different languages ​​or compilation methods, code coverage markers can be normalized according to executable code locations before being written to the historical coverage baseline. This normalization is not numerical standardization, but rather converting code coverage markers from different sources into a unified executable code location representation. The historical coverage baseline is formed by recording the code coverage markers according to the executable code location. The historical coverage baseline is used in subsequent differential reach status calculations to determine whether the current coverage trajectory reaches the executable code location that was not reached before.

[0027] When writing interface constraint descriptions and historical coverage baselines into the test generation basis record, their technical roles need to be maintained. Interface constraint descriptions are oriented towards test case format constraints, while historical coverage baselines are oriented towards coverage status references. After being written into the test generation basis record, they are associated through the same round identifier. Once the round identifier is assigned to the test generation basis record, it can carry the program identifier of the program under test, the reference position of the interface constraint description, the reference position of the historical coverage baseline, and the request trace information of the generation request. The reference position of the interface constraint description is used to read the interface constraint description when building the structured generation prompt record; the reference position of the historical coverage baseline is used to read the historical coverage baseline when building the structured generation prompt record; and the request trace information is used to establish a correspondence between the generation request and the test generation basis record. Subsequently, the test generation basis record with the round identifier and corresponding to the program under test is encapsulated into a test generation context. After the test generation context is read by subsequent steps, it is not necessary to re-infer the source of the interface constraint description and the source of the historical coverage baseline. Instead, the generation constraint content and initial feedback guidance content are directly determined based on the test generation basis record, thereby reducing the confusion of prompt content between different rounds.

[0028] Specifically, in this application, the question bank interface, learning path interface, or grading interface of educational technology products can all serve as examples of the program under test. Taking the grading interface as an example, the interface description text can record the answer text field, question identifier field, scoring result field, and the precondition that question configuration data must exist before calling the grading interface. After reading the interface description text, the answer text field and question identifier field can be written into the request field, the scoring result field into the return field, the text type, identifier type, and return format into the field format constraints, and the existence status of question configuration data into the precondition. The existence status of question configuration data in the precondition is further written into the interface constraint description so that the interface constraint description can constrain the candidate test case batch to carry or reference question configuration data before executing the grading interface. The overwrite flag file can record that the previous round of candidate test case batch has reached the normal scoring branch and the empty answer branch, but has not yet reached the abnormal format branch. After the code coverage markers in the coverage marker file are written into the historical coverage baseline, the historical coverage baseline will participate in the determination of unreached directions in subsequent steps. The executable code location corresponding to the abnormal format branch can also form coverage direction guidance content in the subsequent unreached path guidance block. Thus, the test generation context can simultaneously reflect the callable format of the patch interface and the coverage status after the previous round of execution, giving the subsequent generation process a data foundation that is consistent with the actual running results of the program under test.

[0029] Optionally, based on the test generation context, a structured generation prompt record with block location information is constructed, including: Read the test generation basis record from the test generation context, and determine the generation constraint content and initial feedback guidance content based on the test generation basis record; In the structured generation prompt record, a first generation status block and a second generation status block are set. The generation constraint content is written into the first generation status block, and the initial feedback guidance content is written into the second generation status block. The first generation status block is used to constrain the test case format of the candidate test case batch, and the second generation status block is used to guide the coverage direction and failure avoidance direction of the candidate test case batch. Write a block start marker, a block end marker, and a round marker to the first generated state block and the second generated state block respectively, so that the block location information is formed by the block start marker, the block end marker, and the round marker; The block location information is written into the prompt index table, so that the structured generated prompt record maintains a round correspondence with the test generation context through the prompt index table.

[0030] Specifically, in this application, the structured generated prompt record is a prompt-bearing record for the text generation service to read. The structured generated prompt record differs from ordinary prompt text in that it does not mix interface constraint descriptions, historical coverage baselines, and previous execution failure information into a single input; instead, it separates and carries prompt content for different purposes through block location information. When constructing the structured generated prompt record, the test generation basis record is first read from the test generation context. Since the test generation basis record already includes interface constraint descriptions and historical coverage baselines, the generated constraint content can be determined based on the interface constraint descriptions, and the initial feedback guidance content can be determined based on the historical coverage baselines. The generated constraint content is used to restrict the test case structure of the candidate test case batch, such as the names of input fields, the format of input fields, preconditions for calls, and result validation fields. The initial feedback guidance content is used to indicate the coverage direction and failure avoidance direction of the candidate test case batch, such as the guidance content for unreached paths corresponding to code locations not reached in the previous round, the demonstration guidance content for high-contribution candidate test cases in the previous round, and the failure avoidance content corresponding to the previous execution failure information.

[0031] When setting up the first and second generation state blocks in the structured generation prompt log, the purposes of the two blocks need to be kept separate. The first generation state block is used to carry generation constraint content. The content in the first generation state block generally remains stable during subsequent refactoring because the test case format constraints come from the interface constraint specifications of the program under test, and should not be changed with coverage feedback if the interface of the program under test does not change. The second generation state block is used to carry feedback guidance content. The content in the second generation state block will change with the execution feedback record because the coverage direction and failure avoidance direction need to be updated according to the running status of the candidate test case batch. After writing the generation constraint content into the first generation state block, the first generation state block provides the test case format boundary for the text generation service; after writing the initial feedback guidance content into the second generation state block, the second generation state block provides the coverage direction and failure avoidance direction for the current round for the text generation service. Through the partitioning of the first and second generation state blocks, the structured generation prompt log can maintain the stability of the test case format while allowing the feedback guidance content to participate in subsequent refactoring.

[0032] Block location information is used to record the location of blocks in the structured generation prompt record. After writing the block start marker, block end marker, and round marker for the first generation state block, the block text boundaries and round assignment of the first generation state block in the structured generation prompt record can be determined. After writing the block start marker, block end marker, and round marker for the second generation state block, the block text boundaries and round assignment of the second generation state block in the structured generation prompt record can be determined. The block start marker is used to indicate the beginning position of the block content, the block end marker is used to indicate the end position of the block content, and the round marker is used to indicate the generation round to which the block content belongs. The block start marker, block end marker, and round marker together form the block location information, which is then written to the prompt index table. The prompt index table may include the block name, the position of the block start marker, the position of the block end marker, the round marker, and the reference position of the block content. The block content reference position is used to read the feedback guidance content to be replaced when reconstructing the generated prompt record. When the subsequent prompt reconstructing instruction needs to replace the feedback guidance content in the second generated state block, the second generated state block and the block content reference position can be located through the prompt index table, without having to perform a full-text replacement of the entire structured generated prompt record regardless of content type.

[0033] By maintaining the round-based correspondence between structured generation prompt records and test generation contexts through a prompt index table, the structured generation prompt records from different rounds can be prevented from overwriting each other. Specifically, after the round identifier of the test generation context is entered into the structured generation prompt record, both the first and second generation state blocks carry a round marker corresponding to the test generation context. After the prompt index table records this round marker, subsequent execution feedback records, prompt reconstruction instructions, and reconstructed generation prompt records can all establish a correspondence with the test generation context through the round marker. This approach makes the reconstruction of structured generation prompt records have traceable block text boundaries and ensures that the replacement of feedback guidance content only applies to the second generation state block corresponding to the current round, thereby reducing the risk of erroneous replacement of prompt content across rounds.

[0034] Optionally, the first generated state block includes an interface context block, and the second generated state block includes a high-contribution demonstration block, an unreached path guidance block, and a failed use case negative feedback block. The step of writing the generation constraint content into the first generation state block and writing the initial feedback guidance content into the second generation state block includes: The interface constraint description in the test generation basis record is written into the interface context block in the first generation status block, so that the interface context block carries the generation constraint content; The high-contribution demonstration block and the unreached path guidance block in the second generated state block are initialized according to the historical coverage baseline in the test generation record, so that the coverage direction guidance content in the initial feedback guidance content is carried by the high-contribution demonstration block and the unreached path guidance block; The failed use case negative feedback block in the second generated state block is initialized based on the previous round of execution failure information or empty feedback identifier, so that the failed use case negative feedback block carries the failure avoidance direction guidance content in the initial feedback guidance content.

[0035] Specifically, in this application, the first generation state block further includes an interface context block. The interface context block carries interface constraint descriptions and converts these descriptions into test case format constraints that the text generation service can read. The content of the interface context block may include the interface call entry point, request field format, return field format, assertion acceptance fields, and contextual hints corresponding to the call preconditions. After the interface context block is written into the first generation state block, it participates in the candidate test case batch generation as stable generation constraint content in the current generation round. Since the interface context block originates from the interface constraint descriptions in the test generation basis record, it will not be directly replaced by subsequent execution feedback records, thus ensuring that the candidate test case batch continuously adheres to the call format of the program under test during multiple generation rounds.

[0036] The second generation state block further includes a high-contribution demonstration block, an unreached path guidance block, and a failed test case negative feedback block. The high-contribution demonstration block carries candidate test case content that can serve as demonstration input; the unreached path guidance block carries trigger condition text corresponding to unreached code locations that need to be reached first; and the failed test case negative feedback block carries failure avoidance content corresponding to failed candidate test cases. When initializing the second generation state block, the historical coverage baseline in the test generation basis record can be read first, and then the current round can be judged based on the historical coverage baseline to determine whether there is coverage direction guidance content that can be inherited from the previous round. If there are candidate test cases with high individual coverage contribution from the previous round, the input field record and reach location summary corresponding to that candidate test case are written into the high-contribution demonstration block; if there are no candidate test cases with high individual coverage contribution from the previous round, the high-contribution demonstration block is set to an empty demonstration state or seed demonstration content is written into it. Seed demonstration content can be converted from test cases that conform to the interface constraint description in existing test case files. Seed demonstration content must still be consistent with the generation constraint content in the interface context block, so that the high contribution demonstration block can provide demonstration guidance content that can be read by the text generation service for the second generation state block in the initial round.

[0037] The initialization of the unreached path guidance block relies on the historical coverage baseline. The historical coverage baseline represents the reached executable code locations, while the source code index of the program under test (DUT) represents the overall range of executable code locations within the DUT. By performing position mapping processing on the historical coverage baseline and the source code index, the unreached code locations can be obtained. The unreached path guidance block does not directly write a large amount of source code content; instead, it transforms the unreached code locations into coverage direction guidance content that is easy for the text generation service to understand. This coverage direction guidance content can describe conditional branches, loop entries, or exception entries in a function, giving the candidate test case batch a path reach direction in the next round of generation. Since the unreached path guidance block originates from the historical coverage baseline, the coverage direction guidance content in the unreached path guidance block can form a subsequent continuity relationship with the execution feedback records.

[0038] The initialization of the negative feedback block for failed test cases is based on the previous round's execution failure information or an empty feedback flag. The previous round's execution failure information may include the test case tracking flag of the candidate failed test cases, exception return text, and input field records. If candidate failed test cases existed in the previous round, a failure feature summary can be extracted from the previous round's failure information and written into the negative feedback block, enabling the negative feedback block to carry the failure avoidance direction guidance content in the initial feedback guidance content. If no candidate failed test cases existed in the previous round, the negative feedback block can be initialized based on an empty feedback flag. An empty feedback flag indicates that no failure avoidance basis has been formed in the current round. An empty feedback flag will not be interpreted as a failure feature summary, nor will it guide the text generation service to avoid input patterns that do not exist. Therefore, the negative feedback block for failed test cases can participate in failure avoidance when there is failure basis, and maintain content stability when there is no failure basis, avoiding the influence of unfounded failure avoidance direction guidance content on the batch generation of candidate test cases.

[0039] Optionally, writing the interface constraint description in the test generation basis record into the interface context block of the first generation status block includes: Read the request field, the return field, the field format constraint, and the call precondition from the interface constraint description; The interface call entry point is determined based on the request fields and the call preconditions; the request field format and return field format are determined based on the field format constraints; and the assertion acceptance field is determined based on the return field. Write the interface call entry point into the interface location slot in the interface context block, write the request field format and the return field format into the test case format slot in the interface context block, and write the assertion acceptance field into the result validation slot in the interface context block, so that the interface context block carries the test case format constraints of the structured generated prompt record.

[0040] Specifically, in this application, the interface constraint description needs to be slotted before entering the interface context block. Slotting is used to write different types of content from the interface constraint description into corresponding slots in the interface context block. First, request fields, return fields, field format constraints, and preconditions are read from the interface constraint description. After the request fields are read, they can be used to determine the input fields that the test cases need to construct. After the preconditions are read, they can be used to determine the states that the program under test needs to satisfy before executing the interface call. When determining the interface call entry point based on the request fields and preconditions, the interface path, calling method, preconditions, and dependent data location can be combined to form the interface call entry point. The interface call entry point is not a single URL or function name, but rather call location information that describes how the test case enters the target interface of the program under test. After the interface call entry point is written into the interface location slot, the interface location slot is used to limit the target location of the test case call when generating subsequent candidate test case batches. The interface call entry point also participates in the test case format constraints of subsequent candidate test case batches through the interface location slot.

[0041] After field format constraints are read, they can be applied to request fields and return fields respectively. When determining the request field format based on field format constraints, the field name, field type, field level, required status, and field value boundaries of the request field can be written into the request field format. When determining the return field format based on field format constraints, the field name, field type, field level, and return structure of the return field can be written into the return field format. After the request field format and return field format are written into the test case format slot, the test case format slot can instruct the text generation service to use a test case structure that can be read by the test execution process when generating candidate test case batches. For example, if the request field format requires a certain request field to be an enumeration type, the input field records in the candidate test case batch should use the identifiable field value corresponding to that enumeration type; if the return field format requires a certain return field to be an array structure, the subsequent result validation content should be based on the element position or element field in the array structure. The identifiable field value corresponding to the enumeration type and the element position or element field in the array structure are both contents constrained by the test case format slot and can enter the candidate test case batch generation process along with the request field format and return field format.

[0042] After the returned fields are read, they can be used to determine the assertion acceptance fields. Assertion acceptance fields are the returned fields that need to be referenced in the result validation content. They do not indicate that the expected result is determined, but rather which fields should be read from the actual returned content of the program under test for validation after the test case is executed. After determining the assertion acceptance fields based on the returned fields, they are written into the result validation slot. The result validation slot is used in subsequent executable format encapsulation to bind the result validation content in the generated returned content to the assertion acceptance fields in the actual returned content of the program under test. The interface location slot, test case format slot, and result validation slot together form the internal carrying structure of the interface context block. Through this internal carrying structure, the interface context block can convert the original description in the interface constraint specification into structured generated prompt record test case format constraints, giving the candidate test case batch a clearer call entry point, input format, and result validation acceptance relationship during the generation phase.

[0043] Specifically, in this application, the interface context block remains unchanged in each round of refactoring because it corresponds to the interface constraint description of the program under test, rather than the feedback content generated after batch execution of candidate test cases. When the interface description text of the program under test remains unchanged, the interface call entry point, request field format, return field format, and assertion acceptance field do not change with the coverage trajectory. Conversely, the high-contribution demonstration block, the unreached path guidance block, and the failed test case negative feedback block correspond to the feedback guidance content generated after the execution feedback record is generated, and the feedback guidance content needs to change with the differential reach state. This distinction allows the structured generation prompt record to retain stable test case format constraints in each round of refactoring while allowing the feedback guidance content in the second generation state block to be replaced according to the execution feedback record. Thus, when reading the refactoring generation prompt record, the text generation service can continue to generate test cases according to the same interface format, while also generating more suitable generation return content based on the new coverage direction and failure avoidance direction.

[0044] Optionally, the step of inputting the structured generated prompt record into a text generation service to obtain a batch of candidate use cases, and executing the batch of candidate use cases in an isolated execution environment to generate an execution feedback record, includes: The structured generation prompt record is input into the text generation service, so that the text generation service generates the candidate use case batch corresponding to the structured generation prompt record based on the generation constraint content in the first generation status block and the feedback guidance content in the second generation status block. Assign a use case tracking identifier to each candidate use case in the candidate use case batch; The candidate use cases are loaded one by one according to the use case tracking identifier, and the program under test is driven to run in the isolated execution environment; Collect the code locations reached during the execution of the program under test, and write the code locations into the coverage trajectory corresponding to the use case tracing identifier; Collect the execution status and exception return text of the candidate use cases in the isolated execution environment, and write the execution status and exception return text into the execution receipt; The execution feedback record is generated from the coverage trajectory and the execution receipt.

[0045] Specifically, in this application, the text generation service receives structured generation prompt records and generates a batch of candidate test cases based on the block content in the structured generation prompt records. Before the structured generation prompt records are input to the text generation service, the first generation state block and the second generation state block can be sequentially organized according to the block location information to form a prompt input payload that can be sequentially read by the text generation service. The prompt input payload retains the generation constraint content in the first generation state block and the feedback guidance content in the second generation state block. The generation constraint content is used to limit the test case format of the candidate test case batch, and the feedback guidance content is used to limit the coverage direction and failure avoidance direction of the candidate test case batch. After receiving the prompt input payload, the text generation service first reads the generation constraint content in the first generation state block to determine the request fields, field formats, call preconditions, and assertion acceptance fields that each candidate test case in the candidate test case batch needs to include; then it reads the feedback guidance content in the second generation state block to determine the unreached code locations that the candidate test case batch should prioritize covering and the failure input features that should be avoided. By following the above reading order, the candidate test case batch will not deviate from the interface format of the program under test, and will also inherit the guidance of the previous round of execution feedback records on the coverage direction and failure avoidance direction.

[0046] After the text generation service outputs a batch of candidate test cases, the batch needs to be converted into a data format suitable for execution one by one in an isolated execution environment. A batch of candidate test cases can include multiple candidate test cases, each containing at least an input field record, a call acceptance record, and a result verification record. The input field record indicates the field name, field level, and field value submitted by the candidate test case to the program under test; the call acceptance record indicates the interface call entry point and preconditions for the candidate test case to enter the program under test; and the result verification record indicates the assertion acceptance fields that need to be read after the candidate test case is executed. To ensure that subsequent coverage traces and execution receipts can be traced back to the corresponding candidate test cases, a test case tracking identifier needs to be assigned to each candidate test case in the batch. The test case tracking identifier can be formed by combining the round identifier, the sequential position of the candidate test case in the batch, and the structured content markers of the candidate test case. After the test case tracking identifier is written to the candidate test case, a correspondence can be established between the candidate test case, the coverage trace, and the execution receipt through the same test case tracking identifier.

[0047] When executing candidate test cases in a batch within an isolated execution environment, it's crucial to prevent the execution status of the previous candidate test case from affecting the execution status of the next. The isolated execution environment can be comprised of the runtime dependency files of the program under test (DUT), the test runtime directory, the temporary data directory, and the coverage acquisition configuration. Before loading each candidate test case, the isolated execution environment can clean up any temporary data left over from the previous candidate test case and restore the basic runtime configuration of the DUT. When loading candidate test cases one by one according to the test case tracing identifier, the input field records in the candidate test cases are first read, and then submitted to the corresponding interface call entry point of the DUT according to the call acceptance record, subsequently driving the DUT to run. During the execution of the DUT, the isolated execution environment can enable a coverage acquisition tool to record the reached code locations. This coverage acquisition tool can collect the reached code locations through instrumentation markers, runtime hooks, or interpreted execution tracing. The collected code locations are bound to the test case tracing identifier of the currently executing candidate test case and written into the coverage trajectory corresponding to the test case tracing identifier.

[0048] The coverage trajectory describes the actual code locations reached by candidate test cases in the program under test. The code locations in the coverage trajectory can be expressed using the same code location representation as the historical coverage baseline, such as source file paths, function entry points, branch entry points, statement locations, or instrumentation numbers. Since the code coverage markers in the historical coverage baseline are also recorded according to code locations, the coverage trajectory can be directly compared with the historical coverage baseline later. When executing candidate test cases, it is also necessary to collect the execution status and exception return text of the candidate test cases in the isolated execution environment. The execution status indicates whether the candidate test case has completed loading, completed the interface call, produced a valid return, and stopped at the point of interruption. The exception return text records exception information, formatting errors, missing dependencies, or assertion acceptance errors generated during the execution of the candidate test cases. After the execution status and exception return text are written into the execution receipt, the execution receipt, together with the coverage trajectory, reflects the execution quality of the candidate test cases.

[0049] When generating execution feedback records from coverage traces and execution receipts, it's crucial to maintain the correspondence between them. Execution feedback records can be organized according to use case tracking identifiers, ensuring each candidate use case corresponds to its own coverage trace and execution receipt. Execution feedback records can also carry round identifiers, maintaining a round-based correspondence with test generation basis records and structured generation prompt records. After execution feedback records are generated, coverage traces are used to determine if candidate use cases reach code locations not recorded in the historical coverage baseline, and execution receipts are used to determine if candidate use cases are executable or have failed. Therefore, execution feedback records are not merely execution logs, but rather the direct data basis for generating subsequent refactoring prompts.

[0050] Specifically, in this application, taking the tested program corresponding to the grading interface as an example, a candidate test case in the candidate test case batch can carry an answer text field, a question identifier field, and question configuration data reference information. After assigning a test case tracking identifier to the candidate test case, the candidate test case is loaded in the isolated execution environment according to the test case tracking identifier, and the answer text field, question identifier field, and question configuration data reference information are submitted to the grading interface. During the execution of the tested program, the coverage acquisition tool records the normal scoring branch, empty answer branch, or abnormal format branch reached by the candidate test case, and writes these reached code locations into the coverage trajectory corresponding to the test case tracking identifier. If the candidate test case stops execution because the field format does not conform to the requested field format, the corresponding execution status and abnormal return text are written into the execution receipt. The coverage trajectory and execution receipt are then combined into an execution feedback record, so that subsequent steps can determine whether the candidate test case has generated a new reach status on the historical coverage baseline, and can also determine whether the candidate test case should participate in the update of the high contribution demonstration block or the failed test case negative feedback block.

[0051] Optionally, generating a prompt refactoring instruction based on the differential reach status of the execution feedback record relative to the test generation basis record includes: The coverage trajectory and execution receipt are read from the execution feedback record, and the historical coverage baseline is read from the test generation basis record; The code position in the coverage trajectory is compared with the code coverage marker in the historical coverage baseline to obtain the new reach record representing the new reach status and the repeated reach record representing the repeated reach status. The executable status and failure status are determined based on the execution status and exception return text in the execution receipt; A differential coverage reward record is generated based on the newly added reach record, the repeated reach record, the executable state, and the failure state; The block update object pointing to the second generated state block is determined based on the differential coverage reward record, and the prompt reconstruction instruction is generated based on the block update object and the differential coverage reward record.

[0052] Specifically, in this application, the differential reach status is used to represent the coverage change that occurs in the execution feedback record relative to the historical coverage baseline in the test generation basis record. When generating a prompt refactoring instruction, the coverage trajectory and execution receipt are first read from the execution feedback record, and then the historical coverage baseline is read from the test generation basis record. The coverage trajectory comes from the actual execution process of the current candidate test case batch in the isolated execution environment, and the historical coverage baseline comes from the code coverage markers recorded after the previous round of execution. Since both the code positions in the coverage trajectory and the code coverage markers in the historical coverage baseline are expressed using code positions, they can be compared under the same position expression rules. During the comparison, each code position in the coverage trajectory is matched sequentially with the code coverage marker in the historical coverage baseline. If the code position in the coverage trajectory does not appear in the historical coverage baseline, the code position is written to the new reach record; if the code position in the coverage trajectory already appears in the historical coverage baseline, the code position is written to the repeated reach record. The new reach record is used to represent the new reach status, and the repeated reach record is used to represent the repeated reach status.

[0053] Execution receipts are used to determine the executable and failure states of a batch of candidate test cases. When determining the executable state based on the execution state and exception return text in the execution receipt, candidate test cases that have completed loading, completed the API call, and generated a resolvable return value can be marked as executable. When determining the failure state based on the execution state and exception return text in the execution receipt, candidate test cases that have not completed loading, whose field format does not conform to the generated constraints, whose API call is interrupted, whose dependency data is missing, or who generated exception return text can be marked as failures. Executable and failure states are not isolated state fields; they participate in the generation of differential coverage reward records along with new and repeated reach records. If a candidate test case reaches the code location in a new reach record, but the execution receipt indicates that the candidate test case cannot reliably complete the API call, then the new reach record corresponding to that candidate test case needs to be constrained by the executable state during subsequent demonstration candidate test case selection. If a candidate test case repeatedly reaches the code coverage marker in the historical coverage baseline, and the execution receipt also indicates that the candidate test case has a failure state, then the failure state corresponding to that candidate test case can participate in the update of the failure test case negative feedback block.

[0054] Differential coverage reward records are feedback evaluation records that structurally aggregate new reach records, repeated reach records, executable status, and failure status. Differential coverage reward records can include new reach fields, repeated reach fields, executable fields, and failure fields. The new reach field records the new reach status of the candidate test case batch relative to the historical coverage baseline; the repeated reach field records the repeated reach status of the candidate test case batch relative to the historical coverage baseline; the executable field records the execution status of executable candidate test cases within the candidate test case batch; and the failure field records the failure status of failed candidate test cases within the candidate test case batch. Differential coverage reward records can combine these fields through configurable reach bias rules. These rules can give new reach status higher priority in the block update object determination process, and create a suppressive direction for repeated reach status and failure status during the block update object determination process. The reach bias rule can be pre-set in the test task configuration. The role of the reach bias rule is to convert the coverage change into the block reconstruction direction, rather than changing the internal parameters of the text generation service. After the reach bias rule participates in the differential coverage reward record, the differential coverage reward record can provide a rule basis for determining the block update object.

[0055] When determining the block update object based on the differential coverage reward record, the block update object needs to point to the second generated state block. The second generated state block is used to carry feedback guidance content, which includes demonstration guidance content in the high-contribution demonstration block, unreached path guidance content in the unreached path guidance block, and failure avoidance content in the failed use case negative feedback block. When the differential coverage reward record indicates the existence of a new reach record, the block update object pointing to the high-contribution demonstration block can be determined, allowing candidate use cases related to the new reach record to participate in the update of demonstration guidance content; when the differential coverage reward record indicates the existence of code locations reached by uncovered trajectories, the block update object pointing to the unreached path guidance block can be determined, allowing the unreached path guidance content to point to the unreached code locations; when the differential coverage reward record indicates the existence of a failure state, the block update object pointing to the failed use case negative feedback block can be determined, allowing the failure avoidance content to absorb the exception return text corresponding to the failure state. After the block update object is determined, a prompt refactoring instruction is generated based on the block update object and the differential coverage reward record. The generation process first forms a block update object, and then generates a hint refactoring instruction based on the block update object and the differential coverage reward record. This avoids reversing the technical logic by directly writing the hint refactoring instruction before it has been formed.

[0056] Specifically, in this application, the prompt reconstruction instruction may include a round identifier, a block update object, a reference position of the differential coverage reward record, and a block reconstruction action. The round identifier is used to correspond the prompt reconstruction instruction to the structured generation prompt record of the current round; the block update object is used to point to the block that needs reconstruction in the second generation state block; the reference position of the differential coverage reward record is used to read the newly reached record, repeated reached record, executable state, and failure state when executing the block reconstruction action; the block reconstruction action is used to explain how to subsequently reconstruct the feedback guidance content according to the block location information. After the prompt reconstruction instruction is generated, subsequent steps will reconstruct the feedback guidance content according to the block location information based on the prompt reconstruction instruction. Through this processing order, the generation and execution of the prompt reconstruction instruction are kept technically separate, which can reduce logical confusion between the generation and update stages.

[0057] Specifically, in this application, taking the grading interface as an example, the historical coverage baseline has recorded the normal scoring branch and the empty response branch, and the code position corresponding to the abnormal format branch appears in the current coverage trajectory. After comparing the code position in the coverage trajectory with the code coverage marker in the historical coverage baseline, the code position corresponding to the abnormal format branch is entered into the new reach record, and the code positions corresponding to the normal scoring branch and the empty response branch are entered into the repeated reach record. If the execution receipt indicates that the candidate use case has completed the interface call and returned a parsable result, then the candidate use case corresponds to the executable state; if the execution receipt indicates that the candidate use case has generated an abnormal return text due to missing fields, then the candidate use case corresponds to the failed state. The new reach record, repeated reach record, executable state, and failed state together generate a differential coverage reward record. Based on the differential coverage reward record, the block update object pointing to the high contribution demonstration block, the unreached path guidance block, or the failed use case negative feedback block in the second generated state block can be determined, and a prompt refactoring instruction is generated based on the block update object and the differential coverage reward record.

[0058] Optionally, after generating the differential coverage reward record based on the new reach record, the repeated reach record, the executable state, and the failure state, the method further includes: Each candidate use case is read from the batch of candidate use cases, and the corresponding coverage trajectory is obtained according to the use case tracking identifier corresponding to each candidate use case; The coverage trajectory of a single candidate use case is compared with the code coverage markers in the historical coverage baseline to obtain the new reach record of a single use case; Based on the newly added reach record in the single use case, an individual coverage contribution record is generated; Based on the individual coverage contribution record, a demonstration candidate use case is determined, and the demonstration candidate use case is written into the demonstration update slot in the prompt refactoring instruction; The demonstration update slot is bound to the block update object in the block update object that points to the high contribution demonstration block in the second generated state block, so that the demonstration candidate use case is used to update the demonstration guidance content in the high contribution demonstration block.

[0059] Specifically, in this application, the differential coverage reward record can reflect the overall reach change of the candidate use case batch relative to the historical coverage baseline, but the overall reach change of the candidate use case batch cannot directly indicate the individual contribution of each candidate use case in the batch. To ensure that the demonstration guidance content in high-contribution demonstration blocks originates from candidate use cases with significant generation value, individual coverage contribution records need to be generated after the differential coverage reward record. When generating individual coverage contribution records, each candidate use case is first read from the candidate use case batch, and the corresponding coverage trajectory is obtained based on the use case tracking identifier corresponding to each candidate use case. Since the coverage trajectory is already bound to the use case tracking identifier during the execution phase, the coverage trajectory corresponding to the candidate use case can be read from the execution feedback record based on the use case tracking identifier. In this way, the coverage trajectory of a single candidate use case can form a corresponding relationship with the candidate use case itself, avoiding the misinterpretation of the overall coverage result of the candidate use case batch as the contribution of a single candidate use case.

[0060] When comparing the coverage trajectory of a single candidate use case with the code coverage markers in the historical coverage baseline, the code location expression rules consistent with the differential reach state calculation are used. If a code location in the coverage trajectory of a single candidate use case does not appear in the historical coverage baseline, that code location is added to the single use case new reach record. The single use case new reach record represents the new reach state formed relative to the historical coverage baseline when a single candidate use case is executed independently. Since the single use case new reach record originates from the coverage trajectory of only one candidate use case, it can distinguish the differences in contribution of different candidate use cases in a batch of candidate use cases to the new reach state. When generating an individual coverage contribution record based on the single use case new reach record, the number of code locations, code location type, function where the code location is located, and the executable state of the candidate use case in the single use case new reach record can be written into the individual coverage contribution record. The individual coverage contribution record is used to represent the independent contribution state of the candidate use case to the part not reached by the historical coverage baseline and is used to subsequently determine demonstration candidate use cases.

[0061] When determining demonstration candidate use cases based on individual coverage contribution records, these cases must not only have a new reach status but also an executable status. Individual coverage contribution records can be sorted or stratified according to the new reach contribution status. Candidate use cases with a high new reach contribution status and meeting the executable status requirements can be identified as demonstration candidate use cases. After the demonstration candidate use cases are determined, they are written into the demonstration update slot in the prompt refactoring instruction. The demonstration update slot is the block update position set by the prompt refactoring instruction for high-contribution demonstration blocks, used to carry the demonstration candidate use cases subsequently written into the high-contribution demonstration blocks. After the demonstration candidate use cases are written into the demonstration update slot, the slot is bound to the block update object in the block update object that points to the high-contribution demonstration block in the second generated state block. This binding relationship allows the prompt refactoring instruction to locate the high-contribution demonstration block through the block update object during subsequent execution and convert the demonstration candidate use cases in the demonstration update slot into demonstration guidance content in the high-contribution demonstration block.

[0062] Specifically, in this application, the demonstration candidate use cases are not arbitrarily successful candidate use cases. Demonstration candidate use cases need to come from a batch of candidate use cases and need to be mapped to a coverage trajectory through use case tracking identifiers. If a candidate use case only repeatedly touches the code coverage markers already recorded in the historical coverage baseline, even if the candidate use case is in an executable state, it should not be used as a demonstration candidate use case to enter the demonstration update slot; if a candidate use case touches a code location not recorded in the historical coverage baseline, but the execution receipt indicates that the candidate use case is in a failed state, it should not be used directly as a demonstration candidate use case to enter the demonstration update slot. By jointly determining demonstration candidate use cases through individual coverage contribution records and executable states, the demonstration guidance content in high-contribution demonstration blocks can be more closely aligned with candidate use cases that can generate new touch states and can be run in an isolated execution environment, thereby making the generation direction of the next batch of candidate use cases clearer.

[0063] Specifically, in this application, taking the grading interface as an example, the candidate test case batch can include multiple candidate test cases. The first candidate test case reaches the normal scoring branch, the second candidate test case reaches the abnormal format branch, and the third candidate test case fails to execute due to the lack of a question identifier field. After obtaining the coverage trajectory corresponding to the first, second, and third candidate test cases according to the test case tracking identifier, the coverage trajectory of each candidate test case is compared with the code coverage marker in the historical coverage baseline. If the historical coverage baseline already includes the normal scoring branch but does not include the abnormal format branch, the second candidate test case forms a single test case new reach record. If the execution receipt of the second candidate test case indicates that the candidate test case is in an executable state, an individual coverage contribution record is generated according to the single test case new reach record of the second candidate test case, and the second candidate test case is determined as a demonstration candidate test case based on the individual coverage contribution record. After the demonstration candidate test case is written into the demonstration update slot, the demonstration update slot is bound to the block update object pointing to the high contribution demonstration block, so that the demonstration candidate test case is used to update the demonstration guidance content in the high contribution demonstration block during subsequent refactoring.

[0064] Optionally, determining the exemplary candidate use case based on the individual coverage contribution record includes: The new reach contribution status corresponding to each candidate use case is determined based on the individual coverage contribution record; Based on the executable status, filter out non-executable candidate use cases from the candidate use case batch; Among the retained candidate use cases, candidate use cases that meet the demonstration admission conditions are determined based on the newly added reach contribution status, and the candidate use cases that meet the demonstration admission conditions are determined as the demonstration candidate use cases; The test case tracking identifier, input field record, and single test case new reach record corresponding to the demonstration candidate test case are written into the demonstration update slot, so that the demonstration guidance content corresponds to the new reach record in the differential coverage reward record.

[0065] Specifically, in this application, the newly added reach contribution status is used to indicate the contribution of each candidate use case to the unreached portion of the historical coverage baseline. When determining the newly added reach contribution status based on the individual coverage contribution record, the single use case newly reached record, code location type, and executable status can be read from the individual coverage contribution record. The single use case newly reached record indicates the newly reached code location independently reached by the candidate use case. The code location type is used to distinguish whether the newly reached code location belongs to a conditional branch, loop entry, exception entry, or ordinary statement location. The executable status is used to determine whether the candidate use case can be completely executed by the isolated execution environment. Through the above, a newly added reach contribution status can be formed for each candidate use case. The newly added reach contribution status can be represented by a hierarchical status or a sorted status, and the newly added reach contribution status is used for subsequent demonstration admission condition judgment.

[0066] When filtering out non-executable candidate test cases from the candidate test case batch based on their executable status, it is necessary to first read the execution status and exception return text in the execution receipt. If the execution status indicates that the candidate test case has not completed loading, has not completed the interface call, cannot generate a parseable return, or generates exception return text, then the candidate test case is considered non-executable. Although non-executable candidate test cases may briefly reach certain code locations during runtime, they cannot reliably form input field records and result validation records that can be used for subsequent reference. Therefore, non-executable candidate test cases are filtered out from the candidate test case batch. The remaining candidate test cases after filtering all have an executable status and continue to participate in the demonstration admission condition judgment. This process can prevent the failure input features corresponding to the failure status from entering the high-contribution demonstration block, reducing the probability of the next round of candidate test case batches generating the same type of failure input features.

[0067] Demonstration admission criteria can be pre-configured in the test task configuration corresponding to the generation request, or generated based on the historical coverage baseline and the new reach targets of the current round. Demonstration admission criteria must include at least that the retained candidate test cases have an executable state, and that the new reach record for the single test case corresponding to the retained candidate test case is not empty. Further, demonstration admission criteria may also include that the input field records of the retained candidate test cases conform to the test case format slots in the interface context block, and that the new reach contribution status of the retained candidate test cases points to a branch entry, loop entry, or exception entry not covered by the historical coverage baseline. Among the retained candidate test cases, the new reach contribution status is used to determine whether the demonstration admission criteria are met; candidate test cases that meet the demonstration admission criteria are identified as demonstration candidate test cases. The process of determining demonstration candidate test cases ensures that the demonstration guidance content in the high-contribution demonstration block does not simply reuse successful execution samples, but prioritizes reusing candidate test cases that contribute to the new reach status and meet the generation constraints.

[0068] Once the candidate demonstration use cases are determined, the corresponding use case tracking identifier, input field record, and single use case new reach record are written to the demonstration update slot. The use case tracking identifier is used to locate the candidate demonstration use case when tracing its origin later; the input field record is used to create input examples for the text generation service to reference in the high-contribution demonstration block; and the single use case new reach record indicates which unrecorded code locations in the historical coverage baseline were reached by the candidate demonstration use case. After the demonstration update slot is filled with the above information, a correspondence is established between the demonstration guidance content and the new reach records in the differential coverage reward record. This correspondence ensures that the demonstration guidance content in the high-contribution demonstration block reflects the new reach direction indicated by the differential coverage reward record, rather than just reflecting the input style of the candidate use case itself.

[0069] Specifically, in this application, the input field records in the demonstration update slot must maintain the same format as the request fields in the interface context block. If the request field format in the interface context block requires the answer text field to be of text type and the question identifier field to be of identifier type, then when the demonstration candidate use case is written to the demonstration update slot, the input field records should still retain the field names, field levels, and field values ​​of the answer text field and the question identifier field. The single use case new reach record in the demonstration update slot must maintain the same code location expression as the code coverage marker in the historical coverage baseline, so that the demonstration guidance content can be mapped to the new reach record in the differential coverage reward record later. After the use case tracking identifier, input field records, and single use case new reach records are written together to the demonstration update slot, the demonstration update slot can be converted into the demonstration guidance content in the high contribution demonstration block when the refactoring instruction is executed.

[0070] Specifically, in this application, the demonstration guidance content in the high-contribution demonstration block can adopt natural language use case descriptions or structured use case fragments. Regardless of the expression used, the demonstration guidance content must retain the relationship between the input field records corresponding to the demonstration candidate use cases and the single use case new reach records. Taking the grading interface as an example, the input field records of the demonstration candidate use cases may include the answer text field being a combination of spaces and special symbols, the question identifier field being a valid identifier, and the existence of question configuration data reference information; the single use case new reach record may point to the code location corresponding to the abnormal format branch. After the demonstration update slot writes the above input field records and single use case new reach records together, the demonstration guidance content in the high-contribution demonstration block can guide the text generation service to generate more candidate use cases that can reach the abnormal format branch and conform to the interface constraint description. In this way, the demonstration guidance content reflects both the use case format constraints and the new reach direction, maintaining a technical connection with the new reach records in the differential coverage reward record.

[0071] Specifically, in this application, "high contribution" in the high contribution demonstration block refers to the new reach contribution status formed relative to the historical coverage baseline after a candidate use case meets the demonstration admission conditions in the current round of candidate use case batches. The new reach contribution status can be determined based on the new reach record of a single use case. The more code locations contained in the new reach record of a single use case, and the more concentrated these code locations are in branch entry points, loop entry points, or exception entry points not recorded in the historical coverage baseline, the higher the contribution of the corresponding candidate use case to the coverage direction of the current round. Since this judgment depends on the historical coverage baseline, coverage trajectory, and executable status, "high contribution" is not a subjective evaluation of the candidate use case text itself, but a data-driven screening of the coverage increment formed after the candidate use case is actually executed in the isolated execution environment.

[0072] Specifically, in this application, candidate use cases that simultaneously meet the following conditions can be used as candidate sources for high-contribution demonstration blocks: the candidate use case has an executable state; the single use case new reach record corresponding to the candidate use case is not empty; the input field record of the candidate use case conforms to the use case format slot in the interface context block; and the single use case new reach record corresponding to the candidate use case points to a code location not recorded in the historical coverage baseline. If there are multiple candidate use cases that meet the above conditions in the current round, the candidate use cases can be sorted according to the number of newly reached code locations, the type of code location, and the importance of the path corresponding to the code location in the single use case new reach record, and the candidate use cases ranked higher are determined as demonstration candidate use cases. For example, candidate use cases that reach exception entry points, conditional branch entry points, or loop entry points can be given priority over candidate use cases that only reach ordinary sequential statement locations when entering the demonstration update slot.

[0073] Specifically, in this application, exemplary admission criteria are used to translate "high contribution" into executable screening rules. Exemplary admission criteria may include basic admission criteria and contribution admission criteria. Basic admission criteria are used to exclude unexecutable candidate use cases; for example, a candidate use case should complete loading, complete the interface call, and generate a resolvable return, and the execution receipt should not contain any abnormal return text that would cause the candidate use case to be interrupted. Contribution admission criteria are used to determine whether a candidate use case has generated effective new reach to the historical coverage baseline; for example, the coverage trajectory corresponding to the candidate use case contains code locations that are not present in the historical coverage baseline, and these code locations can be traced back to the corresponding use case tracing identifier through single use case new reach records. Only when both basic and contribution admission criteria are met does the candidate use case enter the screening scope of exemplary candidate use cases.

[0074] Specifically, in this application, "candidate use cases with relatively high generation value" can be understood as candidate use cases that, after being screened by the demonstration admission criteria, can provide input structure references and coverage direction references for the next round of candidate use case batches. The input field record of the candidate use case is used to provide a reusable input organization method to the text generation service, the single use case new reach record of the candidate use case is used to provide a new reach direction to the text generation service, and the executable status of the candidate use case is used to indicate that the input organization method can run in an isolated execution environment. By simultaneously retaining the input field record, the single use case new reach record, and the executable status, the demonstration guidance content in the high contribution demonstration block can avoid merely copying the surface input form, but instead jointly pass the executable input form and the new reach direction to the next round of structured generation prompt records.

[0075] Specifically, in this application, to avoid the meaning of "high contribution" shifting across different rounds, the updating of high contribution demonstration blocks can be limited to the candidate use case batch of the current round. That is, first, the coverage trajectory corresponding to each candidate use case is read based on the use case tracking identifier within the candidate use case batch of the current round. Then, the coverage trajectory corresponding to each candidate use case is compared with the historical coverage baseline to obtain the single-use case new reach record for each candidate use case. Subsequently, demonstration candidate use cases are determined based on the single-use case new reach record and the executable status. Through this round-based screening method, the high contribution demonstration block reflects the new reach contribution status of the current round relative to the historical coverage baseline, rather than a generalized evaluation across projects, interfaces, or programs under test.

[0076] Specifically, this application applies the aforementioned test case generation method to continuous integration / continuous delivery pipelines, regression testing of large software systems, and quality assurance scenarios for educational technology products. In continuous integration / continuous delivery pipelines, the generation request can be triggered by an automated test task after code submission, and the program under test can be a service interface that requires regression testing after this submission. In the quality assurance scenario of educational technology products, the program under test can be a complex business logic interface in a question bank system, learning path engine, or grading system. Taking a grading system as an example, the test generation basis record can be formed jointly by the codebase of the system under test and the interface documentation. The codebase of the system under test provides the executable code line range corresponding to the grading interface, and the interface documentation provides the interface definition, parameter types, and business constraints of the grading interface. The interface constraint descriptions in the test generation basis record can be extracted from the interface documentation or code comments, and the historical coverage baseline in the test generation basis record can be formed by the cumulative coverage code line set obtained after executing the test case set in the previous round in the isolated sandbox environment. Therefore, the test generation context carries both the call format of the grading interface and the overwrite state after the previous round of execution, providing a readable data foundation for the subsequent evolution of structured prompt words.

[0077] Specifically, in this application, the structured generation of prompt records can be implemented as a four-block standardized prompt word structure, which includes [Interface Context Block], [High-Value Demonstration Block], [Uncovered Path Guidance Block], and [Failed Use Case Negative Feedback Block]. The [Interface Context Block] corresponds to the Interface Context Block in the first generation state block mentioned above, and is used to retain the interface definition, parameter types, and business constraints of the system under test; the [High-Value Demonstration Block] corresponds to the High-Contribution Demonstration Block in the second generation state block mentioned above, and is used to write demonstration candidate use cases in the current batch that can contribute to the newly covered code paths; the [Uncovered Path Guidance Block] corresponds to the Unreached Path Guidance Block in the second generation state block mentioned above, and is used to write the key uncovered branch information extracted from the uncovered code line set; the [Failed Use Case Negative Feedback Block] corresponds to the Failed Use Case Negative Feedback Block in the second generation state block mentioned above, and is used to write the use case summary and error reason for the failure in this round of execution. In an optional embodiment, the first generation state block can also be configured with a [generation instruction block]. The [generation instruction block] is used to guide the text generation service to generate a specified number of format specification test cases, such as generating a specified number of test cases. This specified number is represented by the symbol N, where N represents the number of candidate test cases to be generated in each round. It can be pre-configured according to the size of the program under test. For example, N can be configured to 50. Since the [generation instruction block] is only used to maintain format specification instructions and does not change with coverage feedback, the [generation instruction block] can form the generation constraint content together with the [interface context block].

[0078] Specifically, in this application, the text generation service can employ any large language model capable of text generation, such as GPT-type large language models, Claude-type large language models, or other autoregressive language generation models based on the Transformer architecture. In this application, the Transformer architecture refers to a text sequence generation structure composed of a multi-layer attention computation structure and a feedforward transformation structure. After receiving structured generation prompt records, this type of large language model generates candidate test case batches according to the text constraints in the structured generation prompt records. This application does not limit the number of layers, training corpus, or pre-training loss function of the large language model, nor does it perform supervised training, gradient backpropagation, or model parameter fine-tuning on the large language model during test case generation. This is because the policy update in this application is not achieved by changing the internal parameters of the large language model, but rather by structurally reconstructing the [high-value demonstration blocks], [uncovered path guidance blocks], and [failed test case negative feedback blocks]. In other words, the large language model only needs to be able to read text prompts and output text content. The test execution results are fed back to the next round of structured generation prompt records through block content updates. Therefore, this application can be used with any text generation model without depending on a specific large language model structure.

[0079] Specifically, in this application, the structured generation prompt record for the initial round may include an [interface context block], a [high-value demonstration block], and a [generation instruction block]. In the initial round, the [interface context block] contains the interface definition, parameter types, and business constraints of the grading interface, such as the answer text field, question identifier field, scoring result field, and the existence status of question configuration data; the [high-value demonstration block] can be initially empty or filled with a small number of seed test cases; the [generation instruction block] is used to instruct the text generation service to output N test cases with standardized formats. The text generation service generates the first batch of test cases based on the structured generation prompt record of the initial round, and this first batch of test cases corresponds to the aforementioned batch of candidate test cases. After the first batch of test cases is submitted to the isolated sandbox environment for execution, coverage data is collected using a coverage instrumentation tool. The coverage instrumentation tool can be JaCoCo, Istanbul, or Coverage.py, among others. JaCoCo is suitable for code coverage collection of Java programs, Istanbul is suitable for code coverage collection of JavaScript programs, and Coverage.py is suitable for code coverage collection of Python programs. The aforementioned coverage instrumentation tools all record the lines of code, branches, or path locations reached during the execution of test cases and convert these locations into coverage traces.

[0080] Specifically, in this application, the differential coverage reward calculation can adopt the following differential coverage composite reward function: R_i = α·|C_i\C_{i-1}| / |C_total| + β·|C_i| / |C_total| - γ·P_dup.

[0081] Where R_i represents the overall batch reward value of the candidate test case batch in the i-th round, and i represents the current generation round; C_i represents the cumulative covered code lines set after the execution of the i-th round; C_{i-1} represents the cumulative covered set of the previous round, which is an empty set in the initial round; C_total represents the full executable code line set of the tested system, which comes from the range of executable code lines in the code library of the tested system; C_i\C_{i-1} represents the newly added covered lines set in this round, which is the remaining cumulative covered code lines set in the i-th round after removing the cumulative covered set of the previous round. The remaining set of code lines; |C_i\C_{i-1}| represents the number of code lines in the newly added coverage set in this round; |C_total| represents the number of code lines in the full executable code line set of the tested system; α represents the new coverage weight, β represents the cumulative coverage weight, and γ represents the duplication penalty weight. These three can be pre-configured; for example, α can be configured to 0.7, β to 0.2, and γ to 0.1; P_dup represents the coverage duplication rate of this batch of test cases, used to measure the degree to which this batch of candidate test cases repeatedly reaches historically covered code lines. In the above formula, the first term is used to highlight the contribution of the newly added coverage set in this round, the second term is used to retain the description of the cumulative coverage ratio on the overall test progress, and the third term is used to suppress the candidate test case batch from repeatedly covering historically covered code lines.

[0082] Specifically, in this application, the coverage repetition rate P_dup of the test cases in this batch can be calculated based on the actual set of covered code lines, without relying on any model. For each test case j in the candidate test case batch, first read the set of code lines covered by test case j when it is executed independently, c_j, then read the cumulative coverage set C_{i-1} from the previous round, and then calculate the coverage repetition similarity sim_j corresponding to test case j. The formula for calculating the coverage repetition similarity sim_j is: sim_j = |c_j∩C_{i-1}| / |c_j∪C_{i-1}|.

[0083] Where j represents the test case number within the candidate test case batch; c_j represents the set of code lines covered when the j-th test case is executed independently; C_{i-1} represents the cumulative coverage set from the previous round; c_j∩C_{i-1} represents the intersection between the set of code lines covered by the j-th test case and the cumulative coverage set from the previous round, where the code lines in this intersection represent the historically covered code lines repeatedly reached by the j-th test case; c_j∪C_{i-1} represents the union between the set of code lines covered by the j-th test case and the cumulative coverage set from the previous round, where this union represents the total range of code lines participating in the repetition comparison; |c_j∩C_{i-1}| represents the number of code lines in the intersection; and |c_j∪C_{i-1}| represents the number of code lines in the union. The coverage repetition similarity sim_j of all test cases within the candidate test case batch is averaged to obtain the coverage repetition rate P_dup for this batch of test cases. By using this calculation method based on the set of code lines, the coverage repetition rate P_dup of this batch of test cases can reflect the degree to which the candidate test case batch as a whole repeatedly covers the historically covered code lines, and the calculation result comes from the actual execution result in the isolated sandbox environment.

[0084] Specifically, in this application, the individual coverage contribution score of a test case can be calculated using the following formula: r_j = |c_j\C_{i-1}| / |C_total|.

[0085] Where r_j represents the individual coverage contribution score of the j-th test case; c_j represents the set of code lines covered when the j-th test case is executed independently; C_{i-1} represents the cumulative coverage set from the previous round; c_j\C_{i-1} represents the set of newly covered code lines remaining after removing the cumulative coverage set from the previous round from the set of code lines covered by the j-th test case; |c_j\C_{i-1}| represents the number of newly covered code lines generated by the j-th test case; C_total represents the set of all executable code lines of the tested system; |C_total| represents the total number of executable code lines of the tested system. The individual coverage contribution score r_j is used to measure the independent contribution of the j-th test case relative to the historical coverage baseline. For example, if the total number of executable lines of code in the tested system corresponding to the grading system is 10,000, and the previous round of cumulative coverage has covered 6,000 lines, and the j-th test case covers 120 lines when executed independently, of which 30 lines did not appear in the previous round of cumulative coverage, then the individual coverage contribution of the j-th test case is 30 divided by 10,000, which is 0.003. This value is not a subjective evaluation of the quality of the test case text, but rather the proportion of historically uncovered lines of code actually reached by the j-th test case in the isolated sandbox environment.

[0086] Specifically, in this application, the update of the [high-value demonstration block] can directly use the individual coverage contribution score r_j as the sorting basis. After each round of execution, each test case in the candidate test case batch is executed individually, and the individual coverage contribution score r_j is calculated based on the coverage code line set c_j corresponding to each test case. Subsequently, the candidate test cases are sorted in descending order according to the individual coverage contribution score r_j, and the top K test cases in the current batch are taken as positive sample demonstrations and injected into the [high-value demonstration block]. Here, K represents the number of demonstration test cases entering the [high-value demonstration block] in each round, which can be pre-configured. For example, K can be configured as 10. This update rule can be expressed as: after each round of execution, the top K tests in the current batch are unconditionally updated in descending order according to the individual contribution score r_j, without relying on the comparison of R_i. The meaning of this rule is that the demonstration samples of the [high-value demonstration block] come from the individual coverage contribution score sorting within the current candidate test case batch, rather than depending on whether the overall reward value R_i of this round is higher than that of the previous round. In this way, even if the overall batch reward value R_i of a candidate use case batch does not change much in a certain round, use cases with higher individual coverage contribution scores r_j can be extracted from that candidate use case batch as demonstration inputs for the next round, so that high contribution demonstration blocks can continuously absorb candidate use cases that contribute to the new coverage row set.

[0087] Specifically, in this application, the update of the [Uncovered Path Guidance Block] can be based on the set of uncovered lines of code. The set of uncovered lines of code represents the set of lines of code remaining in the full executable code of the system under test after removing the currently accumulated covered parts, which have not yet been executed by any test cases. The set of uncovered lines of code can be obtained by the difference between C_total and C_i, that is, by removing the set of accumulated covered lines of code C_i after the i-th round of execution from the set of full executable code of the system under test C_total. After obtaining the set of uncovered lines of code, key uncovered branch information is extracted from the set of uncovered lines of code. This key uncovered branch information includes the function name, branch condition description, and line number. For conditional branches, the conditional branches can be expressed in natural language as branches when a certain condition in the function is true or false; for loop statements, the loop statements can be expressed in natural language as the iteration path of the loop variable in the function; for exception handling statements, the exception handling statements can be expressed in natural language as the exception catching path of a certain exception type in the function. For example, in a grading system, if the set of uncovered code lines points to the exception branch in the function `checkUserLevel` where the score is less than 0, then the `[Uncovered Path Guiding Block]` can contain the message: "The following code branches are currently not covered. Please prioritize generating test cases that can reach these branches: the exception branch in the function `checkUserLevel` where the score is less than 0." This natural language template is decoupled from specific programming languages, and therefore can be used for programs under test implemented in different languages ​​such as Java, JavaScript, or Python.

[0088] Specifically, in this application, the update of the [Failed Use Case Negative Feedback Block] can be determined based on the failed use cases in this round of execution. Failed use cases can include runtime exception use cases and format error use cases. Runtime exception use cases refer to use cases that trigger exception return text when a candidate use case is executed in an isolated sandbox environment; format error use cases refer to use cases where the input field records of a candidate use case do not conform to the interface definition, parameter type, or business constraints in the [Interface Context Block]. When there are failed use cases in the current round of candidate use case batches, the failure reasons of the failed use cases are extracted, and the summary of the failed use cases and the error reasons are written into the [Failed Use Case Negative Feedback Block] to form failure avoidance content. For example, if the grading interface requires the question identifier field to be a non-empty identifier, and a candidate use case lacks the question identifier field, resulting in a field missing error in the execution receipt, then the input field record and field missing error of that candidate use case can be written into the [Failed Use Case Negative Feedback Block]. When there are no failed use cases in the current round of candidate use case batches, the [Failed Use Case Negative Feedback Block] retains the content from the previous round and does not perform unfounded replacements. This rule can avoid constructing incorrect failure avoidance content when there is no basis for failure.

[0089] Specifically, in this application, the [interface context block] remains unchanged in each iteration, and the [generation instruction block] can also remain unchanged in each iteration. The [high-value demonstration block], [uncovered path guidance block], and [failed test case negative feedback block] are dynamically updated based on the execution results. Therefore, the structured reconstruction of prompt words in this application does not involve updating the model parameter gradients of the text generation service, but rather a regular replacement of the block content in the structured generated prompt record. For the large language model, the input received in each round is still the structured generated prompt record in text form; for the test case generation process, the feedback guidance content in each round of the structured generated prompt record comes from the execution results of the isolated sandbox environment, the coverage data collected by the coverage instrumentation tool, the differential coverage composite reward function, and the test case individual contribution score calculation method. Since the text generation service does not need to receive backpropagation gradients or load new training samples for supervised training, this application can complete multiple rounds of policy updates while keeping the parameters of the large language model unchanged.

[0090] Specifically, in this application, we can continue to use the grading system as an example to illustrate a complete execution round. First, extract the [interface context block] from the interface documentation and code comments of the grading system, such as extracting the answer text field, question identifier field, parameter type, business constraints, and scoring result field. Then, configure the output of N formatted test cases in the [generation instruction block]. For example, N is configured as 50. If this is the initial round, the [high-value demonstration block] is empty or a small number of seed test cases are written, and the [failure test case negative feedback block] is empty. Then, the text generation service generates the first batch of test case sets based on the structured generation prompts. After the first batch of test case sets is submitted to the isolated sandbox environment for execution, the coverage instrumentation tool records the coverage code line set c_j for each test case and records the cumulative coverage code line set C_i after this round of execution. Subsequently, C_i is compared with the cumulative coverage set C_{i-1} from the previous round to obtain the newly added coverage line set C_i\C_{i-1} for this round. Then, the overall reward value R_i of the batch is calculated based on the differential coverage rate composite reward function. At the same time, the individual coverage contribution score r_j is calculated for each test case, and the top K test cases of the current batch are selected in descending order of individual coverage contribution score r_j and written into the [High-Value Demonstration Block]; the function name, branch condition description, and line number are extracted from the set of uncovered code lines and written into the [Uncovered Path Guidance Block]; if there are runtime exceptions or format errors, the corresponding failed test case summary and error reason are written into the [Failed Test Case Negative Feedback Block]. When the text generation service is called again in the next round, the text generation service reads the updated [High-Value Demonstration Block], [Uncovered Path Guidance Block], and [Failed Test Case Negative Feedback Block] to generate a new batch of candidate test cases.

[0091] Specifically, in this application, the termination condition can be configured based on branch coverage, the percentage of newly added coverage in consecutive rounds, and the maximum number of iteration rounds. Branch coverage represents the proportion of branches reached during test case execution to the total number of executable branches in the tested program; the percentage of newly added coverage in consecutive rounds represents the proportion of newly covered lines of code in each round to the total number of executable lines of code; and the maximum number of iteration rounds limits the number of rounds in which the structured generation of prompt records is updated cyclically. For example, iteration can be terminated when the branch coverage reaches a preset threshold, which can be configured to 85%; it can also be terminated after 5 consecutive rounds when the percentage of newly covered lines of code in each round is less than 0.1%; or it can be terminated when the maximum number of iteration rounds is reached, which can be configured to 100 rounds. After termination, the final test case set, coverage report, and reward curves for each round can be output. The final test case set can deduplicate test cases, the coverage report can record the cumulative coverage of each round, and the reward curve for each round can record the change in the overall reward value R_i of each batch. With the above outputs, those skilled in the art can implement the test case generation process of this application based on the interface documentation of the program under test, the code library of the system under test, the coverage instrumentation tool, and the text generation service.

[0092] like Figure 2 As shown, this application also provides a test case generation apparatus, comprising: The context generation unit is used to respond to the generation request, obtain the test generation basis record of the program under test, and generate the test generation context. The prompt record construction unit is used to construct a structured generation prompt record with block positioning information based on the test generation context. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. The candidate execution unit is used to input the structured generated prompt record into the text generation service to obtain a batch of candidate test cases, and execute the batch of candidate test cases in the isolated execution environment to generate execution feedback records; The prompt reconstruction unit is used to generate prompt reconstruction instructions based on the differential reach status of the execution feedback record relative to the test generation basis record, and to reconstruct the feedback guidance content according to the block positioning information based on the prompt reconstruction instructions to obtain the reconstruction generation prompt record. The test case encapsulation unit is used to re-input the refactoring generated prompt record into the text generation service, receive the generated return content output by the text generation service, and encapsulate the generated return content into an executable format to generate test cases.

[0093] like Figure 3As shown, this application also provides an electronic device, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to read and execute the computer instructions to implement any of the above-described test case generation methods.

[0094] like Figure 4 As shown, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement any of the above-described test case generation methods.

[0095] like Figure 5 As shown, this application also provides a test case generation system, including a front-end device and a back-end cloud server, wherein the front-end device and the back-end cloud server are communicatively connected; The front-end device is used to perform the following steps: receive a generation request for the program under test; send the generation request to the back-end cloud server; receive the test cases returned by the back-end cloud server; and display the test cases. The backend cloud server performs the following steps in response to the generation request: It obtains the test generation basis record of the program under test to generate a test generation context; based on the test generation context, it constructs a structured generation prompt record with block location information. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content; it generates a batch of candidate test cases based on the structured generation prompt record and executes the batch of candidate test cases in an isolated execution environment to generate an execution feedback record; it generates a prompt refactoring instruction based on the differential reach state of the execution feedback record relative to the test generation basis record, and refactors the feedback guidance content according to the block location information based on the prompt refactoring instruction to obtain a refactored generation prompt record; it generates generation return content based on the refactored generation prompt record and encapsulates the generation return content in an executable format to generate test cases; and it returns the test cases to the frontend device.

[0096] Figures 2-5 For an exemplary description, please refer to the above. Figure 1 This will not be elaborated upon here.

Claims

1. A test case generation method, characterized in that, include: In response to the generation request, the test generation basis record of the program under test is obtained to generate the test generation context; Based on the test generation context, a structured generation prompt record with block location information is constructed. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. The structured generated prompt record is input into the text generation service to obtain a batch of candidate test cases, and the batch of candidate test cases is executed in the isolated execution environment to generate execution feedback records; Based on the differential reach status of the execution feedback record relative to the test generation basis record, a prompt reconstruction instruction is generated, and the feedback guidance content is reconstructed according to the block positioning information based on the prompt reconstruction instruction to obtain the reconstruction generation prompt record; The reconstructed prompt record is input into the text generation service again, the generated return content output by the text generation service is received, and the generated return content is encapsulated in an executable format to generate test cases.

2. The test case generation method according to claim 1, characterized in that, The step of responding to the generation request by obtaining the test generation basis record of the program under test to generate a test generation context includes: In response to the generation request, determine the program under test to which the generation request is directed; Read the interface description text corresponding to the program under test, extract the request field, return field, field format constraint and call precondition from the interface description text, and write the request field, the return field, the field format constraint and the call precondition into the interface constraint description; Read the coverage marker file generated by the coverage acquisition tool after the previous round of execution, extract code coverage markers from the coverage marker file, and record the code coverage markers to the historical coverage baseline according to the executable code location; The interface constraint description and the historical coverage baseline are written into the test generation basis record, and a round identifier is assigned to the test generation basis record to generate the test generation context with the round identifier and corresponding to the program under test.

3. The test case generation method according to claim 1, characterized in that, The step of constructing a structured generation prompt record with block location information based on the test generation context includes: Read the test generation basis record from the test generation context, and determine the generation constraint content and initial feedback guidance content based on the test generation basis record; In the structured generation prompt record, a first generation status block and a second generation status block are set. The generation constraint content is written into the first generation status block, and the initial feedback guidance content is written into the second generation status block. The first generation status block is used to constrain the test case format of the candidate test case batch, and the second generation status block is used to guide the coverage direction and failure avoidance direction of the candidate test case batch. Write a block start marker, a block end marker, and a round marker to the first generated state block and the second generated state block respectively, so that the block location information is formed by the block start marker, the block end marker, and the round marker; The block location information is written into the prompt index table, so that the structured generated prompt record maintains a round correspondence with the test generation context through the prompt index table.

4. The test case generation method according to claim 3, characterized in that, The first generated state block includes an interface context block, and the second generated state block includes a high contribution demonstration block, an unreached path guidance block, and a failed use case negative feedback block. The step of writing the generation constraint content into the first generation state block and writing the initial feedback guidance content into the second generation state block includes: The interface constraint description in the test generation basis record is written into the interface context block in the first generation status block, so that the interface context block carries the generation constraint content; The high-contribution demonstration block and the unreached path guidance block in the second generated state block are initialized according to the historical coverage baseline in the test generation record, so that the coverage direction guidance content in the initial feedback guidance content is carried by the high-contribution demonstration block and the unreached path guidance block; The failed use case negative feedback block in the second generated state block is initialized based on the previous round of execution failure information or empty feedback identifier, so that the failed use case negative feedback block carries the failure avoidance direction guidance content in the initial feedback guidance content.

5. The test case generation method according to claim 4, characterized in that, The step of writing the interface constraint description in the test generation basis record into the interface context block in the first generation status block includes: Read the request field, the return field, the field format constraint, and the call precondition from the interface constraint description; The interface call entry point is determined based on the request fields and the call preconditions; the request field format and return field format are determined based on the field format constraints; and the assertion acceptance field is determined based on the return field. Write the interface call entry point into the interface location slot in the interface context block, write the request field format and the return field format into the test case format slot in the interface context block, and write the assertion acceptance field into the result validation slot in the interface context block, so that the interface context block carries the test case format constraints of the structured generated prompt record.

6. The test case generation method according to claim 1, characterized in that, The step of inputting the structured generated prompt record into a text generation service to obtain a batch of candidate test cases, and executing the batch of candidate test cases in an isolated execution environment to generate an execution feedback record, includes: The structured generation prompt record is input into the text generation service, so that the text generation service generates the candidate use case batch corresponding to the structured generation prompt record based on the generation constraint content in the first generation status block and the feedback guidance content in the second generation status block. Assign a use case tracking identifier to each candidate use case in the candidate use case batch; The candidate use cases are loaded one by one according to the use case tracking identifier, and the program under test is driven to run in the isolated execution environment; Collect the code locations reached during the execution of the program under test, and write the code locations into the coverage trajectory corresponding to the use case tracing identifier; Collect the execution status and exception return text of the candidate use cases in the isolated execution environment, and write the execution status and exception return text into the execution receipt; The execution feedback record is generated from the coverage trajectory and the execution receipt.

7. The test case generation method according to claim 6, characterized in that, The step of generating a prompt reconstruction instruction based on the differential reach status of the execution feedback record relative to the test generation basis record includes: The coverage trajectory and execution receipt are read from the execution feedback record, and the historical coverage baseline is read from the test generation basis record; The code position in the coverage trajectory is compared with the code coverage marker in the historical coverage baseline to obtain the new reach record representing the new reach status and the repeated reach record representing the repeated reach status. The executable status and failure status are determined based on the execution status and exception return text in the execution receipt; A differential coverage reward record is generated based on the newly added reach record, the repeated reach record, the executable state, and the failure state; The block update object pointing to the second generated state block is determined based on the differential coverage reward record, and the prompt reconstruction instruction is generated based on the block update object and the differential coverage reward record.

8. The test case generation method according to claim 7, characterized in that, After generating the differential coverage reward record based on the new reach record, the repeated reach record, the executable state, and the failure state, the method further includes: Each candidate use case is read from the batch of candidate use cases, and the corresponding coverage trajectory is obtained according to the use case tracking identifier corresponding to each candidate use case; The coverage trajectory of a single candidate use case is compared with the code coverage markers in the historical coverage baseline to obtain the new reach record of a single use case; Based on the newly added reach record in the single use case, an individual coverage contribution record is generated; Based on the individual coverage contribution record, a demonstration candidate use case is determined, and the demonstration candidate use case is written into the demonstration update slot in the prompt refactoring instruction; The demonstration update slot is bound to the block update object in the block update object that points to the high contribution demonstration block in the second generated state block, so that the demonstration candidate use case is used to update the demonstration guidance content in the high contribution demonstration block.

9. The test case generation method according to claim 8, characterized in that, The step of determining exemplary candidate use cases based on the individual coverage contribution records includes: The new reach contribution status corresponding to each candidate use case is determined based on the individual coverage contribution record; Based on the executable status, filter out non-executable candidate use cases from the candidate use case batch; Among the retained candidate use cases, candidate use cases that meet the demonstration admission conditions are determined based on the newly added reach contribution status, and the candidate use cases that meet the demonstration admission conditions are determined as the demonstration candidate use cases; The test case tracking identifier, input field record, and single test case new reach record corresponding to the demonstration candidate test case are written into the demonstration update slot, so that the demonstration guidance content corresponds to the new reach record in the differential coverage reward record.

10. The test case generation method according to claim 7, characterized in that, The step of reconstructing the feedback guidance content according to the block location information based on the prompt reconstruction instruction to obtain the reconstruction generation prompt record includes: Based on the block update object in the prompt reconstruction instruction, search the prompt index table for the block location information corresponding to the second generated state block; Based on the block location information corresponding to the second generated state block, locate the second generated state block to be reconstructed in the structured generation prompt record; The newly reached records, repeated reached records, executable states, and failed states in the differential coverage reward records are mapped to block update payloads respectively. The corresponding feedback guidance content in the second generation state block is replaced by the block update load, while keeping the generation constraint content in the first generation state block unchanged, to obtain the reconstruction generation prompt record.

11. The test case generation method according to claim 4, characterized in that, The step of reconstructing the feedback guidance content according to the block location information based on the prompt reconstruction instruction includes reconstructing the unreached path guidance content of the unreached path guidance block in the second generated state block; The reconstructing of the unreached path guidance content of the unreached path guidance block in the second generated state block includes: Based on the code coverage markers in the historical coverage baseline and the coverage trajectory, determine the location of the unreached code; A source code index is pre-generated based on the source code file of the program under test, and the branch entry, loop entry or exception entry corresponding to the unreached code position is located in the source code index; Generate an unreached path description based on the branch entry, the loop entry, or the abnormal entry; The description of the unreached path is converted into trigger condition text, and the trigger condition text is written into the unreached path guidance block in the second generation state block according to the block location information, so that the unreached path guidance content points to the unreached code location, and the next round of candidate test cases generates test cases pointing to the unreached code location based on the unreached path guidance content.

12. The test case generation method according to claim 4, characterized in that, The step of reconstructing the feedback guidance content according to the block location information based on the prompt reconstruction instruction includes reconstructing the failure avoidance content of the failure use case negative feedback block in the second generated state block; The reconstruction of the failure avoidance content of the failed use case negative feedback block in the second generated state block includes: When the execution status and exception return text in the execution receipt indicate that the candidate use case has failed to execute, the use case tracking identifier of the candidate use case is read, and the input field record is parsed from the candidate use case; A failure feature summary is generated based on the use case tracking identifier, the exception return text, and the input field record. According to the block location information, the failure feature summary is written into the failure use case negative feedback block in the second generation state block, and the failure avoidance content is formed by the failure feature summary, so that the failure avoidance content corresponds to the failure status in the execution receipt. When the execution receipt does not indicate that the candidate use case has failed, the failure avoidance content in the negative feedback block of the failed use case in the previous round is retained.

13. The test case generation method according to claim 1, characterized in that, The step of re-inputting the reconstructed prompt record into the text generation service, receiving the generated return content output by the text generation service, and encapsulating the generated return content into an executable format to generate test cases includes: The reconstructed prompts are recorded and input into the text generation service, which then outputs the generated return content based on the reconstructed feedback guidance content. Verify the use case structure in the generated return content based on the generated constraint content in the first generated state block; When the test case structure in the generated return content conforms to the generated constraint content, extract the test case input content, execution acceptance content, and result verification content from the generated return content; According to the running interface of the program under test, the test case input content, the execution acceptance content, and the result verification content are encapsulated into an executable test case payload; The executable test case payload is written into the test case file to generate the test cases.

14. The test case generation method according to claim 13, characterized in that, Following the generation of test cases, the following is also included: Establish a reference relationship between the test cases and the round identifiers of the refactored prompt records; Write the executable test case payload corresponding to the test case into the test case storage area; Write the generation source record corresponding to the test case into the test case tracking table. The generation source record includes the reference identifier of the test generation basis record, the reference identifier of the execution feedback record, and the reference identifier of the prompt refactoring instruction. Based on the test case tracking table, the test cases are backtracked and located.

15. The test case generation method according to any one of claims 1 to 14, characterized in that, After generating the test cases, the following is also included: The test cases are executed in the isolated execution environment to generate verification execution feedback records; Update the historical coverage baseline in the test generation basis record based on the verification execution feedback record; When the verification execution feedback record meets the iteration termination condition, the test case is output; When the verification execution feedback record does not meet the iteration termination condition, the updated test generation basis record will be used as the input basis for the next round of generation request to continue generating the next round of test cases.

16. A test case generation device, characterized in that, include: The context generation unit is used to respond to the generation request, obtain the test generation basis record of the program under test, and generate the test generation context. The prompt record construction unit is used to construct a structured generation prompt record with block positioning information based on the test generation context. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. The candidate execution unit is used to input the structured generated prompt record into the text generation service to obtain a batch of candidate test cases, and execute the batch of candidate test cases in the isolated execution environment to generate an execution feedback record; The prompt reconstruction unit is used to generate a prompt reconstruction instruction based on the differential reach status of the execution feedback record relative to the test generation basis record, and to reconstruct the feedback guidance content according to the block positioning information based on the prompt reconstruction instruction to obtain the reconstruction generation prompt record; The test case encapsulation unit is used to input the refactoring generation prompt record back into the text generation service, receive the generation return content output by the text generation service, and encapsulate the generation return content into an executable format to generate test cases.

17. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to read and execute the computer instructions to implement the test case generation method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the test case generation method according to any one of claims 1 to 15.

19. A test case generation system, characterized in that, It includes a front-end device and a back-end cloud server, wherein the front-end device and the back-end cloud server are communicatively connected; The front-end device is used to perform the following steps: receiving a generation request for the program under test; sending the generation request to the back-end cloud server; receiving test cases returned by the back-end cloud server; and displaying the test cases. The backend cloud server is used to perform the following steps: in response to the generation request, obtain the test generation basis record of the program under test, so as to generate a test generation context; Based on the test generation context, a structured generation prompt record with block location information is constructed. The structured generation prompt record includes a first generation state block for carrying generation constraint content and a second generation state block for carrying feedback guidance content. Based on the structured generation prompt record, a batch of candidate test cases is generated, and the batch of candidate test cases is executed in an isolated execution environment to generate an execution feedback record; Based on the differential reach status of the execution feedback record relative to the test generation basis record, a prompt reconstruction instruction is generated, and the feedback guidance content is reconstructed according to the block positioning information based on the prompt reconstruction instruction to obtain the reconstruction generation prompt record; Based on the reconstructed prompt record, generate the generated return content, and encapsulate the generated return content into an executable format to generate the test case; The test case is then returned to the front-end device.

Images