Code data generation method and related method, device, medium and product

Abstract

The application discloses a code data generation method and related methods, devices, media and products, wherein the code data generation method comprises: evaluating a patch file generated based on driving an intelligent agent to respond to a code task in a current round to obtain an evaluation result; classifying the evaluation text to obtain a failure category of execution failure, and analyzing the evaluation text to obtain a use case name of execution failure; determining whether the next round can continue based on the exit state, the failure category and the patch file of the current round; constructing a prompt instruction for instructing the intelligent agent to continue the next round based on the failure category, the use case name and the patch file of the current round; and returning to the step of evaluating the patch file generated based on driving the intelligent agent to respond to the code task in the current round to obtain the evaluation result. The above scheme can reduce the repeated consumption of reasoning allowance as much as possible in the code data generation process.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a code data generation method and related methods, devices, media and products. Background Technology

[0002] In fields such as software engineering, the synthesis of training data for code generation models, such as large language models, can typically involve an agent completing programming tasks on a real code repository. The agent's output is then validated through automated evaluation. The validated code data, i.e., "task-pattern" data pairs, can be collected as high-quality training samples.

[0003] Existing synthetic execution systems generally adopt a "single-round execution, fail-and-abandon" model, meaning that the agent performs one round of execution and is immediately evaluated. If the evaluation fails, the task is abandoned and re-executed, resulting in the repeated consumption of inference credits. Therefore, minimizing the repeated consumption of inference credits during code data generation is a pressing issue. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a code data generation method and related methods, devices, media, and products that can minimize the repeated consumption of inference quotas during the code data generation process.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a code data generation method, comprising: evaluating a patch file generated based on the code task driven by the agent in the current round, and obtaining an evaluation result; wherein, after each round of execution, an exit status is recorded, the patch file is evaluated on test cases, and the evaluation result includes whether the code task was executed successfully and the evaluation text output when execution fails; in response to the evaluation result indicating execution failure, classifying the failure category based on the evaluation text, and parsing the evaluation text to obtain the name of the test case that failed execution; determining whether to continue to the next round based on the exit status, failure category, and patch file of the current round; in response to the determination that the next round can continue, constructing a prompt instruction to instruct the agent to continue to the next round based on the failure category, test case name, and patch file of the current round; returning to the step of evaluating the patch file generated based on the code task driven by the agent in the current round, and obtaining an evaluation result, until the evaluation indicates execution success or it is determined that the next round cannot continue, and outputting the code task and the patch file when the code task is evaluated as successful in the case of successful execution.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a method for training a code generation model, comprising: acquiring sample data pairs; wherein the sample data pairs are obtained based on the code data generation method of the first aspect described above, and the sample data pairs include code tasks and patch files when the code tasks are evaluated as successful; and training the code generation model based on the sample data pairs.

[0007] To address the aforementioned technical problems, a third aspect of this application provides a computer device comprising at least a memory and a processor coupled to each other. The memory stores at least program instructions, and the processor executes the program instructions to implement the code data generation method of the first aspect or the training method for the code generation model of the second aspect.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor. These program instructions are used to implement the code data generation method of the first aspect or the training method of the code generation model of the second aspect.

[0009] To address the aforementioned technical problems, the fifth aspect of this application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the code data generation method of the first aspect or the training method for the code generation model of the second aspect.

[0010] The above scheme evaluates the patch file generated by the agent in the current round of code task execution, obtaining the evaluation result. After each round, the exit status is recorded. The patch file is evaluated on test cases, and the evaluation result includes whether the code task was executed successfully and the evaluation text output when it failed. In response to the evaluation result indicating execution failure, the failure text is categorized to obtain the failure category. The evaluation text is then parsed to obtain the name of the failed test case. Based on the exit status, failure category, and patch file of the current round, it is determined whether to continue to the next round. In response to the determination that the next round can continue, a prompt instruction is constructed to instruct the agent to continue to the next round, based on the failure category, test case name, and patch file of the current round. Finally, the system returns the patch file generated by the agent in the current round of code task execution. The estimation process involves obtaining evaluation results until the evaluation indicates successful execution or it is determined that the next round cannot proceed. If the evaluation indicates successful execution, the code task and the patch file for that successful execution are output. On one hand, if the evaluation indicates execution failure, the relevant information generated in the current round is used as a reference to continue to the next round. Compared to directly abandoning the process, continuing to fix the issue with actual patch artifacts improves the utilization rate of fixable defect samples and helps minimize the redundant consumption of inference resources. On the other hand, since the decision to continue to the next round is based on the exit status, failure category, and patch file of the current round, and this decision is made in conjunction with the failure category, use case name, and patch file of the current round, it reduces invalid retries compared to indiscriminate retries without distinguishing the cause of failure, further minimizing the redundant consumption of inference resources. Therefore, the redundant consumption of inference resources can be minimized during the code data generation process. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of the code data generation method of this application; Figure 2a This is a schematic diagram of the framework of an embodiment of the code data generation system of this application; Figure 2b This is a schematic diagram of the framework of an embodiment of the synthetic trajectory protection module of this application; Figure 2c This is a schematic diagram of the framework of an embodiment of the session inheritance module of this application; Figure 2d This is a schematic diagram of an embodiment of the multi-round execution process of this application; Figure 3 This is a flowchart illustrating an embodiment of the training method for the code generation model of this application; Figure 4 This is a schematic diagram of the framework of an embodiment of the code data generation apparatus of this application; Figure 5 This is a schematic diagram of the framework of an embodiment of the training device for the code generation model of this application; Figure 6 This is a schematic diagram of the framework of an embodiment of the computer device of this application; Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application; Figure 8 This is a schematic diagram of the framework of an embodiment of the computer program product of this application. Detailed Implementation

[0012] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0013] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0014] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the symbol " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0015] Please refer to the following: Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the code data generation method of this application. It should be noted that this embodiment can be executed by a computer device. The specific structure and type of the computer device can be found in subsequent related embodiments of this application, and will not be elaborated upon here. Specifically, this embodiment may include the following steps: Step S11: Evaluate the patch file generated based on the driving agent response code task in the current round to obtain the evaluation result.

[0016] In this embodiment, the exit status can be recorded after each round of execution. The patch file can be evaluated on the test cases, and the evaluation result can include whether the code task was executed successfully and the evaluation text output when execution fails. It should be noted that the code task can be executed in a task container. In addition, the code task can also be executed in other execution environments, such as the local operating system (shell), high-performance computing (HPC) schedulers (e.g., Slurm), workflow and task schedulers (e.g., JobScheduler). The execution environment of the code task is not limited here, and will not be listed one by one.

[0017] In one implementation scenario, the code task can be a code generation-related task, such as "generating a piece of code that can decompose a positive integer input by the user into the sum of prime numbers" or "checking whether the following code has any problems, and fixing the code if problems are found." The specific content of the code task is not limited here. In addition, patch files can include program code generated by the agent in response to the code task, used to complete the generation or repair tasks within the code task. See the two examples above for reference; further examples will not be provided here.

[0018] In one implementation scenario, each round can invoke the backend executor to run the task container and record the exit status of each round. It should be noted that exit statuses can include, but are not limited to, rate limiting, timeout, fatal error, and memory overflow; the specific type of exit status is not limited here. In addition, different exit statuses can be categorized based on exit codes, such as using exit code 124 to determine timeout and exit code 137 to determine memory overflow; examples will not be provided here.

[0019] In one implementation scenario, before evaluation, empty completions without any patch output can be identified. If no patch is produced and the final round message is empty, it can be determined as an empty completion. It should be noted that if the current round is determined to be an empty completion, it will not be granted the right to continue, meaning it cannot proceed to the next round. Furthermore, if the code task's continued target round is still evaluated as an execution failure, the continuation of the next round can be terminated to minimize the waste of inference credits from invalid rounds. Of course, if the code task's continued target round is still evaluated as an execution failure, in addition to performing the above operations, evaluation traces can also be cleaned up.

[0020] In one implementation scenario, patch files are used for evaluation on test cases. The specific number and content of test cases are not limited here. Test cases primarily test whether the patch file can complete the code's task. In addition, by evaluating the patch file on test cases, it can be determined whether the code task was executed successfully, and the evaluation text when execution fails. The evaluation text may include, but is not limited to: on which test case the patch file failed to execute, the error message when execution failed (e.g., container error, test case cannot run, test case is invalid, no expected results were obtained when running on the test case, etc.). Specific examples of the evaluation text content will not be provided here.

[0021] In one implementation scenario, please refer to the following: Figure 2a , Figure 2a This is a schematic diagram of the framework of an embodiment of the code data generation system of this application. For example... Figure 2a As shown, the code data generation system may include an executor round-driven module, which drives the agent to execute code tasks according to rounds, and determines whether to proceed to the next round based on the judgment result when the evaluation result is deemed as failing. It should be noted that the executor round-driven module can take a task list (e.g., rounds_manifest) as input, call the backend executor to run the task container round by round, and perform the above-mentioned operations such as recording the exit status, classifying exit codes, and identifying empty completion.

[0022] In one implementation scenario, it can also detect whether the context length of the agent in the current round has reached the compression trigger point. In response to reaching the compression trigger point, it can intercept the compression request signal for the context of the current round and return a blocking decision signal, and persistently store the context of the current round; wherein, the persistently stored context is used to assemble prompt instructions. For example, a length upper limit can be set, and when the context length reaches the upper limit, it can be considered that the compression trigger point has been reached. It should be noted that the assembly process of the prompt instructions can be referred to the subsequent related descriptions, and will not be elaborated here. Please refer to [the relevant documentation / reference]. Figure 2a and Figure 2b , Figure 2b This is a schematic diagram of the framework of an embodiment of the synthetic trajectory protection module of this application. Figure 2b As shown, the synthetic trajectory protection module in the code data generation system can include a compression interception subunit and a trajectory archiving subunit. It should be noted that the synthetic trajectory protection module is used to block automatic compression when the agent's context reaches a compression trigger point and to completely archive the original dialogue trajectory. The synthetic trajectory protection module can set an interception subunit at the compression trigger point. When a compression request signal is intercepted, it can return a blocking decision signal and simultaneously trigger the trajectory archiving subunit to persistently store the current complete session trajectory for subsequent assembly prompts and sample retrieval.

[0023] Step S12: In response to the evaluation result being characterized as an execution failure, classify the failure based on the evaluation text to obtain the failure category, and parse the evaluation text to obtain the use case name of the execution failure.

[0024] In one implementation scenario, when the evaluation result indicates successful execution, the code task and the patch file corresponding to the successful execution can form a sample data pair. This sample data can be used to train the code generation model. In addition, the sample data pair may also include the complete session trace stored persistently as mentioned above; the specific content of the sample data pair is not limited here.

[0025] In one implementation scenario, evidence fragments of execution failure can be extracted based on the evaluation text. For example, text fragments reflecting execution failure can be extracted from the aforementioned error information as evidence fragments. Exemplarily, text fragments within a preset character limit (e.g., 6000) before truncation of the evaluation text can be used as evidence fragments according to a predefined framework for detection and evidence fragment extraction rules. Based on this, the failure category can be determined. Exemplarily, the failure category can belong to any of the following: environment failure, no test, real defect, etc. Specific situations for failure categories are not limited here, nor will they be listed one by one. It should be noted that the environment failure category can represent problems with the execution environment of the code task, such as container, dependency, gateway, etc., and can be denoted as `infra`; the no test category can represent that the evaluation cannot run or there are no effective tests, and can be denoted as `no_tests`; the real defect category can represent the existence of real failed test cases, and can be denoted as `real`. Please refer to the relevant documentation. Figure 2a The code data generation system may include an evaluation evidence classification module, used to classify evaluation texts as failure evidence and output three-state classification results (e.g., the aforementioned environmental failure category, no testing category, and actual defect category). For details, please refer to the aforementioned operations related to determining failure categories; these will not be repeated here. Please refer to Table 1, which is a three-state classification table for failure evidence. As shown in Table 1, different handling operations can be applied to different failure categories; for details, please refer to the subsequent descriptions, which will not be repeated here.

[0026] Table 1. Three-State Classification Table of Failed Evidence

[0027] In one implementation scenario, as one possible example, the evaluation text can be directly parsed to obtain the name of the failed use case. Alternatively, as another possible example, the failure category can be checked first to see if it is a genuine defect class. It should be noted that the specific meaning of a genuine defect class can be found in the aforementioned descriptions. Based on this, in response to the failure category being a genuine defect class, the step of parsing the evaluation text to obtain the name of the failed use case can be performed; conversely, in response to the failure category not being a genuine defect class, the name of the failed use case can be determined to be empty. This way, the parsing operation can be omitted when the failure category is not a genuine defect class. Please refer to [the relevant documentation / reference]. Figure 2a The code data generation system may include a failed test name extraction module, which parses the test case names of failed test cases from the evaluation text, serving as input data for subsequent assembly prompts and progress determination. In addition, the failed test name extraction module can parse the test case names of failed test cases according to the output format of the evaluation framework, and after filtering out duplicate and irrelevant text, can output a structured list of test names (containing at least one test case name).

[0028] Step S13: Based on the exit status, failure category, and patch file of the current round, determine whether to continue to the next round.

[0029] In one implementation scenario, as a possible approach, a large model instruction can be constructed based on the current round's exit status, failure category, use case name, and patch file. This large model instruction instructs the large language model to determine whether to continue to the next round based on the current round's exit status, failure category, use case name, and patch file. Furthermore, the decision result output by the large language model in response to the large model instruction can be used to determine whether to continue to the next round. In this way, the general understanding capability of the large language model can be utilized to determine whether the agent can continue to execute the next round based on the current round's exit status, failure category, use case name, and patch file.

[0030] In another implementation scenario, as another possible approach, we can first check whether the exit status of the current round is a non-renewable state. If the exit status is non-renewable, we can directly determine that we cannot continue to the next round. If the exit status is not non-renewable, we can further determine whether we can continue to the next round based on whether the failure category of the current round indicates renewability and whether the patch file of the current round is not empty. This approach, by first checking the exit status of the current round and then combining the failure category and the patch file to determine whether we can continue to the next round, uses a two-level determination to improve the accuracy of determining substantive progress.

[0031] In a specific implementation scenario, if the exit state of the current round belongs to the set of unrepeatable states, the exit state of the current round can be detected as unrepeatable. It should be noted that the set of unrepeatable states can at least include rate limiting, fatal errors, and timeouts. In other words, when the exit state of the current round is detected to belong to the set of unrepeatable states, it can be directly determined that the next round cannot proceed.

[0032] In a specific implementation scenario, if the failure category indicator for the current round indicates that the process can continue and the patch file for the current round is not empty, it is determined that the process can continue to the next round. Conversely, if the failure category indicator for the current round indicates that the process cannot continue or the patch file for the current round is empty, it is determined that the process cannot continue to the next round. In other words, if the current round meets either of the following conditions: the failure category indicator indicates that the process cannot continue or the patch file for the current round is empty, it is determined that the process cannot continue to the next round. Conversely, if neither of these conditions is met, it is determined that the process can continue to the next round.

[0033] In a specific implementation scenario, if the failure category of the current round is "no test" or "real defect", the failure category of the current round indicates that the run can continue. If the failure category of the current round is "environmental failure", the failure category of the current round indicates that the run cannot continue. "No test" can indicate that the evaluation cannot run or there are no effective tests. "Real defect" can indicate that there are real failed test cases. "Environmental failure" can indicate that there are problems with the execution environment of the code task. For details, please refer to the relevant descriptions above, which will not be repeated here.

[0034] In a specific implementation scenario, please refer to the relevant documents. Figure 2a The code data generation system may include a substantial progress determination module to determine whether a code task is worth continuing. This module takes the classification results (i.e., the aforementioned failure categories), the list of failed test names (i.e., the aforementioned use case names), and the model patch produced in the current round (i.e., the aforementioned patch file) as input, and performs a two-level determination. The specific process of the two-level determination can be found in the aforementioned descriptions and will not be repeated here. In addition, for code tasks with agent session state errors (such as exceeding the maximum round), it can be directly determined that they can continue to the next round, which will not be repeated here. As a possible example, please refer to Table 2, which is the substantial progress determination rule table. The specific rules for the above two-level determination can also be found in Table 2. It should be noted that in Table 2, "∈" indicates belonging to a set, "OR" indicates logical "OR", and "AND" indicates logical "AND". Furthermore, "patch" in Table 2 refers to the aforementioned patch file.

[0035] Table 2 Rules for Determining Substantial Progress

[0036] Step S14: In response to the determination that the next round can continue, construct a prompt instruction to instruct the agent to continue the next round based on the failure category, use case name and patch file of the current round.

[0037] In one implementation scenario, evidence fragments of execution failure can be extracted from the evaluation text, and a first prompt instruction can be constructed based on these evidence fragments and the patch file for the current round. It should be noted that the first prompt instruction can be used to instruct the agent to continue to the next round, referring to the evidence fragments and the patch file. For clarity, please refer to the following example of a first prompt instruction: [Carrying real products] Modified file: <Snapshot Difference List> Failed Tests: <List of Failed Test Names> Failure evidence fragment: <Evaluation output evidence fragment> Requirements: Continue to improve upon existing work; do not start from scratch.

[0038] It should be noted that "carrying actual deliverables" refers to the "patch files of the current round" mentioned above, and "modified files" refers to the list of files that have changed in the current round compared to the previous round (e.g., newly added patch files). This can be obtained by taking a snapshot of the task workspace using set difference; please refer to the relevant description below for details, which will not be elaborated here. In addition, "failure evidence fragments" refers to the "evidence fragments of execution failure" mentioned above. Of course, as shown in the example above, the first prompt instruction can also include a list of failed test names, i.e., the "test case names" mentioned above.

[0039] In one implementation scenario, in response to the failure category of "No Tests" in the current round, a second prompt instruction can be constructed based on the test case name of the current round. It should be noted that "No Tests" can indicate that the evaluation cannot run or there are no valid tests. The second prompt instruction can be used to instruct the agent to first supplement test cases based on the test case name before responding to the code task. It should also be noted that when the failure category is "No Tests," the test case name may be empty. In this case, constructing the second prompt instruction can also use the empty test case name in the second prompt instruction to feedback the information to the agent: the current round of evaluation cannot run or there are no valid tests. In this way, the agent can further supplement test cases accordingly. That is to say, when the failure category of the current round is "No Tests," not only can the aforementioned first prompt instruction be constructed, but a second prompt instruction can also be constructed. The agent can first respond to the second prompt instruction to supplement test cases, and then respond to the first prompt instruction to continue executing the code task. Therefore, subsequent evaluations can be performed on the supplemented test cases.

[0040] In one implementation scenario, a third prompt instruction can be constructed in response to an interruption in the current session. It should be noted that the third prompt instruction can be used to instruct the agent to continue from the breakpoint of the current session until the current session is completed. In other words, if the current session is interrupted, a third prompt instruction can be constructed to require the agent to continue from the breakpoint until the current session is completed, thus allowing the aforementioned evaluation and other operations to continue.

[0041] In an implementation scenario, such as Figure 2a As shown, the code data generation system can include a continuation prompt assembly module, used to assemble differentiated continuation prompts (i.e., prompt instructions) according to failure categories. The continuation prompt assembly module can take the actual progress judgment result (i.e., whether the next round can continue), the list of failed test names (i.e., the aforementioned use case names), and historical artifacts (i.e., the aforementioned patch files) as input, and output three types of prompt instructions. The first type of prompt instruction is a continuation prompt based on the actual patch artifact (i.e., the aforementioned patch files), carrying the generated repair content (i.e., the aforementioned patch files) and failure evidence fragments (i.e., the aforementioned execution failure evidence fragments), requiring the agent to continue repairing based on existing results; the second type of prompt instruction is a test-driven prompt (i.e., the aforementioned second prompt instruction), which, for failure categories without test classes, requires the agent to first supplement test cases before repairing; the third type of prompt instruction is a continue execution prompt (i.e., the aforementioned third prompt instruction), which, for session-interrupted tasks, prompts the agent to continue from the breakpoint. The above prompt instructions embed the list of failed test names (i.e., the aforementioned use case names) and evidence fragments to ensure that the agent can obtain complete context in the next round as much as possible.

[0042] In one implementation scenario, a session identifier for the current round can also be generated. In response to a determination that the next round can proceed, the session identifier for the current round can be written into the session options executed in the next round. Furthermore, when executing the next round, the backend can be driven to restore the session content specified by the session identifier in the session options and continue execution. This method, by writing the session identifier for the current round into the session options executed in the next round and driving the backend to restore the session content specified by the session identifier in the session options and continue execution in the next round, enables session inheritance across multiple rounds.

[0043] In a specific implementation scenario, session identifiers for code tasks can be generated according to deterministic rules. For example, a stable session identifier can be generated using a deterministic hash of the task identifier of the code task.

[0044] In a specific implementation scenario, the session identifier of the current round can be written into the session ID and resume parameters of the next round's execution.

[0045] In a specific implementation scenario, the state of each session can be written to a session log file and marked as recoverable. Additionally, backends that cannot recover sessions can use the prompt word replay channel.

[0046] In a specific implementation scenario, please refer to the relevant documents. Figure 2a and Figure 2c , Figure 2c This is a schematic diagram of the framework of an embodiment of the session inheritance module of this application. Figure 2a As shown, the code data generation system may include a session inheritance module, which is used to inject the session identifier of the current round into the next round of execution, realizing session-level continuation rather than task-level re-run. Figure 2c As shown, the session inheritance module can include a session identifier generation subunit and a session injection subunit. The former takes the task identifier as input and generates a fixed session identifier for the code task according to deterministic rules (e.g., a stable session identifier can be generated using a deterministic hash of the task identifier). The latter writes this session identifier into the session option `session ID` and `resume` parameters of the next round of execution, driving the backend to continue execution in "restore specified session" mode. Figure 2c As shown, the session inheritance module can also include a session persistence subunit, used to write the session state of each round to the session log file and mark it as resumable. Backends that cannot recover the session can use the prompt word replay channel. For example, the session inheritance mechanism of the session inheritance module can be represented by the following chain: Task identifier -> Deterministic hash (e.g., UUIDv5, i.e., Universally Unique Identifier version 5) -> Session identifier -> Inject the resume option for the next round -> Backend executes in session recovery mode -> Session log file is marked as resumable (i.e., recoverable).

[0047] Step S15: Return to the step of evaluating the patch file generated based on the code task that drives the agent to respond in the current round, and obtain the evaluation result until the evaluation is successful or it is determined that the next round cannot be continued. If the evaluation is successful, output the code task and the patch file when the code task is evaluated as successful.

[0048] In one implementation scenario, after constructing the prompt instruction to guide the agent to continue to the next round, the process returns to the aforementioned step of evaluating the patch file generated based on the code task that drives the agent to respond in the current round, obtaining the evaluation result, until a round is evaluated as successful or it is determined that the next round cannot continue. If the evaluation is successful, the code task and the patch file for the successful code task are output. As one possible example, if the evaluation is successful, the code task and the patch file for the successful code task can be combined as a sample data pair for training the code generation model. Specific details can be found in subsequent related embodiments, which will not be elaborated here. As another possible example, the sample data pair may include not only the code task and the patch file for the successful code task, but also the complete session trajectory for the successful code task evaluation, etc., without limitation.

[0049] In one implementation scenario, the file set in the task workspace can be compared before and after each round of execution to generate a snapshot difference for that round. This snapshot difference can be used to identify the start and end of a new session and whether the agent produced anything in that round. Please refer to [reference needed]. Figure 2a The code data generation system can include a round snapshot freeze module, which is used to take a snapshot of the task workspace before and after each round of execution, identify and freeze round boundaries. The round snapshot freeze module can compare the workspace file set before and after (set difference operation), identify files added / changed in the current round, and generate a round snapshot list. In addition, the round snapshot freeze module can also identify the start and end of a new session based on snapshot differences, freeze the round list, minimize the mutual contamination of workspace states between multiple rounds of execution, and provide factual evidence for determining whether there has been output in this round.

[0050] In one implementation scenario, in response to launching a task container, the gateway sidecar can derive a network alias for the task container based on the task identifier (e.g., `--network-alias`). It's important to note that each task container can have a unique network alias, and different task containers can have different network aliases. Based on this, the task container's network alias can be injected into the container network, and then, based on the network alias, model requests within the task can be resolved to the dedicated gateway. This method, deriving a network alias for the task container based on the task identifier and then injecting it into the container network, allows for the resolution of model requests within the task to the dedicated gateway based on the network alias, thus achieving gateway isolation between different tasks.

[0051] In a specific implementation scenario, a gateway ready probe (e.g., using a 200 response from a healthy ready interface as the authoritative criterion) can be used instead of a raw port probe to minimize false positives during cold start.

[0052] In a specific implementation scenario, the task's private configuration can also be written to a private directory outside the task's output directory to minimize the risk of configuration leakage with the output. It should be noted that private configuration can include, but is not limited to, API (Application Programming Interface) keys, etc., and is not limited here.

[0053] In a specific implementation scenario, please refer to the relevant documents. Figure 2a The code data generation system can include a gateway isolation sub-device, used to assign a unique network alias to each task container, ensuring that all model requests for that task are forwarded only through its dedicated gateway instance, thus preventing crosstalk between requests from multiple tasks at the network layer. It should be noted that when the gateway sidecar of a task container starts, the gateway isolation sub-device derives a unique network alias from the task identifier and injects it into the container network. Model requests within the task are resolved to the dedicated gateway through this network alias. In addition, gateway-ready probes can be used instead of raw port probes to minimize false positives during cold starts; and task-specific configurations can be written to a private directory outside the task output directory to prevent configuration leakage with the artifacts.

[0054] In one implementation scenario, please refer to the following: Figure 2a and Figure 2d , Figure 2d This is a schematic diagram illustrating one embodiment of the multi-round execution process of this application. For example... Figure 2a As shown, the actuator round-driven module is connected to the synthetic trajectory protection module, with the former transmitting execution signals to the latter; the synthetic trajectory protection module is connected to the evaluation evidence classification module, with the former transmitting the original trajectory to the latter; the evaluation evidence classification module is connected to the failure test name extraction module, with the former transmitting the failure category to the latter; the evaluation evidence classification module is connected to the substantive progress determination module, with the former transmitting the failure category to the latter; the failure test name extraction module is connected to the substantive progress determination module; the substantive progress determination module is connected to the continuation prompt word assembly module, with the former transmitting a determination signal (i.e., whether to continue to the next round) to the latter; the continuation prompt word assembly module is connected to the session inheritance module, with the former transmitting prompt instructions to the latter; the session inheritance module is connected to the actuator round-driven module, with the former transmitting the session identifier and forwarding the prompt instructions to the latter. In this way, a continuation loop can be formed. In addition, the round snapshot freeze module is bidirectionally connected to the actuator round-driven module; the gateway isolation sub-device is communicatively connected to the task container of the actuator round-driven module. Figure 2dAs shown, the multi-round continuation execution process can include: 1. The executor round-driven module starts the Nth round, and the gateway isolation sub-device injects a network alias into the task container; 2. The synthetic trajectory protection module monitors the context, and when approaching the upper limit, it blocks, compresses, and archives the original trajectory; 3. It evaluates whether the execution was successful; 4. The evaluation evidence classification module outputs a three-state classification, and the failure test name extraction module parses the failed test cases; 5. The substantive progress determination module determines whether to continue running; 6. When it is determined that it can continue running, the continuation prompt word assembly module assembles the prompt instruction with real patches as the base; 7. The session inheritance module injects the session identifier; 8. The executor enters the N+1th round until the determination terminates or the evaluation passes; 9. The round snapshot freeze module takes a snapshot of the set difference throughout the process, and cleans up invalid evaluation traces when terminating. Of course, the above description is only an introduction to the multi-round continuation execution process. For the specific process, please refer to the aforementioned descriptions, which will not be repeated here.

[0055] The above scheme evaluates the patch file generated by the agent in the current round of code task execution, obtaining the evaluation result. After each round, the exit status is recorded. The patch file is evaluated on test cases, and the evaluation result includes whether the code task was executed successfully and the evaluation text output when it failed. In response to the evaluation result indicating execution failure, the failure text is categorized to obtain the failure category. The evaluation text is then parsed to obtain the name of the failed test case. Based on the exit status, failure category, and patch file of the current round, it is determined whether to continue to the next round. In response to the determination that the next round can continue, a prompt instruction is constructed to instruct the agent to continue to the next round, based on the failure category, test case name, and patch file of the current round. Finally, the system returns the patch file generated by the agent in the current round of code task execution. The estimation process involves obtaining evaluation results until the evaluation indicates successful execution or it is determined that the next round cannot proceed. If the evaluation indicates successful execution, the code task and the patch file for that successful execution are output. On one hand, if the evaluation indicates execution failure, the relevant information generated in the current round is used as a reference to continue to the next round. Compared to directly abandoning the process, continuing to fix the issue with actual patch artifacts improves the utilization rate of fixable defect samples and helps minimize the redundant consumption of inference resources. On the other hand, since the decision to continue to the next round is based on the exit status, failure category, and patch file of the current round, and this decision is made in conjunction with the failure category, use case name, and patch file of the current round, it reduces invalid retries compared to indiscriminate retries without distinguishing the cause of failure, further minimizing the redundant consumption of inference resources. Therefore, the redundant consumption of inference resources can be minimized during the code data generation process.

[0056] Please see Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of the training method for the code generation model of this application. It should be noted that this embodiment can be executed by a computer device. The specific structure and type of the computer device can be found in subsequent related embodiments of this application, and will not be elaborated upon here. Specifically, this embodiment may include the following steps: Step S31: Obtain sample data pairs.

[0057] In this embodiment of the disclosure, the sample data pair is obtained based on the process operations in any of the above-described code data generation method embodiments. It should be noted that the sample data pair may include code tasks and patch files when the code task is evaluated as successfully executed; details can be found in the foregoing code data generation method embodiments, and will not be repeated here.

[0058] Step S32: Train the code generation model based on the sample data pairs.

[0059] For example, a code generation model can be used to predict the code task in a sample data pair to obtain a code file. Based on the difference between the code file and the patch file in the sample data pair, the network parameters of the code generation model can be adjusted to force the model to learn to generate executable code files according to the specified code task. It should be noted that the aforementioned difference can be measured using loss functions such as cross-entropy; the specific method of difference measurement is not limited here.

[0060] The above scheme obtains sample data pairs, which are obtained based on the process operations in any of the above code data generation method embodiments. The sample data pairs may include code tasks and patch files when the code tasks are evaluated as successful. Then, the code generation model is trained based on the sample data pairs. Therefore, training the code generation model by combining code tasks and patch files when the code tasks are evaluated as successful helps to improve the code generation capability of the code generation model.

[0061] Please see Figure 4 , Figure 4This is a schematic diagram of the framework of an embodiment of the code generation device of this application. The code generation device 40 includes: an execution evaluation module 41, a classification and parsing module 42, a progress determination module 43, a prompt construction module 44, and a loop iteration module 45. The execution evaluation module 41 is used to evaluate the patch file generated based on the code task driven by the intelligent agent in the current round, and obtain the evaluation result; wherein, after the execution of each round ends, the exit status is recorded, the patch file is evaluated on the test cases, and the evaluation result includes whether the code task was executed successfully and the evaluation text output when the execution fails; the classification and parsing module 42 is used to classify the failure based on the evaluation text in response to the evaluation result being characterized as execution failure, to obtain the failure category of the execution failure, and to parse the evaluation text to obtain the execution failure result. The module includes: a name of the failed use case; a progress determination module 43, used to determine whether to continue to the next round based on the exit status, failure category, and patch file of the current round; a prompt construction module 44, used to construct a prompt instruction to instruct the agent to continue to the next round based on the failure category, use case name, and patch file of the current round in response to the determination that the agent can continue to the next round; and a loop iteration module 45, used to return to the steps of evaluating the patch file generated based on the code task that drives the agent to respond to the code task in the current round, until the evaluation is successful or it is determined that the agent cannot continue to the next round, and outputs the code task and the patch file when the code task is evaluated as successful if the evaluation is successful.

[0062] In the above scheme, the code generation device 40 evaluates the patch file generated by the agent responding to the code task in the current round, and obtains the evaluation result. After each round of execution, the exit status is recorded. The patch file is evaluated on the test cases. The evaluation result includes whether the code task was executed successfully and the evaluation text output when execution fails. In response to the evaluation result indicating execution failure, the failure category is obtained based on the evaluation text. The test case name is obtained based on the evaluation text. Then, based on the exit status, failure category, and patch file of the current round, it is determined whether to continue to the next round. In response to the determination that the next round can continue, a prompt instruction to instruct the agent to continue to the next round is constructed based on the failure category, test case name, and patch file of the current round. Finally, the patch file generated by the agent responding to the code task in the current round is returned. The process involves evaluating code tasks and obtaining evaluation results until either the evaluation indicates successful execution or it is determined that the next round cannot proceed. If the evaluation indicates successful execution, the code task and the patch file for that successful execution are output. On one hand, if the evaluation indicates execution failure, the relevant information generated in the current round is used as a reference to continue to the next round. Compared to directly abandoning the task, continuing to fix it with actual patch artifacts improves the utilization rate of fixable defect samples and helps minimize the redundant consumption of inference resources. On the other hand, since the decision to continue to the next round is based on the exit status, failure category, and patch file of the current round, and this decision is made in conjunction with the failure category, use case name, and patch file of the current round, compared to indiscriminately retries without distinguishing the cause of failure, invalid retries are reduced, further minimizing the redundant consumption of inference resources. Therefore, the redundant consumption of inference resources can be minimized during the code data generation process.

[0063] In some disclosed embodiments, the classification parsing module 42 includes a category detection submodule for detecting whether the failure category is a real defect class; wherein, a real defect class represents the existence of a real failed use case; the classification parsing module 42 includes a text parsing submodule for performing a step of parsing based on the evaluation text to obtain the name of the failed use case in response to the failure category being a real defect class; the classification parsing module 42 includes a direct determination submodule for determining that the name of the failed use case is empty in response to the failure category not being a real defect class.

[0064] In some disclosed embodiments, the progress determination module 43 includes a status detection submodule for detecting whether the exit status of the current round is a non-resumable state; the progress determination module 43 includes a direct determination submodule for directly determining that the next round cannot be continued in response to the exit status of the current round being a non-resumable state; the progress determination module 43 includes a continuation determination submodule for determining whether the next round can be continued in response to the exit status of the current round not being a non-resumable state, based on whether the failure category of the current round indicates that it can be continued and whether the patch file of the current round is not empty.

[0065] In some disclosed embodiments, the continue determination submodule includes a first determination unit, configured to determine that the next round can continue in response to the failure category indication of the current round being able to continue and the patch file of the current round being non-empty; the continue determination submodule includes a second determination unit, configured to determine that the next round cannot continue in response to the failure category indication of the current round being unable to continue or the patch file of the current round being empty.

[0066] In some public embodiments, if the exit state of the current round belongs to the set of non-resumable states, the exit state of the current round is detected as a non-resumable state. The set of non-resumable states includes at least rate limiting, fatal error, and timeout; and / or, if the failure category of the current round is no test or real defect, the failure category of the current round indicates that it can continue; if the failure category of the current round is environment failure, the failure category of the current round indicates that it cannot continue. The no test category indicates that the evaluation cannot run or there are no effective tests; the real defect category indicates that there are real failed use cases; and the environment failure category indicates that there are problems with the execution environment of the code task.

[0067] In some disclosed embodiments, the code generation device 40 includes a compression detection module for detecting whether the context length of the agent in the current round has reached the compression trigger point; the code generation device 40 includes an interception storage module for intercepting the compression request signal for the context of the current round and returning a blocking decision signal in response to reaching the compression trigger point, and persistently storing the context of the current round; wherein the persistently stored context is used to assemble prompt instructions.

[0068] In some disclosed embodiments, the code generation device 40 includes an identifier generation module for generating a session identifier for the current round; the code generation device 40 includes a session inheritance module for, in response to determining that the next round can continue, writing the session identifier of the current round into the session options to be executed in the next round, and driving the backend to restore the session content specified by the session identifier in the session options to continue execution when the next round is executed.

[0069] In some disclosed embodiments, the prompt construction module 44 includes a first construction submodule, used to extract evidence fragments of execution failure based on the evaluation text, and construct a first prompt instruction based on the evidence fragments and the patch file of the current round; wherein, the first prompt instruction is used to instruct the agent to refer to the evidence fragments and the patch file to continue to the next round; the prompt construction module 44 includes a second construction submodule, used to construct a second prompt instruction based on the test case name of the current round in response to the failure category of the current round being the "no test" category; wherein, the "no test" category indicates that the evaluation cannot run or there are no effective tests, and the second prompt instruction is used to instruct the agent to first supplement test cases by referring to the test case name before responding to the code task.

[0070] In some disclosed embodiments, the prompt construction module 44 includes a third construction submodule for constructing a third prompt instruction in response to an interruption of the current round session; wherein the third prompt instruction is used to instruct the agent to continue from the breakpoint of the current round until the current round is completed.

[0071] In some disclosed embodiments, the code task runs in a task container. The code generation device 40 includes an alias derivation module for deriving a network alias for the task container based on the task identifier in response to the gateway side vehicle that starts the task container. The task container has a unique network alias, and different task containers have different network aliases. The code generation device 40 includes an alias injection module for injecting the network alias of the task container into the container network. The code generation device 40 includes a request parsing module for parsing in-task model requests to a dedicated gateway based on the network alias.

[0072] In some disclosed embodiments, the code generation apparatus 40 includes a gateway probe module for replacing bare port probing with a gateway-ready probe.

[0073] In some disclosed embodiments, the code generation apparatus 40 includes a configuration writing module for writing task private configurations to a private directory outside the task output directory.

[0074] In some disclosed embodiments, if the code task is evaluated as successful, a sample data pair is formed between the code task and the patch file at the time of successful evaluation, and the sample data is used to train the code generation model; and / or, if the code task is still evaluated as unsuccessful in the target round, the next round is terminated; and / or, the file set in the task workspace is compared before and after each round to generate a snapshot difference for the corresponding round, and the snapshot difference is used to identify the start and end of a new session and whether the agent has produced anything in the corresponding round.

[0075] It should be noted that the modules (or sub-modules, or units) related to data acquisition / data input in the code generation device 40 can be implemented by hardware devices such as microphones, cameras, keyboards and mice; the modules (or sub-modules, or units) related to data processing can be implemented by hardware devices such as CPUs (Central Processing Units) and NPUs (Neural Processing Units); and the modules (or sub-modules, or units) related to data output can be implemented by hardware devices such as speakers and displays. Examples of each will not be listed here.

[0076] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework of an embodiment of the training device for the code generation model of this application. The training device 50 for the code generation model includes: a sample acquisition module 51 and a model training module 52. The sample acquisition module 51 is used to acquire sample data pairs; wherein, the sample data pairs are obtained based on the above-mentioned code data generation device, and the sample data pairs include code tasks and patch files when the code tasks are evaluated as successful; the model training module 52 is used to train the code generation model based on the sample data pairs.

[0077] In the above scheme, the training device 50 of the code generation model acquires sample data pairs, and the sample data pairs are obtained based on the process operation in any of the above code data generation device embodiments. The sample data pairs may include code tasks and patch files when the code tasks are evaluated as successful. Based on the sample data pairs, the code generation model is trained. Therefore, training the code generation model by combining code tasks and patch files when the code tasks are evaluated as successful helps to improve the code generation capability of the code generation model.

[0078] Please see Figure 6 , Figure 6 This is a schematic diagram of a framework of an embodiment of the computer device of this application. The computer device 60 includes at least a memory 61 and a processor 62 coupled to each other. The memory 61 stores at least program instructions, and the processor 62 is used to execute the program instructions to implement the steps in any of the above-described code data generation method embodiments, or to implement the steps in any of the above-described code generation model training method embodiments. It should be noted that the computer device 60 can be an electronic device capable of executing computer instructions. Exemplarily, the computer device 60 can include, but is not limited to, servers, as well as terminal devices such as smartphones and tablets, and in-vehicle devices such as vehicle-mounted systems. The specific type of the computer device 60 is not limited here.

[0079] Specifically, processor 62 controls itself and memory 61 to implement the steps in any of the code data generation method embodiments described above, or to implement the steps in any of the code generation model training method embodiments described above. Processor 62 can also be referred to as a CPU (Central Processing Unit). Processor 62 may be an integrated circuit chip with signal processing capabilities. Processor 62 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 62 can be implemented using integrated circuit chips.

[0080] In the above scheme, the computer device 60 evaluates the patch file generated based on the code task that drives the agent to respond in the current round, and obtains the evaluation result. After each round of execution, the exit status is recorded. The patch file is evaluated on the test cases. The evaluation result includes whether the code task was executed successfully and the evaluation text output when execution fails. In response to the evaluation result being characterized as execution failure, the failure category is obtained based on the evaluation text. The failure test case name is obtained based on the evaluation text. Then, based on the exit status, failure category, and patch file of the current round, it is determined whether to continue to the next round. In response to the determination that the next round can continue, a prompt instruction to instruct the agent to continue to the next round is constructed based on the failure category, test case name, and patch file of the current round. Finally, the patch file generated based on the code task that drives the agent to respond in the current round is returned. The process involves evaluation, obtaining evaluation results, and continuing until the evaluation indicates successful execution or it is determined that the next round cannot proceed. If the evaluation indicates successful execution, the code task and the patch file for that successful execution are output. On one hand, if the evaluation indicates execution failure, the relevant information generated in the current round is used as a reference to continue to the next round. Compared to directly abandoning the attempt, continuing to fix the issue with a real patch improves the utilization rate of fixable defect samples and helps minimize the redundant consumption of inference resources. On the other hand, since the decision to continue to the next round is based on the exit status, failure category, and patch file of the current round, and this decision is made in conjunction with the failure category, use case name, and patch file of the current round, it reduces invalid retries compared to indiscriminate retries without distinguishing the cause of failure, further minimizing the redundant consumption of inference resources. Therefore, the redundant consumption of inference resources can be minimized during the code data generation process. In addition, the computer device 60 acquires sample data pairs, which are obtained based on the process operations in any of the above code data generation method embodiments. The sample data pairs may include code tasks and patch files when the code tasks are evaluated as successful. Based on the sample data pairs, the code generation model is trained. Therefore, training the code generation model by combining code tasks and patch files when the code tasks are evaluated as successful helps to improve the code generation capability of the code generation model.

[0081] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of this application. The computer-readable storage medium 70 stores program instructions 71 that can be executed by a processor. The program instructions 71 are used to implement the steps in any of the above-described code data generation method embodiments, or to implement the steps in any of the above-described code generation model training method embodiments.

[0082] In the above scheme, the computer-readable storage medium 70 evaluates the patch file generated based on the code task that drives the agent to respond in the current round, and obtains the evaluation result. After each round of execution, the exit status is recorded. The patch file is evaluated on the test cases. The evaluation result includes whether the code task was executed successfully and the evaluation text output when execution fails. In response to the evaluation result being characterized as execution failure, the failure category is obtained based on the evaluation text. The name of the test case that failed is obtained based on the evaluation text. Then, based on the exit status, failure category, and patch file of the current round, it is determined whether to continue to the next round. In response to the determination that the next round can continue, a prompt instruction to instruct the agent to continue to the next round is constructed based on the failure category, test case name, and patch file of the current round. Finally, the patch generated based on the code task that drives the agent to respond in the current round is returned. The process of evaluating the document and obtaining the evaluation result continues until the evaluation indicates successful execution or it is determined that the next round cannot proceed. If the evaluation indicates successful execution, the code task and the patch file for that successful code task are output. On the one hand, if the evaluation result indicates execution failure, the relevant information generated in the current round is used as a reference to continue to the next round. Compared to directly abandoning the process, continuing to fix the issue by carrying real patch artifacts improves the utilization rate of fixable defect samples and helps to minimize the redundant consumption of inference resources. On the other hand, since the decision to continue to the next round is based on the exit status, failure category, and patch file of the current round, and this decision is made in conjunction with the failure category, use case name, and patch file of the current round, compared to indiscriminately retries without distinguishing the cause of failure, invalid retries are reduced, further minimizing the redundant consumption of inference resources. Therefore, the redundant consumption of inference resources can be minimized during the code data generation process. In addition, the computer-readable storage medium 70 acquires sample data pairs, which are obtained based on the process operations in any of the above code data generation method embodiments. The sample data pairs may include code tasks and patch files when the code tasks are evaluated as successful. Based on the sample data pairs, the code generation model is trained. Therefore, training the code generation model by combining code tasks and patch files when the code tasks are evaluated as successful helps to improve the code generation capability of the code generation model.

[0083] Please see Figure 8 , Figure 8 This is a schematic diagram of a framework of an embodiment of the computer program product of this application. The computer program product 80 includes a computer program / instructions 81, which, when executed by a processor, implements the steps in any of the above-described code data generation method embodiments, or implements the steps in any of the above-described code generation model training method embodiments.

[0084] In the above scheme, the computer program product 80 evaluates the patch file generated by the agent responding to the code task in the current round, and obtains the evaluation result. After each round of execution, the exit status is recorded. The patch file is evaluated on the test cases. The evaluation result includes whether the code task was executed successfully and the evaluation text output when execution fails. In response to the evaluation result being characterized as execution failure, the failure category is obtained based on the evaluation text. The failure test case name is obtained based on the evaluation text. Then, based on the exit status, failure category, and patch file of the current round, it is determined whether to continue to the next round. In response to the determination that the next round can continue, a prompt instruction to instruct the agent to continue to the next round is constructed based on the failure category, test case name, and patch file of the current round. Finally, the patch file generated by the agent responding to the code task in the current round is returned. The process involves evaluating code tasks and obtaining evaluation results until either the evaluation indicates successful execution or it is determined that the next round cannot proceed. If the evaluation indicates successful execution, the code task and the patch file for that successful execution are output. On one hand, if the evaluation indicates execution failure, the relevant information generated in the current round is used as a reference to continue to the next round. Compared to directly abandoning the task, continuing to fix it with actual patch artifacts improves the utilization rate of fixable defect samples and helps minimize the redundant consumption of inference resources. On the other hand, since the decision to continue to the next round is based on the exit status, failure category, and patch file of the current round, and this decision is made in conjunction with the failure category, use case name, and patch file of the current round, compared to indiscriminately retries without distinguishing the cause of failure, invalid retries are reduced, further minimizing the redundant consumption of inference resources. Therefore, the redundant consumption of inference resources can be minimized during the code data generation process. In addition, the computer program product 80 acquires sample data pairs, which are obtained based on the process operations in any of the above code data generation method embodiments. The sample data pairs may include code tasks and patch files when the code tasks are evaluated as successful. Based on the sample data pairs, the code generation model is trained. Therefore, training the code generation model by combining code tasks and patch files when the code tasks are evaluated as successful helps to improve the code generation capability of the code generation model.

[0085] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0086] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A method for generating code data, characterized in that, include: The evaluation is performed on the patch file generated by the code task that drives the intelligent agent to respond in the current round, and the evaluation result is obtained. The exit status is recorded after each round of execution. The patch file is evaluated on the test cases. The evaluation result includes whether the code task was executed successfully and the evaluation text output when it failed. In response to the evaluation result being characterized as an execution failure, the failure category is obtained based on the evaluation text, and the use case name of the execution failure is obtained based on the evaluation text. Based on the exit status, failure category, and patch file of the current round, determine whether to continue to the next round; In response to the determination that the next round can continue, a prompt instruction is constructed to instruct the agent to continue the next round based on the failure category, the use case name, and the patch file of the current round; The process returns to the step of evaluating the patch file generated based on the code task that drives the agent to respond in the current round, until the evaluation is successful or it is determined that the next round cannot be continued. If the evaluation is successful, the code task and the patch file when the code task is evaluated as successful are output.

2. The method according to claim 1, characterized in that, The process of parsing the evaluation text to obtain the names of the failed use cases includes: Detect whether the failure category is a real defect class; wherein, the real defect class represents the existence of real failure use cases; In response to the failure category being the actual defect class, the step of parsing the evaluation text to obtain the name of the failed use case is performed; In response to the failure category not being the actual defect class, the name of the failed use case is determined to be empty.

3. The method according to claim 1, characterized in that, The determination of whether to continue to the next round based on the exit status, failure category, and patch file of the current round includes: Detect whether the exit status of the current round is a non-resumable state; In response to the current round's exit status being the "cannot continue" status, it is directly determined that the next round cannot be continued; In response to the exit status of the current round not being the non-resumable state, a determination is made as to whether the next round can continue based on whether the failure category of the current round indicates resumability and whether the patch file of the current round is not empty.

4. The method according to claim 3, characterized in that, The determination of whether to continue to the next round based on whether the failure category of the current round indicates that the process can continue and whether the patch file of the current round is not empty includes: In response to the failure category indication of the current round being able to continue and the patch file of the current round being non-empty, it is determined that the next round can continue; In response to the failure category indicating that the current round cannot continue or the patch file for the current round being empty, it is determined that the next round cannot be continued.

5. The method according to claim 3, characterized in that, If the exit state of the current round belongs to the set of unrepeatable states, the exit state of the current round is detected as the unrepeatable state, and the set of unrepeatable states includes at least rate limiting, fatal error, and timeout. And / or, if the failure category of the current round is the No Tests category or the Real Defects category, the failure category of the current round indicates that the run can continue; if the failure category of the current round is the Environment Failure category, the failure category of the current round indicates that the run cannot continue. The No Tests category indicates that the evaluation cannot run or there are no effective tests; the Real Defects category indicates that there are real failed use cases; and the Environment Failure category indicates that there are problems with the execution environment of the code task.

6. The method according to claim 1, characterized in that, The method further includes: Detect whether the context length of the agent in the current round has reached the compression trigger point; In response to reaching the compression trigger point, the compression request signal for the context of the current round is intercepted and a blocking decision signal is returned, and the context of the current round is persistently stored; wherein the persistently stored context is used to assemble the prompt instruction.

7. The method according to claim 1, characterized in that, The method further includes: Generate the session identifier for the current round; In response to the determination that the next round can continue, the session identifier of the current round is written into the session option to be executed in the next round, and the backend is driven to restore the session content specified by the session identifier in the session option and continue execution when the next round is executed.

8. The method according to claim 1, characterized in that, Based on the failure category, the use case name, and the patch file of the current round, a prompt instruction is constructed to instruct the agent to continue to the next round, including: Based on the evaluation text, evidence fragments of execution failure are extracted, and based on the evidence fragments and the patch file of the current round, a first prompt instruction is constructed; wherein, the first prompt instruction is used to instruct the agent to continue to the next round with reference to the evidence fragments and the patch file; In response to the failure category of the current round being "no test", a second prompt instruction is constructed based on the use case name of the current round; wherein, "no test" indicates that the evaluation cannot run or there are no valid tests, and the second prompt instruction is used to instruct the agent to first supplement the test case with reference to the use case name before responding to the code task.

9. The method according to claim 8, characterized in that, The method of constructing a prompt instruction to instruct the agent to continue to the next round based on the failure category, the use case name, and the patch file of the current round further includes: In response to the interruption of the current session, a third prompt instruction is constructed; wherein the third prompt instruction is used to instruct the agent to continue from the breakpoint of the current session until the current session is completed.

10. The method according to claim 1, characterized in that, The code task is executed in a task container, and the method further includes: In response to the launch of the gateway side vehicle of the task container, a network alias for the task container is derived based on the task identifier; wherein, the task container has a unique network alias, and different task containers have different network aliases; Inject the network alias of the task container into the container network; Based on the network alias, the in-task model request is resolved to the dedicated gateway.

11. The method according to claim 10, characterized in that, The method further includes at least one of the following: replacing bare port probing with gateway-ready probes, and writing task privacy configurations to a private directory outside the task output directory.

12. The method according to any one of claims 1 to 11, characterized in that, If the evaluation indicates successful execution, the code task and the patch file at the time of successful execution form a sample data pair, which is used to train the code generation model. And / or, if the code task continues to the target round and is still evaluated as having failed, terminate the continuation of the next round; And / or, compare the file set in the task workspace before and after each round of execution to generate a snapshot difference for the corresponding round. The snapshot difference is used to identify the start and end of a new session and whether the agent produces anything in the corresponding round.

13. A training method for a code generation model, characterized in that, include: Obtain sample data pairs; wherein, the sample data pairs are obtained based on the code data generation method according to any one of claims 1 to 12, and the sample data pairs include code tasks and patch files when the code tasks are evaluated as successful; The code generation model is trained based on the sample data pairs.

14. A computer device, characterized in that, It includes at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the code data generation method according to any one of claims 1 to 12, or to implement the training method of the code generation model according to claim 13.

15. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the code data generation method according to any one of claims 1 to 12, or to implement the training method for the code generation model according to claim 13.

16. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the code data generation method according to any one of claims 1 to 12, or the training method for the code generation model according to claim 13.

Images