Code reasoning model training method, code reasoning method and electronic equipment
By extracting code instructions from code files, building a multi-agent reasoning link for multi-stage operations, generating reasoning process data, and training a large generative model, the problems of low efficiency and high cost in generating complex logic code in existing technologies are solved, and efficient training and reasoning capabilities are improved.
Patent Information
- Application Number
- CN202510497260.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-09-12
AI Technical Summary
The code generation model in the existing technology performs poorly when processing complex logical code, and has high training costs, large requirements for labeled samples, and long cycles.
Extract code instructions from preset code files, build a multi-agent reasoning link to perform multi-stage reasoning operations, generate reasoning process data, and train generative large models to reduce data labeling. Use multiple agents to perform reasoning operations and testing to improve training efficiency.
It improves the training efficiency of the code reasoning model, reduces training costs, and enhances the model's reasoning capabilities. It also enables fully automatic collection of reasoning process data and reduces reliance on expert data.
Smart Images
Figure CN120633818A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method for training a code reasoning model, a code reasoning method, an electronic device, a storage medium, and a computer program product. Background Art
[0002] In existing technologies, large models are widely used to handle code generation tasks. However, they often perform poorly for code generation problems with complex logic. Some large reasoning models, such as OpenAIO1 and Deepseek's R1, effectively improve their performance on reasoning tasks by incorporating pre-generation thinking. These large models mostly focus solely on the correct answer itself, improving their reasoning capabilities through manually annotated high-quality reasoning data and supervised fine-tuning. While effective, this existing model training method requires a large number of labeled samples, resulting in high implementation costs and a long cycle.
[0003] It can be seen that the training method of the code reasoning model in the existing technology still needs to be improved. Summary of the Invention
[0004] The embodiment of the present application provides a method for training a code reasoning model, which can effectively improve the training efficiency of the code reasoning model, reduce the training cost, and improve the reasoning ability of the trained code reasoning model.
[0005] Correspondingly, an embodiment of the present application also provides a code reasoning method, an electronic device, a storage medium and a computer program product to ensure the implementation and application of the training method of the above-mentioned code reasoning model.
[0006] In order to solve the above problems, the present application discloses a method for training a code reasoning model, which includes:
[0007] Extract code instructions from a preset code file to obtain a data set of code instructions;
[0008] Using a reasoning chain including multiple agents to perform multi-stage reasoning operations on the code instructions in the data set, respectively, to obtain reasoning process data corresponding to each code instruction, the reasoning process data including: code instructions, code, and a description text of the reasoning process of generating the code from the code instructions;
[0009] Based on the code instructions, the code, and a description text of the reasoning process of generating the code by reasoning from the code instructions, a preset generative model is trained to obtain a code reasoning model.
[0010] The present application also discloses a code reasoning method, which includes:
[0011] Get the question text describing the code;
[0012] Generate prompt words based on the question text;
[0013] A preset code inference model is called based on the prompt word to guide the code inference model to infer and generate code for the question text, wherein the code inference model is trained using the aforementioned code inference model training method.
[0014] The embodiment of the present application further discloses a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the embodiment of the present application.
[0015] The embodiments of the present application further disclose a computer program product, including a computer program / computer executable instructions, which implement the method described in the embodiments of the present application when executed by a processor in an electronic device.
[0016] Compared with the prior art, the embodiments of the present application have the following advantages:
[0017] By extracting code instructions from a preset code file, a data set of code instructions is obtained, and a reasoning link including multiple intelligent agents is used to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction. The reasoning process data includes: code instructions, code, and a description text of the reasoning process of generating the code by reasoning from the code instructions. Afterwards, based on the code instructions, the code, and the description text of the reasoning process of generating the code by reasoning from the code instructions, a preset generative large model is trained to obtain a code reasoning model. No data labeling is required, and the reasoning process data is collected automatically, thereby improving the efficiency of training the preset generative large model and reducing the training cost. Moreover, by using a reasoning link including multiple intelligent agents to perform multi-stage reasoning operations to collect reasoning process data, the reasoning ability of the code reasoning model trained based on the reasoning process data can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is one of the step flow charts of the code reasoning model training method disclosed in the embodiment of this application;
[0019] Figure 2 Schematic diagram of a multi-agent framework in the code reasoning model training method disclosed in an embodiment of the present application;
[0020] Figure 3 This is the second step flow chart of the code reasoning model training method disclosed in the embodiment of this application;
[0021] Figure 4 This is a flowchart of the steps of the code reasoning method disclosed in the embodiment of the present application;
[0022] Figure 5 It is a structural diagram of an exemplary device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0023] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0024] like Figure 1 As shown, the training method of the code reasoning model disclosed in the embodiment of the present application includes: steps 102 to 106.
[0025] Step 102: extract code instructions from a preset code file to obtain a data set of code instructions.
[0026] When this application is implemented, code files can be read from the open source code repository.
[0027] For example, several program code files can be read from an open source code repository. Then, each program code file can be parsed separately, and function snippets or code description document snippets can be extracted from the file. The extracted function snippets and code description document snippets can be used as seed data respectively. After that, a pre-trained generative large language model can be used to generate one or more code instructions based on each seed data.
[0028] In some optional embodiments, the extracting code instructions from a preset code file to obtain a data set of code instructions includes: extracting function snippets and / or code description document snippets from the preset code file to obtain seed data; calling a first preset large language model to generate code instructions based on the seed data; scoring the generation difficulty of the code instructions to obtain a difficulty score for each of the code instructions, wherein the difficulty score is positively correlated with the complexity of the generation process of the code instructions; and selecting the code instructions whose difficulty score is greater than a preset score threshold to form a data set of code instructions.
[0029] Taking the application scenario of generating Python code as an example, several Python files can be read from the open source code repository. Then, the file content of each Python file can be parsed separately, and function snippets and / or code description document snippets can be extracted from each code file. Each extracted function snippet and code description document snippet can be used as a seed data.
[0030] In some optional embodiments, the extracting of function snippets and / or code description document snippets from a preset code file to obtain seed data includes: performing syntax analysis on the preset code file to obtain a function snippet and / or a code description document snippet corresponding to the code file; deduplicating the function snippets and / or the code description document snippets corresponding to each of the code files, and using the deduplicated function snippets and the code description document snippets as seed data.
[0031] Optionally, for each python file, the program code in the python file can be read and divided into multiple functions or description documents according to the abstract syntax tree, and then the documents or functions are extracted therefrom to obtain description document fragments or function fragments.
[0032] In some optional embodiments, to balance efficient data utilization and seed data diversity, only one description document snippet or function snippet is retained from each code file and added to the seed data. For example, the description document snippet or function snippet extracted from the code file can be deduplicated based on the file identifier. After deduplication, the retained description document snippet and / or function snippet serves as the seed data.
[0033] Afterwards, based on each seed data, a prompt word is generated according to a preset template, and based on the generated prompt word, a first preset large language model (such as GPT4, mistral large, etc.) is called, guiding the first preset large language model to use the seed data as the answer and generate one or more reasonable questions, which require the seed data as the answer. Taking the seed data as a Python language function as an example, the seed data is assigned to the seed data variable in the prompt word template using the seed data to generate a prompt word. Afterwards, based on the prompt word, the first preset large language model is called, so that the first preset large language model generates a reasonable code instruction question under the guidance of the prompt word, requiring that the code instruction question require the Python language function as the answer.
[0034] Finally, the code instruction problem generated based on the seed data is used as the code instruction.
[0035] Next, the generated code instructions are screened, and the more complex ones are selected to form the code instruction dataset. This reduces the amount of data required for fine-tuning the large language model and improves model training efficiency. For example, the pre-trained second largest language model can be used to score the generation difficulty of each code instruction, resulting in a difficulty score for each code instruction.
[0036] In some optional embodiments, when calling a large language model to score the difficulty of generating code instructions, five difficulty levels can be pre-set in the prompt words, for example, including: very simple, simple, medium, difficult and very difficult, but no specific description of each difficulty level is given. Instead, the large language model is used to analyze the logical complexity of the code instructions and the number of tasks that need to be completed when generating the code, and the large language model is used to evaluate which difficulty level the code instruction belongs to, and the difficulty level obtained by the evaluation is mapped to a difficulty score.
[0037] After obtaining the difficulty score of each code instruction, code instructions with difficulty scores corresponding to medium and above difficulty can be extracted. These extracted more complex code instructions ultimately constitute a code instruction dataset.
[0038] At this point, several code instructions have been automatically generated. Next, these code instructions are used to fine-tune the pre-trained generative large language model, thus enabling the large language model to have code generation capabilities.
[0039] Step 104: Use a reasoning chain including multiple intelligent agents to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction. The reasoning process data includes: code instructions, code, and reasoning process description text for generating the code by reasoning from the code instructions.
[0040] In some optional embodiments, the use of an inference link including multiple intelligent agents to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction, including: performing the following multi-stage reasoning operations on each code instruction in the data set: using an inference link including multiple intelligent agents to perform multiple steps including reasoning operations, reasoning process judgment operations and code testing operations on the code instructions until the code corresponding to the code instruction is obtained; generating reasoning process data corresponding to each code instruction according to the reasoning input and reasoning output of each step of the reasoning operation executed in sequence, and the reflection input and reflection output of each step of the reasoning process judgment operation.
[0041] In the embodiments of this application, a multi-agent framework is designed to synthesize expert data for the code reasoning process. Multiple agent frameworks are used to execute different functional steps in the reasoning process, and multiple agents are called sequentially to form a reasoning chain. Within the reasoning chain, the code corresponding to a code instruction is ultimately derived by performing one or more reasoning operations, one or more reasoning process judgment operations, and one or more code testing operations on a code instruction in an interleaved and looped manner.
[0042] by Figure 2Taking the multi-agent framework shown in the figure as an example, the multi-agent framework includes three agents: a reasoning agent, a reflective agent, and an execution agent. The reasoning agent and the reflective agent form a reasoning-reflection loop, while the reflective agent and the execution agent form an execution-reflection loop. Multiple reasoning-reflection loops and execution-reflection loops form a reasoning chain.
[0043] The functions and implementation plans of each intelligent agent are introduced below.
[0044] The main body of the reasoning agent is a large language model. Its input (i.e., reasoning input) is code instructions and possible partial reasoning processes. Its output (i.e., reasoning output) is the next step in reasoning. By designing special prompts, the reasoning agent can be guided to combine the code instructions and partial reasoning processes to proceed to the next step of reasoning. The reasoning agent only performs a single step of reasoning at a time.
[0045] The reflective agent's main body is a large language model. The reflective agent performs two tasks, each of which implements different functions within the reasoning chain. When the reflective agent performs different tasks, its input (i.e., reflective input) and output (i.e., reflective output) differ. The first task analyzes the input code instructions and the reasoning process data output by the reasoning agent to determine whether the reasoning process is correct and output the next step plan, either continuing the reasoning agent's reasoning or handing the final answer (i.e., code) to the executing agent for verification. The second task analyzes and outputs the cause of any errors in the generated code based on the input code instructions, the answer ultimately obtained through the reasoning agent's reasoning, and the error message returned by the executing agent.
[0046] The execution agent consists of two parts: a large language model and a code compiler. The input of the execution agent is the code instruction and the answer (i.e., code) obtained by the reasoning agent, and the output is the test result. The execution agent first determines whether the currently input code instruction has a preset test case. If there is a test case, the code obtained by reasoning the test case and the code instruction is directly passed to the code compiler, and the code compiler executes the test case to obtain the test result; if there is no test case, the code instruction is input into the large language model to guide the large language model to generate relevant test cases. After that, the generated test case and the code corresponding to the code instruction are passed to the code compiler to execute the instruction test and obtain the test result. If the test result indicates that the test is successful, the reflective agent outputs the code of the successful test; if the test result indicates that the test fails, the reflective agent passes the error message output by the code compiler to the reflective agent.
[0047] In the reasoning chain, the reasoning agent first receives the code instructions and performs step-by-step reasoning. The reflective agent analyzes each step of the reasoning agent's reasoning, determining whether to provide a final answer or allow the reasoning agent to continue reasoning. When the reflective agent concludes that the reasoning agent has inferred the code corresponding to the code instructions, it calls the execution agent to execute the test case to test whether the generated code is correct. If the code generated by the test is correct, the current reasoning process ends and a description of the reasoning process is obtained based on the executed reasoning process. Then, based on the code instructions, the description of the reasoning process, and the generated code, a piece of reasoning expert data is synthesized. If the code generated by the test is incorrect, the test error information is passed to the reflective agent to analyze the problem and the reasoning agent is asked to regenerate the code, ultimately obtaining the correct code or reaching the execution limit of the execution agent.
[0048] As described above, the multiple agents include: reasoning agents, reflective agents, and execution agents. In some optional embodiments, the reasoning chain including multiple agents is used to perform multiple operations including reasoning operations, reasoning process judgment operations, and code testing operations on the code instructions until the code corresponding to the code instructions is obtained, including: performing reasoning operations of generating the code process based on the current step reasoning input by the reasoning agent to obtain the current step reasoning output, wherein the current step reasoning input includes: the code instruction and / or the reasoning output of the reflective agent in the previous step, and the current step reasoning output includes: a descriptive text of the code or reasoning process; performing a judgment operation on the correctness of the reasoning process based on the current step reflection input by the reflective agent. , obtain the current step reflection output, wherein the current step reflection input includes: the reasoning output of the reasoning agent in the previous step or the execution output of the execution agent in the previous step, and the current step reflection output includes one or more of the following outputs: the error information of the previous step reasoning process, the description of the next step reasoning operation, and the code generated by reasoning; in the case that the current step reflection output includes the code generated by reasoning, the code is tested by the execution agent to obtain the execution output corresponding to the test result; in the case that the execution output indicates that the test is successful, the code is used as the code corresponding to the code instruction; in the case that the execution output indicates that the test fails, the execution output is used as the reflection input for the reflective agent to execute the judgment operation of the reasoning process.
[0049] Among them, the execution output is used to trigger the reflective agent to analyze the execution error cause of the code in the previous execution operation; the reasoning output of the reasoning agent in the previous step is used to trigger the reflective agent to analyze whether the reasoning process of the previous reasoning operation is correct.
[0050] During the process of inferring code instructions to obtain code, a first prompt word is first generated based on the code instructions and the initial descriptive text of the reasoning process according to a preset first prompt word template. Then, based on the first prompt word, the large language model built into the reasoning agent is invoked to guide the large language model to perform reasoning operations on the input code instructions according to the initial deduction process, obtaining the inference output for each step. The initial descriptive text of the reasoning process can guide the large language model to generate a chain of multi-step reasoning processes, so that after executing the reasoning process step by step, the code corresponding to the currently input code instruction can be inferred. In this way, the reasoning agent generates one or more reasoning processes to be executed step by step based on the first prompt word and outputs a descriptive text of the reasoning process. Furthermore, the first prompt word is used to guide the large language model to consider every possible aspect when generating the reasoning process, and each reasoning process needs to be re-examined and backtracked when necessary. The content of the first prompt word template is determined based on the test results.
[0051] Next, based on the descriptive text of the reasoning process output by the reasoning agent, a second prompt word is generated according to a preset second prompt word template. Then, based on the second prompt word, the large language model built into the reflective agent is called to guide the large language model to reflect on whether the reasoning process described in the input descriptive text is correct, obtain a judgment result, and execute the next step based on the judgment result. For example, when the judgment result indicates that the reasoning process output by the reasoning agent is correct, a judgment result can be output to instruct the reasoning agent to execute the next reasoning operation; when the judgment result indicates that the reasoning process output by the reasoning agent is incorrect, reasoning error information can be fed back to the reasoning agent, causing the reasoning agent to retrace the reasoning process and recheck the reasoning steps; when the judgment result indicates that the reasoning output of the current step of the reasoning agent is code, the code is output to the execution agent, which then performs the code test operation. The content of the second prompt word template is determined based on the test result.
[0052] Optionally, in the reasoning link, the reflective agent outputs the current code instruction and the code inferred based on the code instruction to the execution agent.
[0053] In an embodiment of the present application, corresponding test cases can be set in advance for code instructions to test the code generated by reasoning for the code instructions, or test cases can be generated by executing an intelligent agent. For example, the executing intelligent agent first retrieves a preset test case. If a test case corresponding to the current code instruction is retrieved, the test case is executed, the generated code is tested and verified, and a test result is obtained. If a test case corresponding to the current code instruction is not retrieved, the preset large prediction model is called to generate a test case for the code instruction, and then the generated test case is executed, the generated code is tested and verified, and a test result is obtained. Among them, the specific implementation method of generating test cases can be found in the prior art, and will not be repeated in the implementation of this application.
[0054] Furthermore, if the test result indicates that the generated code test fails, the execution error cause of the test case output by the test case is fed back to the reflective agent, which combines the execution error cause and performance reflection, and determines the next step for the reasoning agent to perform reasoning process backtracking and recheck the reasoning process; if the test result indicates that the test case is executed successfully, the reasoning agent and the reflective agent can collaborate to complete the reasoning process and reflection process from code instructions to generated code as expert data, and synthesize the code instructions, expert data and instructions into a reasoning process data, which describes the reasoning process of inferring the corresponding code from the code instructions, and the reasoning process includes: forward reasoning process and / or backward reflection process.
[0055] Step 106 , based on the code instructions, the code, and the description text of the reasoning process of generating the code by reasoning from the code instructions, train a preset generative model to obtain a code reasoning model.
[0056] Following this method, we perform inference on each code instruction in the dataset, collect the inference process data corresponding to each code instruction, and add this inference process data to the dataset as expert code inference data. Next, we use this expert code inference data to perform supervised fine-tuning on the pre-trained large language model to obtain a code inference model.
[0057] For the specific implementation method of performing supervised fine-tuning of a pre-trained large language model using code reasoning expert data, please refer to the prior art and will not be repeated in the embodiments of this application.
[0058] Since the code reasoning expert data obtained in the previous steps has a certain paradigm, the code reasoning model obtained after supervised fine-tuning based on this data can follow the corresponding reasoning paradigm and perform reasoning operations, which facilitates the subsequent self-exploration and improvement of the code reasoning model.
[0059] In some optional embodiments, such as Figure 3As shown, based on the code instructions, the code, and the description text of the reasoning process of generating the code by reasoning from the code instructions, a preset generative model is trained to obtain a code reasoning model, and the method further includes: steps 108 to 112.
[0060] Step 108 : calling the code reasoning model to perform reasoning operations on the code instructions, so that single-step multiple reasoning operations are performed at sampling breakpoints preset in the reasoning process of the code reasoning model to sample single-step reasoning process data.
[0061] After fine-tuning the preset generative model based on the code instructions, the code, and the reasoning process description text for generating the code from the code instructions in the code reasoning expert data to obtain the code reasoning model, the fine-tuned code reasoning model uses the code instructions as input, generates prompt words according to the preset prompt word template, and calls the code reasoning model based on the generated prompt words. The code reasoning model can then infer and output the corresponding code according to the reasoning paradigm learned based on the code reasoning expert data.
[0062] In an embodiment of the present application, self-exploration and improvement of the code reasoning model can be performed based on the data set of code instructions obtained in the above steps.
[0063] Optionally, a sampling breakpoint can be pre-set for each reasoning operation step in the reasoning process of the code reasoning model. At the sampling breakpoint, the code reasoning model executes multiple reasoning operations corresponding to the current step, and further executes subsequent reasoning steps based on each reasoning process of the current step until the preset reasoning step depth is reached or the code is inferred. For example, at the sampling breakpoint, the previous reasoning step can be used as the prompt word content, and after the code reasoning model is triggered to conduct multiple explorations based on the previous reasoning step, the reasoning process data of each reasoning step is obtained, thereby sampling more reasoning process data.
[0064] For example, during the first step of the code inference model's inference operation on the input instruction code, sampling breakpoints prompt the large language model to generate multiple inference strategies and perform inference based on each strategy. Subsequently, the next step of the inference operation is performed based on each inference strategy. This method forms multiple inference chains and executes the inference operation. During the execution of the inference operation in each inference chain, if the inference step of a chain reaches a preset upper limit, the subsequent inference steps of that chain can be terminated. If the current inference output of a chain is code, the subsequent inference steps of that chain can be terminated. Otherwise, the next inference step of that chain continues.
[0065] Next, based on the inference process data obtained from the existing sampling and the correctness of the inference results, the pros and cons of single-step reasoning are evaluated, thereby constructing a large amount of single-step preference data.
[0066] Step 110 , evaluating the quality of the single-step reasoning process data to obtain single-step preference data consisting of high-quality single-step reasoning process data and low-quality single-step reasoning process data.
[0067] In some optional embodiments, the single-step reasoning process data is evaluated for quality to obtain single-step preference data consisting of high-quality single-step reasoning process data and low-quality single-step reasoning process data, including: constructing a reasoning process tree with the single-step reasoning process data of the single-step reasoning process as a node; using a tree search algorithm to search for a first path leading to a first node and a second path leading to a second node in the reasoning process tree, wherein the first node is a node of a truth code corresponding to the code instruction, and the second node is a node of a non-truth code corresponding to the code instruction; determining high-quality single-step reasoning process data and low-quality single-step reasoning process data corresponding to the corresponding node based on the number of first paths and the number of second paths induced by the node corresponding to each reasoning process; and constructing single-step preference data of the corresponding single-step reasoning process based on the high-quality single-step reasoning process data and the low-quality single-step reasoning process data.
[0068] The first node and the second node are determined by executing a test code to verify the single-step reasoning process data corresponding to the node.
[0069] First, we can construct an inference process tree using the single-step inference process data as nodes and determining the edges connecting the nodes along each inference link. For example, we can construct edges connecting the nodes according to the order in which the inference process steps are executed within each inference link. Then, starting from the root node of the inference process tree, we traverse the tree to find a leaf node, and use the search path to that leaf node as the complete path from the root node to that leaf node.
[0070] Afterwards, the correctness of each generated complete path is evaluated by executing the test code. For example, after executing the test code, if the leaf node of a path is the truth code of the code instruction corresponding to the root node of the path, then the reasoning steps on the path are considered to have correctly inferred the code. In this case, each node on the path is marked as an "accept" node, and the path is considered the first path. Otherwise, it indicates that there is an error in the node or it is difficult to generate the correct answer from the node. In this case, each node on the path is marked as a "reject" node, and the path is considered the second path.
[0071] Specifically, nodes are first selected, and a layer-order traversal is performed starting from the root node. If the nodes in the current layer have been fully explored, meaning the number of paths originating from these nodes has reached the upper limit, nodes are selected from the child nodes marked "accept" and the reasoning path is explored until the answer node is reached or the exploration depth reaches the upper limit. The remaining paths are then generated from the selected nodes and sampled multiple times until the number of sampled paths reaches a threshold. The answer code in each path is verified. All nodes whose final answers are incorrect are then marked as "rejected" nodes; otherwise, they are classified as "accepted" nodes. All paths generated from the selected nodes are retained, and the path count and correct path count of all nodes originating from the selected node are updated.
[0072] The above search process constructs a large number of tree nodes and finds a large number of first and second paths. Then, based on the number of paths that lead to correct answers (i.e., first paths) and the number of paths that lead to incorrect answers (i.e., second paths) from each tree node, single-step preference data can be constructed.
[0073] Optionally, based on the number of first paths and the number of second paths induced by the node corresponding to each reasoning process, high-quality single-step reasoning process data and low-quality single-step reasoning process data corresponding to the corresponding node are determined, including: obtaining the child nodes of the node corresponding to the current reasoning process; using the single-step reasoning process data from the node to the first child node as high-quality single-step reasoning process data, and using the single-step reasoning process data from the node to the second child node as low-quality single-step reasoning process data, wherein the first child node is the child node that induced the most first paths, and the second child node is the child node that induced the most second paths.
[0074] For example, if the number of first paths from a certain child node Dot_i of a certain node Dot is greater than that of other child nodes, the reasoning process from this node Dot to the child node Dot_i is considered high-quality single-step reasoning process data. If the number of second paths from a certain child node Dot_j is greater than that of other child nodes, the reasoning process from this node Dot to the child node Dot_j is considered low-quality single-step reasoning process data, thus obtaining the single-step reasoning process data corresponding to the node Dot. Using this method, the single-step preference data of the reasoning process of each code instruction is obtained.
[0075] Step 112: Optimize the gait preference of the code inference model based on the single-step preference data.
[0076] Next, using the large amount of single-step preference data constructed in the previous step, the code reasoning model is optimized based on single-step direct preference optimization (Step-DPO) to obtain an enhanced code reasoning model. The above process can be repeated to achieve self-exploration and enhancement of the code reasoning model.
[0077] Optionally, the specific implementation method of optimizing the gait preference of the code inference model based on the single-step preference data can be found in the prior art and will not be described in detail in the embodiments of this application.
[0078] In summary, the training method of the code reasoning model disclosed in the embodiment of the present application extracts code instructions from a preset code file to obtain a data set of code instructions, and adopts a reasoning link including multiple agents to perform multi-stage reasoning operations on the code instructions in the data set respectively, to obtain the reasoning process data corresponding to each of the code instructions, the reasoning process data including: code instructions, code and a description text of the reasoning process of generating the code by reasoning the code instructions. Afterwards, based on the code instructions, the code, and the description text of the reasoning process of generating the code by reasoning the code instructions, the preset generative large model is trained to obtain the code reasoning model. No data labeling is required, and the reasoning process data is collected automatically, thereby improving the efficiency of training the preset generative large model and reducing the training cost. Furthermore, by adopting a reasoning link of multiple agents to perform multi-stage reasoning operations including reasoning operations, reasoning process judgment operations and code testing operations to collect reasoning process data, the reasoning ability of the code reasoning model trained based on the reasoning process data can be improved.
[0079] Furthermore, by setting sampling breakpoints in the reasoning process of the code reasoning model to collect single-step preference data, and performing single-step direct preference optimization on the code reasoning model based on the single-step preference data, while improving the reasoning ability of the code reasoning model, the code reasoning model's code ability is self-improved, reducing dependence on expert data.
[0080] Based on the above embodiment, this embodiment also provides a code reasoning method. Figure 4 As shown, the code reasoning method includes: steps 402 to 406.
[0081] Step 402: Obtain the question text describing the code.
[0082] Optionally, a question text describing the code can be obtained by implementing an application or system of the code reasoning method. The question text is used to describe the purpose and function of the code to be generated. In the embodiments of the present application, the specific method for obtaining the question text describing the code and the content of the question text are not limited.
[0083] Step 404: Generate prompt words based on the question text.
[0084] Next, a prompt word is generated using the question text as input according to a preset third prompt word template. The third prompt word template is used to guide the code reasoning model to generate a multi-step reasoning process, execute reasoning operations step by step according to the reasoning process, determine the correctness of each reasoning step, and plan the next reasoning step based on the judgment results, thereby verifying and backtracking the reasoning process.
[0085] The third prompt word template is determined according to the test results.
[0086] Step 406: Call a preset code reasoning model based on the prompt word to guide the code reasoning model to generate code for the question text by reasoning, wherein the code reasoning model is trained using the following method: extract code instructions from a preset code file to obtain a data set of code instructions; use a reasoning link including multiple intelligent agents to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction, the reasoning process data including: code instructions, code, and a description text of the reasoning process of generating the code by reasoning from the code instructions; based on the code instructions, the code, and the description text of the reasoning process of generating the code by reasoning from the code instructions, train a preset generative large model to obtain the code reasoning model.
[0087] Optionally, after the code reasoning model is obtained by training a preset generative large model based on the code instructions, the code and the reasoning process description text generated by the code instructions, the code reasoning model further includes: calling the code reasoning model to perform reasoning operations on the code instructions, so that single-step multiple reasoning operations are performed at sampling breakpoints preset in the reasoning process of the code reasoning model to sample single-step reasoning process data; evaluating the quality of the single-step reasoning process data to obtain single-step preference data composed of high-quality single-step reasoning process data and low-quality single-step reasoning process data; and optimizing the gait preference of the code reasoning model based on the single-step preference data.
[0088] The training method of the code inference model is described in the previous embodiment and will not be repeated here.
[0089] Next, the preset code reasoning model is called based on the generated prompt words.
[0090] The code reasoning model performs reasoning operations and / or reflective judgment operations on the question text under the guidance of the prompt word until the code corresponding to the question text is obtained.
[0091] In summary, the code reasoning method disclosed in the embodiment of the present application obtains a question text describing the code and generates a prompt word based on the question text. Thereafter, a preset code reasoning model is called based on the prompt word to guide the code reasoning model to reason and generate code for the question text, wherein the code reasoning model is trained using the following method: code instructions are extracted from a preset code file to obtain a data set of code instructions; a reasoning link including multiple intelligent agents is used to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction, and the reasoning process data includes: code instructions, code, and a description text of the reasoning process for generating the code by reasoning from the code instructions; based on the code instructions, the code, and the description text of the reasoning process for generating the code by reasoning from the code instructions, a preset generative large model is trained to obtain the code reasoning model. Since a reasoning link including multiple intelligent agents is used to perform multi-stage reasoning operations including reasoning operations, reasoning process judgment operations, and code testing operations to collect reasoning process data, the ability of the code reasoning model trained based on the reasoning process data to reason about code is improved.
[0092] Based on the above embodiment, this embodiment further provides a device for training a code reasoning model, the device comprising:
[0093] A code instruction data set acquisition module is used to extract code instructions from a preset code file to obtain a code instruction data set;
[0094] an inference module, configured to perform multi-stage inference operations on the code instructions in the data set using an inference chain including multiple agents, to obtain inference process data corresponding to each code instruction, the inference process data including: a code instruction, a code, and a description of the inference process of generating the code from the code instruction;
[0095] The model training module is used to train a preset generative model based on the code instructions, the code, and a description text of the reasoning process of generating the code by reasoning from the code instructions to obtain a code reasoning model.
[0096] Optionally, the adopting of a reasoning chain including multiple agents to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction includes:
[0097] Performing the following multi-stage reasoning operation on each of the code instructions in the data set: using a reasoning chain including multiple agents to perform a multi-step operation including a reasoning operation, a reasoning process judgment operation, and a code testing operation on the code instruction until a code corresponding to the code instruction is obtained;
[0098] Based on the reasoning input and reasoning output of the reasoning operation in each step executed in sequence, and the reflection input and reflection output of the reasoning process judgment operation in each step, the reasoning process data corresponding to each code instruction is generated.
[0099] Optionally, the multiple agents include: a reasoning agent, a reflective agent, and an execution agent, and the use of a reasoning chain including the multiple agents to perform multiple operations including a reasoning operation, a reasoning process judgment operation, and a code testing operation on the code instruction until a code corresponding to the code instruction is obtained includes:
[0100] The reasoning agent performs a reasoning operation of the code generation process based on the current step reasoning input to obtain the current step reasoning output, wherein the current step reasoning input includes: the code instruction and / or the reasoning output of the reflective agent in the previous step, and the current step reasoning output includes: a description text of the code or the reasoning process;
[0101] The reflective agent performs a judgment operation on the correctness of the reasoning process based on the current step reflective input to obtain the current step reflective output, wherein the current step reflective input includes: the reasoning output of the reasoning agent in the previous step or the execution output of the execution agent in the previous step, and the current step reflective output includes one or more of the following outputs: error information of the previous step reasoning process, a description of the next step reasoning operation, and code generated by reasoning;
[0102] When the reflection output at the current step includes code generated by reasoning, the code is tested by the execution agent to obtain an execution output corresponding to the test result;
[0103] In the case where the execution output indicates a success of the test, using the code as the code corresponding to the code instruction;
[0104] In the case where the execution output indicates a test failure, the execution output is used as the reflective input for the reflective agent to perform the reasoning process judgment operation.
[0105] Optionally, after training a preset generative model based on the code instructions, the code, and a description text of the reasoning process of generating the code by reasoning from the code instructions to obtain a code reasoning model, the device further includes:
[0106] a self-improvement module, configured to call the code reasoning model to perform reasoning operations on the code instructions, so as to perform single-step multiple reasoning operations at sampling breakpoints preset in the reasoning process of the code reasoning model to sample single-step reasoning process data; and to evaluate the quality of the single-step reasoning process data to obtain single-step preference data consisting of high-quality single-step reasoning process data and low-quality single-step reasoning process data;
[0107] The self-improvement module is further used to optimize the gait preference of the code reasoning model based on the single-step preference data.
[0108] Optionally, extracting code instructions from a preset code file to obtain a data set of code instructions includes:
[0109] Extract function snippets and / or code description document snippets from preset code files to obtain seed data;
[0110] Calling a first preset large language model to generate code instructions based on the seed data;
[0111] Scoring the difficulty of generating the code instructions to obtain a difficulty score for each code instruction, wherein the difficulty score is positively correlated with the complexity of the generation process of the code instruction;
[0112] The code instructions having the difficulty scores greater than a preset score threshold are selected to form a data set of code instructions.
[0113] Optionally, extracting function snippets and / or code description document snippets from a preset code file to obtain seed data includes:
[0114] By performing syntax analysis on a preset code file, a function snippet and / or a code description document snippet corresponding to the code file is obtained;
[0115] Deduplication processing is performed on the function snippets and / or the code description document snippets corresponding to each of the code files, and the deduplication-processed function snippets and the code description document snippets are used as seed data.
[0116] The code reasoning model training device disclosed in the embodiment of the present application is used to implement the above-mentioned code reasoning model training method. The specific implementation methods of each module of the device can be found in the specific implementation methods of the corresponding steps in the above-mentioned method embodiment, which will not be repeated here.
[0117] In summary, the code reasoning inference model training device disclosed in the example of this application extracts code instructions from a preset code file to obtain a data set of code instructions, and adopts a reasoning link including multiple intelligent agents to perform multi-stage reasoning operations on the code instructions in the data set respectively, to obtain the reasoning process data corresponding to each of the code instructions, the reasoning process data including: code instructions, code, and a description text of the reasoning process of generating the code by reasoning from the code instructions. Afterwards, based on the code instructions, the code, and the description text of the reasoning process of generating the code by reasoning from the code instructions, the preset generative large model is trained to obtain the code reasoning model. No data labeling is required, and the reasoning process data is collected automatically, thereby improving the efficiency of training the preset generative large model and reducing the training cost. Furthermore, by adopting a reasoning link of multiple intelligent agents to perform multi-stage reasoning operations including reasoning operations, reasoning process judgment operations, and code testing operations to collect reasoning process data, the reasoning ability of the code reasoning model trained based on the reasoning process data can be improved.
[0118] Furthermore, by setting sampling breakpoints in the reasoning process of the code reasoning model to collect single-step preference data, and performing single-step direct preference optimization on the code reasoning model based on the single-step preference data, while improving the reasoning ability of the code reasoning model, the code reasoning model's code ability is self-improved, reducing dependence on expert data.
[0119] Based on the above embodiment, this embodiment further provides a code reasoning device, which includes:
[0120] A question text acquisition module is used to obtain the question text describing the code;
[0121] A prompt word generation module, used to generate prompt words based on the question text;
[0122] An inference output module is used to call a preset code inference model based on the prompt word to guide the code inference model to infer and generate code for the problem text, wherein the code inference model is trained using the following method: extracting code instructions from a preset code file to obtain a data set of code instructions, and using a reasoning link including multiple intelligent agents to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction, the reasoning process data including: code instructions, codes, and a description text of the reasoning process of generating the codes by reasoning from the code instructions; thereafter, based on the code instructions, the codes, and the description text of the reasoning process of generating the codes by reasoning from the code instructions, a preset generative large model is trained to obtain a code inference model.
[0123] Optionally, after the code reasoning model is obtained by training a preset generative large model based on the code instructions, the code and the reasoning process description text generated by the code instructions, the code reasoning model further includes: calling the code reasoning model to perform reasoning operations on the code instructions, so that single-step multiple reasoning operations are performed at sampling breakpoints preset in the reasoning process of the code reasoning model to sample single-step reasoning process data; evaluating the quality of the single-step reasoning process data to obtain single-step preference data composed of high-quality single-step reasoning process data and low-quality single-step reasoning process data; and optimizing the gait preference of the code reasoning model based on the single-step preference data.
[0124] The code reasoning device disclosed in the embodiment of the present application is used to implement the above-mentioned code reasoning method. The specific implementation of each module of the device can be found in the specific implementation of the corresponding steps in the above-mentioned method embodiment, which will not be repeated here.
[0125] In summary, the code reasoning device disclosed in the example of the present application obtains a question text describing the code and generates a prompt word based on the question text. Thereafter, based on the prompt word, a preset code reasoning model is called to guide the code reasoning model to reason and generate code for the question text, wherein the code reasoning model is trained by the following method: code instructions are extracted from a preset code file to obtain a data set of code instructions; a reasoning link including multiple intelligent agents is used to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction, and the reasoning process data includes: code instructions, code, and a description text of the reasoning process for generating the code by reasoning from the code instructions; based on the code instructions, the code, and the description text of the reasoning process for generating the code by reasoning from the code instructions, a preset generative large model is trained to obtain the code reasoning model. Since a reasoning link including multiple intelligent agents is used to perform multi-stage reasoning operations including reasoning operations, reasoning process judgment operations, and code testing operations to collect reasoning process data, the ability of the code reasoning model trained based on the reasoning process data to reason about code is improved.
[0126] An embodiment of the present application further provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.
[0127] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the embodiment of the present application.
[0128] The present application also provides an electronic device comprising: a processor and a memory communicatively connected to the processor; the memory storing computer-executable instructions; and the processor executing the computer-executable instructions stored in the memory to implement the method described in the present application. In the present application, the electronic device includes a server, a terminal device, and other devices.
[0129] An embodiment of the present application further discloses a computer program product, including a computer program / computer executable instructions, characterized in that when the computer program / computer executable instructions are executed by a processor in an electronic device, the method described in the embodiment of the present application is implemented.
[0130] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as a server (cluster), a terminal, etc. Figure 5 An exemplary apparatus 500 that can be used to implement various embodiments described in this application is schematically illustrated.
[0131] For one embodiment, Figure 5 An exemplary apparatus 500 is shown having one or more processors 502, a control module (chip set) 504 coupled to at least one of the processor(s) 502, a memory 506 coupled to the control module 504, a non-volatile memory (NVM) / storage device 508 coupled to the control module 504, one or more input / output devices 510 coupled to the control module 504, and a network interface 512 coupled to the control module 504.
[0132] The processor 502 may include one or more single-core or multi-core processors, and the processor 502 may include any combination of general-purpose processors or dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 500 can serve as a server, terminal, or other device described in the embodiments of the present application.
[0133] In some embodiments, the apparatus 500 may include one or more computer-readable media (e.g., memory 506 or NVM / storage 508) having instructions 514 and one or more processors 502 configured in combination with the one or more computer-readable media to execute the instructions 514 to implement a module to perform the actions described in the present disclosure.
[0134] For one embodiment, the control module 504 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 502 and / or any suitable device or component in communication with the control module 504 .
[0135] The control module 504 may include a memory controller module to provide an interface to the memory 506. The memory controller module may be a hardware module, a software module, and / or a firmware module.
[0136] The memory 506 can be used, for example, to load and store data and / or instructions 514 for the device 500. For one embodiment, the memory 506 can include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 506 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).
[0137] For one embodiment, the control module 504 may include one or more input / output controllers to provide interfaces to the NVM / storage device 508 and the input / output device(s) 510 .
[0138] For example, NVM / storage 508 may be used to store data and / or instructions 514. NVM / storage 508 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).
[0139] NVM / storage device 508 may include storage resources that are part of the device on which apparatus 500 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 508 may be accessed via input / output device(s) 510 over a network.
[0140] (One or more) input / output devices 510 may provide an interface for apparatus 500 to communicate with any other appropriate devices. Input / output devices 510 may include communication components, audio components, sensor components, etc. Network interface 512 may provide an interface for apparatus 500 to communicate via one or more networks. Apparatus 500 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard such as Bluetooth, WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.
[0141] For one embodiment, at least one of the processor(s) 502 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 504. For one embodiment, at least one of the processor(s) 502 may be packaged together with the logic of one or more controllers of the control module 504 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 502 may be integrated on the same die with the logic of one or more controllers of the control module 504. For one embodiment, at least one of the processor(s) 502 may be integrated on the same die with the logic of one or more controllers of the control module 504 to form a system-on-chip (SoC).
[0142] In various embodiments, the apparatus 500 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the apparatus 500 may have more or fewer components and / or a different architecture. For example, in some embodiments, the apparatus 500 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.
[0143] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in the memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.
[0144] The present application also provides an electronic device comprising: a processor; and a memory storing executable code, wherein when the executable code is executed, the processor executes one or more methods described in the embodiments of the present application. The memory in the embodiments of the present application can store various data, such as target files, file-application association data, and other data, and can also include user behavior data, thereby providing a data foundation for various processing.
[0145] The embodiments of the present application further provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.
[0146] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0147] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0148] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0149] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0151] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0152] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.
[0153] The above is a detailed introduction to a code reasoning model training method, a code reasoning method, an electronic device, a storage medium and a computer program product provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.
Claims
1. A method for training a code reasoning model, characterized in that: The method comprises: Extract code instructions from a preset code file to obtain a data set of code instructions; Using a reasoning chain including multiple agents to perform multi-stage reasoning operations on the code instructions in the data set, respectively, to obtain reasoning process data corresponding to each code instruction, the reasoning process data including: code instructions, code, and a description text of the reasoning process of generating the code from the code instructions; Based on the code instructions, the code, and a description text of the reasoning process of generating the code by reasoning from the code instructions, a preset generative model is trained to obtain a code reasoning model.
2. The method according to claim 1, characterized in that The method of using a reasoning chain including multiple agents to perform multi-stage reasoning operations on the code instructions in the data set to obtain reasoning process data corresponding to each code instruction includes: Performing the following multi-stage reasoning operation on each of the code instructions in the data set: using a reasoning chain including multiple agents to perform a multi-step operation including a reasoning operation, a reasoning process judgment operation, and a code testing operation on the code instruction until a code corresponding to the code instruction is obtained; Based on the reasoning input and reasoning output of the reasoning operation in each step executed in sequence, and the reflection input and reflection output of the reasoning process judgment operation in each step, the reasoning process data corresponding to each code instruction is generated.
3. The method according to claim 2, characterized in that The multiple agents include: a reasoning agent, a reflective agent, and an execution agent. The reasoning chain including the multiple agents is used to perform multiple operations including reasoning operations, reasoning process judgment operations, and code testing operations on the code instruction until the code corresponding to the code instruction is obtained, including: The reasoning agent performs a reasoning operation of the code generation process based on the current step reasoning input to obtain the current step reasoning output, wherein the current step reasoning input includes: the code instruction and / or the reasoning output of the reflective agent in the previous step, and the current step reasoning output includes: a description text of the code or the reasoning process; The reflective agent performs a judgment operation on the correctness of the reasoning process based on the current step reflective input to obtain the current step reflective output, wherein the current step reflective input includes: the reasoning output of the reasoning agent in the previous step or the execution output of the execution agent in the previous step, and the current step reflective output includes one or more of the following outputs: error information of the previous step reasoning process, a description of the next step reasoning operation, and code generated by reasoning; When the reflection output at the current step includes code generated by reasoning, the code is tested by the execution agent to obtain an execution output corresponding to the test result; In the case where the execution output indicates a success of the test, using the code as the code corresponding to the code instruction; In the case where the execution output indicates a test failure, the execution output is used as the reflective input for the reflective agent to perform the reasoning process judgment operation.
4. The method according to claim 1, wherein After training a preset generative model based on the code instructions, the code, and a description text of the reasoning process of generating the code by reasoning from the code instructions to obtain a code reasoning model, the method further includes: Calling the code reasoning model to perform reasoning operations on the code instructions, so that single-step multiple reasoning operations are performed at sampling breakpoints preset in the reasoning process of the code reasoning model to sample single-step reasoning process data; Evaluating the quality of the single-step reasoning process data to obtain single-step preference data consisting of high-quality single-step reasoning process data and low-quality single-step reasoning process data; The code inference model is optimized for gait preference based on the single-step preference data.
5. The method according to claim 1, wherein The step of extracting code instructions from a preset code file to obtain a data set of code instructions includes: Extract function snippets and / or code description document snippets from preset code files to obtain seed data; Calling a first preset large language model to generate code instructions based on the seed data; Scoring the difficulty of generating the code instructions to obtain a difficulty score for each code instruction, wherein the difficulty score is positively correlated with the complexity of the generation process of the code instruction; The code instructions having the difficulty scores greater than a preset score threshold are selected to form a data set of code instructions.
6. The method according to claim 5, characterized in that The step of extracting function snippets and / or code description document snippets from a preset code file to obtain seed data includes: By performing syntax analysis on a preset code file, a function snippet and / or a code description document snippet corresponding to the code file is obtained; Deduplication processing is performed on the function snippets and / or the code description document snippets corresponding to each of the code files, and the deduplication-processed function snippets and the code description document snippets are used as seed data.
7. A code reasoning method, characterized in that: The method comprises: Get the question text describing the code; Generate prompt words based on the question text; A preset code inference model is called based on the prompt word to guide the code inference model to infer and generate code for the question text, wherein the code inference model is trained using the method according to any one of claims 1 to 6.
8. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.
10. A computer program product comprising a computer program / computer executable instructions, characterized in that When the computer program / computer executable instructions are executed by a processor in an electronic device, the method according to any one of claims 1 to 7 is implemented.