Code generation quality optimization method, terminal device, and storage medium
By constructing standardized training samples and generating a code quality improvement thought chain, the code generation model is guided to undergo supervised fine-tuning. This solves the problems of uninterpretability and instability in the code generation quality optimization process in existing technologies, and improves the accuracy and stability of the code generation results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CENT SOUTH UNIV
- Filing Date
- 2026-04-22
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies lack explicit diagnosis and structured expression of code quality issues in code generation, leading to the uninterpretability and instability of the optimization process, and making it difficult to form a stable closed loop of quality analysis, improvement planning, and generation guidance.
By constructing standardized training samples, executing quality analysis to generate a code quality improvement mindset, using this mindset to guide supervised fine-tuning of the code generation model, forming an explicit improvement plan, and combining reinforcement learning to update the model.
It improves the accuracy, stability, and interpretability of the code generation results, and enhances the model's generalization ability and continuous optimization capability under different tasks.
Smart Images

Figure CN122431709A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing and deep learning technology, and in particular to a method for optimizing code generation quality, a terminal device, and a storage medium. Background Technology
[0002] Code generation technology utilizes large language models to model and generate source code. However, code generation tasks in engineering suffer from typical pain points: 1. Relying solely on prompt word engineering leads to unstable performance. 2. Relying solely on post-processing rules results in weak adaptability. 3. Insufficient ability to jointly optimize compileability, defect rate, semantic consistency, and maintainability.
[0003] The core of the scheme with the publication number CN118468982A lies in fine-tuning the code generation model as a whole through core sample screening, reward model training, and strategy optimization to improve the alignment between the model output and user intent. However, the technical flaw of this scheme is that its optimization mainly relies on reward values and strategy updates to indirectly improve the generation quality. It lacks explicit diagnosis and structured expression for specific quality problems in the current code, and cannot precipitate the problems, causes, and corresponding modification suggestions in the code into reusable intermediate improvement information. Therefore, it is difficult to form a stable closed loop of "quality analysis - improvement planning - guidance generation - re-evaluation" in the application stage.
[0004] The scheme disclosed in CN120386514A constructs a process-supervised dataset and introduces a process-supervised reward model to evaluate rewards for each sub-segment in the code generation process. This allows for the updating of the policy model based on reinforcement learning, thereby improving the quality of code generation. However, the technical flaw of this scheme lies in the fact that it still relies on reward signals as its core, only scoring or providing feedback on code segments. It lacks a systematic analysis and structured expression of the overall code quality issues and cannot clearly provide the current problems, causes, and corresponding modification paths of the code. Therefore, it is difficult to transform the quality evaluation results into improvement information that can directly guide subsequent code generation, resulting in a certain degree of uninterpretability and instability in the optimization process.
[0005] The paper "Improving LLM-Generated Code Quality with GRPO" (Robeyns M, Aitchison L. Improving LLM-generated code quality with GRPO [EB / OL]. (2025) [2026-04-20]. https: / / arxiv.org / abs / 2506.02211) mainly improves the overall quality of generated code during the model training phase by introducing a code quality scoring function into the GRPO reinforcement learning framework, quantifying indicators such as maintainability and security into reward signals. However, this approach still belongs to the typical "reward-driven optimization" paradigm. Its code quality improvement depends on updating parameters based on the scoring feedback of the generated results. Although it enhances the dimensions of quality evaluation, it does not explicitly model the types, causes, and corresponding modification strategies of specific code quality problems, nor does it form a structured intermediate representation that can be directly used to guide subsequent code generation. Therefore, it is still difficult to provide a clear direction for improvement in practical applications, and the interpretability of the optimization process is limited. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a code generation quality optimization method, terminal device and storage medium to improve the accuracy, stability and interpretability of code generation results, in order to address the shortcomings of the existing technology.
[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a code generation quality optimization method, comprising the following steps:
[0008] S1. Obtain training task data and construct standardized training samples; the standardized training samples include task identifiers, task descriptions, reference solutions, and input / output examples.
[0009] S2. Based on the standardized training samples, generate the initial training code results using the basic code generation model;
[0010] S3. Perform quality analysis on the initial training code results to obtain training quality evaluation results;
[0011] S4. Based on standardized training samples, initial training code results, and training quality evaluation results, construct training improvement input data;
[0012] S5. Input the training and improvement input data into the mind chain generation model to generate code quality improvement mind chain training samples; the code quality improvement mind chain training samples are used to characterize the problems existing in the current code, the causes of the problems, and the corresponding modification suggestions.
[0013] S6. Clean and screen the training samples of the code quality improvement mind chain to construct supervised fine-tuning samples;
[0014] S7. Based on the supervised fine-tuning samples, perform supervised fine-tuning on the basic code generation model to obtain the initial improved model;
[0015] S8. Generate reinforcement learning samples based on the initial improved model.
[0016] This solution achieves systematic optimization of code generation quality through an integrated training process encompassing standardized training samples, initial code generation, quality analysis, improvement thought chain generation, and supervised fine-tuning. Compared to existing solutions that rely solely on reward signals or scoring feedback for model optimization, this solution first performs quality analysis on the initial code results to obtain specific quality issue information. Based on this analysis, it generates a code quality improvement thought chain containing problem descriptions, causes, and suggested modifications, thus transforming code quality optimization from implicit parameter adjustment to explicit improvement planning. Furthermore, by cleaning and filtering the training samples from this thought chain and using them for supervised fine-tuning, the model gains the ability to make targeted improvements based on quality issues. Therefore, it improves the accuracy, stability, and interpretability of the code generation results, and enhances the model's generalization and continuous optimization capabilities across different tasks.
[0017] The method of the present invention further includes:
[0018] S9. Perform dynamic evaluation on the reinforcement learning samples to obtain reward evaluation results;
[0019] S10. Based on the reward evaluation results, the initial improved model is updated using reinforcement learning to obtain the target model.
[0020] The method of the present invention further includes:
[0021] S11. Save the target model.
[0022] The method of the present invention further includes:
[0023] For the task data to be processed, using the target model as the thought chain generation model, repeat the operations of steps S1 to S5 to obtain the improved code result.
[0024] The improved code results are evaluated again to obtain the iteration quality evaluation results;
[0025] Based on the iterative quality assessment results, a quality gate judgment is performed to obtain the target code result.
[0026] The specific implementation process of performing quality gate judgment based on the iterative quality evaluation result to obtain the target code result includes: judging whether the improved code result meets the preset quality gate condition based on the iterative quality evaluation result; if the preset quality gate condition is met, the current improved code result is determined as the target code result; if the preset quality gate condition is not met, the improved code result is taken as the new current code result, and the iterative quality evaluation result is taken as the new quality evaluation result, and the process returns to step S4 to continue iterative improvement until the quality gate condition is met or the maximum number of iterations is reached.
[0027] The dynamic evaluation includes one or more of the following: static quality evaluation, task consistency evaluation, dynamic test evaluation, and historical trajectory smoothing evaluation.
[0028] In step S3, the specific process of obtaining the training quality assessment results includes: performing static analysis, task consistency analysis and / or test verification on the initial training code results to generate corresponding training quality assessment results; the training quality assessment results include one or more of code defect information, quality reports and quality scores.
[0029] The specific process of constructing input data for training and improvement includes:
[0030] The standardized training samples, the initial training code results, and the training quality evaluation results are combined to obtain the training improvement input data; the training improvement input data includes a task description, the current code, and a description of the quality problems corresponding to the current code.
[0031] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.
[0032] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon; when the computer program / instructions are executed by a processor, they implement the steps of the above-described method.
[0033] As an inventive concept, the present invention also provides a computer program product, including a computer program / instructions; when the computer program / instructions are executed by a processor, they implement the steps of the above-described method.
[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0035] 1. This invention first performs quality analysis on the initial code results, then generates a code quality improvement mind chain based on the quality analysis results, and uses this mind chain to guide subsequent code generation. This transforms the implicit optimization process that originally relied solely on reward feedback into an explicit improvement process oriented towards code defects. Therefore, it can more effectively overcome the shortcomings of existing solutions in terms of insufficient interpretability, unclear improvement direction, and weak stability of iterative optimization.
[0036] 2. This invention performs quality analysis on the initial code results and further generates a code quality improvement mind chain that includes problem descriptions, cause analysis, and modification suggestions. This mind chain provides explicit guidance for the code generation process, thereby realizing the transformation from "scoring-driven optimization" to "guided generation based on improvement planning". It can effectively overcome the shortcomings of existing solutions in terms of unclear improvement direction, lack of interpretability, and difficulty in forming a stable iterative optimization mechanism.
[0037] 3. This invention performs quality analysis on the initial code and generates a code quality improvement thought chain that includes problem description, cause analysis and modification suggestions. Based on this thought chain, the code generation model is guided to improve the generation. This transforms the quality optimization from a simple scoring feedback mechanism to a guided generation mechanism based on intermediate reasoning results. This can more effectively overcome the shortcomings of existing methods in terms of unclear improvement paths, difficulty in reuse and lack of stable iterative optimization capabilities. Attached Figure Description
[0038] Figure 1 This is a schematic diagram of the method in an embodiment of the present invention;
[0039] Figure 2 This is a schematic diagram of the fine-tuning part in an embodiment of the present invention;
[0040] Figure 3 This is a schematic diagram of the dynamic evaluation part of an embodiment of the present invention. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] Example 1
[0043] like Figure 1 As shown, this embodiment provides a code generation quality optimization method, and the training phase implementation process includes:
[0044] Step 1: Obtain training task data and construct standardized training samples.
[0045] Obtain the raw task data for code generation training, and perform field organization, format standardization, and structure mapping on the raw task data to obtain standardized training samples; wherein, the standardized training samples include at least task identifier, task description, reference solution, and input / output examples.
[0046] Obtain the raw task data for code generation training, which comes from the public dataset MBPP; and perform field sorting, format unification and structure mapping on the raw task data to obtain standardized training samples; wherein, the standardized training samples include at least task identifier, task description, reference answer, input and output examples and test cases.
[0047] Step 2: Generate initial training code results based on standardized training samples.
[0048] Input the task descriptions from the standardized training samples obtained in step 1 into the basic code generation model to generate initial training code results corresponding to each training task.
[0049] Specifically, let the task description be... The basic code generation model is The resulting initial training code is: ;in This refers to the task description. The generated code results. The basic code generation model is a pre-trained large language model that generates corresponding code implementations by semantically understanding the task description.
[0050] Step 3: Perform quality analysis on the initial training code results to obtain the training quality evaluation results.
[0051] Perform static analysis, task consistency analysis, and / or test verification on the initial training code results obtained in step 2 to generate corresponding training quality evaluation results; wherein, the training quality evaluation results include at least one or more of the following: code defect information and quality report.
[0052] Step 4: Construct training improvement input data based on standardized training samples, initial training code results, and training quality evaluation results.
[0053] The standardized training samples obtained in step 1, the initial training code results obtained in step 2, and the training quality evaluation results obtained in step 3 are spliced and organized to form training improvement input data for guiding code improvement learning; the training improvement input data includes at least the task description, the current code, and the description of the quality problems corresponding to the current code.
[0054] Specifically, the training and improvement of input data can be represented as:
[0055]
[0056] in This indicates a task description. Indicates the current code. This indicates a description of quality issues derived from the results of a quality assessment.
[0057] Step 5: Generate code quality improvement training samples based on the input data improved by training.
[0058] The training and improvement input data obtained in step 4 is input into the basic code generation model or the high-performance teacher model (for the initial data generation) to generate code quality improvement thought chains corresponding to the current code quality problems, thus forming training samples for code quality improvement thought chains. The code quality improvement thought chains are used to characterize the problems existing in the current code, the causes of the problems, and the corresponding modification suggestions. The high-performance teacher model is a large language model with stronger capabilities than the basic code generation model, used to generate high-quality training data for code quality improvement thought chains.
[0059] Specifically, the training improves the input data based on the results obtained in step 5. The process of generating a thought chain can be represented as: ,in This indicates a thought chain generation model, implemented in the early stages of training by either a basic code generation model or a high-performance teacher model. This represents a thought process for improving code quality.
[0060] Step 6: Clean and screen the training samples of the code quality improvement mindset chain to construct supervised fine-tuning samples.
[0061] The code quality improvement mind chain training samples obtained in step 5 are subjected to code block extraction, format regularization, abnormal content removal, syntax verification, and validity screening to obtain supervised fine-tuning samples that can be used for supervised fine-tuning.
[0062] Specifically, let the training sample set for the code quality improvement mindset be... After cleaning and screening, a supervised fine-tuning sample set was obtained. .
[0063] Step 7: Perform supervised fine-tuning on the basic code generation model based on supervised fine-tuning samples to obtain the initial improved model.
[0064] like Figure 2As shown, the supervised fine-tuning samples obtained in step 6 are input into the model training framework to perform supervised fine-tuning on the basic code generation model, so that the basic code generation model can generate an initial improved model based on the improvement ideas generated by the quality problem.
[0065] Specifically, let the supervised fine-tuning samples be input-output pairs. The model parameters are optimized by minimizing the difference between the model's generated results and the target thought process. The specific formula is as follows:
[0066]
[0067] in These are the model parameters.
[0068] Step 8: Generate reinforcement learning samples based on the initial improved model.
[0069] Using the initial improved model obtained in step 7, improve thought chains, candidate code results and corresponding quality analysis results are generated for the standardized training samples, and reinforcement learning samples for reinforcement learning optimization are constructed.
[0070] Specifically, let the reinforcement learning samples be: ,in This indicates a task description. This indicates an improvement in the thought process. Indicates the candidate code result. This indicates the corresponding quality assessment result.
[0071] Step 9: Perform dynamic evaluation on the reinforcement learning samples to obtain reward evaluation results.
[0072] like Figure 3 As shown, dynamic evaluation is performed on the candidate code results in the reinforcement learning samples obtained in step 8 to obtain reward evaluation results; wherein, dynamic evaluation includes at least one or more of static quality evaluation, task consistency evaluation, dynamic test evaluation and historical trajectory smoothing evaluation.
[0073] Specifically, static quality assessment score The score is calculated by executing the compilation process using an editor. If the compilation succeeds, the score is 1; otherwise, it is 0. Calculating the task consistency score requires defining a structure consistency score. semantic consistency score ,in Indicates the number of matching syntactic structural units. Indicates the total number of structural units. This represents the scoring function based on CodeBLEU. The task consistency score is then defined as... ,in Denotes the weight coefficients, and satisfies Let the number of test samples corresponding to the i-th task be... The dynamic test evaluation score is defined as:
[0074]
[0075] in This is a flag indicating whether the nth test case passes (1 for passing, 0 for failing). Score the similarity between the output and the expected result. The flag is an exception flag (1 if an exception occurs, 0 otherwise), and a, b, and c are weighting coefficients that satisfy a+b+c=1.
[0076] For ease of explanation, the historical trajectory smoothing evaluation score will be given in step 10.
[0077] Step 10: Update the initial improved model using reinforcement learning based on the reward evaluation results to obtain the target model.
[0078] Based on the reward evaluation results obtained in step 9, a reward signal is constructed, and the parameters of the initial improved model obtained in step 7 are updated using a reinforcement learning optimization algorithm to obtain the target model; wherein, the reinforcement learning optimization algorithm includes PPO, GRPO or other policy optimization algorithms.
[0079] Specifically, the time step t comprehensive reward signal The calculation formula is:
[0080]
[0081] The historical trajectory smoothing score can be calculated based on the reward signals from the previous three time steps:
[0082]
[0083] in, This represents the quality score obtained from static analysis of sample i at time step t. This represents the task consistency score of sample i at time step t. This represents the test validation score of sample i at time step t. The historical trajectory smoothing score of sample i at time step t is represented by , , . These are weighting coefficients. Time steps are indicated by superscripts. This indicates that it is used to characterize different iterative stages in the reinforcement learning process.
[0084] The reward signal is used to characterize the code's standardization and defects, the task consistency score is used to characterize the degree of matching between the generated code and the task description, and the test verification score is used to characterize the correctness of the code's execution under test cases. These can be directly used as feedback signals for reinforcement learning to optimize model parameters and optimize model strategies to be more in line with the requirements.
[0085] Through the above quality analysis process, the correctness, standardization, and task-matching of the generated code can be quantitatively evaluated, thus providing a basis for subsequent code improvement.
[0086] Step 11: Save the target model for subsequent code quality improvements.
[0087] Save the target model obtained in step 10, and deploy the target model as the mind chain generation model in the subsequent code generation quality improvement method; wherein, the mind chain generation model is used to generate the code quality improvement mind chain.
[0088] The inference phase implementation process includes:
[0089] Step 1: Obtain the data of the tasks to be processed and construct standardized task samples.
[0090] Obtain the original task data corresponding to the code generation task, and perform field organization, format standardization, and structure mapping on the original task data to obtain standardized task samples. The standardized task samples should include at least a task identifier, task description, reference solution, and input / output examples. The standardization process is the same as that in step 1 of the training phase.
[0091] Step 2: Generate initial code candidate results based on standardized task samples.
[0092] Input the task description from the standardized task sample obtained in step 1 into the basic code generation model to generate initial code candidate results corresponding to the task description.
[0093] Step 3: Perform quality analysis on the initial code candidate results to obtain the initial quality assessment results.
[0094] Perform static analysis, task consistency analysis, and / or testing on the initial code candidate results obtained in step 2 to generate corresponding initial quality assessment results. These initial quality assessment results shall include at least one or more of the following: code defect information, quality reports, and quality scores. The quality analysis method is the same as the quality assessment method in step 9 of the training phase.
[0095] Step 4: Construct quality improvement input data based on standardized task samples, initial code candidate results, and initial quality assessment results.
[0096] The standardized task samples obtained in step 1, the initial code candidate results obtained in step 2, and the initial quality assessment results obtained in step 3 are combined and organized to form quality improvement input data to guide code improvement. The quality improvement input data includes at least a task description, the current code, and a description of the quality issues corresponding to the current code.
[0097] Step 5: Generate code quality improvement mindset based on quality improvement input data.
[0098] Input the quality improvement input data obtained in step 4 into the expert model (thinking chain generation model) to generate a code quality improvement thinking chain corresponding to the current code quality problem; wherein, the improvement planning model is a part of the target model obtained in step 11 or its derived model, and the code quality improvement thinking chain is used to characterize the current code problems, the causes of the problems, and the corresponding modification suggestions.
[0099] Step 6: Generate improved code results based on the code quality improvement mindset.
[0100] The initial code candidate results obtained in step 2 and the code quality improvement mind chain obtained in step 5 are input together into the basic code generation model to generate improved code results.
[0101] In some implementations, the improved code result is the result of rewriting, repairing, or optimizing the initial code candidate result.
[0102] Step 7: Re-evaluate the improved code results to obtain the iteration quality evaluation results.
[0103] Perform quality analysis again on the improved code obtained in step 6 to obtain the iterative quality evaluation results corresponding to the improved code results; the iterative quality evaluation results include static quality score, task consistency score, and dynamic test score. The quality analysis method is the same as the quality analysis method in step 3 of the training period.
[0104] Step 8: Perform quality gate judgment based on the iterative quality assessment results and output the target code result.
[0105] Based on the iterative quality assessment results obtained in step 7, determine whether the improved code results meet the preset quality access control conditions; the preset quality access control conditions include at least the number of code smells, code vulnerabilities, and code defects meeting the preset threshold requirements.
[0106] If the preset quality threshold conditions are met, the current improved code result is determined as the target code result and output; if the preset quality threshold conditions are not met, the improved code result is taken as the new current code result, and the iterative quality evaluation result obtained in step 7 is taken as the new quality evaluation result. Then, the process returns to step 4 to continue iterative improvement until the quality threshold conditions are met or the maximum number of iterations is reached.
[0107] Example 2
[0108] Embodiment 2 of the present invention provides a terminal device corresponding to Embodiment 1 above. The terminal device can be a processing device for a client, such as a mobile phone, a laptop, a tablet computer, a desktop computer, etc., to execute the method of the above embodiments.
[0109] The terminal device in this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in Embodiment 1 described above.
[0110] In some implementations, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.
[0111] In other implementations, the processor can be any type of general-purpose processor, such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation here.
[0112] Example 3
[0113] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to Embodiment 1 above, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, they implement the steps of the method of Embodiment 1 above.
[0114] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.
[0115] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0118] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0119] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for optimizing code generation quality, characterized in that, Includes the following steps: S1. Obtain training task data and construct standardized training samples; The standardized training samples include task identifiers, task descriptions, reference solutions, and input / output examples; S2. Based on the standardized training samples, generate the initial training code results using the basic code generation model; S3. Perform quality analysis on the initial training code results to obtain training quality evaluation results; S4. Based on standardized training samples, initial training code results, and training quality evaluation results, construct training improvement input data; S5. Input the training and improvement input data into the mind chain generation model to generate code quality improvement mind chain training samples. The code quality improvement mindset training samples are used to characterize the problems existing in the current code, the causes of the problems, and the corresponding modification suggestions; S6. Clean and screen the training samples of the code quality improvement mind chain to construct supervised fine-tuning samples; S7. Based on the supervised fine-tuning samples, perform supervised fine-tuning on the basic code generation model to obtain the initial improved model; S8. Generate reinforcement learning samples based on the initial improved model.
2. The code generation quality optimization method according to claim 1, characterized in that, Also includes: S9. Perform dynamic evaluation on the reinforcement learning samples to obtain reward evaluation results; S10. Based on the reward evaluation results, the initial improved model is updated using reinforcement learning to obtain the target model.
3. The code generation quality optimization method according to claim 2, characterized in that, Also includes: S11. Save the target model.
4. The code generation quality optimization method according to claim 2, characterized in that, Also includes: For the task data to be processed, using the target model as the thought chain generation model, repeat the operations of steps S1 to S5 to obtain the improved code result. The improved code results are evaluated again to obtain the iteration quality evaluation results; Based on the iterative quality assessment results, a quality access control judgment is performed to obtain the target code result.
5. The code generation quality optimization method according to claim 4, characterized in that, The specific implementation process of performing quality gate judgment based on the iterative quality assessment results to obtain the target code results includes: Based on the iterative quality assessment results, determine whether the improved code result meets the preset quality threshold conditions; if the preset quality threshold conditions are met, then the current improved code result is determined as the target code result; if the preset quality threshold conditions are not met, then the improved code result is taken as the new current code result, and the iterative quality assessment result is taken as the new quality assessment result, and return to step S4 to continue iterative improvement until the quality threshold conditions are met or the maximum number of iterations is reached.
6. The code generation quality optimization method according to claim 2, characterized in that, The dynamic evaluation includes one or more of the following: static quality evaluation, task consistency evaluation, dynamic test evaluation, and historical trajectory smoothing evaluation.
7. The code generation quality optimization method according to claim 1, characterized in that, In step S3, the specific process of obtaining the training quality assessment results includes: performing static analysis, task consistency analysis and / or test verification on the initial training code results to generate corresponding training quality assessment results; the training quality assessment results include one or more of the following: code defect information and quality reports.
8. The code generation quality optimization method according to claim 1, characterized in that, The specific process of constructing input data for training and improvement includes: The standardized training samples, the initial training code results, and the training quality evaluation results are combined to obtain the training improvement input data; the training improvement input data includes a task description, the current code, and a description of the quality problems corresponding to the current code.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory; characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program / instructions stored thereon; characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.