Code automatic generation method and device, electronic equipment and storage medium
By repeatedly inputting large language models to identify highly controversial error codes and setting up multi-role guidance, the problems of low accuracy and repair coverage in code generation by large language models are solved, achieving more efficient and stable code generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INST OF SOFTWARE - CHINESE ACAD OF SCI
- Filing Date
- 2026-03-05
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies have low accuracy and repair coverage for code generated by large language models, and a single expert prompting framework cannot fully capture and resolve diverse code error types.
By repeatedly inputting programming task samples into the target large language model, highly controversial error codes are identified, clustered, and multi-role guidance is set. Code is generated using multiple large language models, and an example set is constructed to provide role descriptions for multiple roles, guiding the model to generate code.
It improves the accuracy and fix coverage of code generation, overcomes the bias of a single expert framework, and enhances the model's adaptability to complex tasks and the stability of generation.
Smart Images
Figure CN121785607B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for automatic code generation. Background Technology
[0002] Large language models are typically based on the Transformer architecture, including encoder-decoder or decoder-only structures (such as the GPT series models). In context learning, the model's input consists of a prompt containing a few examples and a test input. Examples are usually presented in the form of "input-output" pairs, and the model captures contextual dependencies through a self-attention mechanism. Context learning allows the model to infer task patterns from the provided examples and generate predictions for new inputs without gradient updates or fine-tuning. A core technique is prompt engineering, where examples are randomly sampled from the dataset, typically only a small subset of samples due to limitations in the input length of the large language model (e.g., token constraints). The code generation steps include: data preparation: randomly sampling a small number of examples from the training set; prompt construction: concatenating the sampled examples into a prompt string and inputting it into the large language model; inference generation: the large language model generates outputs based on the prompts, such as code fixes or optimization suggestions; evaluation: validating the generated results using metrics such as accuracy or execution success rate.
[0003] Existing technologies typically employ a single expert prompting framework (such as ExpertPrompting) to achieve automatic code generation. This framework assigns a single "expert" role to the large language model, providing targeted prompts to guide code generation. Researchers use context learning techniques to automatically synthesize detailed, customized expert identity descriptions for each specific instruction, and then ask the large language model to provide an answer based on this context.
[0004] While a single expert suggestion framework improves model performance, its limitation lies in relying on guidance from only one role, which may not be able to fully capture and resolve diverse code error types, resulting in low accuracy and fix coverage of code generated by large language models. Summary of the Invention
[0005] This invention provides a method, apparatus, electronic device, and storage medium for automatic code generation, in order to solve the technical problems of low accuracy and repair coverage in code generated from large language models in the prior art.
[0006] This invention provides a method for automatic code generation, comprising:
[0007] The programming task sample is input into the target large language model multiple times to obtain multiple versions of the code corresponding to the programming task sample output by the target large language model;
[0008] The degree of controversy for the programming task sample is determined based on the number of target code versions in multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions.
[0009] Cluster the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold to obtain various code error types corresponding to the target large language model;
[0010] The target large language model is input with prompt words containing programming tasks and role descriptions corresponding to various code error types to obtain the code output by the target large language model; the role descriptions are used to assign the role of the code error type handling expert to the target large language model.
[0011] According to a code automatic generation method provided by the present invention, before determining the degree of controversy of the programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample, the method further includes:
[0012] The programming task sample is input into another large language model that is different from the target large language model, and the version code corresponding to the programming task sample is output by the other large language model.
[0013] According to the code automatic generation method provided by the present invention, determining the degree of controversy of the programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample includes:
[0014] Calculate the ratio of the number of target code versions to the total number of code versions;
[0015] The degree of controversy is obtained by subtracting the ratio from 1.
[0016] According to the code automatic generation method provided by the present invention, after determining the controversy level of the programming task sample based on the number of target code versions in multiple versions of code corresponding to the programming task sample, and before clustering the error code versions corresponding to each programming task sample with a controversy level greater than a preset threshold to obtain various code error types corresponding to the target large language model, the method further includes:
[0017] Correct non-critical errors in the incorrect code version based on the correct code version. The non-critical errors include incorrect variable names.
[0018] According to the code automatic generation method provided by the present invention, after clustering the error code versions corresponding to each programming task sample with a controversy degree greater than a preset threshold to obtain various code error types corresponding to the target large language model, before inputting prompt words containing programming tasks and role descriptions corresponding to various code error types into the target large language model to obtain the code output by the target large language model, the method further includes:
[0019] Construct an example set for the target large language model, the example set including the correct code version and the error code version corresponding to various code error types;
[0020] The prompt words also include the correct code version and the incorrect code version in the example set.
[0021] According to the code automatic generation method provided by the present invention, the step of clustering the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold includes:
[0022] The error code versions are clustered using the target large language model.
[0023] The present invention also provides a code automatic generation device, comprising:
[0024] The input module is used to input the programming task sample multiple times into the target large language model to obtain multiple versions of the code corresponding to the programming task sample output by the target large language model;
[0025] The determination module is used to determine the degree of controversy of the programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions;
[0026] The clustering module is used to cluster the error code versions corresponding to each programming task sample with a dispute degree greater than a preset threshold, so as to obtain various code error types corresponding to the target large language model.
[0027] The input module is further configured to input prompts containing programming tasks and role descriptions corresponding to various code error types into the target large language model to obtain the code output by the target large language model; the role descriptions are used to assign the role of the code error type handling expert to the target large language model.
[0028] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic code generation method described above.
[0029] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the code automatic generation method as described above.
[0030] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the code automatic generation method as described above.
[0031] The code automatic generation method, apparatus, electronic device, and storage medium provided by this invention run the same programming task sample multiple times and identify the most uncertain programming task sample of the large language model based on the degree of controversy. The error code of the highly controversial sample is clustered to automatically identify and summarize the core error type of the model. When generating new code, corresponding expert roles are set for the large language model according to the identified error type. Through multi-role guidance, the large language model can solve potential errors from multiple perspectives, overcome the bias of a single expert framework, and thus improve the accuracy of code generation and the repair coverage. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0033] Figure 1 This is a flowchart illustrating the automatic code generation method provided by the present invention.
[0034] Figure 2 This is a schematic diagram of the correct code version and the incorrect code version provided by this invention.
[0035] Figure 3 This is a schematic diagram illustrating the principle of the automatic code generation method provided by this invention.
[0036] Figure 4 This is a schematic diagram showing the comparative experimental results between the present invention and the prior art.
[0037] Figure 5 This is a schematic diagram of the structure of the automatic code generation device provided by the present invention.
[0038] Figure 6This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0040] First, we will introduce the technologies related to large language models.
[0041] Large Language Model (LLM): A large language model is a deep learning-based artificial intelligence model specifically designed for processing and generating natural language text. By training on massive amounts of text data (typically billions to trillions of words), large language models learn statistical relationships in language, enabling them to perform various natural language processing tasks such as text generation, translation, summarization, and semantic analysis.
[0042] In-context learning is a method that does not require updating or fine-tuning model parameters. Instead, the model learns patterns from the context of the input to perform new tasks. In this paradigm, the model receives a set of examples containing input-output pairs and uses the structure and content of these examples to infer how to process new inputs, thereby generating the appropriate output.
[0043] Demonstration: In contextual learning, a demonstration refers to an input-output pair used to guide the language model's reasoning. These demonstrations show how to obtain the desired output from a specific input, helping the model better understand and perform similar tasks.
[0044] Error Factors: Error factors refer to factors or variables that may lead to errors in code generation or other tasks. Identifying and understanding these factors is an important step in improving model performance, helping the model to more accurately identify and correct errors.
[0045] Multi-role Guidance: Multi-role guidance is a strategy that provides diverse guidance and advice through different roles (such as different experts or perspectives) when dealing with complex tasks, avoiding the bias of a single perspective, thereby improving the comprehensiveness and accuracy of task completion.
[0046] Existing technologies typically employ few-shot random sampling and analogical prompting to achieve automatic code generation. These approaches attempt to improve performance through example selection in general generation tasks, but they have significant shortcomings.
[0047] Traditional few-shot random sampling methods directly draw examples from the dataset to build hints without requiring additional role guidance, but they face sensitivity and instability issues, especially in the code domain where there is a lack of targeted research on error types.
[0048] Analogy hints improve performance by automatically guiding the reasoning process of large language models. They eliminate the need for labeled or retrieved examples, offering high versatility and convenience, and can be customized for each problem, providing strong adaptability and flexibility. However, this method still requires further optimization in code generation tasks to address issues related to multi-role collaboration and deep mining of error factors.
[0049] In representing code error examples, traditional methods typically provide examples to a large language model in a few-shot manner. However, a single expert framework falls short when dealing with open-ended code errors for which multiple reasonable fixes exist. For example, for the problem of "poor function performance," the "expert suggestion method" might assign the large language model the role of a performance optimization expert, providing an overly simplistic improvement suggestion, such as simply adding caching, while ignoring other possible solutions, such as algorithm optimization or data structure adjustments. This approach can introduce bias and overlook other effective fixes. Ideally, code error fixes should encompass a variety of approaches, addressing multiple aspects of the problem, such as readability and maintainability. This highlights the potential for bias from a single expert and limits the depth required to consider different fixes when dealing with open-ended code errors.
[0050] Based on the preceding introduction, the following shortcomings can be identified in the existing technology:
[0051] High instability: Existing technologies rely on random sampling to construct prompts, making large language models highly sensitive to examples. This leads to significant performance fluctuations, a high error rate, and an inability to guarantee stable output. This instability makes it difficult to meet the requirements for efficient and stable code generation in practical applications.
[0052] Low accuracy and incomplete coverage: Existing single expert suggestion frameworks introduce bias. These methods address errors from only a single perspective, failing to comprehensively cover multiple error types and neglecting potential alternative remediation solutions. The lack of customized solutions for different tasks, error types, and large language models results in low remediation coverage and overall accuracy.
[0053] Inefficiency and poor interpretability: Existing methods do not systematically mine for error factors, relying on manual error identification or analogy hints to automatically generate examples. This results in high computational complexity, slow response speed, and a lack of transparency and interpretability in the generation process, increasing debugging costs and security risks.
[0054] The following is combined Figures 1 to 6 The present invention describes a method, apparatus, electronic device, and storage medium for automatic code generation.
[0055] Figure 1 This is a flowchart illustrating the automatic code generation method provided by the present invention, such as... Figure 1 As shown, the method includes, but is not limited to, steps S1, S2, S3 and S4.
[0056] Step S1: Input the programming task sample into the target large language model multiple times to obtain multiple versions of code corresponding to the programming task sample output by the target large language model.
[0057] Multiple programming task samples can be randomly selected from code repositories or public datasets (such as HumanEVal). For each programming task sample, it can be input into the target large language model (such as GPT-4o) multiple times to obtain multiple versions of the code corresponding to that programming task sample generated by the target large language model. In this way, each programming task sample has multiple versions of code. For example, each programming task sample can be input into the target large language model 3 times to obtain 3 versions of code.
[0058] Step S2: Determine the degree of controversy of the programming task sample based on the number of target code versions in the multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions.
[0059] Understandably, if the number of correct code versions is greater than the number of incorrect code versions, then the number of target code versions equals the number of correct code versions. The fewer the number of correct code versions, the higher the controversy surrounding the programming task sample. Conversely, if the number of correct code versions is less than the number of incorrect code versions, then the number of target code versions equals the number of incorrect code versions. The fewer the number of incorrect code versions, the higher the controversy surrounding the programming task sample. Therefore, the closer the number of correct code versions is to the number of incorrect code versions, the greater the controversy surrounding the programming task sample. The closer the number of correct code versions is to the number of incorrect code versions for a given programming task sample, the greater the uncertainty of the large language model regarding that programming task sample.
[0060] Step S3: Cluster the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold to obtain the various code error types corresponding to the target large language model.
[0061] Selecting programming task samples with a controversy level greater than a preset threshold is equivalent to sorting the controversy levels of each programming task sample from highest to lowest, and selecting the top-ranked samples. Large language models exhibit significant uncertainty regarding these programming task samples, resulting in typical code errors.
[0062] The various code error types corresponding to the target large language model can include condition judgment errors, method call errors, incomplete code, and so on.
[0063] Step S4: Input the prompt words containing programming tasks and role descriptions corresponding to various code error types into the target large language model to obtain the code output by the target large language model; the role description is used to set the role of code error type handling expert for the target large language model.
[0064] For example, if the various code error types corresponding to the target large language model include condition judgment errors, method call errors, and incomplete code, then the role descriptions corresponding to condition judgment errors, method call errors, and incomplete code can be "You are a condition judgment error handling expert", "You are a method call error handling expert", and "You are a code completion expert", respectively.
[0065] After inputting the prompt words containing role descriptions corresponding to various code error types into the target large language model, the target large language model will automatically generate code in the identity of multiple expert roles.
[0066] As described above, the automatic code generation method of the present invention runs the same programming task sample multiple times and identifies the most uncertain programming task sample of the large language model based on the degree of controversy. It clusters the erroneous code of the highly controversial sample, automatically identifies and summarizes the core error types of the model, and sets corresponding expert roles for the large language model according to the identified error types when generating new code. Through multi-role guidance, the large language model can solve potential errors from multiple perspectives, overcome the bias of a single expert framework, and thus improve the accuracy of code generation and the coverage of fixes.
[0067] In one embodiment, prior to step S2, the method of the present invention may further include:
[0068] Input the programming task sample into another large language model that is different from the target large language model, and obtain the version code corresponding to the programming task sample output by the other large language model.
[0069] For example, if there are two other large language models, the programming task sample can be input into these two large language models respectively to obtain two versions of code. By superimposing the three versions of code generated by the target large language model, each programming task sample will have five versions of code. The degree of controversy of the programming task sample can then be calculated based on the five versions of code.
[0070] Considering that the inherent architecture and training biases of the target large language model may prevent the full exposure of all potential error types, this invention introduces multiple large language models with different architectures or training backgrounds when generating multiple code versions. Each model generates a code version for the same task. By utilizing the differences between different models, more diverse code outputs can be generated, which is more likely to trigger and expose potential error patterns that are difficult for a single model to detect. The degree of controversy calculated based on more diverse code can more accurately reflect the inherent complexity and ambiguity of a programming task sample itself, rather than just the weakness of a single model.
[0071] In one embodiment, step S2 of the present invention may further include:
[0072] Calculate the ratio of the number of target code versions to the total number of code versions;
[0073] Subtract the ratio from 1 to get the degree of controversy.
[0074] The formula for calculating the degree of controversy can be:
[0075] Controversy level = 1 - number of target code versions / total number of code versions.
[0076] For example, if a programming task sample has 3 correct code versions and 2 incorrect code versions, then the number of target code versions is 3, the total number of code versions is 5, and the controversy level of this programming task sample is 1 - 3 / 5 = 0.4.
[0077] The ratio of the number of target code versions to the total number of code versions reflects the consistency of the model output. Subtracting this ratio from 1 yields the controversy level. Thus, the more consistent the model output, the lower the controversy level; the more dispersed the model output, the higher the controversy level, achieving a negative correlation between controversy level and the number of target code versions. This formula makes the controversy level a calculable and rankable indicator, enabling automatic sorting of all programming task samples and precise selection of the samples with the highest controversy level for subsequent error clustering analysis.
[0078] In one embodiment, after step S2 and before step S3, the method of the present invention may further include:
[0079] Correct non-critical errors in the incorrect code version based on the correct code version. Non-critical errors include incorrect variable names.
[0080] Non-critical errors can also include differences in coding style. For example... Figure 2 The correct code version shown corrects the incorrect code version. In the incorrect code, "sum" and "i" are incorrect variable names, and "if i>0 and i % 2 != 0" is a condition judgment error. Condition judgment errors, method call errors, and incomplete code are all critical errors. Therefore, "sum" should be corrected to "ans" in the correct code, and "i" should be corrected to "num" in the correct code.
[0081] Before performing error clustering, a code standardization step is added. Based on the correct code version, non-critical errors unrelated to the core logic in the erroneous code version are corrected, such as unifying variable names. By eliminating non-critical error clustering, we can focus more on the fundamental errors in the logical structure and algorithm level of the code, thereby generating more core and meaningful error types, laying a solid foundation for building high-quality expert roles and examples in the future.
[0082] In one embodiment, after step S3 and before step S4, the method of the present invention may further include:
[0083] Construct an example set for the target large language model, which includes the correct and incorrect code versions corresponding to various code error types;
[0084] The prompts also include the correct and incorrect code versions in the example set.
[0085] like Figure 2 As shown, you can add the correct code version (correct solution example) and the incorrect code version (incorrect solution example) for the conditional statement error to the prompt words. Similarly, you can add the correct code version and the incorrect code version for other code error types to the prompt words.
[0086] This is equivalent to adopting the few-shot strategy in context learning. In addition to describing the model's role, it also adds specific examples to the prompts. Each example contains a pair of correct and incorrect codes, which intuitively shows the model a specific type of error and how to correct it. By combining abstract role descriptions with concrete positive and negative examples, it provides clearer and more direct guidance for large language models, enabling them to better understand task requirements and error patterns. The model can learn and imitate correct code patterns more effectively, thus having a higher probability of generating correct code when faced with new tasks.
[0087] In one embodiment, step S3, clustering the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold, may further include:
[0088] Clustering of various error code versions is performed using a target large language model.
[0089] Specifically, multiple error code versions extracted from highly controversial samples can be input into a large language model, which is then instructed to classify them based on the root cause of the errors. By leveraging the powerful natural language understanding and code analysis capabilities of the large language model itself to perform the clustering task, the error clustering process is fully automated, overcoming the inefficiency and slow response speed of manual intervention.
[0090] Based on the above, we can conclude that... Figure 3 The complete technical solution is shown.
[0091] This invention addresses various existing technologies and Figure 3 The method was used in the experiment, and the results are as follows: Figure 4 As shown.
[0092] Experimental results show that:
[0093] On the HumanEval dataset, the accuracy of the gpt-4o model reached 95.7% after using our method, which is 5.5% higher than the ordinary Few-Shot method.
[0094] On the BigCodeBench (hard) dataset, the accuracy of the gpt-4o model was improved to 29.0% after using our method, which is 5.4% higher than the Few-Shot method.
[0095] On the LeetCode dataset, the gpt-4o model achieved an accuracy of 60.7% using our method, a 5.1% improvement over the Few-Shot method.
[0096] These results demonstrate that the controversy-based strategy is better able to identify samples that can improve model performance, especially on more challenging datasets. By precisely selecting samples with high information gain, this invention effectively overcomes the instability of random sampling.
[0097] Experimental results show that:
[0098] After using our method, the claude-3.5 model achieved an accuracy of 96.3% on the HumanEval dataset, which is 15.8% higher than the Analogical method.
[0099] After using our method, the accuracy of the claude-3.5 model on the LeetCode dataset reached 81.3%, which is 21.9% higher than the Analogical method.
[0100] These data validate the superiority and versatility of this invention in complex code scenarios. The model's self-driven error identification makes error localization more accurate, reduces the need for manual intervention, and improves efficiency and response speed.
[0101] Experimental results show that:
[0102] On the deepseek-v3 model:
[0103] After using our method, the accuracy on the HumanEval dataset was improved to 94.5%, which is 5.4% higher than the ExpertPrompting method.
[0104] After using our method, the accuracy on the LeetCode dataset is 87.3%, which is a significant improvement of 17.2% over the ExpertPrompting method.
[0105] These results demonstrate that the multi-role guidance mechanism has significant advantages in improving the model's adaptability and diversity to code tasks. Through the collaborative work of multiple roles, comprehensive optimization of different types of code errors is achieved, overcoming the bias problem of a single-role framework.
[0106] In theoretical analysis, this invention enhances the model's ability to explore decision boundaries by selecting highly controversial samples and employing multi-role guidance, enabling it to capture deeper error patterns and thus improve the accuracy and diversity of code generation. This method not only improves the quality of generated code but also reduces the probability of generating erroneous code.
[0107] Experimental data fully validates the effectiveness of this invention on different datasets and models. Compared with existing technologies, this invention significantly improves accuracy on various language models and datasets, especially on more challenging datasets (such as BigCodeBench(hard)), demonstrating its advantages in complex tasks.
[0108] This invention automatically selects information-rich and representative samples to construct an efficient example set by comparing positive and negative sample analysis, rather than relying on random sampling. This solves the sampling instability and uncertainty problems of existing technologies, ensures the relevance and stability of the examples, improves the generation accuracy and reduces the generation error rate.
[0109] This invention utilizes the inherent capabilities of large language models to automatically identify, classify, and express key errors in the code domain without manual intervention. This overcomes the inefficiency of existing technologies, achieves accurate error location, and provides an interpretable basis for subsequent guidance, thus addressing the problems of incomplete coverage and slow response speed.
[0110] This invention assigns error factors a role based on different error types and provides customized correctness generation guidance for specific tasks, error types, and large language models. It solves the problems of bias introduction and low accuracy of a single expert framework. By comprehensively optimizing all aspects of the code from multiple perspectives, it enhances adaptability and generalization ability, and achieves correct code generation on the first try.
[0111] This invention utilizes error factor mining and controversy strategies to accurately select high-information-gain prime examples, avoiding the instability caused by random sampling. By leveraging the inherent capabilities of large language models for error identification, classification, and representation, it overcomes the inefficiency of manual intervention and the volatility caused by random examples, ensuring the stability of the generated results.
[0112] This invention employs a multi-role approach, constructing roles based on different error types to provide targeted solutions for each error. This avoids the bias problems of a single expert suggestion framework, comprehensively improves the coverage and accuracy of repairs, and can better adapt to the needs of different tasks, error types, and large language models.
[0113] This invention significantly improves processing efficiency and reduces computational complexity through systematic error factor mining and multi-role guidance. Simultaneously, the generation process is more transparent and interpretable, helping to reduce debugging costs and security risks.
[0114] The automatic code generation apparatus provided by the present invention is described below. The automatic code generation apparatus described below and the automatic code generation method described above can be referred to in correspondence.
[0115] like Figure 5 As shown, the present invention provides a code automatic generation device, including but not limited to:
[0116] The input module is used to input the programming task sample multiple times into the target large language model to obtain multiple versions of the code corresponding to the programming task sample output by the target large language model;
[0117] The determination module is used to determine the degree of controversy of a programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions;
[0118] The clustering module is used to cluster the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold, so as to obtain the various code error types corresponding to the target large language model.
[0119] The input module is also used to input prompts containing programming tasks and role descriptions corresponding to various code error types into the target large language model, and obtain the code output by the target large language model; the role description is used to set the role of code error type handling expert for the target large language model.
[0120] Figure 6 A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can call logical instructions in the memory to execute an automatic code generation method. This method includes: inputting programming task samples multiple times into a target large language model to obtain multiple versions of code corresponding to the programming task samples output by the target large language model; determining the controversy level of the programming task sample based on the number of target code versions among the multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the controversy level is negatively correlated with the number of target code versions; clustering the incorrect code versions corresponding to each programming task sample with a controversy level greater than a preset threshold to obtain various code error types corresponding to the target large language model; inputting prompt words containing role descriptions corresponding to the programming task and various code error types into the target large language model to obtain the code output by the target large language model; the role descriptions are used to assign the role of code error type handling expert to the target large language model.
[0121] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0122] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the code automatic generation method provided by the above methods. The method includes: inputting programming task samples multiple times into a target large language model to obtain multiple versions of code corresponding to the programming task samples output by the target large language model; determining the controversy level of the programming task sample based on the number of target code versions among the multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the controversy level is negatively correlated with the number of target code versions; clustering the various incorrect code versions corresponding to each programming task sample with a controversy level greater than a preset threshold to obtain various code error types corresponding to the target large language model; inputting prompt words containing role descriptions corresponding to the programming task and various code error types into the target large language model to obtain the code output by the target large language model; the role descriptions are used to set the role of code error type handling expert for the target large language model.
[0123] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the automatic code generation method provided by the above methods. This method includes: inputting programming task samples multiple times into a target large language model to obtain multiple versions of code corresponding to the programming task samples output by the target large language model; determining the degree of controversy of a programming task sample based on the number of target code versions among the multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions; clustering the incorrect code versions corresponding to each programming task sample with a degree of controversy greater than a preset threshold to obtain various code error types corresponding to the target large language model; inputting prompt words containing role descriptions corresponding to the programming task and various code error types into the target large language model to obtain the code output by the target large language model; the role descriptions are used to assign the role of a code error type handling expert to the target large language model.
[0124] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for automatically generating codes, characterized by, include: The programming task sample is input into the target large language model multiple times to obtain multiple versions of the code corresponding to the programming task sample output by the target large language model; The degree of controversy for the programming task sample is determined based on the number of target code versions in multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions. Cluster the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold to obtain various code error types corresponding to the target large language model; The target large language model is input with prompt words containing programming tasks and role descriptions corresponding to various code error types to obtain the code output by the target large language model; the role descriptions are used to assign the role of the code error type handling expert to the target large language model.
2. The code automatic generation method according to claim 1, wherein Before determining the degree of controversy of the programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample, the method further includes: The programming task sample is input into another large language model that is different from the target large language model, and the version code corresponding to the programming task sample is output by the other large language model.
3. The method of claim 1, wherein, The step of determining the degree of controversy of the programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample includes: Calculate the ratio of the number of target code versions to the total number of code versions; The degree of controversy is obtained by subtracting the ratio from 1.
4. The method of claim 1, wherein, After determining the controversy level of a programming task sample based on the number of target code versions in multiple versions of code corresponding to the programming task sample, and before clustering the error code versions corresponding to each programming task sample with a controversy level greater than a preset threshold to obtain various code error types corresponding to the target large language model, the method further includes: Correct non-critical errors in the incorrect code version based on the correct code version. The non-critical errors include incorrect variable names.
5. The method of claim 1, wherein, After clustering the error code versions corresponding to each programming task sample with a dispute level greater than a preset threshold to obtain various code error types corresponding to the target large language model, and before inputting prompt words containing the programming task and the role descriptions corresponding to various code error types into the target large language model to obtain the code output by the target large language model, the method further includes: Construct an example set for the target large language model, the example set including the correct code version and the error code version corresponding to various code error types; The prompt words also include the correct code version and the incorrect code version in the example set.
6. The method of claim 1, wherein, The clustering of each error code version corresponding to each programming task sample with a dispute level greater than a preset threshold includes: The error code versions are clustered using the target large language model.
7. An apparatus for automatically generating codes, characterized by comprising: include: The input module is used to input the programming task sample multiple times into the target large language model to obtain multiple versions of the code corresponding to the programming task sample output by the target large language model; The determination module is used to determine the degree of controversy of the programming task sample based on the number of target code versions among multiple versions of code corresponding to the programming task sample; the number of target code versions is the larger of the number of correct code versions and the number of incorrect code versions, and the degree of controversy is negatively correlated with the number of target code versions; The clustering module is used to cluster the error code versions corresponding to each programming task sample with a dispute degree greater than a preset threshold, so as to obtain various code error types corresponding to the target large language model. The input module is further configured to input prompts containing programming tasks and role descriptions corresponding to various code error types into the target large language model to obtain the code output by the target large language model; the role descriptions are used to assign the role of the code error type handling expert to the target large language model.
8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the automatic code generation method as described in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the automatic code generation method as described in any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the automatic code generation method as described in any one of claims 1 to 6.