Code generation model training method, code generation method, device and equipment
By combining negative log-likelihood loss and annotation contrast loss to train the code generation model, the problem of insufficient code quality for small and medium-scale models is solved, and more efficient annotated code generation is achieved.
Patent Information
- Application Number
- CN202410288490.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-13
- Publication Date
- 2025-09-16
AI Technical Summary
Due to the lack of capabilities of small and medium-scale code generation models, it is difficult to improve code quality through direct splitting.
By combining negative log-likelihood loss and annotation contrast loss to train the code generation model, it can learn the difference between annotated and unannotated codes and improve the preference for annotations.
Improved the quality of annotated code generation and training efficiency for small and medium-sized models.
Smart Images
Figure CN120653249A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a code generation model training method, code generation method, device and equipment. Background Art
[0002] Code generation is the task of generating runnable code based on a natural language description of a problem. This typically requires the model to be able to understand complex descriptions. The current mainstream approach is to generate high-quality code by breaking down complex tasks into simpler intermediate steps. This approach has been successful for very large-scale language models. However, for small and medium-sized models, direct decomposition often fails to improve code quality due to insufficient model capabilities. Therefore, how to enable small and medium-sized models to generate high-quality code is an urgent problem to be solved. Summary of the Invention
[0003] The present invention provides a method, apparatus, and device for training a code generation model. The method enables the code generation model to increase its preference for annotations during training, enabling better generation of annotated code, thereby improving the training efficiency of the code generation model. The technical solution is as follows:
[0004] In one aspect, a method for training a code generation model is provided, the method comprising:
[0005] Processing sample task information in sample data based on a code generation model to obtain a target code snippet, the sample data including the sample task information, a positive sample code snippet, and a negative sample code snippet, the sample task information being used to describe conditions that the generated code must satisfy, the positive sample code snippet including code and code comments that satisfy the sample task information, and the negative sample code snippet being obtained by removing comments from the positive sample code snippet;
[0006] Determining a negative log-likelihood loss of the code generation model based on the sample task information and the target code snippet;
[0007] Determining an annotation contrast loss of the code generation model based on the positive sample code snippet and the negative sample code snippet, where the annotation contrast loss is used to represent the difference between the positive sample code snippet and the negative sample code snippet;
[0008] The code generation model is trained based on the negative log-likelihood loss and the annotation contrast loss.
[0009] In another aspect, a code generation method is provided, the method comprising:
[0010] Acquire first task information, where the first task information includes a target logic annotation prompt, where the target logic annotation prompt is used to stimulate the model to generate code annotations, and where the target logic annotation prompt contains more information than the standard logic annotation prompt;
[0011] The first task information is processed based on a target code generation model to obtain a first code fragment including code comments, and the target code generation model is trained by the code generation model training method described in any of the above aspects.
[0012] In another aspect, a device for training a code generation model is provided, the device comprising:
[0013] a processing module configured to process sample task information in sample data based on a code generation model to obtain a target code snippet, wherein the sample data includes the sample task information, a positive sample code snippet, and a negative sample code snippet, wherein the sample task information is used to describe conditions that the generated code must meet, the positive sample code snippet includes code and code comments that meet the sample task information, and the negative sample code snippet is obtained by removing comments from the positive sample code snippet;
[0014] a first loss determination module, configured to determine a negative log-likelihood loss of the code generation model based on the sample task information and the target code snippet;
[0015] A second loss determination module is configured to determine an annotation contrast loss of the code generation model based on the positive sample code snippet and the negative sample code snippet, wherein the annotation contrast loss is used to represent the difference between the positive sample code snippet and the negative sample code snippet;
[0016] A training module is used to train the code generation model based on the negative log-likelihood loss and the annotation contrast loss.
[0017] In some embodiments, the second loss determination module is used to determine a first logarithmic probability and a second logarithmic probability based on the positive sample code snippet and the negative sample code snippet, the first logarithmic probability being the logarithmic probability of the positive sample code snippet, and the second logarithmic probability being the logarithmic probability of the negative sample code snippet; when the difference between the first logarithmic probability and the second logarithmic probability is not less than a threshold, determining that the annotation comparison loss of the code generation model is zero; when the difference between the first logarithmic probability and the second logarithmic probability is less than the threshold, determining that the annotation comparison loss of the code generation model is the difference between the threshold and the difference.
[0018] In some embodiments, the first loss determination module is used to determine the logarithmic probability of any code string in the target code snippet based on the sample task information and at least one code string that is generated before the code string; sum and invert the logarithmic probabilities of multiple code strings in the target code snippet to obtain the negative log-likelihood loss of the code generation model.
[0019] In some embodiments, the apparatus further comprises:
[0020] An acquisition module is used to acquire code corpus, wherein the code corpus includes code and code comments;
[0021] A division module, configured to divide the code corpus into a plurality of positive sample code segments;
[0022] A parsing module, configured to parse any positive sample code snippet to obtain code comments in the positive sample code snippet;
[0023] The parsing module is further configured to delete comments in the positive sample code snippet to obtain a negative sample code snippet corresponding to the positive sample code snippet.
[0024] In another aspect, a code generation device is provided, the device comprising:
[0025] An acquisition module is configured to acquire first task information, wherein the first task information includes a target logic annotation prompt, wherein the target logic annotation prompt is used to stimulate the model to generate code annotations, and the target logic annotation prompt contains more information than the standard logic annotation prompt;
[0026] A processing module is used to process the first task information based on a code generation model to obtain a first code fragment including code comments, and the target code generation model is trained by the code generation model training method described in any of the above aspects.
[0027] In some embodiments, the apparatus further comprises:
[0028] The acquisition module is further configured to acquire feedback information on the first code snippet;
[0029] a fusion module, configured to fuse the feedback information and the first task information into second task information when the feedback information is other information than confirmation information;
[0030] The processing module is further configured to process the second task information based on the target code generation model to obtain a second code fragment including code comments.
[0031] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded and executed by the processor to implement the training method or code generation method of the code generation model in the embodiment of the present application.
[0032] On the other hand, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the training method or code generation method of the code generation model in the embodiment of the present application.
[0033] On the other hand, a computer program product is provided, which includes at least one program segment, wherein the at least one program segment is stored in a computer-readable storage medium, and a processor of a computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, so that the computer device executes the code generation model training method or code generation method described in any of the above implementation methods.
[0034] An embodiment of the present application provides a method for training a code generation model. The method combines negative log-likelihood loss with annotation contrast loss during the model training process, and trains the code generation model through two loss functions, so that the code generation model can learn the difference between annotated code and unannotated code during the training process, thereby improving the model's preference for annotations, and can better generate annotated code, thereby improving the training efficiency of the code generation model. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0036] Figure 1 A schematic diagram of an implementation environment for a code generation model training method provided in an embodiment of the present application;
[0037] Figure 2 A flowchart of a method for training a code generation model provided in an embodiment of the present application;
[0038] Figure 3 A flowchart of another method for training a code generation model provided in an embodiment of the present application;
[0039] Figure 4 This is a flowchart of a code generation method provided by an embodiment of the present application;
[0040] Figure 5 This is a flowchart of another code generation method provided by an embodiment of the present application;
[0041] Figure 6 This is a block diagram of a code generation model training device provided according to an embodiment of the present application;
[0042] Figure 7 is a block diagram of a code generation device provided according to an embodiment of the present application;
[0043] Figure 8 This is a block diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0045] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.
[0046] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more.
[0047] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the positive sample code snippets and sample task information involved in this application were obtained with full authorization.
[0048] The following is an introduction to the professional terms involved in this application:
[0049] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field of computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI domains. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.
[0050] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by demonstration. Pretrained models are the latest development in deep learning, integrating these techniques.
[0051] Code LLMs: Code Large Langage Models, code large language model.
[0052] WizardCoder: A large model series based on open source CodeLLaMA instructions fine-tuned by pre-trained models in multiple code language corpora, including versions with multiple model parameter sizes.
[0053] StarCoder: An open-source series of large models pre-trained on a variety of coding language corpora, including versions with different model parameter sizes.
[0054] MANGO: code as natural logic pivOt, code as a bridge between nature and logic, this application proposes a solution to stimulate code generation capabilities to improve code generation quality.
[0055] LCP: Logical Comment Prompt, a decoding strategy that uses prompts to stimulate the model to add comments.
[0056] CoT: Chain of Thought, a strong baseline that uses step-by-step thinking prompts to improve the performance of models on complex reasoning tasks.
[0057] Pass@k: A metric for evaluating the pass rate of code generation. After multiple sampling, it calculates the probability that at least k sample results can pass the test.
[0058] The following is an introduction to the implementation environment involved in this application:
[0059] The code generation model training method provided in the embodiment of the present application can be executed by a computer device, which can be provided as a server or a terminal. The following is a schematic diagram of the implementation environment of the code generation model training method provided in the embodiment of the present application.
[0060] See also Figure 1 , Figure 1 A schematic diagram of an implementation environment for a training method for a code generation model provided in an embodiment of the present application, the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, and the present application is not limited thereto. In some embodiments, the server 102 is used to train the code generation model, and the trained code generation model is used to generate code with annotations. A target application is installed on the terminal 101, and the target application is used to generate code. In some embodiments, the terminal 101 is embedded with a trained code generation model, and the terminal 101 generates code through the code generation model. In other embodiments, the terminal 101 generates code through the code generation model on the server 102.
[0061] In some embodiments, the terminal 101 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a VR (Virtual Reality) device, an AR (Augmented Reality) device, etc., but is not limited thereto. In some embodiments, the server 102 is an independent server or a server cluster or distributed system composed of multiple servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some embodiments, the server 102 is primarily responsible for computing tasks, and the terminal 101 is responsible for secondary computing tasks; alternatively, the server 102 is responsible for secondary computing services, and the terminal 101 is responsible for primary computing tasks; alternatively, the server 102 and the terminal 101 use a distributed computing architecture for collaborative computing.
[0062] See also Figure 2 , Figure 2 A flowchart of a method for training a code generation model provided in an embodiment of the present application, the method comprising the following steps.
[0063] 201. The computer device processes the sample task information in the sample data based on the code generation model to obtain a target code fragment.
[0064] In an embodiment of the present application, a code generation model is used to generate code snippets that meet the requirements based on task information. For example, the task information is to calculate the average value given a set of numbers and output Python code. The code generation model will then generate the corresponding Python code. The sample data includes sample task information, positive sample code snippets, and negative sample code snippets. The sample task information is used to describe the conditions that the generated code needs to meet. The positive sample code snippet includes the code and code comments that meet the sample task information. The negative sample code snippet is obtained by deleting the comments from the positive sample code snippet. Among them, the positive sample code snippet is a positive sample, including code and code comments. The negative sample code snippet is a negative sample, including code but not including code comments. That is, after deleting the code comments from the positive sample code snippet, the negative sample code snippet is obtained.
[0065] 202. The computer device determines a negative log-likelihood loss of a code generation model based on the sample task information and the target code snippet.
[0066] In an embodiment of the present application, a computer device performs model training by using negative log-likelihood loss, wherein the negative log-likelihood loss is the inverse of the log-likelihood loss. Log-likelihood loss (Log-likelihood Loss), also known as logistic regression loss (Logistic Loss) or cross-entropy loss (cross-entropy Loss), is defined on probability estimation. Log-likelihood loss is commonly used in multinomial logistic regression and neural networks, as well as some variants of expectation maximization algorithms. Log-likelihood loss quantifies the accuracy of the classifier by penalizing incorrect classifications. Minimizing the log-likelihood loss is essentially equivalent to maximizing the accuracy of the classifier.
[0067] 203. The computer device determines the annotation comparison loss of the code generation model based on the positive sample code snippet and the negative sample code snippet.
[0068] In the embodiment of the present application, the model training is also performed by using annotation contrast loss, which is used to represent the difference between the positive sample code snippet and the negative sample code snippet.
[0069] 204. Computer equipment trains a code generation model based on negative log-likelihood loss and annotation contrast loss.
[0070] In an embodiment of the present application, the computer device may sum the two types of losses determined in the above steps, and adjust the parameters of the code generation model based on the sum of the losses to achieve iterative training of the code generation model.
[0071] The training scheme for the code generation model provided in this application combines the negative log-likelihood loss with the annotation contrast loss during the model training process, and trains the code generation model through two loss functions, so that the code generation model can learn the difference between annotated code and unannotated code during the training process, thereby improving the model's preference for annotations, and can better generate annotated code, thereby improving the training efficiency of the code generation model.
[0072] above Figure 2 The basic process of the training method for the code generation model is as follows: Figure 3 Further introduction to the training method of the code generation model. Figure 3 , Figure 3 A flowchart of another method for training a code generation model provided in an embodiment of the present application, the method comprising the following steps.
[0073] 301. The computer device obtains sample data.
[0074] In an embodiment of the present application, sample data includes sample task information, positive sample code snippets, and negative sample code snippets. The sample task information describes the conditions that the generated code must meet. The positive sample code snippets include code and code comments that meet the sample task information. The negative sample code snippets are obtained by removing the comments from the positive sample code snippets. Accordingly, a computer device first obtains a code corpus. The code corpus includes code and code comments. The computer device then divides the code corpus into multiple positive sample code snippets. Optionally, each positive sample code snippet can implement a complete function. For each positive sample code snippet, the computer device obtains the sample task information corresponding to the positive sample code snippet. The sample task information indicates the conditions that the positive sample code snippet must meet, such as the type of code to be generated and the function to be implemented. Optionally, the sample task information can be compiled by a person skilled in the art based on the positive sample code snippet. For each positive sample code snippet, the computer device parses the positive sample code snippet to obtain the code comments within the positive sample code snippet. By removing the comments within the positive sample code snippet, the negative sample code snippet corresponding to the positive sample code snippet can be obtained. In other words, the negative code snippets are obtained by removing the code comments in the positive code snippets.
[0075] For example, the positive sample code snippet is:
[0076] “#Calculate the average of the numbers
[0077] total=sum(numbers)
[0078] average=total / len(numbers)
[0079] return average”.
[0080] Based on the functionality of the positive code snippet above, the developer writes a task description. This task description consists of two parts: one is the task description: Given a list of numbers, calculate their average; the other is the related instruction information: Generate Python code.
[0081] The negative sample code snippet is:
[0082] "total = sum(numbers)
[0083] average=total / len(numbers)
[0084] return average”.
[0085] 302. The computer device processes the sample task information in the sample data based on the code generation model to obtain a target code segment.
[0086] In an embodiment of the present application, for code generation tasks, different code generation models are used to generate different types of code. For example, a C++ generation model is used to generate C++ code, a Python generation model is used to generate Python code, and a Java generation model is used to generate Java code. The computer device can determine the required code generation model based on the sample task information, and then process the task description and related instruction information in the sample task information based on the code generation model to generate the corresponding target code snippet.
[0087] It should be noted that the higher the similarity between the target code snippet and the positive sample code snippet, the better the code generation model.
[0088] 303. The computer device determines a negative log-likelihood loss of the code generation model based on the sample task information and the target code snippet.
[0089] In an embodiment of the present application, the computer device can train the model based on a negative log-likelihood loss function to reduce the difference between the generated code snippet and the code snippet used as a label.
[0090] In some embodiments, for any code string in a target code segment, the logarithmic probability of the code string is determined based on sample task information and at least one code string that precedes the code string in the generation order. The logarithmic probabilities of multiple code strings in the target code segment are then summed and negated to obtain a negative log-likelihood loss for the code generation model. For any code string, by determining the logarithmic probability of the code string based on previously generated code strings relative to the code string, the contextual relationship between the code strings can be determined, thereby enabling the model to learn the continuity between the code strings and improving the quality of the model.
[0091] Optionally, the negative log-likelihood loss function is shown in the following formula (1).
[0092] Formula (1):
[0093]
[0094] Among them, L lm represents the negative log-likelihood loss; N represents the total number of characters in the target code snippet; P(y i |y i-1 ,…,y1,X) represents the i-1,…,y1,X determines y i The probability of y i represents the i-th character in the target code snippet; X represents task information.
[0095] 304. The computer device determines the annotation comparison loss of the code generation model based on the positive sample code snippet and the negative sample code snippet.
[0096] In an embodiment of the present application, in order to enable the model to learn the difference between annotated code snippets and unannotated code snippets, the computer device can train the model based on an annotation contrast loss function, rewarding the model's preference for annotated code snippets by the difference between positive and negative samples, so that the model can be biased towards generating annotated code snippets, thereby improving the quality of the code snippets generated by the model. The annotation contrast loss is used to represent the difference between the positive sample code snippet and the negative sample code snippet.
[0097] In some embodiments, the step of determining the annotation contrast loss by the computer device using the annotation contrast loss function includes: the computer device determines a first logarithmic probability and a second logarithmic probability based on the positive sample code snippet and the negative sample code snippet. The first logarithmic probability is the logarithmic probability of the positive sample code snippet, and the second logarithmic probability is the logarithmic probability of the negative sample code snippet. When the difference between the first logarithmic probability and the second logarithmic probability is not less than a threshold, the computer device determines that the annotation contrast loss of the code generation model is zero; when the difference between the first logarithmic probability and the second logarithmic probability is less than a threshold, the computer device determines that the annotation contrast loss of the code generation model is the difference between the threshold and the difference.
[0098] Optionally, the annotation contrast loss function is shown in the following formula (2).
[0099] Formula (2):
[0100]
[0101] Among them, L cl Indicates annotation comparison loss; max() indicates the maximum value; m indicates the threshold; Indicates determination based on X probability; Determine based on X probability; Represents the negative sample code snippet, that is, the negative sample corresponding to the target code snippet; represents the positive sample code snippet, that is, the positive sample corresponding to the target code snippet; X represents the task information.
[0102] 305. Computer equipment trains a code generation model based on negative log-likelihood loss and annotation contrast loss.
[0103] In an embodiment of the present application, the computer device may sum the negative log-likelihood loss and the annotation contrast loss to obtain a total training loss, and then train the code generation model using the total training loss.
[0104] Optionally, the total training loss is calculated as shown in the following formula (3):
[0105] Formula (3):
[0106] L=L lm +L cl
[0107] Among them, L represents the total training loss, L lm represents the negative log-likelihood loss; L cl denotes the annotation contrast loss.
[0108] It should be noted that the above steps are based on a single training run. The code generation model requires multiple iterations to complete training. For any iteration, if the trained code generation model meets the requirements, training ends; otherwise, the next iteration continues.
[0109] It should be noted that this application uses annotation contrast loss as an example for illustration. Optionally, other contrastive learning scoring strategies, such as InfoNCE, can also be used for training. This embodiment of the application is not limited to this.
[0110] An embodiment of the present application provides a method for training a code generation model. The method combines negative log-likelihood loss with annotation contrast loss during the model training process, and trains the code generation model through two loss functions, so that the code generation model can learn the difference between annotated code and unannotated code during the training process, thereby improving the model's preference for annotations, and can better generate annotated code, thereby improving the training efficiency of the code generation model.
[0111] Figure 2 and 3 The embodiment of the present invention is the process of training the code generation model. Figure 4 The embodiment of the present invention introduces the use process of the code generation model. Figure 4 , Figure 4 This is a flowchart of a code generation method provided in an embodiment of the present application. The code generation model used in this method is the code generation model trained by any of the above embodiments. The method includes the following steps.
[0112] 401. A computer device obtains first task information, where the first task information includes a target logic annotation prompt. The target logic annotation prompt is used to stimulate a model to generate code annotations. The target logic annotation prompt contains more information than a standard logic annotation prompt.
[0113] In an embodiment of the present application, a user may input first task information through an annotation input interface. A computer device may obtain the first task information input by the user. The first task information may include target logic annotation prompts for stimulating the model to generate code annotations. The target logic annotation prompts may include a large amount of prompt information, enabling the code generation model to better generate annotations.
[0114] 402. The computer device processes the first task information based on the target code generation model to obtain a first code segment including code comments.
[0115] In an embodiment of the present application, a computer device generates a first code snippet using a trained target code generation model, wherein the first code snippet includes code comments.
[0116] This embodiment of the present application provides a code generation method that generates code snippets including code comments by inputting task information. This method can be applied to scenarios such as online code generation. Because the code snippets are generated using a trained code generation model, and the task information includes more prompt information, it can better stimulate the code generation model to generate more detailed comments, thereby improving the efficiency of code snippet generation.
[0117] above Figure 4 The basic process of the code generation method is as follows: Figure 5 The embodiment of the code generation method is further introduced. Figure 5 , Figure 5 This is a flowchart of another code generation method provided in an embodiment of the present application, which includes the following steps.
[0118] 501. The computer device displays a task setting page, where the task setting page is used to set a code generation task.
[0119] In this embodiment of the present application, users can create and set code generation tasks through the task settings interface. Code generation tasks include task time and task information. Task information describes the conditions that the generated code must meet, such as the type of code to be generated and the functionality to be implemented. Task information consists of two parts: a task description and related instruction information.
[0120] 502. The computer device obtains first task information, where the first task information includes a target logic annotation prompt. The target logic annotation prompt is used to stimulate the model to generate code annotations. The target logic annotation prompt contains more information than the standard logic annotation prompt.
[0121] In an embodiment of the present application, after a user submits a code generation task, the computer device parses the code generation task to obtain first task information. The first task information includes instructions related to the target logic annotation prompt.
[0122] For example, see the following Table 1. Table 1 exemplarily shows the difference between the standard annotation logic prompt and the target logic annotation prompt.
[0123] Table 1
[0124]
[0125] As shown in Table 1, the target logic annotation prompt has an additional bold part compared to the standard annotation logic prompt. It should be noted that the above target logic annotation prompt is an exemplary example. Accordingly, other prompts with the same semantics can also be used, and the embodiments of the present application are not limited to this. For example: "The following Python script addresses the given problem, with complex logic broken down using straightforward comments:" or "To clarify the logic, it's preferable to include annotations in the script. Here's the Python code for the given issue:".
[0126] 503. The computer device processes the first task information based on the target code generation model to obtain a first code segment including code comments.
[0127] In an embodiment of the present application, a computer device may generate a code snippet that satisfies first task information using a target code generation model. For ease of description, this is referred to as a first code snippet. The first code snippet includes code comments. The computer device may display the generated first code snippet. A user may provide feedback on the first code snippet.
[0128] 504. The computer device obtains feedback information on the first code segment.
[0129] In an embodiment of the present application, a computer device may obtain user feedback regarding a first code snippet. The feedback information indicates whether the user is satisfied with the first code snippet. If the user is satisfied with the first code snippet, the feedback information serves as confirmation. If the user is dissatisfied with the first code snippet, the user may enter additional information. The additional information may be used to instruct the user to regenerate or add new constraints.
[0130] 505. When the feedback information is information other than confirmation information, the computer device merges the feedback information and the first task information into second task information.
[0131] In the embodiment of the present application, the computer device may fuse the first task information and the feedback information as new task information, which is referred to as second task information for ease of description.
[0132] 506. The computer device processes the second task information based on the target code generation model to obtain a second code segment including code comments.
[0133] In the embodiment of the present application, the computer device generates a new code snippet based on the second task information generated in the previous step through the target code generation model. For ease of description, the second code snippet is referred to as the second code snippet. The second code snippet is more in line with the user's needs than the first code snippet.
[0134] It should be noted that the user can continue to input feedback information until the generated code snippet meets the requirements.
[0135] The following further introduces the process and effect of the code generation method provided in the embodiment of the present application.
[0136] See Table 2, which exemplarily shows the differences between different code generation methods.
[0137] Table 2
[0138]
[0139] Depend on Figure 2 It can be seen that the code generation method that includes annotation comparative learning (the model training method provided in this application) can generate code snippets containing more comments. The code generation method that includes annotation comparative learning and target logic annotation prompts (the code generation method provided in this application) can generate code snippets containing even more comments.
[0140] See Table 3, which uses the Pass@K metric for evaluation. Pass@1 is a standard method for evaluating code generation. Higher values indicate better performance, and an improvement of more than two points generally indicates a significant improvement. Pass@1 indicates accurate results generated in a single pass. Pass@5 indicates at least one accurate result in five passes. Pass@10 indicates at least one accurate result in ten passes. SFT and COT are existing code generation strategies, while LCP is a method that only adds logic annotation hints. MANGO represents the method proposed in this application. HumanEval and MBPP represent datasets. StarCoder and WizardCoder represent model architectures. 3B and 7B represent model sizes, where B = billion and 7B = 7 billion represent 7 billion parameters. These two model sizes require the V100 and A100 graphics cards for training or inference, respectively. The V100 graphics card has 16GB or 32GB of HBM2 memory, with a memory bandwidth of 900GB / s. The A100's larger memory capacity and higher bandwidth enable it to handle larger datasets and complex computational tasks.
[0141] Table 3
[0142]
[0143] As shown in Table 3, the solution (MANGO) provided in this application has significant improvements over the existing technology in various models and datasets.
[0144] The embodiment of the present application provides a code generation method, which generates code snippets including code comments by inputting task information and feedback information, and can be applied to scenarios such as online code generation. Since the code snippets are generated by the code generation model obtained through training, and the task information includes more prompt information, the code generation model can be better stimulated to generate more detailed comments. The code snippets are then regenerated by combining the feedback information, so that the generated code snippets can better meet the needs of users, improve the quality of the generated code, and thus improve the generation efficiency of the code snippets.
[0145] Figure 6 This is a block diagram of a code generation model training device provided according to an embodiment of the present application. Figure 6 The device includes: a processing module 601, a first loss determination module 602, a second loss determination module 603 and a training module 604.
[0146] Processing module 601 is configured to process sample task information in sample data based on a code generation model to obtain target code snippets. The sample data includes sample task information, positive sample code snippets, and negative sample code snippets. The sample task information describes the conditions that the generated code must meet. The positive sample code snippet includes code that meets the sample task information and code comments. The negative sample code snippet is obtained by removing comments from the positive sample code snippet.
[0147] A first loss determination module 602 is configured to determine a negative log-likelihood loss of a code generation model based on sample task information and a target code snippet;
[0148] A second loss determination module 603 is configured to determine an annotation comparison loss of the code generation model based on the positive sample code snippet and the negative sample code snippet, where the annotation comparison loss is used to represent the difference between the positive sample code snippet and the negative sample code snippet;
[0149] The training module 604 is used to train the code generation model based on negative log-likelihood loss and annotation contrast loss.
[0150] In some embodiments, the second loss determination module 603 is used to determine a first logarithmic probability and a second logarithmic probability based on the positive sample code snippet and the negative sample code snippet, the first logarithmic probability being the logarithmic probability of the positive sample code snippet, and the second logarithmic probability being the logarithmic probability of the negative sample code snippet; when the difference between the first logarithmic probability and the second logarithmic probability is not less than a threshold, the annotation comparison loss of the code generation model is determined to be zero; when the difference between the first logarithmic probability and the second logarithmic probability is less than a threshold, the annotation comparison loss of the code generation model is determined to be the difference between the threshold and the difference.
[0151] In some embodiments, the first loss determination module 602 is used to determine the logarithmic probability of any code string in the target code snippet based on sample task information and at least one code string that is generated before the code string; sum and invert the logarithmic probabilities of multiple code strings in the target code snippet to obtain the negative log-likelihood loss of the code generation model.
[0152] In some embodiments, the apparatus further comprises:
[0153] The acquisition module is used to obtain code corpus, which includes code and code comments;
[0154] The division module is used to divide the code corpus into multiple positive sample code fragments;
[0155] A parsing module is used to parse any positive sample code snippet to obtain code comments in the positive sample code snippet;
[0156] The parsing module is also used to delete the comments in the positive sample code snippet to obtain the negative sample code snippet corresponding to the positive sample code snippet.
[0157] An embodiment of the present application provides a training device for a code generation model. The device combines negative log-likelihood loss with annotation contrast loss during the model training process, and trains the code generation model through two loss functions, so that the code generation model can learn the difference between annotated code and unannotated code during the training process, thereby improving the model's preference for annotations, and can better generate annotated code, thereby improving the training efficiency of the code generation model.
[0158] Figure 7 is a block diagram of a code generation device provided according to an embodiment of the present application. Figure 7 The device includes: an acquisition module 701 and a processing module 702.
[0159] An acquisition module 701 is configured to acquire first task information, wherein the first task information includes a target logic annotation prompt, wherein the target logic annotation prompt is used to stimulate the model to generate code annotations, and the target logic annotation prompt contains more information than the standard logic annotation prompt;
[0160] The processing module 702 is used to process the first task information based on the target code generation model to obtain a first code fragment including code comments. The target code generation model is trained by the code generation model training method of any aspect above.
[0161] In some embodiments, the apparatus further comprises:
[0162] The acquisition module 701 is further configured to obtain feedback information on the first code snippet;
[0163] a fusion module, configured to fuse the feedback information and the first task information into second task information when the feedback information is other information than confirmation information;
[0164] The processing module 702 is further configured to process the second task information based on the target code generation model to obtain a second code segment including code comments.
[0165] An embodiment of the present application provides a code generation device that generates code snippets including code comments by inputting task information and feedback information, which can be applied to scenarios such as online code generation. Since the code snippets are generated by the trained code generation model, and the task information includes more prompt information, the code generation model can be better stimulated to generate more detailed comments. The code snippets are then regenerated by combining the feedback information, so that the generated code snippets can better meet the needs of users, improve the quality of the generated code, and thus improve the generation efficiency of the code snippets.
[0166] In the embodiments of the present application, the computer device may be a terminal or a server. When the computer device is a terminal, the terminal serves as the execution subject to implement the technical solution provided in the embodiments of the present application; when the computer device is a server, the server serves as the execution subject to implement the technical solution provided in the embodiments of the present application; or, the technical solution provided in the present application may be implemented through interaction between the terminal and the server, which is not limited in the embodiments of the present application.
[0167] Take the computer device configured as a server as an example. Figure 8 It is a structural diagram of a server provided in accordance with an embodiment of the present application. The server 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 801 and one or more memories 802, wherein the memory 802 is used to store executable program code, and the processor 801 is configured to execute the above-mentioned executable program code to implement the training method or code generation method of the code generation model provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.
[0168] An embodiment of the present application also provides a computer-readable storage medium, in which at least one program is stored, and the at least one program is loaded and executed by a processor to implement a code generation model training method or a code generation method of any of the above-mentioned implementation methods.
[0169] An embodiment of the present application also provides a computer program product, which includes at least one program segment, and the at least one program segment is stored in a computer-readable storage medium. The processor of the computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, so that the computer device executes the code generation model training method or code generation method of any of the above-mentioned implementation methods.
[0170] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here. The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. A method for training a code generation model, characterized in that: The method comprises: Processing sample task information in sample data based on a code generation model to obtain a target code snippet, the sample data including the sample task information, a positive sample code snippet, and a negative sample code snippet, the sample task information being used to describe conditions that the generated code must satisfy, the positive sample code snippet including code and code comments that satisfy the sample task information, and the negative sample code snippet being obtained by removing comments from the positive sample code snippet; Determining a negative log-likelihood loss of the code generation model based on the sample task information and the target code snippet; Determining an annotation contrast loss of the code generation model based on the positive sample code snippet and the negative sample code snippet, where the annotation contrast loss is used to represent the difference between the positive sample code snippet and the negative sample code snippet; The code generation model is trained based on the negative log-likelihood loss and the annotation contrast loss.
2. The method according to claim 1, characterized in that The determining, based on the positive sample code snippet and the negative sample code snippet, the annotation comparison loss of the code generation model includes: Determining a first logarithmic probability and a second logarithmic probability based on the positive sample code fragment and the negative sample code fragment, wherein the first logarithmic probability is the logarithmic probability of the positive sample code fragment, and the second logarithmic probability is the logarithmic probability of the negative sample code fragment; When a difference between the first logarithmic probability and the second logarithmic probability is not less than a threshold, determining that the annotation comparison loss of the code generation model is zero; When the difference between the first logarithmic probability and the second logarithmic probability is smaller than the threshold, the annotation comparison loss of the code generation model is determined to be the difference between the threshold and the difference.
3. The method according to claim 1, characterized in that Determining the negative log-likelihood loss of the code generation model based on the sample task information and the target code snippet includes: For any code string in the target code segment, determining a logarithmic probability of the code string based on the sample task information and at least one code string that is generated before the code string; The logarithmic probabilities of multiple code strings in the target code segment are summed and inverted to obtain a negative log-likelihood loss of the code generation model.
4. The method according to claim 1, wherein The method further comprises: Obtaining code corpus, wherein the code corpus includes code and code comments; Dividing the code corpus into a plurality of positive sample code snippets; For any positive sample code snippet, the positive sample code snippet is parsed to obtain code comments in the positive sample code snippet; The comments in the positive sample code snippet are deleted to obtain a negative sample code snippet corresponding to the positive sample code snippet.
5. A code generation method, characterized in that: The method comprises: Acquire first task information, where the first task information includes a target logic annotation prompt, where the target logic annotation prompt is used to stimulate the model to generate code annotations, and where the target logic annotation prompt contains more information than the standard logic annotation prompt; The first task information is processed based on a target code generation model to obtain a first code fragment including code comments, and the target code generation model is trained by the code generation model training method described in any one of claims 1 to 4.
6. The method according to claim 5, characterized in that The method further comprises: Obtaining feedback information on the first code snippet; In a case where the feedback information is information other than confirmation information, fusing the feedback information and the first task information into second task information; The second task information is processed based on the target code generation model to obtain a second code fragment including code comments.
7. A training device for a code generation model, characterized in that: The device comprises: a processing module configured to process sample task information in sample data based on a code generation model to obtain a target code snippet, wherein the sample data includes the sample task information, a positive sample code snippet, and a negative sample code snippet, wherein the sample task information is used to describe conditions that the generated code must meet, the positive sample code snippet includes code and code comments that meet the sample task information, and the negative sample code snippet is obtained by removing comments from the positive sample code snippet; a first loss determination module, configured to determine a negative log-likelihood loss of the code generation model based on the sample task information and the target code snippet; A second loss determination module is configured to determine an annotation contrast loss of the code generation model based on the positive sample code snippet and the negative sample code snippet, wherein the annotation contrast loss is used to represent the difference between the positive sample code snippet and the negative sample code snippet; A training module is used to train the code generation model based on the negative log-likelihood loss and the annotation contrast loss.
8. A code generating device, characterized in that: The device comprises: An acquisition module is configured to acquire first task information, wherein the first task information includes a target logic annotation prompt, wherein the target logic annotation prompt is used to stimulate the model to generate code annotations, and the target logic annotation prompt contains more information than the standard logic annotation prompt; A processing module is used to process the first task information based on a target code generation model to obtain a first code fragment including code comments, and the target code generation model is trained by the training method of the code generation model described in any one of claims 1 to 4.
9. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one computer program, the at least one computer program is loaded by the processor and executes the training method of the code generation model described in any one of claims 1 to 4, or the at least one computer program is loaded by the processor and executes the code generation method described in claim 5 or 6.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is used to execute the training method of the code generation model described in any one of claims 1 to 4, or the at least one computer program is used to execute the code generation method described in claim 5 or 6.
11. A computer program product, comprising at least one program segment, wherein the at least one program segment is stored in a computer-readable storage medium, and a processor of a computer device reads the at least one program segment from the computer-readable storage medium, and the processor executes the at least one program segment, so that the computer device executes the code generation model training method described in any one of claims 1 to 4, or the code generation model training method described in claim 5 or 6.