Training method of code generation model and code generation method and system

Abstract

The invention provides a code generation model training method and a code generation method and system.The code generation model training method comprises the steps that a first training data set is obtained, the first training data set comprises a first data pair, and the first data pair comprises a first code problem sample and a first test case corresponding to the first code problem sample; inputting the first code question sample into a pre-trained basic network model to obtain a first prediction code answer corresponding to the first code question sample, and performing correctness verification on the first prediction code answer based on the first test case to obtain a first verification result, and performing reinforcement learning training on the basic network model according to the first verification result to obtain a code generation model. The basic network model is trained by combining the first verification result and reinforcement learning, so that the correctness and robustness of generating corresponding codes by the code generation model can be better improved. And the code capability of the code generation model is improved.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and particularly to a training method for a code generation model, a code generation method, and a system. Background Art

[0002] With the development of artificial intelligence technology, large language models have gradually been more widely applied. The improvement of the code ability of large language models has always been an important direction that the industry pays more attention to.

[0003] In related technologies, mainly the schemes of code model distillation or data strong supervised learning are used to improve the corresponding model effects. However, this is relatively limited in improving the code ability of large language models.

[0004] For example, taking code model distillation as an example, usually a large number of code instruction problems are used to call the open-source large-parameter and powerful models in the industry to obtain question-answer (QA) data pairs, and supervised learning is used to improve the own model. This method can improve the model's various list metrics, but the model's generalization ability is weak.

[0005] It should be noted that the content of the above related technologies is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the filing date of this specification, nor does it mean that it can become the prior art of this specification. Summary of the Invention

[0006] This specification provides a training method for a code generation model, a code generation method, and a system to avoid at least one of the above technical problems.

[0007] In a first aspect, this specification provides a training method for a code generation model, including:

[0008] Obtain a first training data set, where the first training data set includes first data pairs, and the first data pairs include first code problem samples and first test cases corresponding to the first code problem samples;

[0009] Input the first code problem sample into a pre-trained basic network model to obtain a first predicted code answer corresponding to the first code problem sample;

[0010] Verify the correctness of the first predicted code answer based on the first test case to obtain a first verification result; and

[0011] Perform reinforcement learning training on the basic network model according to the first verification result to obtain a code generation model, where the code generation model is used to generate code corresponding to code problems.

[0012] Second aspect, this specification provides a code generation method, including:

[0013] Obtain the code generation problem to be generated;

[0014] Input the code generation problem to be generated into a code generation model to obtain the code corresponding to the code generation problem to be generated, where the code generation model is obtained based on the training method described in the first aspect.

[0015] Third aspect, this specification provides a training system for a code generation model, including:

[0016] At least one storage medium storing at least one instruction set for training a code generation model;

[0017] At least one processor communicatively connected to the at least one storage medium, where when the at least one processor runs, it reads the at least one instruction set and executes the training method described in the first aspect according to the indication of the at least one instruction set.

[0018] Fourth aspect, this specification provides a code generation system, including:

[0019] At least one storage medium storing at least one instruction set for generating code;

[0020] At least one processor communicatively connected to the at least one storage medium, where when the at least one processor runs, it reads the at least one instruction set and executes the code generation method described in the second aspect according to the indication of the at least one instruction set.

[0021] Fifth aspect, this specification provides a computer-readable non-transitory storage medium, where at least one instruction set is stored in the computer-readable non-transitory storage medium, and the at least one instruction set is executed by at least one processor to implement the method described in the first aspect or the second aspect.

[0022] As can be seen from the above technical solutions, the training method, code generation method and system of the code generation model provided by this specification can better improve the correctness and robustness of the code generation model to generate corresponding code by combining the first verification result and reinforcement learning. Furthermore, the code ability of the code generation model is improved.

[0023] Other functions of the training method, code generation method and system of the code generation model provided by this specification will be partially listed in the following description. The creative aspects of the training method, code generation method and system of the code generation model provided by this specification can be fully explained by practice or using the methods, devices and combinations described in the following detailed examples. Description of the Drawings

[0024] To more clearly illustrate the technical solutions in the embodiments of this specification, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0025] Figure 1 Schematic diagram of the application scenario of the training method for the code generation model provided in the embodiments of this specification;

[0026] Figure 2 Schematic diagram of the structure of the training system for the code generation model provided in the embodiments of this specification;

[0027] Figure 3 Schematic diagram of the flow of the training method for the code generation model provided in an embodiment of this specification;

[0028] Figure 4 Schematic diagram of the flow of the training method for the code generation model provided in another embodiment of this specification; <able>

[0029] Figure 5 Schematic diagram of the flow of the code generation method provided in the embodiments of this specification. Specific implementation manners

[0030] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are only examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0031] It should be understood that the terms "including" and "having" and any variations thereof in the embodiments of this specification are intended to cover but not be exclusive of inclusion. For example, a product or device including a series of components does not necessarily have to be limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0032] The term "and / or" in the embodiments of this specification describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0033] In the embodiments of this specification, the term "plurality" refers to two or more, and other quantifiers are similar thereto.

[0034] The terms "first", "second", "third", etc. in this specification are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be interchanged under appropriate circumstances, for example, it is possible to implement in an order other than those given in the illustrations or descriptions of the embodiments of this specification.

[0035] The term "unit / module" used in this specification refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to the element.

[0036] In order to avoid at least one of the technical problems mentioned in the above background art, this specification presents a technical concept through creative labor:

[0037] First, a network model with relatively basic code capabilities (which can be called a basic network model) is trained to predict code answers corresponding to code problem samples based on the basic network model.

[0038] Then, the correctness of the predicted code answers is verified based on test cases corresponding to the code problem samples, such as verifying whether the predicted code answers are executable codes, correct codes, etc. For example, according to an executable code checker, verify whether the code answers are executable codes and are correctly executed codes to obtain the result of the correctness verification.

[0039] Finally, reinforcement learning training is performed on the basic network model with basic code capabilities according to the result of the correctness verification, so as to obtain a code generation model with relatively high code capabilities. For example, if the result of the correctness verification indicates that the code answer is an executable code and is correctly executed, then use the result of the correctness verification as a positive feedback signal to adjust the code generation strategy of the basic network model, such as the parameters of the basic network model, etc., to iteratively train and finally obtain the code generation model. On the contrary, if the result of the correctness verification indicates that the code answer is an unexecutable code or an incorrectly executed code, then use the result of the correctness verification as a negative feedback signal to adjust the code generation strategy of the basic network model, such as the parameters of the basic network model, etc., to iteratively train and finally obtain the code generation model.

[0040] Correspondingly, the code generation model can be used to generate codes corresponding to code problems.

[0041] Among them, code ability can include the ability to generate, understand, and manipulate code. Relatively speaking, the stronger the code ability, the more accurate and reliable the code generated based on this code ability.

[0042] A code problem sample can be understood as a sample of a relatively specific programming task or problem description. For example, "write a function to calculate the greatest common divisor of two integers". Correspondingly, a code answer can be understood as the specific code corresponding to the code problem sample. For example, the code for "a function to calculate the greatest common divisor of two integers".

[0043] A test case can be understood as the input and expected output corresponding to the code problem sample, used to verify the correctness of the generated code answer.

[0044] A basic network model can be understood as a pre-trained network model (such as GPT, Codex, etc.), which can generate candidate code answers according to the input natural language or code problem. That is, the basic network model can transform the problem description into a code prediction result. Relatively speaking, the code ability of the basic network model is weaker than that of the code generation model.

[0045] Reinforcement Learning (RL) is a machine learning method and a field of machine learning. It learns how to take actions to maximize the cumulative reward by interacting with the environment. In reinforcement learning, an agent gradually optimizes its decision-making strategy through exploration (trying different behaviors) and exploitation (selecting the known best behavior). The behavior strategy of the model is optimized through a reward mechanism. Exemplarily, reinforcement learning uses the result of correctness verification as a feedback signal to adjust the parameters of the basic network model to make the generated code more accurate and robust.

[0046] The technical solution provided in this specification is implemented based on the above technical concept. Based on the description of the above technical concept, in the technical solution provided in this specification, by using the correctness verification result as a signal feedback in the training process of reinforcement learning, the correctness and robustness of the code generated by the code generation model can be better improved. Furthermore, the code ability of the code generation model can be improved.

[0047] For the convenience of the reader's understanding of this specification, the application scenarios of this specification are introduced below.

[0048] Taking the training method of the code generation model provided in this specification as an example, the technical solution provided in this specification is applicable to scenarios where it is necessary to train to obtain specific code corresponding to code problems. For example, the technical solution provided in this specification can be applied to scenarios such as education and training, automated software development assistants, open source community contribution support, and enterprise internal development process optimization.

[0049] Taking the above educational and training scenarios as an example:

[0050] Through the technical solution provided in this specification, corresponding code content can be generated and output according to the code problems for generating corresponding code proposed by teachers or students.

[0051] For example, a teacher or student can propose a code problem such as "Write a function to reverse a string". Through the technical solution provided in this specification, the code corresponding to this code problem can be generated and output. Such as the code for writing a function to reverse a string.

[0052] To avoid cumbersome statements, the principles of applying the technical solution of this specification to other scenarios will not be listed one by one in this specification.

[0053] Figure 1 It is a schematic diagram of the application scenario of the training method of the code generation model according to the embodiments of this specification (hereinafter simply referred to as the training method). Among them, the training method of this specification can be applied to, for example, Figure 1 the scenario 100 shown in the figure. As Figure 1 shown, the scenario 100 may include a target user 101, a client 102, a server 103, and a network 104.

[0054] The target user 101 may be a user who triggers the training of the code generation model. For example, the target user 101 may perform a target operation on the client 102 to trigger the training of the code generation model.

[0055] The client 102 may be an electronic device that provides an interaction function to the target user 101. For example, the client 102 may provide an interaction interface to the target user 101, and the target user 101 may perform interaction operations on the interaction page. In some embodiments, the client 102, in response to detecting an operation triggered by the target user 101 to train the code generation model, executes the training method described in this specification. At this time, the client 102 may store data or instructions for executing the training method described in this specification and may execute or be used to execute the data or instructions. In some embodiments, the client 102 may include a hardware device with data information processing functions and necessary programs for driving the hardware device to work to execute the training method described in this specification.

[0056] In some embodiments, the client 102 may include a mobile device, a tablet computer, a laptop computer, a built-in device of a motor vehicle, or the like, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home device may include a smart TV, a desktop computer, etc., or any combination. In some embodiments, the smart mobile device may include a smart phone, a personal digital assistant, a gaming device, a navigation device, etc., or any combination thereof. In some embodiments, the built-in device in a motor vehicle may include an in-vehicle computer, an in-vehicle TV, etc.

[0057] In some embodiments, the client 102 may be installed with one or more applications (APPs). The APP can provide the target user 101 with the ability and interface to interact with the outside world through the network 104. The APP includes but is not limited to: web browser APP programs, search APP programs, chat APP programs, shopping APP programs, video APP programs, financial management APP programs, instant messaging tools, email clients, social platform software, and so on.

[0058] As Figure 1 shown, the client 102 can communicate with the server 103. Among them, the server 103 can communicate with one client 102, or can communicate with multiple clients 102. In some embodiments, the client 102 can interact with the server 103 through the network 104 to receive or send messages, etc.

[0059] The server 103 can be a server that provides various services. For example, the server 103 can be a cloud server or a local server. The server 103 can communicate with one client 102 and receive the data sent by this client 102, or can communicate with multiple clients 102 and receive the data sent by each client 102 respectively.

[0060] In some embodiments, the training method described in this specification can be executed on the server 103. At this time, the server 103 can store the data or instructions for executing the training method described in this specification, and can execute or be used to execute the data or instructions. The server 103 can include a hardware device with data information processing functions and the necessary programs for driving the hardware device to work.

[0061] The network 104 is a medium for providing a communication connection between the client 102 and the server 103. The network 104 can facilitate the exchange of information or data. As Figure 1As shown, the client 102 and the server 103 can be respectively connected to the network 104, and transmit information or data to each other through the network 104.

[0062] In some embodiments, the network 104 can be any type of wired or wireless network, or a combination thereof. For example, the network 104 can include a cable network, a wired network, an optical fiber network, a telecommunication network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network TM, a short-range wireless network (ZigBee TM), a near field communication (NFC) network, or a similar network.

[0063] In some embodiments, the network 104 can include one or more network access points. For example, the network 104 can include a wired or wireless network access point, such as a base station or an Internet exchange point, through which one or more components of the client 102 and the server 103 can be connected to the network 104 to exchange data or information.

[0064] It should be noted that Figure 1 the numbers of the client 102, the server 103, and the network 104 in [[ ]] are merely illustrative. According to the implementation requirements, there can be any number of clients 102, servers 103, and networks 104. Moreover, the training method provided in this specification can be executed entirely on the client 102, entirely on the server 103, or partially on the client 102 and partially on the server 103.

[0065] That is to say, Figure 1 and the above description for [[ ]] is only used to exemplarily illustrate the application scenarios to which the training method of this specification may be applicable, and should not be construed as a limitation on the application scenarios. Figure 1

[0066] Figure 2 ​The hardware structure diagram of a training system 200 provided according to an embodiment of this specification is shown. The training system 200 can execute the training method described in this specification. The training method is introduced in other parts of this specification. When the training method is executed on the client 102, the training system 200 can be the client 102. When the training method is executed on the server 103, the training system 200 can be the server 103. When the training method is partially executed on the client 102 and partially executed on the server 103, the training system 200 can be a system including the client 102 and the server 103.

[0067] As Figure 2 shown, the training system 200 may include at least one storage medium 203 and at least one processor 202. In some embodiments, the training system 200 may further include a communication port 204 and an internal communication bus 201. The training system 200 may further include I / O components 205.

[0068] The internal communication bus 201 can connect different system components. For example, the internal communication bus 201 can connect the storage medium 203, the processor 202, the communication port 204, and the I / O components 205.

[0069] The I / O components 205 support input / output between the training system 200 and other components.

[0070] The communication port 204 is used for data communication between the training system 200 and the outside world. For example, the communication port 204 can be used for data communication between the training system 200 and the network 104. The communication port 204 can be a wired communication port or a wireless communication port.

[0071] The storage medium 203 may include a data storage device. The data storage device can be a non-transitory storage medium or a transitory storage medium. For example, the data storage device can include one or more of a magnetic disk 2031, a read-only storage medium (ROM) 2032, or a random access storage medium (RAM) 2033. The storage medium 203 further includes at least one instruction set stored in the data storage device. The instruction set includes computer program code, and the computer program code can include programs, routines, objects, components, data structures, processes, modules, etc. for executing the training method provided in this specification.

[0072] At least one processor 202 can be communicatively connected to at least one storage medium 203. The at least one processor 202 is configured to execute the above-mentioned at least one instruction set. When the training system 200 runs, the at least one processor 202 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the training method provided in this specification. The processor 202 can execute all steps included in the training method. The processor 202 can be in the form of one or more processors. In some embodiments, the processor 202 can include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field-programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of executing one or more functions, etc., or any combination thereof.

[0073] For illustrative purposes only, only one processor 202 is shown in the training system 200 in the drawings. However, it should be noted that the training system 200 in this specification can also include multiple processors. Therefore, the operations and / or method steps disclosed in this specification can be executed by one processor or jointly executed by multiple processors. For example, if it is described in this specification that the processor 202 of the training system 200 executes step A and step B, it should be understood that step A and step B can also be jointly or separately executed by two different processors 202 (for example, the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0074] Please refer to Figure 3 , Figure 3 is a schematic flowchart of the training method for the code generation model provided by an embodiment of this specification. Among them, Figure 3 The execution subject of the shown training method can be the training system. For the description of the training system, reference can be made to the above examples and will not be elaborated here.

[0075] As Figure 3 shown, the method includes the following S301 to S304:

[0076] S301: Obtain a first training data set, where the first training data set includes first data pairs, and the first data pairs include first code problem samples and first test cases corresponding to the first code problem samples.

[0077] Exemplarily, the first training data set can be understood as a data set used to train a code generation model. In this data set, there are first data pairs. The number of first data pairs can be multiple, which is not limited in this embodiment and can be determined by the training system based on requirements, historical records, experiments, etc.

[0078] Each first data pair can at least include two parts: a programming task or problem description (i.e., the first code problem sample), and the input and expected output related to the first code problem sample, which are used to verify the correctness of the first predicted code answer corresponding to the first code problem sample generated later (i.e., the first test case).

[0079] The acquisition of the first training data set can be implemented by the following examples:

[0080] In one example, the training system can be connected to a data acquisition device to receive the first training data set collected and sent by the data acquisition device.

[0081] In another example, the training system can provide a tool for loading data, and the user can use this tool for loading data to transfer the first training data set to the training system.

[0082] Among them, the tool for loading data can be an interface for connecting to an external device, such as an interface for connecting to other storage devices, and the first training data set transmitted by the external device can be obtained through this interface; the tool for loading data can also be a display device. For example, the training system can output an interface for the function of loading data on the display device, and the user can import the first training data set into the training system through this interface.

[0083] S302: Input the first code problem sample into a pre-trained basic network model to obtain a first predicted code answer corresponding to the first code problem sample.

[0084] The architecture of the basic network model is not limited in this embodiment. As combined with the above analysis, the basic network model can be a pre-trained large language model (such as GPT, Codex, etc.). The basic network model can generate candidate code answers according to the input natural language or code problem.

[0085] Correspondingly, in this step, the input of the basic network model is the first code problem sample, and the output is the predicted code answer (i.e., the first predicted code answer).

[0086] S303: Verify the correctness of the first predicted code answer based on the first test case to obtain a first verification result.

[0087] Exemplarily, the training system may verify the correctness of the first predicted code answer based on the first test case, and the correctness may include two dimensions: whether it can be executed, and if it can be executed, whether it can be correctly executed.

[0088] S304: Perform reinforcement learning training on the basic network model according to the first verification result to obtain a code generation model, where the code generation model is used to generate code corresponding to the code problem.

[0089] Correspondingly, after obtaining the first verification result, the training system may perform reinforcement learning training on the basic network model in combination with the first verification result to finally obtain a code generation model.

[0090] Combining the above analysis of S301 to S304, it can be seen that in this embodiment, the training system trains the basic network model by combining the first verification result and reinforcement learning, which can better improve the correctness and robustness of the code generation model in generating corresponding code. Furthermore, the code ability of the code generation model is improved.

[0091] Combining the above analysis, it can be known that the basic network model is a pre-trained network model with relatively basic code ability. The code generation model is a network model with relatively strong code ability obtained by training the basic network model based on the data set. To enable readers to more deeply understand the technical principle of the training method provided in this specification, the Figure 4 training method provided in this specification will be elaborated in more detail.

[0092] Figure 4 is a schematic flowchart of the training method of the code generation model provided in another embodiment of this specification. As Figure 4 shown, the method includes S401 to S408:

[0093] S401: Obtain a second training data set, where the second training data set includes second data pairs, and the second data pairs include second code problem samples, code answer samples corresponding to the second code problem samples, and second test cases.

[0094] It can be understood that, in order to avoid cumbersome statements, for the same or similar technical features in this embodiment and the above examples, reference can be made to the above examples, and this embodiment will not be elaborated again.

[0095] For example, for the understanding of the second training data set, reference can be made to the relevant description of the first training data set in the above example S301, and details will not be described here.

[0096] Specifically, for the understanding of the second code problem sample in this example, reference can be made to the description of the first code problem sample in S301 in the above example; for the understanding of the second test case in the example, reference can be made to the description of the first test case in S301 in the above example.

[0097] Relatively speaking, the second training dataset also includes code answer samples. The code answer samples can be understood as the specific code (i.e., the reference answer) corresponding to the second code problem sample.

[0098] In some embodiments, S401 may include the following steps 11 to 13:

[0099] Step 11: Collect code problems from sources such as public websites, open-source data, and code examination websites as the first code problem samples.

[0100] Exemplarily, the training system can obtain the first code problem samples in various ways.

[0101] Step 12: Use the first large language model to generate the first test cases and code answer samples based on the first code problem samples obtained in step 11, so as to obtain a second training dataset including the second data pair <first code problem sample, code answer sample, second test case>.

[0102] Among them, the first large language model includes but is not limited to open-source models such as deepseek and GPT.

[0103] In this embodiment, the training system determines the first test cases and code answer samples by using the first large language model, which can improve the efficiency of determining the first test cases and code answer samples. And it can reduce manual intervention to improve the intelligence and automation of determining the first test cases and code answer samples.

[0104] Step 13: Filter the second training dataset obtained in step 12 to obtain the filtered second training dataset.

[0105] That is to say, the second training dataset used to train the basic network model can be the filtered dataset.

[0106] In some embodiments, the filtering process may include at least one of quality filtering, incomplete data filtering, sensitive data filtering, and duplicate generation filtering.

[0107] Exemplarily, for the first code problem samples, code answer samples, and second test cases obtained in step 12, the training system can perform quality filtering on this part of the data to filter out the data with relatively low quality. The training system can also perform incomplete data filtering on this part of the data to filter out the incomplete data. The training system can also perform sensitive data filtering on this part of the data to filter out sensitive data such as privacy and security involved therein. The training system can also perform duplicate generation filtering on this part of the data to filter out the redundant data.

[0108] Relatively speaking, through the filtering process, the training system can make the data remaining after the filtering process have high effectiveness, accuracy, and reliability. Thus, when training a basic network model based on the filtered data, the training efficiency and reliability can be improved. Furthermore, the basic network model can have high basic code capabilities. That is, the accuracy and reliability of the basic network model are improved.

[0109] S402: Verify the correctness of the code answer samples based on the second test cases.

[0110] Exemplarily, the code answer samples and the second test cases can be the data respectively after the above-mentioned filtering process.

[0111] The training system can use an executable code validator to determine the correctness of the code answer samples based on the second test cases, that is, verify the correctness of the code answer samples. To determine whether the code answer samples are executable codes, and in the case that they are executable codes, whether they are correctly executable codes.

[0112] The executable code validator can be understood as a validator for judging codes to determine whether the codes are executable and whether they are correctly executable when executable.

[0113] Correspondingly, in this example, the executable code validator can be understood as a validator for judging the code answer samples based on the second test cases to determine whether the code sample answers are executable and whether they are correctly executable when executable. That is, verify the correctness and robustness of the code answer samples based on the second test cases.

[0114] S403: Train a basic network model according to the dataset corresponding to the code answer samples that have passed the correctness verification.

[0115] Exemplarily, if the code answer samples are executable codes and are correctly executable codes, they are determined as the code answer samples that have passed the correctness verification. On the contrary, if the code answer samples are non-executable codes, or are executable but cannot be correctly executed, they are determined as the code answer samples that have not passed the correctness verification.

[0116] In this embodiment, it is mainly based on obtaining a sample of code answers that have passed the correctness verification, and training a basic network model based on the second data corresponding to this part of the code answer sample.

[0117] That is to say, the training system can obtain a dataset corresponding to the sample of code answers that are executable and can be correctly executed from the second training dataset through an executable code validator (including the sample of code answers that have passed the correctness verification, the corresponding second code question sample, and the second test case), and train a basic network model to filter out non-executable code and incorrectly executed code, thereby improving the accuracy and reliability of training.

[0118] S404: Obtain a first training dataset, where the first training dataset includes first data pairs, and the first data pairs include first code question samples and first test cases corresponding to the first code question samples.

[0119] Similarly, for the implementation principle of S404, reference can be made to the description of S301 in the above example, which will not be elaborated here.

[0120] In some embodiments, S404 may include the following steps 21 and 22:

[0121] Step 21: The training system can search for publicly available code competition questions and corresponding test cases to obtain first code question samples and first test cases.

[0122] In some embodiments, for the obtained code competition questions, the training system can first clean and filter them to obtain corresponding first code question samples.

[0123] This embodiment does not limit the method of cleaning and filtering. For example, reference can be made to the filtering process in the above example.

[0124] Step 22: The training system can also generate more first test cases based on a second large language model to construct a first training dataset.

[0125] Exemplarily, the training system can collect first code question samples through online or offline means. During this process, the training system may also obtain corresponding first test cases.

[0126] In order to increase the data volume and richness, the training system can, in addition to the first test cases obtained by the search method, also obtain first test cases by using the method of generating with a large language model to improve the effectiveness and reliability of subsequent training.

[0127] In some embodiments, when the number of collected test cases is less than a preset number threshold, the training system may generate test cases corresponding to the first code problem sample based on the second large language model.

[0128] Combined with the above example, when the training system collects the first code problem sample, it may also collect the first test case corresponding to the first code problem sample (i.e., the collected test case).

[0129] However, the number of collected test cases may be large or small. To further improve the comprehensiveness and effectiveness of training without consuming too many training resources, the training system can determine whether more test cases are needed based on the preset number threshold. So that in the case of no need, training can be performed based on the collected test cases; in the case of need, the corresponding number of test cases can be generated based on the second large language model.

[0130] Similarly, the preset number threshold can be determined by the training system based on requirements, historical records, and experimental methods, which are not limited in this embodiment.

[0131] S405: Input the first code problem sample into the pre-trained basic network model to obtain the first predicted code answer corresponding to the first code problem sample.

[0132] Exemplarily, the training method provided in this specification may include two stages. The first stage is the training stage of the basic network model described in S401 to S403, that is, the stage of training the basic network model. The second stage is the reinforcement learning training stage of obtaining the code generation model based on the basic network model described in S404 to S408.

[0133] Correspondingly, this step can be understood as that the training system can input the first code problem sample into the basic network model obtained through the first stage of training to obtain the first predicted code answer for the reinforcement learning in the second stage.

[0134] Similarly, for the implementation principle of S405, reference can be made to the description of S302 in the above example, which will not be elaborated here.

[0135] S406: Verify the correctness of the first predicted code answer based on the first test case to obtain the first verification result.

[0136] Similarly, for the implementation principle of S406, reference can be made to the description of S303 in the above example, which will not be elaborated here.

[0137] S407: Determine the reward information according to the first verification result.

[0138] S408: Adjust the code generation strategy of the basic network model according to the reward information to train a code generation model.

[0139] Exemplarily, the training system can be trained based on reinforcement learning. For the training result (i.e., the first predicted code answer), an executable code validator can be used to feedback the code executability, correctness, and robustness, and be used as a reward to feedback to the trained model.

[0140] For example, if the first verification result indicates that the generated first predicted code answer passes the first test case, a positive reward is given. Conversely, if the first verification result indicates that the generated first predicted code answer fails (such as output error or runtime exception), a negative reward is given.

[0141] Correspondingly, the training system takes maximizing the cumulative reward information as the optimization goal, so that the code generated by the basic network model better meets the requirements of the first test case. Thus, a code generation model is obtained.

[0142] Specifically, in the current reinforcement learning training, the training system can use the executable code validator to verify the correctness of the first predicted code answer obtained in the current iteration based on the first test case, and obtain the first verification result of the current iteration.

[0143] If the first verification result of the current iteration indicates that the first predicted code answer obtained in the current iteration is an unexecutable code, or although it is an executable code, it is an incorrectly executed code, a negative feedback signal is provided to the basic network model.

[0144] Correspondingly, based on this negative feedback signal, the basic network model adjusts the code generation strategy (such as model parameters, etc.) for generating the first predicted code answer in the current iteration, and enters the next reinforcement learning training. It continues until a satisfactory solution (such as a model that can relatively accurately generate the code corresponding to the code problem) is found or the predetermined learning cycle (such as the preset number of iterations) is reached and then it ends.

[0145] Conversely, if the first verification result of the current iteration indicates that the first predicted code answer obtained in the current iteration is executable and is correctly executed, a positive feedback signal is provided to the basic network model.

[0146] Correspondingly, based on this positive feedback signal, the basic network model adjusts the code generation strategy (such as model parameters, etc.) for generating the first predicted code answer in the current iteration, and enters the next reinforcement learning training. It continues until a satisfactory solution (such as a model that can relatively accurately generate the code corresponding to the code problem) is found or the predetermined learning cycle (such as the preset number of iterations) is reached and then it ends.

[0147] In this embodiment, the training system determines corresponding reward information by combining the first verification result of verifying the correctness of the first prediction code, and adjusts the code generation strategy, model parameters, etc. of the basic network model based on the reward information, so as to train and obtain a code generation model. This can enable the code generation model to generate more accurate and robust code. In addition, manual intervention can also be reduced. For example, by automatically generating code through the code generation model, the time and effort of developers to write code manually can be reduced. Moreover, it has relatively wide applicability. For example, it is applicable to a variety of programming tasks, from simple algorithm implementation to complex system development.

[0148] According to another aspect of this specification, this specification also provides a code generation method. What can execute the code generation method provided in this specification can be a code generation system.

[0149] Regarding the code generation system, reference can be made to the description of the training system in the above examples, which will not be elaborated here.

[0150] In some embodiments, the code generation system can be the training system or a system other than the training system. In the case where the code generation system is a system other than the training system, the training system can transmit the trained code generation model to the code generation system based on the communication link between it and the code generation system. So that the code generation system can implement the code generation method provided in this specification based on the code generation model.

[0151] Alternatively, the training system can externally provide a service interface for invoking the code generation model. When the code generation system has a need to generate code, it can call the service provided by the code generation model through this service interface to obtain the corresponding code.

[0152] In addition, for the application scenarios of the code generation method provided in this specification, reference can also be made to the description of the application scenarios of the training method in the above examples, which will not be elaborated here.

[0153] Please refer to Figure 5 , Figure 5 which is a schematic flowchart of the code generation method provided in the embodiments of this specification. As Figure 5 shown, this method includes the following S501 and S502:

[0154] S501: Obtain the code generation problem to be generated.

[0155] The code generation problem to be generated can be understood as a programming task or problem description for which the user needs to obtain the corresponding code.

[0156] Similarly, for the manner in which the code generation system obtains the code generation problem to be generated, reference can be made to the relevant description in S301 of the above examples, which will not be elaborated here.

[0157] S502: Input the code generation problem to be generated into a code generation model to obtain the code corresponding to the code generation problem to be generated, where the code generation model is obtained based on the training method described in any of the above embodiments.

[0158] Combined with the above example, if a code generation model is deployed in the code generation system, the code generation system can input the code generation problem to be generated into the code generation model deployed therein, and obtain and output the code corresponding to the code generation problem to be generated. If a code generation model is not deployed in the code generation system, the code generation system can call the service interface of the training system to call the service provided by the code generation model through this service interface, such as inputting the code generation problem to be generated into the code generation model, and obtaining and outputting the code corresponding to the code generation problem to be generated.

[0159] Based on the above analysis, it can be seen that the code generation model has high accuracy and reliability. Therefore, the code corresponding to the code generation problem to be generated generated and output based on the code generation model also has relatively high accuracy and reliability.

[0160] In this specification, the Large Language Model (LLM) can also be abbreviated as the large model. The large language model is a natural language processing model based on deep learning technology, and its parameter scale usually reaches billions to hundreds of billions or even higher, with powerful language understanding and generation capabilities. The large language model can adopt the Transformer architecture or its variants (such as GPT, BERT, etc.). This architecture uses the Attention Mechanism to achieve global modeling of sequence data, can efficiently handle long-distance dependencies, and thus performs well in natural language tasks. The large language model learns the statistical features and semantic correlations of language by pre-training on a large-scale corpus, enabling it to have excellent generalization capabilities. The core capabilities of the large language model include but are not limited to: understanding context semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage methods usually include two modes: direct inference and fine-tuning. In the direct inference mode, the user guides the large language model to generate specific outputs by designing prompts. The prompt can be a task description or instruction in text form, used to stimulate the semantic understanding and generation capabilities of the large language model. In the fine-tuning mode, the large language model is further trained on a small-scale dataset in a specific domain to optimize its performance on specific tasks. The powerful generalization capabilities and flexibility of the large language model make it an important tool in the field of artificial intelligence technology, providing an efficient and accurate solution for automated text generation and understanding.

[0161] In some embodiments, large language models can also have the ability to understand and generate data of other modalities (such as vision, audio, etc.). In this case, large language models can also be called Multimodal Large Language Models (MLLMs). By integrating various types of inputs and outputs such as text, images, and sounds, MLLMs provide a richer and more natural interaction experience. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation ability makes MLLMs have broad application prospects in multiple fields.

[0162] It should be noted that the key technologies of large language models can be found in the detailed description in the paper "A Survey of Large Language Models" (paper number: arXiv:2303.18223v16, publication date: March 11, 2025, publication link: https: / / doi.org / 10.48550 / arXiv.2303.18223). This specification will not elaborate on it here.

[0163] It is worth noting that the above examples are only used to exemplarily illustrate the possible implementation methods of the training method and code generation method of this specification, and should not be understood as a limitation on the implementation methods of the training method and code generation method of this specification. Exemplarily, based on the above technical concepts, some of the above technical features can be combined to obtain new embodiments; new technical features can also be added based on the above examples to obtain new embodiments; some technical features can also be reduced based on the above examples to obtain new embodiments; some of the technical features in the above examples can be replaced with other technical features; some of the technical features and their order in the above examples can also be adjusted to obtain new embodiments, etc. There is no need to list them all here.

[0164] Based on the above technical concept, this specification also provides a computer-readable non-transitory storage medium. At least one instruction set is stored in the computer-readable non-transitory storage medium. When the at least one instruction set is executed by a processor, the steps of the training method and code generation method described in this specification are implemented.

[0165] In some possible embodiments, various aspects of this specification can also be implemented in the form of a program product, which includes program code. Taking the training method as an example, when the program product runs on the training system 200, the program code is used to cause the training system 200 to execute the steps of the training method described in this specification. The program product for implementing the above method can be a portable compact disc read-only memory (CD-ROM) that includes program code and can run on the training system 200. However, the program product of this specification is not limited to this. In this specification, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system. The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of the readable storage medium include: an electrical connection with one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or component. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. The program code for performing the operations of this specification can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the training system 200, partially on the training system 200, executed as an independent software package, partially on the training system 200 and partially on a remote training system, or entirely on the remote training system 200.

[0166] It can be understood that for the form of the program product regarding the code generation method, reference can be made to the examples of the form of the program product of the training method, and details will not be elaborated here.

[0167] It should be noted that in the technical solutions of this specification, the collection, storage, use, processing, transmission, provision, disclosure, and other processing of the relevant information of users (such as relevant codes or relevant code problems, etc.) all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0168] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require a specific order or a sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0169] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure may be presented only by way of example and may not be restrictive. Although not explicitly stated herein, those skilled in the art can understand that this specification is intended to encompass various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0170] In addition, certain terms in this specification have been used to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" mean that the specific features, structures, or characteristics described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it should be emphasized and understood that two or more references to "an embodiment" or "one embodiment" or "alternative embodiments" in various parts of this specification do not necessarily all refer to the same embodiment. Additionally, the specific features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0171] It should be understood that in the foregoing description of the embodiments of this specification, for the purpose of helping to understand a feature, for the purpose of simplifying this specification, this specification combines various features in a single embodiment, drawing, or its description. However, this does not mean that the combination of these features is necessary. Those skilled in the art may very well mark out some of the devices as separate embodiments when reading this specification. That is to say, the embodiments in this specification can also be understood as the integration of multiple sub - embodiments. And the content of each sub - embodiment is also valid when it has fewer features than all the features of a single foregoing disclosed embodiment.

[0172] Each patent, patent application, publication of patent application, and other materials cited herein, such as articles, books, specifications, publications, documents, references, etc. (excluding any historical prosecution files associated therewith), are hereby incorporated by reference for all purposes relevant hereto, e.g., in the specification and claims of this application. However, in the event of any inconsistency or conflict between the descriptions, definitions, and / or terms of the above materials and those used in this application, the descriptions, definitions, and / or terms used in this application shall prevail.

[0173] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Accordingly, the embodiments disclosed in this specification are presented by way of example only and not by way of limitation. Those skilled in the art may implement the application in this specification by taking alternative configurations based on the embodiments in this specification. Therefore, the embodiments of this specification are not limited to the embodiments precisely described in the application.

Claims

1. A training method for a code generation model, comprising: Obtaining a first training data set, wherein the first training data set includes first data pairs, and each first data pair includes a first code problem sample and a first test case corresponding to the first code problem sample; Inputting the first code problem sample into a pre-trained basic network model to obtain a first predicted code answer corresponding to the first code problem sample; Verifying the correctness of the first predicted code answer based on the first test case to obtain a first verification result; And Performing reinforcement learning training on the basic network model according to the first verification result to obtain a code generation model, wherein the code generation model is used to generate code corresponding to a code problem.

2. The method according to claim 1, wherein, The performing reinforcement learning training on the basic network model according to the first verification result to obtain a code generation model includes: Determining reward information according to the first verification result; and Adjusting the code generation strategy of the basic network model according to the reward information to train and obtain the code generation model.

3. The method according to claim 1, wherein The method further includes: Obtaining a second training data set, wherein the second training data set includes second data pairs, and each second data pair includes a second code problem sample, a code answer sample corresponding to the second code problem sample, and a second test case; Verifying the correctness of the code answer sample based on the second test case; and Training and obtaining the basic network model according to the data set corresponding to the code answer sample that passes the correctness verification.

4. The method according to claim 3, wherein, The code answer sample and the second test case are generated according to the second code problem sample by using a first large language model.

5. The method according to claim 3, wherein The second training data set is a filtered data set; Wherein, the filtering process includes at least one of quality filtering, incomplete data filtering, sensitive data filtering, and duplicate generation filtering.

6. The method according to any one of claims 1 to 5, wherein The first test case includes the collected test cases and / or the test cases generated based on a second large language model.

7. The method according to claim 6, wherein The method further includes: If the number of the collected test cases is less than a preset number threshold, generating a test case corresponding to the first code problem sample based on the second large language model.

8. A code generation method, comprising: Obtaining a code problem to be generated; Inputting the code problem to be generated into a code generation model to obtain code corresponding to the code problem to be generated, wherein the code generation model is obtained based on the training method according to any one of claims 1 to 7.

9. A training system for a code generation model, comprising: At least one storage medium storing at least one instruction set for training and obtaining a code generation model; At least one processor communicatively connected to the at least one storage medium, wherein when the at least one processor runs, it reads the at least one instruction set and executes the training method according to any one of claims 1 to 7 according to the indication of the at least one instruction set.

10. A code generation system, comprising: At least one storage medium storing at least one instruction set for generating code; At least one processor, communicatively coupled to the at least one storage medium, wherein, when the at least one processor runs, it reads the at least one instruction set and executes the code generation method according to the instruction of the at least one instruction set as claimed in claim 8.

Images