An interactive code generation method, system and storage medium

Abstract

The application provides an interactive code generation method, system and storage medium. The interactive code generation method comprises the following steps: based on an initial language model π θ sampling is performed to generate a candidate code x0; then, a natural language feedback result f of a user is obtained according to the candidate code x0, so as to generate a revised code x1 based on the natural language feedback result f; it is judged whether the revised code x1 meets a preset code requirement, so that when the revised code meets the preset code requirement, a target code R is obtained based on the revised code. By introducing the natural language feedback of the user, significant improvement can be achieved with a small amount of feedback, and the data preparation cost in the application is greatly reduced. Moreover, by introducing the feedback of the user in the use process, the model training becomes interactive.

Description

TECHNICAL FIELD

[0001] The application relates to a code generation method, in particular to an interactive code generation method, system and storage medium, and belongs to the technical field of code generation. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have shown great capabilities in many natural language processing (NLP) tasks. In particular, in the code generation task, models like BLOOM, GitHub Copilot, etc. have shown the potential to assist programmers in writing code more efficiently. However, there are still some problems in the training method of existing code generation models.

[0003] The training data of existing pre-training models is mostly from code libraries crawled from the network, which contains a large number of low-quality and security-vulnerable code samples. Moreover, the training of the model only relies on these static offline data, lacking correction feedback on the output results of the model. This leads to uneven quality of the code generated by the model, which may cause security risks when directly applied to actual projects. Improving the effective training method of the model and enhancing its ability to generate reliable high-quality code has become a very urgent problem to be solved. SUMMARY

[0004] Therefore, the application provides an interactive code generation method, system and storage medium to solve or alleviate the technical problems in the prior art, and at least provides a beneficial choice.

[0005] The technical scheme of the application embodiment is implemented as follows:

[0006] S100: based on an initial language model π θ performing a sampling to generate a candidate code x0.

[0007] S200: obtaining a natural language feedback result f of a user according to the candidate code x0, to generate a revised code x1 based on the natural language feedback result f.

[0008] And S300: judging whether the revised code x1 meets a preset code requirement, to obtain a target code R based on the revised code when it meets the preset code requirement.

[0009] Further preferably, the S100 further comprises: obtaining a task description data set D={(t,u)}; wherein the task description data set D at least includes a task description t and a corresponding test set u.

[0010] Further preferably, the S200 includes:

[0011] Based on the test set u, it is determined whether the candidate code x0 passes the test, so as to obtain the data set C = {(x0, t, u) | x0 ~ π according to the test judgment result θ (·|t),EVAL(x0,t)=0,(t,u)∈D}.

[0012] Obtain the natural language feedback result f based on the data set C, and obtain the sample set C according to the natural language feedback result f annotated ={(x0,f,t)|(x0,t,u)∈C}.

[0013] A model improvement distribution q is constructed according to the natural language feedback result f.

[0014] And, based on the sample set C annotated And the model improvement distribution q obtains the target distribution of the initial language model Where R(x1,t) is the quality scoring function of the modified code x1 corresponding to the task description t, R(x1,t)=EVAL(x1,t), and β is a symbolic constant parameter.

[0015] Further preferably, the S200 further includes:

[0016] According to the target distribution Initial language model π θ And the model improves the distribution q to obtain the sampling estimation loss function

[0017] Estimate the loss function L for the sample θ (t) Perform minimization processing to enhance the initial language model π N times θ ; Where N is initialized to 1.

[0018] Further preferably, the S200 further includes:

[0019] Based on the initial language model π after N times of enhancement θ Perform subsampling to construct a feedback distribution p based on the subsampling results F .

[0020] By modifying the model π ψ According to the feedback distribution p F Generate the corrected code x1.

[0021] Further preferably, the S300 further includes:

[0022] The initial language model π after N times of enhancement is trained based on the target code θ, the initial language model is enhanced for N+1 times θ .

[0023] Further preferably, the method further comprises:

[0024] When the modified code x1 does not meet the preset code requirement, an error code x0 is obtained based on the modified code x1.

[0025] And the modified model π ψ is trained based on the error code x0. ψ .

[0026] Further preferably, the method further comprises:

[0027] Steps S100 to S300 are repeated to obtain an enhanced language model based on the initial language model π θ enhanced for N+1 times.

[0028] Further preferably, the application also provides an interactive code generation system, which comprises:

[0029] a first generation module configured to perform one sampling based on an initial language model π θ to generate a candidate code x0.

[0030] a second generation module configured to obtain a natural language feedback result f of a user according to the candidate code x0, and generate a modified code x1 based on the natural language feedback result f.

[0031] And a third generation module configured to determine whether the modified code x1 meets a preset code requirement, and obtain a target code R based on the modified code x1 when the modified code meets the preset code requirement.

[0032] Further preferably, the application also provides a storage medium having a computer program stored therein, wherein the computer program is configured to execute the interactive code generation method when running.

[0033] The application has the following advantages due to the above technical solutions:

[0034] 1. The application can achieve significant improvement with a small amount of feedback by introducing user natural language feedback, thereby greatly reducing the data preparation cost in the application.

[0035] 2. The application makes the model training interactive by introducing the feedback of the user in the use process.

[0036] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a flow chart of the interactive code generation method described in this application;

[0039] Figure 2 To adopt Figure 1 System framework diagram of the interactive code generation method. DETAILED DESCRIPTION

[0040] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0041] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] like Figure 1 As shown, the embodiment of the present application provides an interactive code generation method, comprising the following steps:

[0043] S100: Perform a sampling based on the initial language model to generate candidate codes.

[0044] S200: Obtaining a user's natural language feedback result according to the candidate code, so as to generate a revised code based on the natural language feedback result.

[0045] And, S300: judging whether the correction code meets the preset code requirement, and obtaining the target code based on the correction code if it meets the requirement.

[0046] In this embodiment, specifically: S100 further includes: obtaining a task description data set; wherein the task description data set at least includes a task description and a corresponding test set.

[0047] In a feasible implementation, the initial language model π is inputθ , encoding task description dataset D = {(t, u)}, where:

[0048] π θ : a pre-trained language model parameterized by θ, defined as π θ : V * → [0, 1], mapping token sequences on vocabulary V to the interval [0, 1].

[0049] t: task description, such as "write a function to calculate prime numbers".

[0050] u: test set UNITTESTS(t) corresponding to task t.

[0051] Further, the initial language model π θ is sampled to generate candidate code.

[0052] Specifically:

[0053] For task description t ~ p(t), candidate code x0 is generated by sampling from π θ (·|t). θ .

[0054] In this embodiment, specifically, the S200 includes:

[0055] Based on the test set, it is determined whether the candidate code passes the test, so as to obtain the dataset according to the test determination result.

[0056] In one possible implementation, it is checked whether the candidate code x0 passes the test set u: EVAL(x0, t) is calculated, where EVAL(x, t) is defined as:

[0057]

[0058] Further, those codes that do not pass the test are filtered to obtain the dataset

[0059] C = {(x0, t, u) | x0 ~ π θ (·|t), EVAL(x0, t) = 0, (t, u) ∈ D}

[0060] Where the low-quality code sample C obtained from the initial language model prepares for the next user feedback.

[0061] Based on the dataset, the natural language feedback result is obtained, and the sample set is obtained according to the natural language feedback result.

[0062] In one possible implementation, for each code sample (x0, t, u) in C, the user is asked to examine the code x0 and give a natural language feedback result f describing the code error and how to fix it. The resulting set of samples with feedback is C

[0063] C annotated = {(x0, f, t) | (x0, t, u) ∈ C}

[0064] Further, the natural language feedback result f comes from a feedback distribution p F (f | t, x0, EVAL(x0, t) = 0), which represents the feedback distribution for an incorrect code x0.

[0065] A model improvement distribution is constructed according to the natural language feedback result.

[0066] In one possible implementation, a model improvement distribution q that improves the code quality is constructed using the natural language feedback result f provided by the user:

[0067] where q is defined as a distribution over all possible fixed codes x1, q guides how to fix the initial code x0 using the natural language feedback result f.

[0068] The specific form of q is as follows:

[0069]

[0070] where δ0 and δ1 are Dirac delta distributions centered at 0 and 1, π ψ is a fixing model that generates fixed code x1 according to the natural language feedback result f.

[0071] In addition, a target distribution of the initial language model is obtained based on the sample set and the model improvement distribution.

[0072] In one possible implementation, the sample set C annotated with user feedback is obtained, and the model improvement distribution q of the improvement model for generating fixed code defines the target distribution of the initial language model π θ The form of the target distribution is as follows:

[0073]

[0074] where R(x1, t) is a quality scoring function of code x1 for task t, and here it is set to the unit test result:

[0075] R(x1, t) = EVAL(x1, t)

[0076] where β is a symbolic constant parameter.

[0077] ​In the embodiment, specifically, the S200 further includes:

[0078] According to the target distribution, the initial language model and the model improvement distribution, a sampling estimation loss function is obtained.

[0079] And, the sampling estimation loss function is minimized to enhance the initial language model for N times, wherein N is initialized as 1.

[0080] Wherein, N is initialized as 1, which means that when the minimization processing is performed for the first time, it is one enhancement, if the initial language model is continuously enhanced subsequently, it is two enhancements, and so on to express the number of enhancements.

[0081] In a feasible implementation, π θ By minimizing the following KL divergence approximation

[0082]

[0083] The target cross-entropy loss function is obtained by expansion:

[0084] L(θ)=-E t～p(t) [L θ (t)]

[0085] Wherein,

[0086]

[0087] Further, the importance sampling estimation loss function is:

[0088]

[0089] Wherein, q is the model improvement distribution constructed by using the feedback.

[0090] Further, by minimizing the sampling estimation loss function L(θ), the π θ approximation fuses the feedback knowledge So as to enhance the code generation quality.

[0091] In the embodiment, specifically, the S200 further includes:

[0092] Based on the initial language model enhanced for N times, secondary sampling is performed, and a feedback distribution is constructed based on the secondary sampling result.

[0093] And, the correction model generates a corrected code according to the feedback distribution.

[0094] In a feasible implementation, from π θSample initial code x0, keep those EVAL(x0, t) = 0, ask the user to give natural language feedback results f for x0, build feedback distribution p F， Use model π ψ Generate revised code x1 based on natural language feedback results f, approximate sampling distribution π ψ , only keep x1 that passes the test, that is, EVAL(x1, t) = 1.

[0095] It should be noted that π θ in this step has been enhanced initial language model.

[0096] In this embodiment, specifically, the S300 further includes:

[0097] Based on the target code, train the N times enhanced initial language model to enhance the initial language model by N+1 times.

[0098] In a feasible implementation, use π ψ According to the natural language feedback results f, generate the revised code set R (that is, the target code), fine-tune the initial model π θ on R to obtain the enhanced

[0099] Wherein, the optimization algorithm used in fine-tuning:

[0100] Input: initial model π θ , revised code set R = {(t, x1)}.

[0101] For each sample (t, x1), input task description t and target code x1.

[0102] Calculate cross-entropy loss: loss = -log π θ (x1|t).

[0103] Minimize the loss by gradient descent algorithm to update the parameters θ of π θ .

[0104] Get fine-tuned

[0105] In this embodiment, specifically, it further includes:

[0106] When the revised code does not meet the preset code requirements, obtain error code based on the revised code.

[0107] And train the revised model through the error code to enhance the revised model.

[0108] In an implementation, the training correction model π ψ , comprises:

[0109] Input: error code x0, natural language feedback result f.

[0110] Target: corresponding artificial correction code x1.

[0111] Optimization goal: maximize π ψ The likelihood of generating x1.

[0112] In this embodiment, specifically: further comprising:

[0113] Repeat S100 to S300 to obtain an enhanced language model based on the initial language model after N+1 times of enhancement.

[0114] In an implementation, the above process is repeated to continuously improve and obtain an enhanced language model.

[0115] Wherein, Learn how to correct common errors of π θ , Integrate the knowledge of user feedback.

[0116] Further, the performance on the unit test can be evaluated :

[0117] For a task description t on the test set, sample Generate code; check if the code passes the corresponding test case; and calculate evaluation indicators such as pass@k.

[0118] It should be noted that the above embodiment uses natural language feedback to interact and train, and the natural language feedback is expressive and has a large amount of information, and it is easier to point out the root cause of the problem; using such feedback to treat the symptoms according to the current output of the model makes the learning more targeted, and the feedback is also easier to collect and accumulate. Other feedback forms such as scoring are difficult to express rich information and are not direct enough to guide model improvement. Natural language feedback makes the interactive training method of the present application more efficient.

[0119] Further, the embodiments of the present application train a model that can fully utilize feedback to correct code first, and then enhance the original model with the generated code. This two-stage training method fully utilizes the guidance of natural language feedback and avoids the problem of poor stability when directly training. The two-stage training works together to make feedback knowledge better accumulated into the model, which is the key to the significant effect of the present application.

[0120] Further preferably: as Figure 2As shown, the application further provides an interactive code generation system, which comprises:

[0121] a first generation module, configured to perform a sampling based on an initial language model π θ to generate a candidate code x0.

[0122] a second generation module, configured to acquire a natural language feedback result f of a user according to the candidate code x0, to generate a revised code x1 based on the natural language feedback result f.

[0123] and a third generation module, configured to judge whether the revised code x1 meets a preset code requirement, to acquire a target code R based on the revised code when it meets the preset code requirement.

[0124] Further preferably, the application further provides a storage medium, wherein a computer program is stored in the storage medium, and the computer program is configured to execute the interactive code generation method when running.

[0125] The above merely describes a specific implementation of the application, but the protection scope of the application is not limited to this. Any person skilled in the art can easily think of various changes or replacements within the technical range disclosed in the application, and these should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. An interactive code generation method, characterized in that: The following steps are involved: S100: Based on the initial language model π θ Perform a sampling to generate a candidate code x0; S2 00: Obtain the user's natural language feedback result f according to the candidate code x0, and generate a revised code x1 based on the natural language feedback result f; And, S300: judging whether the correction code x1 meets the preset code requirement, and if so, obtaining the target code R based on the correction code; The S100 further includes: obtaining a task description dataset D={(t,u)}; wherein the task description dataset D at least includes a task description t and a corresponding test set u; The S200 includes: Based on the test set u, it is determined whether the candidate code x0 passes the test, so as to obtain the data set C = {(x0, t, u) | x0 ~ π according to the test judgment result θ (·|t),EVAL(x0,t)=0,(t,u)∈D}; Obtain the natural language feedback result f based on the data set C, and obtain the sample set C according to the natural language feedback result f annotated ={(x0,f,t)|(x0,t,u)∈C}; Constructing a model improvement distribution q according to the natural language feedback result f; And, based on the sample set C annotated And the model improvement distribution q obtains the target distribution of the initial language model Where R(x1,t) is the quality scoring function of the modified code x1 corresponding to the task description t, R(x1,t)=EVAL(x1,t), and β is a symbolic constant parameter; The S200 further includes: According to the target distribution Initial language model π θ And the model improves the distribution q to obtain the sampling estimation loss function Estimate the loss function L for the sample θ (t) Perform minimization processing to enhance the initial language model π N times θ ; Where N is initialized to 1; The S200 further includes: Based on the initial language model π after N times of enhancement θ Perform subsampling to construct a feedback distribution p based on the subsampling results F ; By modifying the model π ψ According to the feedback distribution p F Generate correction code x1; Also includes: When the correction code x1 does not meet the preset code requirement, obtaining an error code x0 based on the correction code x1; And, the correction model π is trained by the error code x0 ψ , to enhance the modified model π ψ .

2. The interactive code generation method according to claim 1, characterized in that The S300 further includes: The initial language model π after N times of enhancement is trained based on the target code θ , enhance the initial language model π by N+1 times θ .

3. The interactive code generation method according to claim 2, characterized in that Also includes: Repeat S100 to S300 to obtain the initial language model π after N+1 enhancements. θ Get the enhanced language model 4. A system using the interactive code generation method according to any one of claims 1 to 3, characterized in that: The system comprises: The first generation module is used to generate the language based on the initial language model π θ Perform a sampling to generate a candidate code x0; A second generating module is configured to obtain a user's natural language feedback result f according to the candidate code x0, so as to generate a corrected code x1 based on the natural language feedback result f; and a third generating module for determining whether the correction code x1 meets the preset code requirements, so as to obtain the target code R based on the correction code if it meets the requirements; The first generation module further includes: obtaining a task description dataset D={(t,u)}; wherein the task description dataset D at least includes a task description t and a corresponding test set u; The second generation module includes: Based on the test set u, it is determined whether the candidate code x0 passes the test, so as to obtain the data set C = {(x0, t, u) | x0 ~ π according to the test judgment result θ (·|t),EVAL(x0,t)=0,(t,u)∈D}; Obtain the natural language feedback result f based on the data set C, and obtain the sample set C according to the natural language feedback result f annotated ={(x0,f,t)|(x0,t,u)∈C}; Constructing a model improvement distribution q according to the natural language feedback result f; And, based on the sample set C annotated And the model improvement distribution q obtains the target distribution of the initial language model Where R(x1,t) is the quality scoring function of the modified code x1 corresponding to the task description t, R(x1,t)=EVAL(x1,t), and β is a symbolic constant parameter; The second generation module further includes: According to the target distribution Initial language model π θ And the model improves the distribution q to obtain the sampling estimation loss function Estimate the loss function L for the sample θ (t) Perform minimization processing to enhance the initial language model π N times θ ; Where N is initialized to 1; The second generation module further includes: Based on the initial language model π after N times of enhancement θ Perform subsampling to construct a feedback distribution p based on the subsampling results F ; By modifying the model π ψ According to the feedback distribution p F Generate correction code x1; Also includes: When the correction code x1 does not meet the preset code requirement, obtaining an error code x0 based on the correction code x1; And, the correction model π is trained by the error code x0 ψ , to enhance the modified model π ψ .

5. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the interactive code generation method according to any one of claims 1 to 3 when running.

Images