Path-oriented diffusion type code pre-training method and system

Abstract

The invention discloses a path-oriented diffusion type code pre-training method and system, and the method comprises the steps: 1) carrying out the diffusion of existing code training data through a preset path diffusion mode, and obtaining a diffused code data set; wherein the method for diffusing the code training data x by utilizing the path diffusion mode comprises the following steps of: shielding or damaging the code training data x to different degrees to obtain n diffusion samples which are associated with the code training data to serve as a diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to the code training data form the diffused code data set; 2) training a code pre-training model by using the diffused code data set; according to the method, the use of a single code is enhanced, and the association among a plurality of training data samples is expanded, so that the trained model can process downstream tasks more effectively.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a path-oriented diffusion code pre-training method and system. Background Art

[0002] Code pre-training models play a vital role in modern software development. Through pre-training on large-scale code corpora, these models can automatically understand and generate code, thereby significantly improving development efficiency. As the complexity of software development continues to increase, the importance of code pre-training models has become increasingly prominent. However, since the parameter scale of pre-training models can be expanded indefinitely, the available data in reality may quickly reach its upper limit, so how to make full use of every available high-quality code data is a crucial issue.

[0003] Existing code pre-training technology mainly uses data in two ways: 1) predicting the obscured content in the data; 2) restoring the original data from the damaged data. The first method first sets a certain probability (such as 15%) to obscure the content in the original input, and then predicts the content or category of the obscured position through a deep learning model, such as predicting the value corresponding to an obscured token or a continuous token, or predicting whether the obscured token belongs to the identifier of the code. The pre-training model predicts the obscured content inside the data by learning the code context, thereby perceiving the composition and specific content of the code data. The second method destroys the content in the original input data with a certain probability. Unlike simply obscuring the data content, the way to destroy the original data content may include inserting additional tokens or deleting existing tokens. After introducing a series of destruction operations to add noise to the data, the pre-training model needs to restore the original input data content from the destroyed content in order to learn the structure and organization that should be in the correct data.

[0004] The shortcomings of the existing technology mainly include two aspects: 1) Insufficient data utilization. At present, whether predicting the occluded content in the data or restoring the original data from the damaged data, only a single data sample is processed once in a training round. However, there is a large randomness in processing the data only once, which is equivalent to only using most of the retained content of the data to predict a small part of the missing or damaged content, and the data utilization efficiency only depends on a small part of the content that needs to be predicted. Therefore, this method lacks sufficient learning of a single data, resulting in the need for a large amount of data for model learning. At the same time, this also leads to the situation that the existing data volume will soon be unable to support the training of larger-scale parameter models when the model parameters continue to expand. 2) Weak correlation between data. The existing code pre-training tasks mainly focus on capturing the internal structure and characteristics of a single code training sample, without considering the possible correlation between multiple training samples. The lack of utilization of possible correlations between pre-training data often leads to low learning efficiency.

[0005] In summary, on the issue of code pre-training, existing methods mainly guide the pre-training process based on two specific methods: predicting the occluded content in the data or recovering the original data from the damaged data. However, these two methods still face the challenges of insufficient data utilization and weak correlation between data. These challenges have led to the current need for a large amount of data for code pre-training, which may soon lead to the situation where the existing data scale is difficult to support the training of models with larger parameter scales. Therefore, in an era when large-scale parameter pre-training models are booming, how to efficiently use existing data to improve the performance of pre-training models has become a crucial issue. Summary of the invention

[0006] In view of the above problems, the present invention provides a path-oriented diffusion code pre-training method and system.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A path-oriented diffusion code pre-training method, the steps of which include:

[0009] 1) Diffusion of the existing code training data by a preset path diffusion method to obtain a diffused code data set; wherein, the method of diffusing a code training data x by using the path diffusion method is: blocking or destroying the code training data x to different degrees to obtain n diffusion samples (x1, x2, x3, ..., x n ) as the diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code data set;

[0010] 2) Using the diffused code data set to train the code pre-training model; wherein, for the diffused sample set corresponding to the code training data x in the diffused code data set, using a plurality of diffused samples x generated by blocking the code training data x to different degrees mask Train the code pre-trained model and calculate the corresponding loss value

[0011] Multiple diffusion samples x generated by damaging the code training data x to different degrees noise Train the code pre-trained model and calculate the corresponding loss value Then, by minimizing the total loss function value Optimize the code pre-trained model.

[0012] Furthermore, the path diffusion method is linear path diffusion, and the linear diffusion path refers to the continuous occlusion or destruction of the code training data x to different degrees during the diffusion process, and the occlusion or destruction degree between the generated diffusion samples follows a linear relationship, that is, the degree of occlusion or destruction gradually decreases or increases.

[0013] Furthermore, the path diffusion method is a tree-shaped path diffusion composed of multiple linear path diffusions, and the tree-shaped path diffusion refers to blocking or destroying the code training data x, and the blocking degree or destruction degree between the generated diffusion samples has a tree-shaped correlation relationship.

[0014] Furthermore, the path diffusion method is a graphic path diffusion composed of multiple linear path diffusions, and the graphic path diffusion refers to blocking or destroying the code training data x, and the blocking degree or destruction degree between the generated diffusion samples has a ring structure.

[0015] Furthermore, the code pre-training model is an Encoder-Decoder framework based on Transformer.

[0016] A code generation method, characterized in that a code requirement description or a code to be edited is input into a code pre-training model trained by the above method to generate a required target program.

[0017] A path-oriented diffusion code pre-training system, characterized by comprising a code training data diffusion module and a model training module; wherein:

[0018] The code training data diffusion module is used to diffuse the existing code training data by a preset path diffusion method to obtain a diffused code data set; wherein the method of diffusing a code training data x by using the path diffusion method is: the code training data x is blocked or destroyed to different degrees to obtain n diffusion samples (x1, x2, x3, ..., x n ) as the diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code data set;

[0019] The model training module is used to train the code pre-training model using the diffused code data set; wherein, for the diffused sample set corresponding to the code training data x in the diffused code data set, a plurality of diffused samples x generated by masking the code training data x to different degrees are used. mask Train the code pre-trained model and calculate the corresponding loss value Multiple diffusion samples x generated by damaging the code training data x to different degrees noise Train the code pre-trained model and calculate the corresponding loss value Then, by minimizing the total loss function value Optimize the code pre-trained model.

[0020] A code generation system, characterized by comprising a code training data diffusion module, a model training module and a code generation module; wherein,

[0021] The code training data diffusion module is used to diffuse the existing code training data by a preset path diffusion method to obtain a diffused code data set; wherein the method of diffusing a code training data x by using the path diffusion method is: the code training data x is blocked or destroyed to different degrees to obtain n diffusion samples (x1, x2, x3, ..., x n ) as the diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code data set;

[0022] The model training module is used to train the code pre-training model using the diffused code data set; wherein, for the diffused sample set corresponding to the code training data x in the diffused code data set, a plurality of diffused samples x generated by masking the code training data x to different degrees are used. mask Train the code pre-trained model and calculate the corresponding loss value Multiple diffusion samples x generated by damaging the code training data x to different degrees noise Train the code pre-trained model and calculate the corresponding loss value Then, by minimizing the total loss function value Optimizing the code pre-training model;

[0023] The code generation module is used to input the code requirement description or the code to be edited into the trained and optimized code pre-training model to generate the required target program.

[0024] A server, characterized in that it comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the above method.

[0025] A computer-readable storage medium stores a computer program, wherein the computer program implements the above method when executed by a processor.

[0026] The beneficial effects of the present invention are as follows:

[0027] During the design phase, the present invention comprehensively rethought the code pre-training problem and obtained a new perspective to improve the efficiency of code pre-training data utilization. Using this new perspective, the present invention enhances the use of a single code and expands the association between multiple training data samples. This new perspective also brings new challenges to the code pre-training model. Specifically, the new perspective requires us to reasonably expand each piece of data and associate the expanded data. At the same time, it also requires the model to accurately generate the target code content given different contexts, so that downstream tasks can be processed more efficiently. In order to address the new challenges, the present invention designs a path-oriented diffusion code pre-training method and system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is the basic process of the path-oriented diffusion code pre-training method and system. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific implementation cases and drawings.

[0030] The present invention implements a path-oriented diffusion code pre-training method and system, which means: at the input end, the user diffuses the existing code training data through a pre-set path; at the output end, the model uses the diffused code data set to pre-train the code pre-training model. The trained code pre-training model receives the input code requirement description or the code to be edited, and then automatically synthesizes the required target program to meet the user's requirements or solve the user's modification intention.

[0031] Specifically, based on the importance of making full use of data and code pre-training models, the present invention designs a system to automatically solve the problems related to code data and models. The input and output of this system are described as follows:

[0032] On the input side, in the data diffusion stage, the user inputs the original data, and the system diffuses the data according to the set path; on the output side, after the code pre-training model training is completed and enters the application stage, the user provides the code requirement description or the code that needs to be edited. When the code requirement description is entered, the model will automatically generate code that meets the requirement description; when the code that needs to be edited is entered, the model will automatically edit the existing code and improve the code quality.

[0033] At the output end, the system of the present invention will train the model based on the diffused large-scale corpus, and automatically generate target code that meets the requirements based on the knowledge learned by the model.

[0034] In order to improve the existing pre-training technology, the present invention: 1. First, a path-oriented diffusion strategy is designed to process pre-training data; 2. Then, a pre-training technology integrating multiple diffusion strategies is used to train the code-related deep learning model; 3. Finally, the generation results of the path-oriented diffusion code pre-training system are evaluated. The following is a detailed introduction to the performance:

[0035] 1. Path-oriented diffusion strategy

[0036] In order to better utilize the existing data in the pre-training process, the present invention adopts Figure 1 The basic process shown in applies a path-oriented flooding strategy to the data.

[0037] The core idea of ​​the path-oriented diffusion strategy is to achieve the purpose of data diffusion by applying continuous occlusion or destruction operations to the same data sample with different probabilities, and to learn the association between training samples by using different paths and internal association information of the paths when a single data sample is diffused. In the present invention, operations such as occlusion or destruction on data are collectively referred to as data processing methods, and the diffusion path is defined as an ordered set from one data point to another data point after processing the same sample to different degrees. Specific path diffusion methods include linear path diffusion, tree path diffusion, graphic path diffusion and other path diffusion.

[0038] according to Figure 1 As shown in Figure 2, a linear diffusion path refers to a linear relationship between a data point and another data point during the diffusion process, such as the probability of being blocked gradually decreasing or increasing. Figure 1 The left side shows that the proportion of noise and occlusion gradually decreases, and the latter diffusion sample is the result of processing based on the previous sample. Similarly, tree path diffusion refers to a tree-like relationship between one data point and other data points, such as Figure 1 The middle position is a case of diffusion in a binary tree manner. Graph path diffusion refers to the existence of a ring structure on a data diffusion path, such as Figure 1 The right side shows the ring structure caused by the replacement of processing positions and the addition of additional processing. In addition, the diffusion methods that follow a certain path association other than linear, tree and graph are collectively referred to as other path diffusion methods.

[0039] For each input data x, when constructing a diffusion path, we first select a higher initial probability P to block or destroy the original input to obtain x1, and then gradually reduce the probability of blocking or destroying the original data based on x1 (similarly, if the initial probability is low, the diffusion path can also gradually increase the probability of blocking or destroying the original data), so as to obtain a series of diffusion samples x1, x2, x3, ..., x corresponding to the data x n . where x1,x2,x3,…,x n There is the following internal relationship: x2 is the probability of being blocked or damaged reduced on the basis of x1 (that is, the blocked or damaged parts contained in x2 are a subset of the blocked or damaged parts contained in x1), x3 is the probability of being blocked or damaged reduced on the basis of x2 (that is, the blocked or damaged parts contained in x3 are a subset of the blocked or damaged parts contained in x2), and so on (that is, x i The area that is blocked or destroyed is x i-1 The subset of the occluded or damaged areas is included), and the diffusion samples x1,x2,x3,…,x nConstitute a diffusion path. If all diffusion samples constitute a diffusion path, it is called linear path diffusion; similarly, the diffusion sample can also contain multiple diffusion paths, forming a variety of path diffusion methods such as tree path diffusion and graphic path diffusion. Assuming that the original K% content is restored on the basis of the initial probability P each time during the data diffusion process, the proportion of occluded content in the first diffused sample x1 is (PK)%. Similarly, the proportion of occluded content in the second diffused sample is (P-2K)%, and the proportion of occluded content in the nth diffused sample is (Pn*K)%. For the convenience of display, the present invention sets the processing ratio of each reduction to K, but the processing ratio of each reduction or increase can also be flexible, and it is not necessary to follow the same reduction or increase ratio each time. In the end, the original 1 data sample is diffused into n additional samples, that is, in one round of training, the model will be trained on n+1 variants of the data sample containing different processing ratios.

[0040] In the pre-training task of predicting the occluded content in the data, the existing pre-training task goal is to only occlude a small part of the data in a single data, such as occluding 15% of the code content. However, the path-oriented diffusion strategy can choose a higher occlusion probability, such as occluding 60% of the code content. Then gradually reduce the occlusion probability on this basis. For example, each time the occlusion probability is reduced by 15%, three additional data samples will be obtained, namely, data samples with 45%, 30% and 15% of the code content respectively occluded. Based on this, each data sample is used 4 times instead of 1 time in a training round. The path association between these 4 data samples is reflected in the data sample with 30% of the content being occluded. 15% of the content is restored on the basis of the data sample with 45% of the content being occluded, rather than randomly occluding 30% of the content. The specific initial occlusion probability and the probability of each reduction can be adjusted according to different needs and situations.

[0041] In the pre-training task of recovering the original data from the damaged data, the existing pre-training usually adds noise to the code data through operations such as insertion, deletion, and occlusion, and then recovers the original data content from the data containing a certain probability (such as 20%) of noise. However, this pre-training task also only adds noise once to a single data sample, which leads to the fact that the efficiency of data utilization depends largely on the location where the noise is randomly added. In the path-oriented diffusion strategy, the initial state of the data may be that a large degree of noise is added, for example, the proportion of noise in the data reaches 60%. Similarly, by gradually reducing the noise proportion on this basis, the path-oriented diffusion strategy can expand multiple different data variants. By using different degrees of noise data to recover the original data content, the data utilization rate can be effectively improved.

[0042] In addition to the two common pre-training tasks mentioned above, the path-oriented diffusion strategy can also be adapted according to the characteristics of different pre-training tasks. Different pre-training tasks only provide a way to process the original data under a certain probability, while the path-oriented diffusion strategy first increases (or decreases) the processing ratio of the original data, and then enhances the use of data by diffusing this processing ratio. It is worth noting that the different diffused samples have specific path correlations. By learning multiple diffused data samples and their path information, the model can make full use of the data and improve the final performance on downstream tasks.

[0043] 2. Code Pre-training Model

[0044] The code pre-training model in the present invention uses a linear path diffusion method and combines two common processing operations, occlusion and destruction, as examples to apply the above path-oriented diffusion strategy for training and development. The specific development of the code generation pre-training model can be seen as a combination of the following three parts: model structure, input and output representation, and pre-training tasks. It is worth noting that the final code pre-training model is one of the implementation methods corresponding to the path-oriented diffusion strategy proposed in the present invention.

[0045] 1) Code pre-training model structure

[0046] The path-oriented diffusion strategy involved in the present invention is a general method. The classic Transformer-based Encoder-Decoder framework is selected for the code pre-training model design. On this basis, the present invention will continue pre-training on the model weights of CodeT5 with a scale of 600 million parameters, which allows the model to benefit from the knowledge and representation learned by CodeT5, which is particularly beneficial in code-related tasks.

[0047] 2) Input and output representation

[0048] The code pre-training model proposed in the present invention accepts input in a formalized manner as follows, where w i Represents the token of the i-th input, [CLS] and [SEP] represent special tokens in the input.

[0049] x=[CLS],w1,w2,…,w m , [SEP]

[0050] Accordingly, the output can be formally described as follows, where y i The token representing the ith output, [BOS] and [EOS] represent the start and end markers in the output, respectively.

[0051] y=[BOS],y1,y2,…,y z , [EOS]

[0052] It is worth noting that, depending on the different requirements of different tasks, the input x can be a natural language description text (such as a code generation task) or a code that needs to be modified (such as a code enhancement task).

[0053] 3) Pre-training tasks

[0054] The pre-training tasks involved in the present invention are intended to effectively combine different data diffusion strategies. In order to enable the code pre-training model to effectively generate target code, the present invention proposes two specific path-oriented diffusion data processing strategies and designs two pre-training tasks for this purpose: code generation based on occlusion content and code generation based on denoising.

[0055] a) Code generation based on occlusion content

[0056] This pre-training task is based on the linear diffusion of the occluded content in the predicted code. Unlike the traditional method of randomly masking about 15% of the content in the input, the present invention increases the initial occlusion probability to 60% in this task. The present invention diffuses based on the initial occluded content, and during each diffusion process, the present invention reduces the occlusion probability by 15% each time. It is worth noting that the initial probability and the probability of each reduction designed in the present invention can be adjusted according to different tasks.

[0057] Formally, the loss for this task can be described as follows:

[0058]

[0059] Among them, t represents the current generated step, y t Represents the tth token of the generated edited code. mask Indicates that there is some degree of obscuration in the data. (θ) Represents the probability distribution fitted by the code pre-training model.

[0060] b) Denoising-based code generation

[0061] The second pre-training of the present invention is code generation based on denoising, specifically restoring the original input content based on a certain probability of noise. Traditional denoising methods usually only add 20% of noise, while the present invention can add 60% or even more noise and diffuse it on this basis. Consistent with the previous pre-training task, in this task, the present invention also uses a specific diffusion operation of reducing the noise probability by 15% each time.

[0062]

[0063] Among them, x noiseRepresents the data after adding noise, where the introduced noise types include code token insertion, token deletion, and token replacement.

[0064] c) Combination of pre-training objectives

[0065] The present invention treats each pre-training task equally and adds the loss value of each task to obtain the overall loss. The final loss function is described as follows:

[0066]

[0067] Application phase: Input the code requirement description or the code to be edited into the trained code pre-training model to generate the required target program.

[0068] III. Evaluation of the technical effects of the present invention

[0069] Finally, in order to evaluate the functionality of the present invention, the present invention selects three code generation-related downstream tasks to evaluate the capabilities of the code pre-training model. These tasks are:

[0070] Code translation: The goal of this task is to convert the code of one programming language into the code of another programming language. In order to evaluate the performance of the present invention, the present invention selects the mutual translation between Java and CSharp programming languages ​​as a test. The dataset CodeTrans used in the evaluation process includes both code translation from Java to CSharp and code translation from CSharp to Java. These test data are used to verify the ability of the present invention in processing and translating existing codes.

[0071] Code Enhancement: The goal of this task is to input defective code, and the model automatically fixes the errors and outputs correct code. This task aims to evaluate the ability of the present invention to automatically edit code without natural language guidance. The dataset used for evaluation is divided into two versions: small and medium. The small version contains shorter code, while the medium version contains longer code.

[0072] Code generation: This task is to input a natural language description of a piece of code, and the model automatically generates the corresponding code according to the requirements of the natural language description. This task is used to evaluate the ability of the present invention in understanding natural language and generating corresponding code. The specific data set evaluated is the Concode code generation data set.

[0073] The evaluation results show that in the code translation task, the present invention achieves an accuracy of 68.8% and 71.7% in the Java to CSharp translation and CSharp to Java translation, respectively, significantly exceeding the code pre-training model of the same scale (which can achieve an accuracy of 64.10% and 65.60%, respectively); in the code enhancement task, it can achieve an accuracy of 21.95% on the small version dataset, exceeding the 20.94% accuracy of the original code pre-training model; in the code generation task, the present invention can achieve an accuracy of 21.85% and a CodeBLEU value of 42.44 on the Concode dataset, while the model of the same parameter scale can only achieve an accuracy of 21.55% and a CodeBLEU value of 41.39.

[0074] Practical application scenarios of the present invention:

[0075] With the increasing complexity of modern software, software development has become more and more time-consuming and error-prone. Code generation is a method that can effectively improve the efficiency of software development. Automatically generating the required target code using natural language or existing code is also an important application scenario. The code generation pre-training model plays an indispensable role in realizing software automation. However, with the continuous expansion of the scale of model parameters, the existing data scale gradually cannot support the continuous expansion of model parameters. The method of the present invention can improve the utilization efficiency of existing data, enhance the correlation between data samples, and ultimately improve the performance of the code pre-training model, thereby better improving the programming efficiency of developers.

[0076] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A path-oriented diffusion code pre-training method, the steps of which include: 1) Diffusion of the existing code training data by a preset path diffusion method to obtain a diffused code data set; wherein, the method of diffusing a code training data x by using the path diffusion method is: blocking or destroying the code training data x to different degrees to obtain n diffusion samples (x1, x2, x3, ..., x n ) as the diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code data set; 2) Using the diffused code data set to train the code pre-training model; wherein, for the diffused sample set corresponding to the code training data x in the diffused code data set, using a plurality of diffused samples x generated by blocking the code training data x to different degrees mask Train the code pre-trained model and calculate the corresponding loss value Multiple diffusion samples x generated by damaging the code training data x to different degrees noise Train the code pre-trained model and calculate the corresponding loss value Then, by minimizing the total loss function value Optimize the code pre-trained model.

2. The method according to claim 1, characterized in that The path diffusion method is linear path diffusion. The linear diffusion path refers to the continuous occlusion or destruction of the code training data x to different degrees during the diffusion process. The occlusion or destruction degree between the generated diffusion samples follows a linear relationship, that is, the degree of occlusion or destruction gradually decreases or increases.

3. The method according to claim 2, characterized in that The path diffusion method is a tree-shaped path diffusion composed of multiple linear path diffusions. The tree-shaped path diffusion refers to blocking or destroying the code training data x, and the blocking degree or destruction degree between the generated diffusion samples has a tree-shaped correlation relationship.

4. The method according to claim 2, characterized in that: The path diffusion method is a graphic path diffusion composed of multiple linear path diffusions. The graphic path diffusion refers to blocking or destroying the code training data x, and the blocking degree or destruction degree between the generated diffusion samples has a ring structure.

5. The method according to claim 1, characterized in that The code pre-training model is an Encoder-Decoder framework based on Transformer.

6. A code generation method, characterized in that: The code requirement description or the code to be edited is input into the code pre-training model trained by the method described in claim 1 to generate the required target program.

7. A path-oriented diffusion code pre-training system, characterized in that: It includes code training data diffusion module and model training module; among them, The code training data diffusion module is used to diffuse the existing code training data by a preset path diffusion method to obtain a diffused code data set; wherein the method of diffusing a code training data x by using the path diffusion method is: the code training data x is blocked or destroyed to different degrees to obtain n diffusion samples (x1, x2, x3, ..., x n ) as the diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code data set; The model training module is used to train the code pre-training model using the diffused code data set; wherein, for the diffused sample set corresponding to the code training data x in the diffused code data set, a plurality of diffused samples x generated by masking the code training data x to different degrees are used. mask Train the code pre-trained model and calculate the corresponding loss value Multiple diffusion samples x generated by damaging the code training data x to different degrees noise Train the code pre-trained model and calculate the corresponding loss value Then, by minimizing the total loss function value Optimize the code pre-trained model.

8. A code generation system, characterized in that: It includes code training data diffusion module, model training module and code generation module; among them, The code training data diffusion module is used to diffuse the existing code training data by a preset path diffusion method to obtain a diffused code data set; wherein the method of diffusing a code training data x by using the path diffusion method is: the code training data x is blocked or destroyed to different degrees to obtain n diffusion samples (x1, x2, x3, ..., x n ) as the diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code data set; The model training module is used to train the code pre-training model using the diffused code data set; wherein, for the diffused sample set corresponding to the code training data x in the diffused code data set, a plurality of diffused samples x generated by masking the code training data x to different degrees are used. mask Train the code pre-trained model and calculate the corresponding loss value Multiple diffusion samples x generated by damaging the code training data x to different degrees noise Train the code pre-trained model and calculate the corresponding loss value Then, by minimizing the total loss function value Optimizing the code pre-training model; The code generation module is used to input the code requirement description or the code to be edited into the trained and optimized code pre-training model to generate the required target program.

9. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images