A path-oriented diffused code pre-training method and system

Abstract

This invention discloses a path-oriented diffusion-based code pre-training method and system. The method includes the following steps: 1) Diffusion of existing code training data using a pre-set path diffusion method to obtain a diffused code dataset; wherein, the method for diffusion of a code training data x using the path diffusion method is: occluding or destroying the code training data x to different degrees to obtain n diffusion samples that are related to the code training data x as a diffusion sample set corresponding to the code training data x; the diffusion sample sets corresponding to each code training data constitute the diffused code dataset; 2) Training a code pre-training model using the diffused code dataset. This invention enhances the use of individual code and expands the correlation between multiple training data samples, thereby enabling the trained model to handle downstream tasks more effectively.

Description

Technical Field

[0001] This invention belongs to the field of information technology, specifically relating to a path-oriented diffusion-based code pre-training method and system. Background Technology

[0002] Code pre-trained models play a crucial role in modern software development. By pre-training on large-scale code corpora, these models can automatically understand and generate code, significantly improving development efficiency. As the complexity of software development continues to increase, the importance of code pre-trained models becomes increasingly apparent. However, while the parameter size of pre-trained models can expand infinitely, the amount of data available in reality may quickly reach its limit. Therefore, how to fully utilize every available high-quality code dataset is a critical issue.

[0003] Existing code pre-training techniques primarily utilize data in two ways: 1) predicting occluded content in the data; and 2) recovering the original data from corrupted data. The first method first occludes the content of the original input with a certain probability (e.g., 15%), then uses a deep learning model to predict the content or category of the occluded location. For example, it predicts the value corresponding to an occluded token or a sequence of tokens, or whether the occluded token belongs to the code's identifier. The pre-trained model learns the code context to predict the occluded content within the data, thus perceiving the components and specific content of the code data. The second method corrupts the content of the original input data with a certain probability. Unlike simply occluding data content, this corruption may involve inserting additional tokens or deleting existing tokens. After introducing a series of corrupting operations to add noise to the data, the pre-trained model needs to recover the original input data content from the corrupted content to learn the correct structure and organization of the data.

[0004] The shortcomings of existing technologies mainly include two aspects: 1) Insufficient data utilization. Currently, whether predicting occluded content in the data or recovering the original data from corrupted data, a single data sample is processed only once in a training epoch. However, processing the data only once introduces significant randomness, meaning that only a large portion of the retained data is needed to predict the missing or corrupted parts. Data utilization efficiency depends only on the small portion of content that needs to be predicted. Therefore, this approach lacks sufficient learning from individual data, leading to a model that often requires a large amount of data to learn. Simultaneously, this also results in a situation where, as model parameters continue to expand, the existing data volume quickly becomes insufficient to support training larger-scale parameter models. 2) Weak correlation between data. Existing code pre-training tasks primarily focus on capturing the internal structure and characteristics of a single code training sample, without considering the potential correlations between multiple training samples. This lack of utilization of potential correlations between pre-training data often leads to low learning efficiency.

[0005] In summary, existing methods for code pre-training primarily guide the pre-training process by either identifying occluded content in the prediction data or recovering the original data from corrupted data. However, both methods face challenges such as insufficient data utilization and weak correlations between data points. These challenges lead to the current requirement for large amounts of data for code pre-training, potentially resulting in situations where the existing data scale is insufficient to support training models with larger parameter sizes. Therefore, in an era of rapid development of large-scale parameter pre-trained models, how to efficiently utilize existing data to improve the performance of pre-trained models has become a crucial issue. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a path-oriented diffusion-based code pre-training method and system.

[0007] The technical solution adopted in this invention is as follows:

[0008] A path-oriented diffusion-based code pre-training method, comprising the following steps:

[0009] 1) Diffusion is performed on existing code training data using a pre-defined path diffusion method to obtain a diffused code dataset. Specifically, the method for diffusing code training data x using path diffusion involves occluding or disrupting the code training data x to varying degrees, resulting in n diffused samples (x1, x2, x3, ..., x...) that are related to the code training data. n The set of diffused samples corresponding to the code training data x is used as the training data set for the code; the set of diffused samples corresponding to each training data set of the code constitutes the diffused code dataset.

[0010] 2) Train a pre-trained code model using the diffused code dataset; wherein, for the diffused sample set corresponding to the code training data x in the diffused code dataset, multiple diffused samples x generated by occluding the code training data x to different degrees are used. mask Train the code to pre-train the model and calculate the corresponding loss value.

[0011] Multiple diffusion samples x were generated by subjecting the training data x of the code to different degrees of corruption. noise Train the code to pre-train the model and calculate the corresponding loss value. Then, by minimizing the total loss function value Optimize the pre-trained model of the code.

[0012] Furthermore, the path diffusion method is linear path diffusion, which means that the code training data x is continuously occluded or damaged to different degrees during the diffusion process, and the degree of occlusion or damage between the generated diffusion samples follows a linear relationship, that is, the degree of occlusion or damage gradually decreases or increases.

[0013] Furthermore, the path diffusion method is a tree-shaped path diffusion composed of multiple linear path diffusions. The tree-shaped path diffusion refers to occluding or destroying the code training data x, and the degree of occlusion or destruction between the generated diffusion samples has a tree-shaped correlation.

[0014] Furthermore, the path diffusion method is a graphical path diffusion composed of multiple linear path diffusions. The graphical path diffusion refers to occluding or destroying the code training data x, and the degree of occlusion or destruction between the generated diffusion samples has a loop structure.

[0015] Furthermore, the code pre-training model is an Encoder-Decoder framework based on Transformer.

[0016] A code generation method is characterized by inputting a code requirement description or the code to be edited into a code pre-training model trained by the above method to generate the required target program.

[0017] A path-oriented diffusion-based code pre-training system, characterized by comprising a code training data diffusion module and a model training module; wherein,

[0018] The code training data diffusion module is used to diffuse existing code training data using a pre-set path diffusion method to obtain a diffused code dataset. Specifically, the method for diffusion of a piece of code training data x using the path diffusion method involves occluding or damaging the code training data x to varying degrees, resulting in n diffused samples (x1, x2, x3, ..., xn) that are related to the code training data. n The set of diffused samples corresponding to the code training data x is used as the training data set for the code; the set of diffused samples corresponding to each training data set of the code constitutes the diffused code dataset.

[0019] The model training module is used to train a code pre-training model using the diffused code dataset; wherein, for the diffused sample set corresponding to the code training data x in the diffused code dataset, multiple diffused samples x generated by occluding the code training data x to different degrees are used. mask Train the code to pre-train the model and calculate the corresponding loss value. Multiple diffusion samples x were generated by subjecting the training data x of the code to different degrees of corruption. noise Train the code to pre-train the model and calculate the corresponding loss value. Then, by minimizing the total loss function value Optimize the pre-trained model of the code.

[0020] A code generation system, characterized in that it includes a code training data diffusion module, a model training module, and a code generation module; wherein,

[0021] The code training data diffusion module is used to diffuse existing code training data using a pre-set path diffusion method to obtain a diffused code dataset. Specifically, the method for diffusion of a piece of code training data x using the path diffusion method involves occluding or damaging the code training data x to varying degrees, resulting in n diffused samples (x1, x2, x3, ..., xn) that are related to the code training data. n The set of diffused samples corresponding to the code training data x is used as the training data set for the code; the set of diffused samples corresponding to each training data set of the code constitutes the diffused code dataset.

[0022] The model training module is used to train a code pre-training model using the diffused code dataset; wherein, for the diffused sample set corresponding to the code training data x in the diffused code dataset, multiple diffused samples x generated by occluding the code training data x to different degrees are used. mask Train the code to pre-train the model and calculate the corresponding loss value. Multiple diffusion samples x were generated by subjecting the training data x of the code to different degrees of corruption. noise Train the code to pre-train the model and calculate the corresponding loss value. Then, by minimizing the total loss function value Optimize the pre-trained model of the code;

[0023] The code generation module is used to input the code requirement description or the code that needs to be edited into the trained and optimized code pre-trained model to generate the required target program.

[0024] A server is characterized by comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.

[0025] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention, during its design phase, comprehensively reconsidered the code pre-training problem and arrived at a novel perspective for improving the efficiency of code pre-training data utilization. Utilizing this new perspective, the invention enhances the use of individual code snippets and expands the correlations between multiple training data samples. However, this new perspective also presents new challenges to code pre-training models. Specifically, it requires us to reasonably expand each data point and correlate the expanded data. It also requires the model to accurately generate target code content given different contexts, thereby enabling more effective handling of downstream tasks. To address these new challenges, this invention designs a path-oriented, diffusion-based code pre-training method and system. Attached Figure Description

[0028] Figure 1 It is the basic process of a path-oriented diffusion-based code pre-training method and system. Detailed Implementation

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below through specific embodiments and accompanying drawings.

[0030] This invention implements a path-oriented diffusion-based code pre-training method and system, which refers to: at the input end, the user diffuses existing code training data through a pre-defined path; at the output end, the model uses the diffused code dataset to pre-train a code pre-training model. The trained code pre-training model receives input code requirement descriptions or code to be edited, and then automatically synthesizes the required target program to meet the user's requirements or address the user's modification intentions.

[0031] Specifically, recognizing the importance of fully utilizing data and pre-trained models based on code, this invention designs a system to automatically solve problems related to code data and models. The input and output of this system are described below:

[0032] At the input end, during the data diffusion phase, the user inputs raw data, and the system diffuses the data according to the set path. At the output end, after the code pre-training model is trained and enters the application phase, the user provides a code requirement description or code that needs to be edited. When a code requirement description is input, the model automatically generates code that matches the requirement description; when code that needs to be edited is input, the model automatically edits existing code and improves code quality.

[0033] At the output end, the system of this invention trains a model based on the large-scale corpus that has been spread, and automatically generates target code that meets the requirements based on the knowledge learned by the model.

[0034] To improve upon existing pre-training techniques, this invention: 1. First, designs a path-oriented diffusion strategy to process pre-training data; 2. Then, utilizes a pre-training technique integrating multiple diffusion strategies to train code-related deep learning models; 3. Finally, evaluates the generation results of the path-oriented diffusion code pre-training system. The performance details are described below:

[0035] I. Path-oriented diffusion strategy

[0036] To better utilize existing data during pre-training, this invention employs... Figure 1 The basic process shown applies a path-oriented diffusion strategy to the data.

[0037] The core idea of ​​path-oriented diffusion strategies is to achieve data diffusion by applying continuous occlusion or destruction operations with different probabilities to the same data sample, and to learn the associations between samples by utilizing the different paths and internal correlation information during the diffusion of a single data sample. In this invention, occlusion or destruction operations on data are collectively referred to as data processing methods, and a diffusion path is defined as an ordered set of data points from one data point to another after different degrees of processing of the same sample. Specific path diffusion methods include linear path diffusion, tree-like path diffusion, graphical path diffusion, and other path diffusion methods.

[0038] according to Figure 1 As shown, a linear diffusion path refers to a path where the diffusion process follows a linear relationship between data points, such as a gradual decrease or increase in the probability of occlusion. Figure 1 The left side shows a gradual decrease in the proportion of noise and occlusion, and each subsequent diffused sample is the result of processing the previous sample. Similarly, tree-like path diffusion refers to a tree-like relationship between data points, such as in... Figure 1 The middle section is an example of diffusion using a binary tree approach. Graph path diffusion refers to a data diffusion path containing a loop structure, for example... Figure 1 The right side illustrates the loop structure resulting from handling position replacement and additional processing. Furthermore, diffusion patterns that follow a specific path association, other than linear, tree-like, and graphical patterns, are collectively referred to as other path diffusion patterns.

[0039] For each input data x, when constructing a diffusion path, a relatively high initial probability P is first chosen to occlude or destroy the original input to obtain x1. Then, based on x1, the probability of occluding or destroying the original data is gradually decreased (similarly, if the initial probability is low, the probability of occluding or destroying the original data can be gradually increased along the diffusion path). This process is used to obtain a series of diffused samples x1, x2, x3, ..., x1 corresponding to the data x. n Where x1, x2, x3, ..., x n The internal relationships are as follows: x2 is based on x1 with a reduced probability of being occluded or damaged (i.e., the occluded or damaged areas included in x2 are a subset of the occluded or damaged areas included in x1), x3 is based on x2 with a reduced probability of being occluded or damaged (i.e., the occluded or damaged areas included in x3 are a subset of the occluded or damaged areas included in x2), and so on (i.e., x... i The area that is obscured or damaged is x. i-1 (Contains a subset of occluded or damaged areas), where the diffused samples x1, x2, x3, ..., x nA diffusion path is formed. If all diffusion samples form a single diffusion path, it is called linear path diffusion; similarly, diffusion samples can also contain multiple diffusion paths, forming various path diffusion methods such as tree-shaped path diffusion and graph-shaped path diffusion. Assuming that during the data diffusion process, each time K% of the original content is recovered based on the initial probability P, the proportion of occluded content in the first diffusion sample x1 is (PK)%. Similarly, the proportion of occluded content in the second diffusion sample is (P-2K)%, and the proportion of occluded content in the nth diffusion sample is (Pn*K)%. For ease of demonstration, this invention sets the reduction ratio to K each time, but the reduction or increase ratio can be flexible and does not need to follow the same reduction or increase ratio each time. Ultimately, the original 1 data sample is diffused into n additional samples, meaning that in one training round, the model will be trained on n+1 variants of that data sample containing different processing ratios.

[0040] In pre-training tasks involving predicting occluded content in the dataset, existing pre-training tasks aim to occlude only a small portion of the data in a single dataset, such as occluding 15% of the code. However, a path-oriented diffusion strategy can choose a higher occlusion probability, such as occluding 60% of the code. Then, the occlusion probability is gradually reduced, for example, by 15% each time, resulting in three additional data samples: those occluding 45%, 30%, and 15% of the code, respectively. Based on this, each data sample is utilized four times in a training epoch instead of once. The path-related relationship between these four data samples is that the data sample with 30% occlusion recovers 15% of the content based on the data sample with 45% occlusion, rather than randomly occluding 30%. The specific initial occlusion probability and the probability of each reduction can be adjusted according to different needs and situations.

[0041] In pre-training tasks that recover original data from corrupted data, existing pre-training methods typically introduce noise into the code data through operations such as insertion, deletion, and occlusion, and then recover the original data content from data containing a certain probability (e.g., 20%) of noise. However, this pre-training task also introduces noise only once for each data sample, which means that the data utilization efficiency largely depends on the location where the noise is randomly introduced. In a path-oriented diffusion strategy, the initial state of the data may be that it has been introduced with a high degree of noise, for example, the noise ratio in the data reaches 60%. Similarly, by gradually reducing the noise ratio, the path-oriented diffusion strategy can be extended to multiple different data variants. By utilizing data with different levels of noise to recover the original data content, the data utilization rate can be effectively improved.

[0042] Besides the two common pre-training tasks mentioned above, path-oriented diffusion strategies can be adapted to the characteristics of different pre-training tasks. Different pre-training tasks only provide a certain probability of processing the original data, while path-oriented diffusion strategies first increase (or decrease) the proportion of processing the original data, and then enhance data utilization by diffusing this proportion. It is worth noting that the different samples diffused out have specific path-related relationships. By learning the multiple diffused data samples and their path information, the model can fully utilize the data and improve its performance on downstream tasks.

[0043] II. Code Pre-trained Model

[0044] In this invention, the code pre-training model uses a linear path diffusion approach, taking occlusion and destruction as examples of common processing operations, and applies the above path-oriented diffusion strategy for training and development. The specific development of the code generation pre-training model can be viewed as a combination of the following three parts: model structure, input / output representation, and pre-training task. It is worth noting that the final code pre-training model is one implementation of the path-oriented diffusion strategy proposed in this invention.

[0045] 1) Code pre-trained model structure

[0046] The path-oriented diffusion strategy involved in this invention is a general approach, and the code pre-training model design adopts the classic Transformer-based Encoder-Decoder framework. Building upon this, this invention further pre-trains on the 600 million parameter scale of the CodeT5 model weights, allowing the model to benefit from the knowledge and representations learned by CodeT5, which is particularly advantageous in code-related tasks.

[0047] 2) Input / output representation

[0048] The formal description of the input received by the code pre-training model proposed in this invention is as follows, where w i [CLS] represents the token of the i-th input, and [SEP] represents special tokens in the input.

[0049] x = [CLS], w1, w2, ..., w m [SEP]

[0050] Accordingly, the output can be formally described as follows, where y i [BOS] represents the token of the i-th output, and [EOS] represents the start and end markers in the output, respectively.

[0051] y = [BOS], y1, y2, ..., y z [EOS]

[0052] It is worth noting that, depending on the different requirements of different tasks, the input x can be natural language description text (such as code generation tasks) or code that needs to be modified (such as code enhancement tasks).

[0053] 3) Pre-training task

[0054] The pre-training tasks involved in this invention aim to effectively combine different data diffusion strategies. To enable the code pre-training model to effectively generate target code, this invention proposes two specific path-oriented diffusion data processing strategies and designs two pre-training tasks for this purpose: code generation based on occlusion content and code generation based on denoising.

[0055] a) Code generation based on occluded content

[0056] This pre-training task is based on linear diffusion of occluded content in the prediction code. Unlike traditional methods that randomly occlude approximately 15% of the input, this invention increases the initial occlusion probability to 60% in this task. The invention diffuses based on the initial occluded content, reducing the occlusion probability by 15% with each diffusion iteration. It is noteworthy that both the initial probability and the probability reduction each time can be adjusted according to different tasks.

[0057] In a formal sense, the loss of this mission can be described as follows:

[0058]

[0059] Where t represents the current generation step, y t This represents the t-th token in the generated edited code. mask This indicates that some content in the data is obscured. (P) (θ) This represents the probability distribution fitted by the code pre-trained model.

[0060] b) Denoising-based code generation

[0061] The second pre-training step of this invention is based on denoised code generation, specifically recovering the original input content from noise with a certain probability. Traditional denoising methods typically add only 20% noise, while this invention can add 60% or even more noise, and then diffuse it. Consistent with the previous pre-training task, this invention also uses a 15% reduction in noise probability for each diffusion operation.

[0062]

[0063] Where, x noiseThis represents the data after noise has been added, where the types of noise introduced include token insertion, token deletion, and token replacement.

[0064] c) Combination of pre-training objectives

[0065] This invention treats each pre-training task equally and sums the loss values ​​of each task to obtain the overall loss. The final loss function is described as follows:

[0066]

[0067] Application phase: Input the code requirement description or the code that needs to be edited into the pre-trained code model to generate the required target program.

[0068] III. Evaluation of the Technical Effects of the Invention

[0069] Finally, to evaluate the functionality of this invention, three downstream tasks related to code generation were selected to assess the capabilities of the code pre-trained model. These tasks are:

[0070] Code Translation: The goal of this task is to translate code from one programming language into code from another. To evaluate the performance of this invention, mutual translation between Java and C# programming languages ​​was chosen as the test. The CodeTrans dataset used in the evaluation process includes both Java to C# and C# to Java code translations. These test data are used to verify the invention's ability to process and translate existing code.

[0071] Code Augmentation: The goal of this task is to input flawed code, have the model automatically correct the errors, and output the correct code. This task aims to evaluate the ability of this invention to automatically edit code without natural language guidance. The evaluation dataset is divided into two versions: small and medium. The small version contains shorter code, while the medium version contains longer code.

[0072] Code Generation: This task involves inputting a natural language description of a code snippet, and the model automatically generates the corresponding code based on the requirements of the natural language description. This task is used to evaluate the invention's ability to understand natural language and generate corresponding code. The specific dataset used for evaluation is the Concode code generation dataset.

[0073] Evaluation results show that in code translation tasks, this invention achieves accuracy rates of 68.8% and 71.7% in Java-to-C++ and C++-to-Java translation, respectively, significantly exceeding the accuracy rates of pre-trained code models of the same scale (64.10% and 65.60%, respectively). In code enhancement tasks, it achieves an accuracy rate of 21.95% on the small version dataset, surpassing the 20.94% accuracy rate of the original pre-trained code model. In code generation tasks, this invention achieves an accuracy rate of 21.85% and a CodeBLEU score of 42.44 on the Concode dataset, while models of the same parameter scale only achieve an accuracy rate of 21.55% and a CodeBLEU score of 41.39.

[0074] Practical application scenarios of this invention:

[0075] With the increasing complexity of modern software, software development has become increasingly time-consuming and error-prone. Code generation is an effective method to improve software development efficiency, and automatically generating the required target code using natural language or existing code is an important application scenario. Code generation pre-trained models play an indispensable role in achieving software automation; however, as the scale of model parameters continues to expand, the existing data scale gradually becomes insufficient to support the continuous expansion of model parameters. The method of this invention can improve the utilization efficiency of existing data, enhance the correlation between data samples, and ultimately improve the performance of the code pre-trained model, thereby better improving the programming efficiency of developers.

[0076] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A path-oriented diffusion-based code pre-training method, comprising the following steps: 1) Diffusion is performed on existing code training data using a pre-defined path diffusion method to obtain a diffused code dataset; wherein, the path diffusion method is used to diffuse a set of code training data... The method for diffusion is: using the training data of this code By performing varying degrees of occlusion or disruption, n diffusion samples that are correlated with the training data of this code are obtained. ) as training data for this code The corresponding set of diffusion samples; the set of diffusion samples corresponding to each code training data constitutes the diffused code dataset; wherein, the path diffusion method is linear path diffusion, which refers to the diffusion process of code training data... By performing continuous occlusion or disruption to varying degrees, the degree of occlusion or disruption among the generated diffused samples follows a linear relationship, meaning the degree of occlusion or disruption gradually decreases or increases; or the path diffusion method is a tree-like path diffusion composed of multiple linear path diffusions, where tree-like path diffusion refers to the diffusion of code training data... The occlusion or destruction process generates diffused samples with a tree-like correlation in terms of the degree of occlusion or destruction; or the path diffusion method is a graphical path diffusion composed of multiple linear path diffusions, where graphical path diffusion refers to the diffusion of code training data. The degree of occlusion or destruction between the generated diffused samples exhibits a ring structure. 2) Train a pre-trained code model using the diffused code dataset; wherein, for the code training data in the diffused code dataset... The corresponding diffusion sample set, using the training data of this code Multiple diffusion samples generated by varying degrees of occlusion Train the code to pre-train the model and calculate the corresponding loss value. Using the training data of this code Multiple diffusion samples generated by varying degrees of destruction Train the code to pre-train the model and calculate the corresponding loss value. Then, by minimizing the total loss function value... Optimize the pre-trained model of the code.

2. The method according to claim 1, characterized in that, The code pre-training model is an Encoder-Decoder framework based on Transformer.

3. A code generation method, characterized in that, Input the code requirement description or the code that needs to be edited into the code pre-trained model trained by the method described in claim 1 to generate the required target program.

4. A path-oriented diffusion-based code pre-training system, characterized in that, This includes a code training data diffusion module and a model training module; among which, The code training data diffusion module is used to diffuse existing code training data using a pre-set path diffusion method to obtain a diffused code dataset; wherein, the path diffusion method is used to diffuse a set of code training data... The method for diffusion is: using the training data of this code By performing varying degrees of occlusion or disruption, n diffusion samples that are correlated with the training data of this code are obtained. ) as training data for this code The corresponding set of diffusion samples; the set of diffusion samples corresponding to each code training data constitutes the diffused code dataset; The model training module is used to train a pre-trained code model using the diffused code dataset; wherein, for the code training data in the diffused code dataset... The corresponding diffusion sample set, using the training data of this code Multiple diffusion samples generated by varying degrees of occlusion Train the code to pre-train the model and calculate the corresponding loss value. Using the training data of this code Multiple diffusion samples generated by varying degrees of destruction Train the code to pre-train the model and calculate the corresponding loss value. Then, by minimizing the total loss function value... Optimize the pre-trained model of the code; The path diffusion method is linear path diffusion, which refers to the diffusion process involving code training data. When continuous occlusion or destruction is performed, the degree of occlusion or destruction between the generated diffusion samples follows a linear relationship, that is, the degree of occlusion or destruction gradually decreases or increases. Alternatively, the path diffusion method may be a tree-like path diffusion composed of multiple linear path diffusions, where tree-like path diffusion refers to the diffusion of code training data. The degree of occlusion or destruction generated by the occlusion or destruction of the samples creates a tree-like correlation between them. Alternatively, the path diffusion method may be a graphical path diffusion consisting of multiple linear path diffusions, where graphical path diffusion refers to the diffusion of code training data. The degree of occlusion or destruction between the generated diffuse samples exhibits a ring structure.

5. A code generation system, characterized in that, It includes a code training data diffusion module, a model training module, and a code generation module; among which, The code training data diffusion module is used to diffuse existing code training data using a pre-set path diffusion method to obtain a diffused code dataset; wherein, the path diffusion method is used to diffuse a set of code training data... The method for diffusion is: using the training data of this code By performing varying degrees of occlusion or disruption, n diffusion samples that are correlated with the training data of this code are obtained. ) as training data for this code The corresponding set of diffusion samples; the set of diffusion samples corresponding to each code training data constitutes the diffused code dataset; The model training module is used to train a pre-trained code model using the diffused code dataset; wherein, for the code training data in the diffused code dataset... The corresponding diffusion sample set, using the training data of this code Multiple diffusion samples generated by varying degrees of occlusion Train the code to pre-train the model and calculate the corresponding loss value. Using the training data of this code Multiple diffusion samples generated by varying degrees of destruction Train the code to pre-train the model and calculate the corresponding loss value. Then, by minimizing the total loss function value... Optimize the pre-trained model of the code; The code generation module is used to input the code requirement description or the code that needs to be edited into the trained and optimized code pre-trained model to generate the required target program; The path diffusion method is linear path diffusion, which refers to the diffusion process involving code training data. When continuous occlusion or destruction is performed, the degree of occlusion or destruction between the generated diffusion samples follows a linear relationship, that is, the degree of occlusion or destruction gradually decreases or increases. Alternatively, the path diffusion method may be a tree-like path diffusion composed of multiple linear path diffusions, where tree-like path diffusion refers to the diffusion of code training data. The degree of occlusion or destruction generated by the occlusion or destruction of the samples creates a tree-like correlation between them. Alternatively, the path diffusion method may be a graphical path diffusion consisting of multiple linear path diffusions, where graphical path diffusion refers to the diffusion of code training data. The degree of occlusion or destruction between the generated diffuse samples exhibits a ring structure.

6. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 3.

Images