Code generation method, device and system based on deep learning

Abstract

The invention discloses a code generation method, device and system based on deep learning. The method comprises the following steps: receiving code data input by a user; standardizing the code data to obtain standardized code data; dividing the standardized code data into morphemes, and encoding the morphemes to obtain an integer index; inputting the integer index into an embedding layer to obtain a fixed dimension vector; inputting the fixed dimension vector into a pre-trained encoder to obtain a feature vector sequence; the encoder firstly adopts a self-attention mechanism to obtain a long-distance dependency relationship in a fixed-dimension vector, and obtains a feature vector sequence through feature transformation and aggregation operation; and generating a code according to the feature vector sequence. According to the code generation method based on deep learning, a large number of codes do not need to be understood, the code generation accuracy and efficiency can be remarkably improved, and the cost is low.

Description

Technical Field

[0001] The present application relates to the field of code generation technology, and specifically to a code generation method, device and system based on deep learning. Background Art

[0002] An intelligent code assistant is a tool that uses artificial intelligence technology to help developers write, debug, and optimize code. It can provide code suggestions, automatically complete code snippets, detect errors, and generate documents, aiming to improve development efficiency and code quality. Common intelligent code assistants include GitHub Copilot, TabNine, etc., which provide real-time help to users by analyzing large code bases and contexts.

[0003] GitHub Copilot is mainly used for code writing, refactoring, and debugging in software development. It can achieve rapid development of cross-language projects and enable code sharing and style unification in team collaboration. Therefore, GitHub Copilot has the following advantages:

[0004] 1. Intelligent completion: Based on OpenAI's GPT model, GitHub Copilot can learn a large number of code bases and developers' coding styles, provide developers with instant code suggestions, greatly reduce the time of writing boilerplate code, and enable developers to focus more on solving problems and implementing functions.

[0005] 2. Multi-language support: Support for multiple programming languages, including JavaScript, Python, Java, etc., enables Copilot to play a role in different fields and projects.

[0006] 3. Personalized learning: It has a learning mode that can gradually adapt to the user's coding style and provide more personalized code suggestions.

[0007] 4. Team collaboration: In team collaboration, GitHub Copilot can quickly share code snippets, promote more efficient collaboration, while maintaining consistency in coding standards and style within the team.

[0008] GitHub Copilot applies advanced natural language processing and machine learning technologies to code writing, achieving intelligent code completion and personalized recommendations. Through deep integration with platforms such as GitHub and training using a rich open source code base, it improves the accuracy and generalization ability of the model.

[0009] However, GitHub Copilot also has the following flaws:

[0010] 1. Learning curve: Copilot relies heavily on understanding a large amount of code, so it takes time to learn and adapt to new programming styles and patterns.

[0011] 2. Poor accuracy: Although Copilot tries to provide accurate code suggestions, it is impossible to be 100% accurate and sometimes provides suggestions that are not completely correct.

[0012] 3. Strong dependency: Copilot relies on machine learning models. If the model is no longer updated or becomes outdated, it may no longer provide the latest features.

[0013] 4. High cost: Copilot may increase the workload of developers, especially when a lot of refactoring or complex logic code is required.

[0014] 5. Privacy issues: Copilot requires access to the user's code base, which may raise privacy and security issues. Summary of the invention

[0015] To this end, the present application provides a code generation method, device and system based on deep learning to solve the problem that the code generation tools in the prior art rely on the understanding of a large amount of code, resulting in poor accuracy and high cost.

[0016] In order to achieve the above objectives, this application provides the following technical solutions:

[0017] In a first aspect, a code generation method based on deep learning is provided, wherein the code generation method based on deep learning is applied to a backend service, comprising:

[0018] Step 1: Receive code data input by the user;

[0019] Step 2: Standardizing the code data to obtain standardized code data;

[0020] Step 3: Segment the standardized code data into morphemes and encode them to obtain integer indexes;

[0021] Step 4: Input the integer index into the embedding layer to obtain a fixed-dimensional vector;

[0022] Step 5: Input the fixed-dimensional vector into a pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain the long-distance dependency relationship in the fixed-dimensional vector, and obtains a feature vector sequence through feature transformation and aggregation operations;

[0023] Step 6: Generate code according to the feature vector sequence.

[0024] Preferably, after step 5, the method further includes:

[0025] Input the feature vector sequence into the decoder to calculate the attention weight, and generate a context vector according to the attention weight;

[0026] The next feature vector is predicted according to the context vector and the feature vector, and the probability distribution of each feature vector is calculated according to the activation function, so as to obtain a complete code sequence.

[0027] Preferably, the activation function is a softmax activation function.

[0028] Preferably, in step 2, the standardization process includes removing comments, spaces and line breaks.

[0029] Preferably, in step 5, the feature transformation and aggregation operations include linear transformation, nonlinear activation function and layer normalization.

[0030] Preferably, the method further comprises: receiving a user's evaluation result on the generated code, and optimizing the encoder according to the evaluation result.

[0031] It also includes automatic code completion, error checking, and refactoring suggestions.

[0032] In a second aspect, a code generation device based on deep learning includes:

[0033] A data receiving module, used for receiving code data input by a user;

[0034] A standardization processing module, used for performing standardization processing on the code data to obtain standardized code data;

[0035] The encoding module is used to: =segment the standardized code data into morphemes and encode them to obtain integer indexes;

[0036] An embedding module, configured to input the integer index into an embedding layer to obtain a fixed-dimensional vector;

[0037] An encoder module is used to input the fixed-dimensional vector into a pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain the long-distance dependency in the fixed-dimensional vector, and obtains a feature vector sequence through feature transformation and aggregation operations;

[0038] A code generation module is used to generate code according to the feature vector sequence.

[0039] In the third aspect, a deep learning-based code generation system comprises a front-end interface and a back-end service, wherein the front-end interface and the back-end service communicate with each other via an application layer protocol, the front-end interface is implemented via Web technology and is located in a browser or IDE plug-in of a user device, and the back-end service comprises a deep learning-based code generation device as described in claim 8.

[0040] Preferably, the Web technology is HTML, CSS or JavaScript.

[0041] Compared with the prior art, this application has at least the following beneficial effects:

[0042] 1. The present application provides a code generation method based on deep learning, which receives code data input by a user; standardizes the code data to obtain standardized code data; divides the standardized code data into morphemes and encodes them to obtain integer indexes; inputs the integer indexes into an embedding layer to obtain a fixed-dimensional vector; inputs the fixed-dimensional vector into a pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain long-distance dependencies in the fixed-dimensional vector, and obtains a feature vector sequence through feature transformation and aggregation operations; and generates code according to the feature vector sequence. The code generation method based on deep learning provided by the present application does not require understanding of a large amount of code, can significantly improve the accuracy and efficiency of code generation, and is low-cost.

[0043] 2. This application provides a code generation system based on deep learning, including a front-end interface and a back-end service. The front-end interface is implemented through Web technology, which can achieve localized and lightweight deployment, solving the privacy leakage problem existing in traditional cloud service code assistants. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more intuitively illustrate the prior art and the present application, exemplary drawings are given below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing the present application; for example, those skilled in the art are capable of easily making conventional adjustments or further optimizations to the addition / reduction / attribution division, specific shapes, positional relationships, connection methods, dimensional ratios, etc. of certain units (components) based on the technical concepts and exemplary drawings disclosed in the present application.

[0045] Figure 1 A flow chart of a code generation method based on deep learning provided in Example 1 of the present application. DETAILED DESCRIPTION

[0046] The present application is further described below in detail through specific embodiments in conjunction with the accompanying drawings.

[0047] In the description of this application: unless otherwise specified, the meaning of "plurality" is two or more. The terms "first", "second", "third", etc. in this application are intended to distinguish the objects referred to, and do not have special meanings in terms of technical connotations (for example, they should not be understood as emphasizing the importance or order, etc.). Expressions such as "including", "comprising", "having", etc. also mean "not limited to" (certain units, components, materials, steps, etc.).

[0048] The terms such as "upper", "lower", "left", "right", "middle", etc. cited in this application are usually used to indicate the general relative position relationship for the purpose of intuitive understanding by referring to the drawings, and are not absolute limitations on the position relationship in the actual product.

[0049] Embodiment 1

[0050] See also Figure 1 This embodiment provides a code generation method based on deep learning, which is applied to a backend service. The deep learning architecture is an encoder-only (only the encoder part is used), a decoder-only (only the decoder part is used), or an encoder-decoder (both the encoder and the decoder are used) architecture. The method includes:

[0051] S1: receiving code data input by the user;

[0052] Specifically, the code data is a code snippet or a natural language description.

[0053] S2: standardize the code data to obtain standardized code data;

[0054] Specifically, the received code data is standardized, including removing comments, spaces, line breaks and other elements that do not affect the code logic, to ensure that the model can focus on the code structure itself.

[0055] S3: Split the standardized code data into morphemes and encode them to obtain integer indices;

[0056] Specifically, the code is split into tokens and encoded so that the model can understand and process it. This usually involves converting the code into a series of integer indices, each index corresponding to a specific token.

[0057] S4: Input the integer index into the embedding layer to obtain a fixed-dimensional vector;

[0058] Specifically, the model first receives preprocessed code data, which is usually converted into a format suitable for model processing (e.g., tokens), such as a sequence of morphemes after word segmentation or a sequence of encoded integers. The integer index is input into the embedding layer, and each morpheme or integer index is mapped to a vector of fixed dimension. These fixed-dimensional vectors capture the semantic information of the morphemes, allowing the model to understand the contextual relationships in the code. The dimensions of the embedding layer are pre-set and can be adjusted according to the model complexity and task requirements.

[0059] S5: Input the fixed-dimensional vector into the pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain the long-distance dependency in the fixed-dimensional vector, and obtains a feature vector sequence through feature transformation and aggregation operations;

[0060] Specifically, the encoder layer first uses a self-attention mechanism to process the embedded vector sequence. The self-attention mechanism allows the model to focus on different parts of the input sequence and generate weighted representations based on the context. In this way, the model is able to capture long-distance dependencies in the code.

[0061] After the self-attention mechanism, the encoder layer usually performs a series of feature transformation and aggregation operations, such as linear transformation, nonlinear activation function (such as ReLU), and layer normalization. Feature transformation and aggregation operations help the model extract higher-level feature representations and enhance the model's expressiveness.

[0062] To increase the depth and understanding of the model, multiple encoder layers can be stacked. Each encoder layer outputs a sequence of feature vectors, which are passed to the next encoder layer or decoder (in the encoder-decoder architecture).

[0063] S6: Generate code based on the feature vector sequence.

[0064] In this step, when generating output code, CodeShell-7B (large language model) adopts an autoregressive decoding method. That is, the model generates each token in turn and uses the generated token as the context for subsequent generation, including:

[0065] S601: Initial context setting: At the beginning of the generation process, the model will receive an initial context, which can be an existing code, a function signature, or some user-provided keywords. The initial context will be converted into a token embedding sequence by the model's embedding layer as the starting point for the model to generate subsequent tokens.

[0066] S602: Tokens are generated sequentially;

[0067] Model input: In each generation step, the model takes the currently generated token sequence (including the initial context and the token generated in the previous step) as input.

[0068] Self-attention mechanism: The model uses the self-attention mechanism to process these input tokens and capture the dependencies between them. The self-attention mechanism allows the model to predict the next token based on considering the entire sequence.

[0069] Predict the next Token: Based on the current input and internal state, the model predicts the next possible Token. This is usually achieved through a softmax layer, which outputs a probability distribution representing the probability of the next Token over all possible Tokens.

[0070] Selection or sampling: Based on this probability distribution, the Token with the highest probability can be selected as the next generated Token (greedy search), or other strategies such as random sampling or beam search can be used to balance diversity and accuracy.

[0071] S603: Update context and iterate: Once a new token is generated, it will be added to the generated token sequence to form a new context. This new context will be used as the input for the next generation of the model, and the iteration will continue until the stop condition is met (such as reaching the preset generation length, generating a specific end token, etc.).

[0072] S604: Processing details and optimization: In practical applications, in order to improve efficiency, batch generation is usually adopted, that is, predicting multiple tokens at a time instead of predicting only one at a time.

[0073] In addition, various optimization techniques can be used to speed up the generation process, such as using GPU acceleration, optimizing the model structure to reduce the amount of calculation, etc.

[0074] The output code is post-processed, such as deduplication and formatting, and presented in a form that is easier for humans to understand.

[0075] The present embodiment provides a method for generating code based on deep learning, further comprising: receiving a user's evaluation result of the generated code, and optimizing the encoder according to the evaluation result. Specifically, the user can evaluate the generated code, check its correctness and applicability, and provide feedback on the evaluation result, so as to further optimize the encoder.

[0076] It should be noted that in the encoder-decoder architecture, the decoder uses the attention mechanism to interact with the encoder's output. The decoder calculates the attention weights for the encoder's output sequence at each time step and generates a context vector based on these weights; the decoder also predicts the next morpheme based on the context vector and the previous output morpheme (feature vector), and this process is carried out step by step until the complete code sequence is generated or the preset length limit is reached; the last layer of the decoder is the output layer, which usually uses a softmax activation function to calculate the probability distribution of each possible morpheme, which is used to guide the final prediction of the model.

[0077] In this embodiment, the deep learning architecture, whether it is an encoder-only, decoder-only or encoder-decoder architecture, needs to be pre-trained to determine the weights and parameters.

[0078] The training process specifically includes:

[0079] Step 1: Data preparation: First, collect a large and diverse code corpus, including open source code and self-built high-quality data sets, and then filter and screen the data sets to remove duplicate and low-quality code snippets to ensure the high quality of the training data set.

[0080] In this step, high-quality pre-training data is formed by crawling Github data, Stack Overflow and StarCoder datasets, as well as a small amount of high-quality Chinese and English data.

[0081] Step 2: Model architecture selection: Select a suitable model architecture according to the task requirements, such as encoder-only, decoder-only or encoder-decoder. This embodiment prefers to select a model architecture with high computational efficiency and moderate number of parameters.

[0082] Step 3: Pre-training: Use a large-scale code corpus for pre-training to learn programming knowledge and patterns. The pre-training stage usually uses unsupervised learning methods, such as language model training, so that the model can effectively understand various programming languages ​​and scenarios.

[0083] Step 4: Fine-tuning: Based on the pre-trained model, supervised learning is performed using a specific fine-tuning dataset. The fine-tuning dataset is built based on actual software development tasks and problems, and is designed to enable the model to better understand and respond to the specific needs of developers. Through the fine-tuning process, the model can gradually adapt to specific software engineering tasks, such as code generation, defect detection, etc.

[0084] Step 5: Optimization and evaluation: During the training process, continuously optimize the model's hyperparameters, such as learning rate, batch size, etc., to improve the model's performance. Regularly evaluate the model's performance, such as evaluating it on a validation set or test set, to ensure the model's effectiveness and generalization ability.

[0085] The weight and parameter determination process specifically includes:

[0086] Step 1: Initialization: Before training begins, you need to set initial values ​​for the model's weights and parameters. This can be achieved by random initialization, pre-trained model loading, etc.

[0087] Step 2: Forward propagation and back propagation: During the training process, the model calculates the predicted value through forward propagation and compares it with the true value to get the error. Then, the gradient is calculated through the back propagation algorithm, and the weights and parameters of the model are updated to reduce the error.

[0088] Step 3: Gradient Descent and Optimizer: Update the weights and parameters of the model using the gradient descent algorithm and its variants such as Adam, RMSprop, etc. The optimizer is responsible for adjusting the value of each parameter based on the calculated gradient to minimize the loss function.

[0089] Step 4: Iteration and convergence: Through multiple iterations of training, the weights and parameters of the model gradually converge to the optimal solution. This is usually manifested as the loss function value gradually decreasing and stabilizing, while the performance of the model on the validation set is improved.

[0090] Step 5: Save and Load: After training is complete, the model's weights and parameters can be saved to a file for subsequent use or deployment. When needed, the saved weights and parameters can be loaded to restore the model's state.

[0091] In this example, only 500 billion tokens were trained to achieve excellent performance, thanks to its efficient training architecture and algorithm optimization, such as deep customization based on Megatron-LM and Flash Attention2 acceleration.

[0092] The deep learning-based code generation method provided in this embodiment can also perform automatic code completion, error checking, and refactoring suggestions:

[0093] 1. Code auto-completion

[0094] Context-aware: By analyzing the context of the code, including variable names, function names, class names, and the relationships between them, it predicts the code that may be written next. For example, when a developer enters a function name and adds parentheses, the parameter list of the function will be automatically completed.

[0095] Combining syntax and semantics: In addition to considering the grammatical structure of the code, semantic information is also used to provide more accurate completion suggestions. By understanding the intent and logic in the code, it is able to recommend the code snippets that are most relevant to the current context.

[0096] Learning and Optimization: As usage time increases, the model will continue to learn and optimize its auto-completion capabilities. By collecting user feedback and usage data, it can gradually adjust its prediction model to provide completion suggestions that are more in line with developer habits. It has high accuracy and efficiency in code generation and completion.

[0097] 2. Error Checking

[0098] Real-time checking: It can perform real-time error checking while developers are writing code. This means that it can immediately find and highlight grammatical errors, spelling errors or other potential problems, so that developers can correct them in time.

[0099] Type inference and verification: With powerful type inference capabilities, the type of a variable can be predicted and verified when necessary. If a developer tries to assign a value of the wrong type to a variable, a warning or error message will be issued.

[0100] Logical error detection: In addition to syntax and type errors, logical errors can also be detected to a certain extent. For example, by analyzing the execution path and conditional judgment of the code, developers can be prompted for possible logical loopholes or inconsistencies.

[0101] 3. Refactoring Suggestions

[0102] Code quality assessment: Ability to assess the quality of code, including complexity, readability, maintainability, etc. Based on these assessment results, targeted refactoring suggestions can be provided to help developers improve code structure.

[0103] Duplicate code detection and elimination: It can detect duplicate code segments in the code and suggest developers to merge or abstract them into reusable functions or classes. This helps reduce code redundancy and improve code maintainability and reusability.

[0104] Best Practice Recommendations: Based on the best practices of programming languages ​​and development frameworks, we recommend more elegant and efficient code implementations for developers. This helps developers learn and master the best programming practices in the industry.

[0105] The code generation method based on deep learning provided in this embodiment has the following advantages:

[0106] (1) It does not require understanding of a large amount of code, can significantly improve the accuracy and efficiency of code generation, and is low-cost.

[0107] (2) Supports multiple programming languages, such as Python, Java, C#, C++, JavaScript, HTML, CSS, SQL, etc., providing developers with extensive programming support.

[0108] (3) It has high accuracy and efficiency in code generation and completion.

[0109] (4) It can detect potential security risks in the code, such as SQL injection, cross-site scripting attacks, etc., and provide repair suggestions to improve the security of the code.

[0110] (5) It can automatically generate code comments to provide relevant information about code functions, parameters, and return values. At the same time, it can also conduct in-depth analysis of the code, detect potential errors, redundant code, and performance bottlenecks, and provide optimization suggestions.

[0111] Embodiment 2

[0112] This embodiment provides a code generation device based on deep learning, including:

[0113] A data receiving module, used for receiving code data input by a user;

[0114] A standardization processing module, used for performing standardization processing on the code data to obtain standardized code data;

[0115] The encoding module is used to: =segment the standardized code data into morphemes and encode them to obtain integer indexes;

[0116] An embedding module, configured to input the integer index into an embedding layer to obtain a fixed-dimensional vector;

[0117] An encoder module is used to input the fixed-dimensional vector into a pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain the long-distance dependency in the fixed-dimensional vector, and obtains the feature vector sequence through feature transformation and aggregation operations;

[0118] A code generation module is used to generate code according to the feature vector sequence.

[0119] For the specific implementation content of each module in a code generation device based on deep learning, please refer to the above definition of a code generation method based on deep learning, which will not be repeated here.

[0120] Embodiment 3

[0121] This embodiment provides a deep learning-based code generation system (referred to as: intelligent code assistant), including a front-end interface and a back-end service. The front-end interface and the back-end service communicate through an application layer protocol (HTTP request or WebSocket). The back-end service includes a deep learning-based code generation device provided in Example 2.

[0122] Specifically, the front-end interface is a window for users to interact with the system, usually including code editors, execution buttons, result display areas, etc. The front-end interface is implemented through Web technologies (such as HTML, CSS, or JavaScript) and is located in the browser or IDE plug-in of the user's device.

[0123] The backend service is the core processing unit of the system, responsible for processing requests sent by the frontend, performing tasks such as code analysis, generation, and completion, and returning the results to the frontend interface. The backend service may be deployed on a cloud server or run on a local machine.

[0124] The intelligent functions of code assistants rely on pre-trained large code models, which are located at the core of backend services and are the key to performing tasks such as code analysis and generation. Models and algorithms are usually encapsulated in specific libraries, such as PyTorch, TensorFlow, etc., and interact with backend services through API interfaces.

[0125] The deep learning-based code generation system provided in this embodiment also includes a database and cache component for storing user data, model parameters, historical records, etc. The database and cache work closely with the backend service to provide data persistence and fast access capabilities.

[0126] The intelligent code assistant provided in this embodiment has the characteristics of lightweight, localized deployment, IDE plug-in support, and multi-task evaluation system:

[0127] Lightweight design: Code Assistant supports 4-bit / 8-bit quantization. The quantized model consumes less memory and is faster while ensuring performance. For example, the 4-bit quantized version of Code Assistant can also run efficiently on GPUs with smaller video memory.

[0128] Localized deployment: Code Assistant provides a C / C++ version of the reasoning interface, which supports developers to use it on personal computers without GPUs. This localized deployment method not only ensures data security, but also enables developers to use Code Assistant for programming assistance anytime and anywhere.

[0129] IDE plug-in support: Code Assistant provides plug-in support for mainstream development environments such as VS Code and JetBrains IDE, allowing developers to directly use the functions of Code Assistant in a familiar development environment.

[0130] Multi-task evaluation system: Code Assistant provides a multi-task evaluation system that supports common software development activities such as complete project context, coverage code generation, code defect detection and repair, and test case generation, providing developers with comprehensive software development support.

[0131] The deep learning-based code generation system provided in this embodiment also has the following advantages:

[0132] (1) Real-time: The code assistant uses real-time communication technologies such as WebSocket to achieve instant interaction between the front-end and the back-end. After the user enters the code, he can immediately see the execution results or completion suggestions, which improves programming efficiency.

[0133] (2) Intelligence: The code assistant uses a pre-trained code model to implement intelligent functions such as automatic code completion, error checking, and refactoring suggestions. These functions can significantly reduce the burden on developers and improve code quality.

[0134] (3) Scalability: The architecture design of the code assistant takes scalability into consideration. As technology develops and needs change, new models, algorithms, or functional components can be easily added to meet programming needs in more scenarios.

[0135] Code Assistant provides developers with efficient and intelligent programming support through its powerful code generation and completion capabilities, intelligent programming assistance, efficient data management and training, lightweight, localized deployment, and comprehensive ecosystem (IDE plug-in support and multi-task evaluation system). These technical effects not only improve developers' programming efficiency and quality, but also promote the intelligent development of software engineering.

[0136] In the code assistant, each component works closely together and interacts with each other to realize the operation of the overall function. The front-end interface provides an entry for user interaction and a window for displaying results; the back-end service, as the core processing unit, is responsible for receiving front-end requests, calling model algorithms, processing data and returning results; models and algorithms provide intelligent code analysis and generation capabilities; databases and caches provide support for data persistence and fast access. These components work together and form the code assistant, an efficient and intelligent programming support platform.

[0137] The technical features of the above embodiments may be arbitrarily combined (as long as there is no contradiction in the combination of these technical features). To make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A code generation method based on deep learning, characterized in that: The deep learning-based code generation method is applied to backend services, including: Step 1: Receive code data input by the user; Step 2: Standardizing the code data to obtain standardized code data; Step 3: Segment the standardized code data into morphemes and encode them to obtain integer indexes; Step 4: Input the integer index into the embedding layer to obtain a fixed-dimensional vector; Step 5: Input the fixed-dimensional vector into a pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain the long-distance dependency relationship in the fixed-dimensional vector, and obtains a feature vector sequence through feature transformation and aggregation operations; Step 6: Generate code according to the feature vector sequence.

2. The code generation method based on deep learning according to claim 1, characterized in that: After step 5, also include: Input the feature vector sequence into the decoder to calculate the attention weight, and generate a context vector according to the attention weight; The next feature vector is predicted according to the context vector and the feature vector, and the probability distribution of each feature vector is calculated according to the activation function, so as to obtain a complete code sequence.

3. The code generation method based on deep learning according to claim 2, characterized in that: The activation function is a softmax activation function.

4. The code generation method based on deep learning according to claim 1, characterized in that: In step 2, the standardization process includes removing comments, spaces, and line breaks.

5. The code generation method based on deep learning according to claim 1, characterized in that: In step 5, the feature transformation and aggregation operations include linear transformation, nonlinear activation function and layer normalization.

6. The code generation method based on deep learning according to claim 1, characterized in that: Also includes: An evaluation result of a user on the generated code is received, and the encoder is optimized according to the evaluation result.

7. The code generation method based on deep learning according to claim 1, characterized in that: It also includes code completion, error checking, and refactoring suggestions.

8. A code generation device based on deep learning, characterized in that: include: A data receiving module, used for receiving code data input by a user; A standardization processing module, used for performing standardization processing on the code data to obtain standardized code data; The encoding module is used to: =segment the standardized code data into morphemes and encode them to obtain integer indexes; An embedding module, configured to input the integer index into an embedding layer to obtain a fixed-dimensional vector; An encoder module is used to input the fixed-dimensional vector into a pre-trained encoder to obtain a feature vector sequence; the encoder first uses a self-attention mechanism to obtain the long-distance dependency in the fixed-dimensional vector, and obtains a feature vector sequence through feature transformation and aggregation operations; A code generation module is used to generate code according to the feature vector sequence.

9. A code generation system based on deep learning, characterized in that: It includes a front-end interface and a back-end service, wherein the front-end interface and the back-end service communicate with each other via an application layer protocol, the front-end interface is implemented via Web technology and is located in a browser or IDE plug-in of a user device, and the back-end service includes a deep learning-based code generation device as described in claim 8.

10. The deep learning-based code generation system according to claim 9, characterized in that: The web technology is HTML, CSS or JavaScript 。

Images