Code generation method and related device based on grammar discriminator

Abstract

Existing code generation models rely on learning the semantic association between natural language descriptions and codes to generate code, and lack explicit constraints on the grammatical structure of the code, which may lead to grammatical errors in the generated code. In order to solve the problem of grammatical errors in the code, some methods perform grammatical corrections on the generated code through post-processing, but post-processing is an independent step, separated from the code generation model itself, and cannot directly affect the generation ability of the model during the code generation process. In addition, most post-processing methods are based on static rules, which may be difficult to adapt to complex code structures or flexible grammars. In this regard, the present application discloses a code generation method and related equipment based on a grammar discriminator, which includes: inputting the generation description information corresponding to the code to be generated into the code generation model, and obtaining the target code output by the code generation model. In the present application, grammatically accurate code is generated by the code grammar optimization function of the code generation model.

Description

Technical Field

[0001] The present application belongs to the technical field of code generation, and specifically relates to a code generation method based on a grammar discriminator and related equipment. Background Art

[0002] Automatic code generation is of great significance to improving software development efficiency and software quality.

[0003] In the exemplary technology, the description information of the code to be generated is input into the model, and the code generated by the model can be obtained.

[0004] However, the descriptive information input into the model is in natural language, while the code contains clear and rich information about the programming language and control logic structure, that is, the code has a grammatical structure. The model generates code based on this descriptive information, but the descriptive information does not contain the code's structural information. This may lead to grammatical errors in the code generated by the model, resulting in low accuracy. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a code generation method and related devices based on a grammar discriminator to solve the problem of low accuracy of the generated code.

[0006] In a first aspect, an embodiment of the present application provides a code generation method based on a grammar discriminator, comprising:

[0007] Get the generation description information corresponding to the code to be generated;

[0008] The generation description information is input into a code generation model to obtain a target code output by the code generation model, wherein the code generation model has a code syntax optimization function.

[0009] In some possible implementations, before obtaining the generation description information corresponding to the code to be generated, the method further includes:

[0010] Acquire a plurality of first training samples, where the first training samples include training codes and corresponding code description information;

[0011] The preset model is trained according to each of the first training samples to obtain a code generation model with code syntax optimization function.

[0012] In some possible implementations, the preset model includes a code generation module, a sequence extraction module, a loss calculation module, and a grammar discriminator, wherein:

[0013] The code generation module is used to generate a hidden state feature vector according to the code description information of the training code;

[0014] The sequence extraction module is configured to perform linear sequence extraction on the training code to obtain a target abstract syntax tree sequence, and to parse the hidden state feature vector to obtain a generated abstract syntax tree sequence;

[0015] The loss calculation module is configured to determine an abstract syntax tree loss parameter according to the target abstract syntax tree sequence and the generated abstract syntax tree sequence;

[0016] The grammar discriminator is used to determine a grammatical correctness score of the hidden state feature vector based on the abstract syntax tree loss parameter and the hidden state feature vector, and the grammatical correctness score is used by the code generation module to learn code grammar optimization.

[0017] In some possible implementations, the code generation method based on a grammar discriminator further includes:

[0018] Training the grammar discriminator according to second training samples corresponding to each training code, wherein the second training samples include an abstract syntax tree loss parameter and a hidden state feature vector corresponding to the training code;

[0019] When the network parameters of the grammatical discriminator no longer converge, a grammatical correctness score corresponding to the current second training sample is generated according to the current second training sample.

[0020] In some possible implementations, the loss calculation module is configured to determine the abstract syntax tree loss parameter according to an edit distance between the target abstract syntax tree sequence and the generated abstract syntax tree sequence.

[0021] In some possible implementations, after the code generation module updates its parameters based on the grammatical correctness score, the cross-entropy loss parameter corresponding to the code generation module is determined according to the hidden state feature vector, and the code generation module after the parameter update is updated according to the cross-entropy loss parameter.

[0022] In some possible implementations, the code generation module is a pre-trained model obtained by training with a plurality of the first training samples.

[0023] In a second aspect, an embodiment of the present application provides a code generation device, comprising:

[0024] An acquisition module is used to obtain generation description information corresponding to the code to be generated;

[0025] An input module is used to input the generation description information into a code generation model to obtain the target code output by the code generation model, wherein the code generation model has a code syntax optimization function.

[0026] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0029] In an embodiment of the present application, by obtaining generation description information of the code to be generated and inputting the generation description information into a code generation model, a target code output by the code generation model is obtained. In the present application, the code generation model has a code syntax optimization function. When generating code, the code generation model optimizes the grammatical structure of the code, thereby improving the correctness of the grammatical structure of the generated code, improving the accuracy of code generation, and thus improving the quality of the generated code. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is one of the flow charts of a code generation method based on a grammar discriminator according to an exemplary embodiment;

[0031] Figure 2 This is a second flow chart of a code generation method based on a grammar discriminator according to an exemplary embodiment;

[0032] Figure 3 This is a diagram illustrating a training process of a code generation model according to an exemplary embodiment. Figure 1 ;

[0033] Figure 4 This is a diagram illustrating a training process of a code generation model according to an exemplary embodiment. Figure 2 ;

[0034] Figure 5 This is a diagram illustrating a training process of a code generation model according to an exemplary embodiment. Figure 3 ;

[0035] Figure 6 This is a diagram illustrating a training process of a code generation model according to an exemplary embodiment. Figure 4 ;

[0036] Figure 7is a structural block diagram of a code generating device according to an exemplary embodiment;

[0037] Figure 8 The figure is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0039] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally refers to before and after.

[0040] Automated code generation technology is of great significance to improving development efficiency and software quality. Automated code generation tasks involve inputting code generation requirements described in natural language into a code generation model, which then automatically generates the corresponding programming language code.

[0041] Automated code generation can be viewed as a translation task similar to natural language to programming language. Programming language code, like natural language, is highly repeatable and predictable, and possesses statistical properties similar to natural language, which can be captured using statistical language models. Therefore, general-purpose large language models from the field of natural language processing are used for automated code generation. However, unlike natural language, programming code contains more explicit and rich information about programming language syntax and control logic structures. This structural information is crucial for the syntactic correctness of model-generated code.

[0042] In recent years, code generation technology based on large-scale pre-trained models has made significant progress. Code generation models based on deep learning, especially those based on large-scale pre-trained models, have shown great potential in various code generation scenarios. However, existing deep learning-based code generation models primarily rely on learning the semantic associations between natural language descriptions and code to generate code, lacking explicit constraints on the code's grammatical structure. This can lead to syntax errors in the generated code, such as missing necessary brackets, mismatched indentation, or broken grammatical rules. These errors not only directly affect the compilability of the code but can also significantly increase the cost of subsequent debugging and modification, resulting in low practical usability of the generated code. To alleviate the problem of syntax errors in code generation models, post-processing can be used to correct the syntax of the generated code. For example, some methods use code formatting tools or rule-based syntax checkers to check and modify the generated results. However, these post-processing methods have limitations: first, post-processing is an independent step, separate from the code generation model itself, and cannot directly affect the model's generation capabilities during the code generation process; second, most of these post-processing methods are based on static rules and may not be adaptable to complex code structures or flexible syntax. In this regard, this application proposes a code generation method and system that combines a grammar discriminant network to support the generation of high-quality and high-availability code, so as to solve the problem that the existing deep learning-based code generation model lacks explicit constraints on the code grammatical structure and cannot guarantee grammatical correctness during the generation process.

[0043] In order to solve the above problems, the present application proposes a code generation method based on a grammar discriminator, which obtains the generation description information of the code to be generated, inputs the generation description information into a code generation model, and obtains the target code output by the code generation model. The code generation model has a code grammar optimization function. When generating code, the code generation model optimizes the grammatical structure of the code, thereby improving the correctness of the grammatical structure of the generated code, improving the accuracy of code generation, and thus improving the quality of the generated code. In addition, the code generation model has a code grammar optimization function, so that during the code generation process, the code generation model optimizes the grammatical structure of the code based on the grammar optimization function, so that the code generation model directly generates grammatically correct code without the need to correct the grammatical structure of the generated code through post-processing. Moreover, since the code generation model has a code grammar optimization function, even if the code structure of the code required to generate the description information is complex, the code generation model can generate code with a correct grammatical structure, thereby supporting the generation of code with a complex code structure. That is, the code generation method based on a grammar discriminator of the present application can generate high-quality and grammatically correct code.

[0044] The following describes in detail the code generation method based on the grammar discriminator and related devices provided in the embodiments of the present application with specific embodiments.

[0045] The following is a detailed description of the code generation method based on the grammar discriminator provided in the embodiment of the present application.

[0046] Reference Figure 1 , Figure 1 One of the flow charts of the code generation method based on grammar discriminator provided in this application is as follows: Figure 1 As shown, the code generation method based on the grammar discriminator includes the following steps:

[0047] Step S101: Obtain generation description information corresponding to the code to be generated.

[0048] In this embodiment, the execution subject is a code generating device, which can be a device, system or server for generating code. For ease of description, the code generating device is referred to as a device below.

[0049] When the user needs to generate class code, he inputs the generation description information corresponding to the code to be generated into the device. The code to be generated includes but is not limited to class code, which refers to class-level code, which includes multiple member methods, member variable definitions, and relationships between classes.

[0050] The generation description information refers to a textual description of the code to be generated in natural language. Natural language is different from programming language. Natural language refers to a language composed of text. That is, the user uses text to describe the code to be generated to obtain the generation description information. The generation description information can be input into the device by the user.

[0051] Step S102 : inputting the generation description information into a code generation model to obtain a target code output by the code generation model, wherein the code generation model has a code syntax optimization function.

[0052] The device is provided with a code generation model, which is a model for learning the syntax structure of the code, that is, the code generation model has a code syntax optimization function. The code generation model makes the syntax structure of the generated code correct through the code syntax optimization function.

[0053] After obtaining the generation description information, the device inputs the generation description information into a code generation model, which then generates preliminary code based on the generation description information. In one example, because the code generation model has a code syntax optimization function, the code syntax optimization function is used to perform syntax optimization on the preliminary generated code, and the syntax optimization corrects the grammatical structure of the preliminary generated code, thereby generating target code with a correct grammatical structure. In another example, the code generation model can generate target code with a correct grammatical structure based on the syntax optimization function and the generation description information.

[0054] In this embodiment, generation description information of the code to be generated is obtained and input into a code generation model to obtain target code output by the code generation model. In this embodiment, the code generation model has a code syntax optimization function. When generating code, the code generation model optimizes the grammatical structure of the code, thereby improving the grammatical correctness of the generated code, improving the accuracy of code generation, and thus improving the quality of the generated code.

[0055] Reference Figure 2 , Figure 2 This is the second flow chart of the code generation method based on the grammar discriminator provided by this application, based on Figure 1 In the embodiment shown, before step S101, the following steps are further included:

[0056] Step S201: Acquire a plurality of first training samples, where the first training samples include training codes and corresponding code description information.

[0057] In this embodiment, the device can also be used to train a code generation model. The device obtains multiple training samples, which are defined as first training samples. The first training sample includes training code and code description information corresponding to the training code. The training code is grammatically correct code, and the code description information includes a textual description of the training code, which is a natural language description. In addition, the first training sample may also include parsed information about the grammatical structure of the training code.

[0058] Step S202: training a preset model according to each first training sample to obtain a code generation model with code syntax optimization function.

[0059] After obtaining each first training sample, the preset model is trained to obtain a generation model with code syntax optimization function. The preset model can be a deep learning model. The preset model can be set with a model structure for parsing the training code and a structure for generating the code. The structure for generating the code generates the code based on the generation description information. The parsing structure parses the grammatical structure of the generated code. The parsed grammatical structure is compared with the parsing information in the first training sample. The parameters of the code generation structure are updated based on the comparison information, so that the code generation structure is updated, thereby enabling the code generation structure to learn the correct grammatical structure of the code; in addition, the code generated by the code generation structure is compared with the training code in the first training sample to generate a loss function, and the model parameters of the code generation structure are updated through the loss function.

[0060] In this embodiment, the device trains a preset model using a plurality of first training samples to obtain a code generation model with a code syntax optimization function.

[0061] In one embodiment, referring to Figure 3 ,The preset model includes a code generation module, a sequence extraction module, a loss calculation module, and a grammar discriminator.

[0062] The code generation module is used to generate a hidden state feature vector h according to the code description information corresponding to the training code;

[0063] The sequence extraction module is used to linearize the training code to extract the target AST (Abstract Syntax Tree) sequence, and parse the hidden feature state vector to generate the AST sequence;

[0064] The loss calculation module is used to determine the AST loss parameters based on the target AST sequence and the generated AST sequence;

[0065] The syntax discriminator is used to determine the syntax correctness score of the hidden feature vector based on the AST loss parameter and the hidden state feature vector. The syntax correctness score is used for learning code syntax optimization in the code generation module.

[0066] Specifically, the code description information of the first training sample is input into the code generation module, and the code generation module generates a hidden state feature vector based on the code description information to characterize the generated code. The generated code and the training code are input into the sequence extraction module, and the sequence extraction module performs linear sequence extraction on the generated code to obtain a generated AST sequence, and parses the hidden state feature vector and linearizes the sequence extraction to obtain a target AST sequence. The loss calculation module calculates the AST loss parameter based on the difference between the target AST sequence and the generated AST sequence. The AST loss parameter and the hidden state feature vector serve as inputs to the grammar discriminator. The grammar discriminator determines the grammatical correctness score of the hidden state feature vector based on the abstract grammar loss parameter and the hidden state feature vector, that is, the grammar discriminator is used to determine the grammatical correctness of the generated code. The grammar discriminator feeds back the grammatical correctness score to the code generation module, so that the code generation module generates and learns the correct grammatical structure.

[0067] In this embodiment, the code generation module in the preset model is provided with a code syntax optimization function through a syntax discriminator, so that the trained code generation model can generate code with a correct grammatical structure.

[0068] In one embodiment, the code generation module is trained with a plurality of first training samples to obtain a pre-trained model.

[0069] Exemplarily, each first training sample constitutes a <NL description, training code> training set. The NL description is the code description information, and the training code can be class-level code or other types of code. The device uses a pre-trained code model as the pre-training model. The pre-trained code model is, for example, the CodeT5 model. The pre-trained code model generates a hidden state feature vector h through the code description information, and calculates a loss function based on the hidden state feature vector h and the target code sequence (the target AST sequence extracted from the training code by the sequence extraction module), so as to update the parameters of the pre-trained code model based on the loss function.

[0070] In this embodiment, the device inputs the <NL description, training code> training set to the pre-training model, and obtains the pre-trained code model (CodeT5) with the fine-tuned and saved parameter being 1 as the code generation module. This process performs preliminary code generation fine-tuning training on the pre-trained code model with an encoder-decoder architecture on the <NL description, training code> training set, enabling the code generation module to have a certain code generation ability, providing a basis for the subsequent training of the syntax discriminator and the optimization of the code generation model itself.

[0071] In one embodiment, the loss calculation module determines the abstract syntax tree loss parameter based on the edit distance between the target abstract syntax tree sequence and the generated abstract syntax tree sequence.

[0072] Exemplarily, in order to quantify the syntactic difference between the generated code and the training code, a loss calculation module is designed for the preset model. The input of the loss calculation module is the target AST sequence and the generated AST sequence, and the output of the loss calculation module is the normalized edit distance between the two. The normalized edit distance is the syntactic loss of the abstract syntax tree, which is used as the abstract syntax tree loss parameter. The range of the abstract syntax tree loss parameter is [0, 1]. The closer the value is to 0, the closer the structures and syntactic information of the two AST sequences are; the closer the value is to 1, the greater the difference between the two.

[0073] The edit distance measures the similarity in syntactic structure between the generated code and the training code by calculating the minimum number of edit operations between the two AST sequences. The edit distance can measure and quantify the difference degree between two strings, that is, taking words as the minimum edit unit and applying it to the difference comparison between AST sequences.

[0074] In this embodiment, in order to effectively quantify the syntactic differences between codes and provide feedback information for the code generation model, the edit distance is used to measure the similarity of the code in the syntactic structure by calculating the minimum number of edit operations between two AST sequences. Compared with the traditional syntactic discrimination method, this embodiment establishes a connection between the AST and the syntactic discrimination of the code, reflects the differences in the syntactic structure information between codes, and can not only determine the syntactic rationality, but also provide a fine-grained difference quantification result, further guiding the optimization of the class-level code generation task.

[0075] In one embodiment, the syntactic discriminator is trained according to the second training samples corresponding to each training code. The second training samples include the abstract syntax tree loss parameters and hidden state feature vectors corresponding to the training codes; in the case where the network parameters of the syntactic discriminator no longer converge, according to the current second training samples, a syntactic correctness score corresponding to the current second training samples is generated.

[0076] Specifically, the syntactic discriminator can be trained through each second training sample. After the training is completed, the syntactic correctness score corresponding to the current second training sample can be generated, enabling the code generation module to learn the syntactic structure optimization based on the syntactic correctness score.

[0077] Exemplarily, referring to Figure 4 , the input is the training data for the class-level code generation task, that is, the class-level generation task training data set composed of data pairs in the form of <NL description, class-level code>, and the output is the syntactic discriminator training data set composed of data pairs in the form of <decoder hidden state h, AST loss>. The data set generated through this process can be used to train the syntactic discriminator alone, effectively separating the training process of the syntactic discriminator and avoiding the negative impact of the complex decoding and parsing operations in the direct calculation of the syntactic loss on the overall training efficiency of the code generation model. This process converts the syntactic correctness score (i.e., the AST loss) into an optimization target that can be learned by the code pre-training code generation model, avoiding the non-differentiable problem faced by the direct use of the AST edit distance calculation, so that the non-differentiable syntactic loss can be converted into a differentiable optimization signal through the prediction ability of the syntactic discriminator neural network.

[0078] In this data processing and collection process, first, use the AST sequence extraction algorithm of the sequence extraction module to obtain the linearized target code AST sequence from the code sequence of the original training data, and use the pre-trained code model (parameters are 1) that has completed preliminary fine-tuning to perform an evaluation process on the data set for one epoch (set the model to the eval mode). During this evaluation process, obtain the hidden state feature vector h output by the decoder end of the pre-trained code model. The shape of the hidden state feature vector h is ( ) of the three-dimensional tensor ( ), two operations are performed on the hidden state feature vector h. One is The average pooling operation of the dimension is obtained as follows ( ) is cached into the decoder hidden state h in the pair <decoder hidden state h, AST loss>; the other performs batch decoding operations and parses them into ASTs, which are converted into linear generated code AST sequences using the sequence extraction algorithm of the sequence extraction module. Next, the loss calculation module compares the generated AST sequence with the AST of the target code, obtaining a grammatical loss with a value of [0, 1]. This serves as the label data for training the grammatical discriminator and is cached into the AST loss in the pair <decoder hidden state h, AST loss>. Finally, the cached <decoder hidden state h, AST loss> data pairs obtained after the evaluation are used as the training dataset for the grammatical discriminator.

[0079] Reference Figure 5 , the grammatical discriminator is designed as a three-layer fully connected neural network. The input layer FC receives the hidden state h and the target AST loss; the hidden layer BN ( ) is a two-layer fully connected layer, and each layer is used And the activation function, the activation function can be (Linear rectification function); the output layer is a linear transformation that maps the hidden layer features to a scalar value, representing the predicted AST loss value. Hidden state vector The forward propagation process in the grammatical discriminator is shown in the following formula.

[0080]

[0081] in, For the grammatical discriminator, the hidden state vector The grammatical loss of the predicted output, For the The weights of the layers, Indicates the The bias value of the layer, for Normalization operation, is the activation function.

[0082] The grammatical loss prediction training process of the grammatical discriminator and the components used are as follows Figure 5 As shown, the input is the grammatical discriminator training set consisting of <decoder hidden state h, AST loss> data pairs, and the output is the predicted grammatical loss And a syntax discriminator with the ability to predict syntax loss and with the saved parameter being W1. After the training of this process, the syntax discriminator has the ability to predict the corresponding syntax loss by using the hidden output state h of the decoder of the pre-trained code model. Thus, in the subsequent training process of the code generation model, the feedback signal generated by the syntax discriminator predicting the syntax loss can be used to guide the optimization of the pre-trained code model, providing an explicit constraint for syntax correctness guarantee.

[0083] During the process of training the syntax discriminator to learn to predict syntax loss, the loss function Loss is determined through the Mean Squared Error (MSE) function perdictor , and the loss function is as shown in the following formula:

[0084]

[0085] Where is the batch size, is the AST loss predicted for the th sample, is the AST loss label (i.e., the target AST loss) of the i-th sample.

[0086] In this embodiment, in order to convert the syntax correctness score (i.e., the predicted AST loss value) into an optimization target that can be learned by the pre-trained model and apply it to the training process of the code generation model, this embodiment proposes a syntax discriminator network and its syntax loss prediction training process. Training the syntax discriminator to have the ability to predict the corresponding syntax loss by using the hidden output state h of the decoder of the pre-trained code model. By training the prediction ability of the syntax discriminator neural network, the non-differentiable and non-learnable syntax loss is converted into a differentiable and learnable optimization signal, so that in the training process of the code generation model, the feedback signal generated by the syntax discriminator predicting the syntax loss can be used to guide the optimization of the pre-trained code model, providing an explicit constraint for syntax correctness guarantee.

[0087] In one embodiment, after the code generation module updates the parameters based on the syntax correctness score, the cross-entropy loss parameter of the code generation module is determined based on the hidden state feature vector, and then the code generation module with updated parameters is updated based on the cross-entropy loss parameter.

[0088] In this embodiment, referring to Figure 6 , during the training process of the code generation module, the input is a training set composed of <NL description, class-level code> data pairs (the first training sample), and the output is a code generation module with the explicit guarantee ability of syntax correctness and with the saved parameter being 2 pre-trained code model (post-trained code generation module). This process uses a syntax discriminator on a class-level code training dataset to perform two learning tasks: AST loss prediction and code generation. This training phase trains the pre-trained code model, which has undergone preliminary fine-tuning, so that the code generation model learns to generate code with guaranteed grammatical correctness. During this process, the syntax discriminator's parameter W1 remains frozen because the syntax discriminator has been independently trained and has the ability to predict syntax loss using the decoder hidden vector h. The syntax discriminator generates a syntax loss feedback signal based on the pre-trained code model's decoder hidden vector h, guiding the optimization of the pre-trained code model towards code generation with correct grammar.

[0089] After fine-tuning, the pre-trained code model is trained using the syntax discriminator as a bridge, transforming the grammatical correctness score (i.e., the predicted AST loss) into a learnable optimization objective for the pre-trained code model. This cleverly circumvents the non-differentiability issues faced by directly calculating and using the AST edit distance. Leveraging the predictive power of the syntax discriminator's neural network, the non-differentiable syntax loss is transformed into a differentiable optimization signal. Furthermore, the syntax score provided by the syntax discriminator acts as a soft constraint, combined with the objective of the code generation task, forming a multi-task learning training paradigm.

[0090] AST loss and code generation loss have different back propagation paths, which makes it impossible to directly weight the two. When back propagating, AST loss needs to first flow through the syntax discriminator network, and then pass the syntax score to the parameters of the pre-trained code model to achieve syntax-related optimization goals. The code generation loss acts directly on the pre-trained code model parameters based on the generation results of the target sequence. Therefore, these two losses not only have different action paths, but also have significant differences in the semantics and optimization goals of their gradients. Directly weighting the combination of the two losses will destroy the independence and logical consistency of the optimization process, which may lead to instability in model training or performance degradation. Therefore, for these two learning tasks, this embodiment introduces a staged gradient back propagation method to achieve collaborative training of syntax optimization and generation optimization, and converts the syntax loss ( ) and code generation loss ( ) are used sequentially to update the model parameters in the same training batch, rather than the traditional weighted combined loss function, making full use of the independent optimization effects of the two losses.

[0091] During the training process, the hidden state output by the code generation module is , then in the first stage, the parameters W1 of the syntax discriminator are frozen, and the parameters of the code generation module are optimized using the predicted AST loss; in the second stage, the loss of the code generation task (cross entropy loss) is used to optimize the model parameters. This process can be expressed by the following formula.

[0092]

[0093]

[0094] in, is the hidden state feature vector output by the code generation module, Perform an average pooling operation on it to convert the three-dimensional tensor into two-dimensional and then input it into the grammar discriminator ,pass The parameters of the gradient update code generation module .

[0095]

[0096]

[0097] in, is the predicted probability distribution of the token generated by the model at time t, is the actual token in the target sequence at time t, is the length of the sequence. Use code to generate tasks The parameters of the gradient update code generation module .

[0098] In this embodiment, to introduce grammatical correctness assurance into the code generation process based on a deep learning code generation model, a two-stage code generation training process based on dual-task learning, combined with a grammatical discriminator, is proposed. Through a phased gradient backpropagation approach, collaborative training of grammatical optimization and code generation optimization is achieved. The grammatical loss predicted by the grammatical discriminator and the code generation loss are sequentially used to update model parameters within the same training batch, rather than using a traditional weighted combined loss function. This fully utilizes the independent optimization effects of the two losses, effectively training a code generation model capable of ensuring grammatical correctness.

[0099] Based on the same inventive concept, the present application also provides a code generation device. Figure 7 The code generation device provided in the embodiment of the present application is described in detail.

[0100] Figure 7 The figure is a structural block diagram of a code generating device according to an exemplary embodiment.

[0101] like Figure 7 As shown, the code generating device 700 may include:

[0102] An acquisition module 710 is used to acquire generation description information corresponding to the code to be generated;

[0103] The input module 720 is used to input the generation description information into the code generation model to obtain the target code output by the code generation model, wherein the code generation model has a code syntax optimization function.

[0104] In one embodiment, the code generating device 700 is further configured to:

[0105] Acquire a plurality of first training samples, where the first training samples include training codes and corresponding code description information;

[0106] The preset model is trained according to each first training sample to obtain a code generation model with code syntax optimization function.

[0107] In one embodiment, the preset model includes a code generation module, a sequence extraction module, a loss calculation module and a grammar discriminator, wherein:

[0108] A code generation module is used to generate a hidden state feature vector based on the code description information of the training code;

[0109] The sequence extraction module is used to extract the linear sequence of the training code to obtain the target abstract syntax tree sequence, and parse the hidden state feature vector to obtain the generated abstract syntax tree sequence;

[0110] A loss calculation module, configured to determine an abstract syntax tree loss parameter based on a target abstract syntax tree sequence and a generated abstract syntax tree sequence;

[0111] The syntax discriminator is used to determine the syntax correctness score of the hidden state feature vector based on the abstract syntax tree loss parameter and the hidden state feature vector. The syntax correctness score is used for learning code syntax optimization in the code generation module.

[0112] In one embodiment, the code generating device 700 is further configured to:

[0113] Training a grammar discriminator according to second training samples corresponding to each training code, the second training samples including an abstract syntax tree loss parameter and a hidden state feature vector corresponding to the training code;

[0114] When the network parameters of the grammatical discriminator no longer converge, a grammatical correctness score corresponding to the current second training sample is generated according to the current second training sample.

[0115] In one embodiment, the code generation module is a pre-trained model obtained by training with a plurality of first training samples.

[0116] In one embodiment, the loss calculation module is configured to determine an abstract syntax tree loss parameter according to an edit distance between a target abstract syntax tree sequence and a generated abstract syntax tree sequence.

[0117] In one embodiment, after the code generation module updates its parameters based on the grammatical correctness score, the cross entropy loss parameter corresponding to the code generation module is determined according to the hidden state feature vector, and the parameters of the code generation module after the parameter update are updated according to the cross entropy loss parameter.

[0118] The code generation device provided in the embodiment of the present application can achieve Figure 1-Figure 7 The various processes implemented in the illustrated embodiments achieve the same technical effects and will not be described again here to avoid repetition.

[0119] In some embodiments, as Figure 8 As shown, an embodiment of the present application further provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores a program or instruction that can be run on the processor 801, and when the program or instruction is executed by the processor 801, the various steps of the above-mentioned embodiment of the code generation method based on the grammar discriminator are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0120] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0121] An embodiment of the present application also provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned embodiment of the code generation method based on the grammar discriminator are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0122] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0123] An embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-mentioned embodiment of the code generation method based on the grammar discriminator, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0124] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0126] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A code generation method based on a grammar discriminator, characterized in that: include: Acquire a plurality of first training samples, where the first training samples include training codes and corresponding code description information; Training a preset model based on each of the first training samples to obtain a code generation model with code syntax optimization function, wherein the preset model includes a code generation module, a sequence extraction module, a loss calculation module, and a syntax discriminator; Get the generation description information corresponding to the code to be generated; Inputting the generation description information into a code generation model to obtain a target code output by the code generation model, wherein the code generation model has a code syntax optimization function; Training a preset model based on each of the first training samples to obtain a code generation model with code syntax optimization function, including: Inputting the code description information of the training code into the code generation module, and the code generation module generating a hidden state feature vector according to the code description information of the training code; Inputting the training code and the hidden state feature vector into the sequence extraction module, the sequence extraction module linearizing the training code to obtain a target abstract syntax tree sequence, and parsing the hidden state feature vector to obtain a generated abstract syntax tree sequence; Inputting the target abstract syntax tree sequence and the generated abstract syntax tree sequence into the loss calculation module, the loss calculation module determining an abstract syntax tree loss parameter according to the edit distance between the target abstract syntax tree sequence and the generated abstract syntax tree sequence; Inputting the abstract syntax tree loss parameter and the hidden state feature vector into the syntax discriminator, wherein the syntax discriminator determines a syntax correctness score of the hidden state feature vector based on the abstract syntax tree loss parameter and the hidden state feature vector, and the syntax correctness score is used by the code generation module to perform code syntax optimization learning; After the code generation module performs parameter update based on the grammatical correctness score, a cross entropy loss parameter corresponding to the code generation module is determined according to the hidden state feature vector, and the code generation module after parameter update is updated according to the cross entropy loss parameter.

2. The code generation method based on a grammar discriminator according to claim 1, characterized in that: The code generation module is used to generate a hidden state feature vector according to the code description information of the training code; The sequence extraction module is configured to perform linear sequence extraction on the training code to obtain a target abstract syntax tree sequence, and to parse the hidden state feature vector to obtain a generated abstract syntax tree sequence; The loss calculation module is configured to determine an abstract syntax tree loss parameter according to the target abstract syntax tree sequence and the generated abstract syntax tree sequence; The grammar discriminator is used to determine a grammatical correctness score of the hidden state feature vector based on the abstract syntax tree loss parameter and the hidden state feature vector, and the grammatical correctness score is used by the code generation module to learn code grammar optimization.

3. The code generation method based on grammar discriminator according to claim 2, characterized in that: Also includes: Training the grammar discriminator according to second training samples corresponding to each training code, wherein the second training samples include an abstract syntax tree loss parameter and a hidden state feature vector corresponding to the training code; When the network parameters of the grammatical discriminator no longer converge, a grammatical correctness score corresponding to the current second training sample is generated according to the current second training sample.

4. The code generation method based on grammar discriminator according to claim 2, characterized in that: The code generation module is a pre-trained model obtained by training with a plurality of the first training samples.

5. The code generation method based on grammar discriminator according to claim 2, characterized in that: The loss calculation module is configured to determine the abstract syntax tree loss parameter according to an edit distance between the target abstract syntax tree sequence and the generated abstract syntax tree sequence.

6. A code generating device, characterized in that: include: A training sample acquisition unit, configured to acquire a plurality of first training samples, wherein the first training samples include training codes and corresponding code description information; a model training unit, configured to train a preset model based on each of the first training samples to obtain a code generation model with a code syntax optimization function, wherein the preset model includes a code generation module, a sequence extraction module, a loss calculation module, and a syntax discriminator; An acquisition module is used to obtain generation description information corresponding to the code to be generated; An input module, configured to input the generation description information into a code generation model to obtain a target code output by the code generation model, wherein the code generation model has a code syntax optimization function; When the model training unit trains a preset model according to each of the first training samples to obtain a code generation model with a code syntax optimization function, the model training unit includes: Inputting the code description information of the training code into the code generation module, and the code generation module generating a hidden state feature vector according to the code description information of the training code; Inputting the training code and the hidden state feature vector into the sequence extraction module, the sequence extraction module linearizing the training code to obtain a target abstract syntax tree sequence, and parsing the hidden state feature vector to obtain a generated abstract syntax tree sequence; Inputting the target abstract syntax tree sequence and the generated abstract syntax tree sequence into the loss calculation module, the loss calculation module determining an abstract syntax tree loss parameter according to the edit distance between the target abstract syntax tree sequence and the generated abstract syntax tree sequence; Inputting the abstract syntax tree loss parameter and the hidden state feature vector into the syntax discriminator, wherein the syntax discriminator determines a syntax correctness score of the hidden state feature vector based on the abstract syntax tree loss parameter and the hidden state feature vector, and the syntax correctness score is used by the code generation module to perform code syntax optimization learning; After the code generation module performs parameter update based on the grammatical correctness score, a cross entropy loss parameter corresponding to the code generation module is determined according to the hidden state feature vector, and the code generation module after parameter update is updated according to the cross entropy loss parameter.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the code generation method based on the grammar discriminator are implemented as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the code generation method based on a grammar discriminator according to any one of claims 1 to 5 are implemented.

Images