Knowledge and data double-wheel driven code generation method

Abstract

The invention relates to a knowledge and data double-wheel driven code generation method, in particular to a method for automatically generating a programming language based on knowledge and data double-wheel driven, and belongs to the technical field of natural language processing. The code generation method comprises the following steps: S1, constructing a programming language knowledge base; s2, setting a classifier model; s3, setting a code generation model; and S4, performing knowledge and data double-wheel driven code generation according to the knowledge base, the classifier model and the code generation model. Compared with the prior art, the method has the advantages that information in two aspects of knowledge and data is combined for code generation, related model structures are relatively independent, and a user can freely select a classifier model and a code generation model with proper sizes to complete training according to actual training resources; the method still has good performance under the conditions that the code data volume is small and the code generation model parameter quantity is small.

Description

Technical Field

[0001] The present invention relates to a code generation method, in particular to a method for automatically generating a programming language based on knowledge and data dual-wheel drive, belonging to the technical field of natural language processing. Background Art

[0002] With the development of artificial intelligence technology and the popularization of large models, natural language processing related technologies have also achieved innovative breakthroughs. Various large language models with hundreds of millions to hundreds of billions of parameters pre-trained on massive data have demonstrated unprecedented powerful capabilities in various natural language processing.

[0003] Code generation is the process of generating program code snippets from natural language text. The development of code generation can be divided into three main stages: initially, programmers or experts manually wrote code generation rules and templates based on their programming knowledge to achieve some simple and primary code generation related tasks; then, machine learning methods were used to achieve code generation using probability and statistical methods; and finally, deep neural network models were used to automatically extract code language features. During this development process, the accuracy of code generation has been qualitatively improved.

[0004] Traditional template- and rule-based methods and machine learning methods have good interpretability and low resource requirements, but they require experts in the field to inject knowledge by manually writing templates and rules, that is, they rely on "knowledge" drive. In addition, this method lacks flexibility and adaptability, and the code generation results are unsatisfactory. The method based on the deep neural network model can learn the characteristics of the corresponding programming language from a large amount of text data or code data, and has achieved good results in the generation of mainstream programming languages, but it requires a huge amount of annotated data as a data set to obtain better training results, that is, it relies on "data" drive. For some rare programming languages ​​that are difficult to obtain a large amount of annotated data, it is difficult for the baseline model to obtain a training data volume that matches its volume, and it is difficult to achieve satisfactory code generation capabilities. Summary of the invention

[0005] In view of the shortcomings of the code generation methods in the prior art, namely, either over-reliance on knowledge or over-reliance on data, the inventors creatively proposed a code generation method driven by both knowledge and data, so as to better complete the code generation task.

[0006] The innovation of the present invention is that when performing code generation tasks, expert rules and templates as well as labeled data for training are fully utilized to organically combine knowledge with data, thereby completing the code generation task with less data and hardware resources.

[0007] A knowledge and data dual-wheel driven code generation method, comprising the following steps:

[0008] Step 1: Build a programming language knowledge base module.

[0009] Specifically, the method includes the following sub-steps:

[0010] Step 1.1: Learn and acquire knowledge of the target programming language.

[0011] Learn the target programming language and how to write it by reading books related to the programming language, learning how to write the programming language through online blogs or videos, reading related engineering source code, and other methods.

[0012] Step 1.2: Set the code category and corresponding knowledge representation.

[0013] According to what has been learned and the understanding of the content, the knowledge of each major data type or statement form that appears in the programming language (such as data structures, class definitions, function definitions, algorithm implementations in general high-level programming languages, or similar concepts in the target language to be generated, collectively referred to as "categories") is summarized and summarized to form the "knowledge" manually summarized and refined in this category. Specifically, if the category form is relatively fixed and has a unified code template, the template and some prompts that instruct the large model to generate corresponding codes according to the template can be combined as the "knowledge" set by experts in this category; if there is no template for this category, some useful information of this category (such as type definition, writing characteristics, keyword requirements, etc.) can be extracted from the official documents of the language, or some information that helps the code model generate correct code can be manually summarized according to the learning content (such as the "comment" category can prompt comment symbols and usage methods, and the "identifier" category can prompt naming style and context context, etc.) as the knowledge and rules of this category. These templates and prompt information can also be set by analogy with the characteristics of mainstream programming languages ​​such as C / C++, Python, and Java. This can better enable the model to mobilize pre-trained programming language information, which is conducive to allowing the code generation model to better understand the new programming language.

[0014] Step 1.3: Form a knowledge base module.

[0015] Organize the above-obtained code categories and corresponding knowledge representations in the form of key-value pairs (k, v), and summarize these key-value pairs into a dictionary as the knowledge base K of the programming language, that is:

[0016] K={(k1,v1),(k2,v2),…,(k n ,v n )} (1)

[0017] Among them, (k i ,v i ) represents the i-th category and the corresponding knowledge.

[0018] Step 2: Set up the classifier model module.

[0019] For natural language instructions input by users to generate target programming language code, a trained classifier model is needed to parse out the specific category that the instruction is to generate, so as to determine the expert knowledge corresponding to the category in the knowledge base. Only then can the prompts, rules and templates be incorporated into the input through splicing and other methods, thereby guiding the code generation model to better generate the corresponding code segment.

[0020] Step 2.1: Get a text classification dataset.

[0021] According to the characteristics of the target programming language and the difficulty of data acquisition, search for existing data sets on the Internet, or build a classification data set yourself. The format of the text classification data set should be:

[0022] D t ={(X1,y1),(X2,y2),…,(X n ,y n )} (2)

[0023] Among them, D t represents a text classification dataset, X i represents the i-th natural language description text, y i Indicates the category of the generated code corresponding to the natural language description text.

[0024] Step 2.2: Train the classifier model.

[0025] The classifier model is trained using a programming language instruction text classification dataset, so that the latter can learn to generate instructions based on the natural language code of the target programming language and determine the category of the target code that the user wants to generate. This method has no requirements for the baseline model. Users can choose a suitable text classification model based on their hardware resources and preferences. Commonly used models include support vector machines (SVM) based on machine learning, decision trees, recurrent neural networks (RNN) based on deep learning, BERT, etc.

[0026] Step 3: Set up the code generation model module.

[0027] For the code generation baseline model, before performing code generation tasks in a new programming language, it is pre-trained or fine-tuned twice on the target programming language data, so that the model can learn the characteristics of the language before prediction, further improving the accuracy of generated code, especially for those categories without templates.

[0028] Step 3.1: Get the code generation dataset.

[0029] Similar to the method of obtaining text classification datasets, the training dataset is obtained by directly selecting an existing dataset or constructing a dataset by yourself, depending on the difficulty of obtaining code generation data in the target programming language. The format of each sample is different depending on the dataset format required by the selected code generation model; the common formats can be divided into sequence-to-sequence task dataset formats ({input, output}) with only input and output, and instruction fine-tuning dataset formats ({input, instruction, output}) that additionally contain "instruction" fields. Each sample is represented as sample, and the training dataset can be represented as:

[0030] D c ={sample1,sample2,…,sample n} (3)

[0031] Among them, D c Indicates the code generation data set, sample i Represents the i-th sample in the data set.

[0032] Step 3.2: Train / fine-tune the code generation model.

[0033] Generate dataset D using the acquired target programming language code c , train or fine-tune the baseline code generation model according to the hardware resource level and actual usage requirements. The difference between training and fine-tuning is that training generally refers to updating all parameters of the model, which has very high requirements for hardware resources and training data volume, but allows the model to better learn the writing characteristics of the target programming language, and is suitable for situations with abundant hardware resources and sufficient data volume; while fine-tuning generally refers to updating only a small number of model parameters and freezing other parameters. Fine-tuning has significantly lower requirements for hardware resources than training, and only requires fewer training samples of new programming languages ​​to achieve it. It also allows the model to maintain its original ability to generate code in other programming languages, and is suitable for situations where hardware resources and data volume are limited and incremental development is required.

[0034] Step 4: Code generation driven by knowledge and data.

[0035] Step 4.1: Combine the modules.

[0036] The three prepared model modules, namely the classifier, knowledge base and code generator, are connected end to end to form an overall architecture. Specifically, the input end of the classifier model module is used to input natural language description text, the output end of the classifier model module is connected to the knowledge base and the generator respectively, the output end of the knowledge base is connected to the code generator, and the code generator outputs the code.

[0037] Sub-step 4.2: Generate code using the code generation model.

[0038] Enter the natural language description text of the corresponding code you want to generate, recorded as description. The natural language description text is first parsed by the classifier model to obtain the target code category you want to generate, recorded as category. d ,Right now:

[0039] category d =classifier(description) (4)

[0040] classifier() means inference through the classifier network.

[0041] After that, search for the key category in the programming language knowledge base K d If found, take out its mapping value knowledge d , which is "knowledge", is concatenated with the input natural language description text to form the input of the code generation model, denoted as input d ; If no search is found, the user input is directly used as the input of the code generation model d ,Right now:

[0042]

[0043] Finally, the obtained input is fed into the code generation model for the text-to-code generation task, namely:

[0044] output d =generator(input d ) (6)

[0045] Among them, output d That is, the target programming language code segment is finally generated, and generator() means using the code generation model for inference.

[0046] Beneficial Effects

[0047] Compared with the prior art, the method of the present invention has the following advantages:

[0048] 1. This method makes full use of the learning content of both knowledge and data, and proposes a code generation method that combines knowledge and data information, so that the model can not only learn encoding knowledge from data, but also learn the characteristics of the target language through templates, rules, etc.

[0049] 2. The model structures involved in this method are relatively independent. Users can freely choose classifier models and code generation models of appropriate sizes to complete training based on actual training resource conditions.

[0050] 3. Compared with the current large models that require massive amounts of data for training, this method still performs well when the amount of code data is small and the number of code generation model parameters is small. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the architecture of the method of the present invention. DETAILED DESCRIPTION

[0052] The method of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] Example

[0054] As the instruction manual Figure 1 As shown in the figure, the present invention consists of three main modules, taking the construction of a code generation model for the simulation modeling language AFSIM as an example. First, by learning the AFSIM code knowledge and writing methods, the various code categories are divided, and each category is associated with the corresponding knowledge representation to form a knowledge base K; then, the AFSIM text classification and code generation datasets are constructed, and BERT is selected as the text classification baseline model and CodeT5+ as the code generation baseline model, and they are trained / fine-tuned respectively; finally, these models are combined according to Figure 1 As shown in the figure, the natural language description text description is fed to the input of BERT, and the output from CodeT5+ is the AFSIM code segment corresponding to description.

[0055] Specifically, the following steps are included:

[0056] Step 1: Learn and acquire knowledge related to the target programming language, which includes the following sub-steps:

[0057] Step 1.1: Learn and acquire knowledge of the target programming language.

[0058] AFSIM is a framework for simulation modeling of military scenarios. It has a certain degree of confidentiality and its knowledge can only be learned through the official website introduction, official documents and training tutorials. Through these resources, you can learn AFSIM knowledge and writing methods.

[0059] Step 1.2: Set the code category and corresponding knowledge representation.

[0060] AFSIM is a declarative simulation modeling language that defines a variety of common military simulation platform types, such as specific platform type platform, abstract platform type platform_type, code type script, weapon type weapon, etc. Templates for some platforms can be found in the document, and almost all types have descriptions such as definition information. This information is collated and compiled manually to form a knowledge representation corresponding to the type.

[0061] Step 1.3: Form a knowledge base.

[0062] The key-value pairs consisting of categories and corresponding knowledge representations are summarized into a dictionary format to form the AFSIM programming language knowledge base K. In particular, because the number of types in AFSIM is small (100 to 300), the key value is encoded from the type name into a serial number representation, starting from 0 and increasing by 1 for each key-value pair.

[0063] Step 2: Set up the classifier model, which includes the following sub-steps:

[0064] Step 2.1: Get a text classification dataset.

[0065] As mentioned above, due to the confidentiality and sensitivity of AFSIM, the relevant datasets cannot be obtained on the Internet. The software directory contains dozens of engineering file source codes. Python is used to write text parsing scripts to parse and extract code segments from code files. After deduplication, filtering, screening, and editing, the AFSIM text classification dataset is obtained. The AFSIM code generation dataset can be obtained in a similar way, which will not be repeated below.

[0066] Step 2.2: Train the classifier model.

[0067] In this embodiment, BERT is selected as the classifier baseline model, and BERT is trained using the AFSIM text classification dataset.

[0068] Step 3: Set up the code generation model, which includes the following sub-steps:

[0069] Step 3.1: Get the code generation dataset.

[0070] Step 3.2: Train / fine-tune the code generation model.

[0071] In this embodiment, the CodeT5+ model is selected as the code generation baseline model, and the AFSIM code generation dataset is used to train CodeT5+.

[0072] Step 4: Code generation driven by knowledge and data, including the following sub-steps:

[0073] Step 4.1: Combine the modules.

[0074] According to the instruction manual Figure 1 The three main modules are connected in the manner shown.

[0075] Step 4.2: Code Generation

[0076] The natural language text of the AFSIM code generation instructions is fed to the BERT input, and after the overall feed-forward processing of the model, the target AFSIM code is finally obtained at the CodeT5+ output.

[0077] In order to illustrate the content and implementation method of the present invention, this specification gives a specific embodiment. The purpose of introducing details in the embodiment is not to limit the scope of the claims, but to help understand the content of the present invention. Those skilled in the art should understand that: without departing from the spirit of the present invention, various technical solutions or embodiments obtained by various modifications, changes or equivalent replacements of technical features to the technical solutions or embodiments of the present invention all fall within the protection scope of the present invention.

Claims

1. A knowledge and data dual-wheel driven code generation method, characterized in that: The code generation method comprises the following steps: S1, constructing a programming language knowledge base; S2, setting a classifier model; S3, setting a code generation model; and S4, knowledge and data dual-wheel driven code generation.

2. The knowledge and data dual-wheel driven code generation method according to claim 1 is characterized in that: The step S1 includes the following sub-steps: S1.1, learning and acquiring knowledge related to the target programming language; S1.2, setting code categories and corresponding knowledge representations; S1.3, forming a knowledge base.

3. The knowledge and data dual-wheel driven code generation method according to claim 2 is characterized in that: The specific content of the sub-step S1.2 is: summarizing the knowledge in the form of major data categories appearing in the programming language to form the "knowledge" manually summarized and extracted from the categories; the specific content of the sub-step S1.3 is: organizing the obtained code categories and corresponding knowledge representations in the form of key-value pairs (k, v), and aggregating these key-value pairs into a dictionary as the knowledge base K of the programming language, that is: K = {(k1, v1), (k2, v2), ..., (k n ,v n )}, where (k i ,v i ) represents the i-th category and the corresponding knowledge.

4. The knowledge and data dual-wheel driven code generation method according to claim 3 is characterized in that: The specific content of the sub-step S2.1 is: according to the characteristics of the target programming language and the difficulty of data acquisition, search for existing data sets from the Internet, or write a script to automatically extract and generate a classification data set from the existing project source code. The format of the classification data set should be: t ={(X1,y1),(X2,y2),…,(X n ,y n )}, where D t represents a text classification dataset, X i represents the i-th natural language description text, y i Indicates the category of the generated code corresponding to the natural language description text; the specific content of the sub-step S2.2 is: use the programming language instruction text classification data set to train the classifier model, so that the classifier model learns to generate instructions according to the natural language code of the target programming language, and determines the category of the target code that the user wants to generate.

5. The knowledge and data dual-wheel driven code generation method according to claim 4 is characterized in that: The specific content of the sub-step S3.1 is: according to the difficulty of obtaining the target programming language code generation data, obtain the training data set by directly selecting an existing data set or constructing a data set by yourself; the specific content of the sub-step S3.2 is: use the training data set to train or fine-tune the baseline code generation model according to the hardware resource level and actual usage requirements.

6. The knowledge and data dual-wheel driven code generation method according to claim 5 is characterized in that: The sub-step S4 specifically includes the following contents: S4.1, combining the knowledge base module, the classifier module and the code generation module; S4.2, input the natural language description text that needs to generate the corresponding code, denoted as description. The natural language description text is first parsed by the classifier model to obtain the target code category that the description wants to generate, denoted as category. d , that is: category d = classifier(description), classifier() means to infer through the classifier network and search for the key category in the programming language knowledge base K d If found, take out its mapped value knowledge d , which is "knowledge", is concatenated with the natural language description text input by the user to form the input of the code generation model, denoted as input d ; If no search is found, the input is directly used as the input of the code generation model d ; Finally, the obtained input is fed into the code generation model to perform the text-to-code generation task, that is: output d =generator(input d ), where output d That is, the target programming language code segment is finally generated, and generator() means using the code generation model for inference.

Images