Natural language programming method, device, equipment and storage medium

Abstract

The present invention relates to a natural language programming method, device, equipment and storage medium, the method comprising: receiving a target scene described in natural language input by a user, the target scene comprising a character and a target task; inputting the target scene into a pre-trained language model; using the language model, generating text corresponding to the result of the character performing the target task as programming result text. Using the natural language programming method of the present invention, programming using natural language is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a natural language programming method, device, equipment and storage medium. Background Art

[0002] The statements herein merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In order to drive the computer to run, a programming language is needed. For example, in order to drive the computer to perform addition operations, you need to tell the computer 1+1=, and then the computer runs to get 2. Initially, Turing designed a paper tape model, which encoded 0 and 1 through punched paper tape to input instructions to the computer. Later, assembly language was formed, and instructions were simply memorized with words, such as ADD 11, which means 1+1. Later, people felt that assembly language was still troublesome, so high-level languages ​​were formed, such as C language and Python. Programming languages ​​are becoming simpler and closer to natural languages, but users still need to learn and understand them before they can use them.

[0004] On the other hand, most computer languages ​​require English coding, which is not friendly to other languages. Users in many countries cannot use their own language to program. Therefore, several Chinese programming methods have emerged, such as "Zhou Python" and classical Chinese programming, which can be used to program in Chinese. As an example of programming using "Zhou Python", enter:

[0005] >>>Print out 'Hello World'

[0006] After executing the above command, the text "Hello, World" will be displayed on the screen. As an example of using classical Chinese programming, enter:

[0007] I have a saying: "Ask the heaven and earth what is good"

[0008] After executing the above command, the output will be "Ask the heaven and earth where is good".

[0009] In addition, traditional programming languages ​​need to follow strict syntax, and even a slight syntax error will cause a runtime error. And traditional programming languages ​​cannot directly understand complex tasks, such as writing a Spring Festival couplet, which is easy for us humans to understand, but not for computers. Summary of the invention

[0010] The purpose of the present invention is to provide a new programming method, device, equipment and storage medium to realize programming using natural language.

[0011] The purpose of the present invention is achieved by the following technical solution. According to a natural language programming method proposed by the present invention, the method comprises the following steps: receiving a target scene described in natural language input by a user, the target scene comprising a character and a target task; inputting the target scene into a pre-trained language model; using the language model, generating text corresponding to the result of the character performing the target task as programming result text.

[0012] The purpose of the present invention can be further achieved by adopting the following technical measures.

[0013] In the aforementioned natural language programming method, during the process of training the language model, the training data input into the language model includes large-scale natural language description scenes, wherein the scenes include characters and tasks.

[0014] In the aforementioned natural language programming method, in the process of training the language model, the training data input into the language model includes corpus of the target genre; wherein the corpus of the target genre includes novel corpus, story corpus and / or news corpus.

[0015] The aforementioned natural language programming method further includes, before inputting the training data into the language model for training, splitting the training data into a plurality of sub-training data to distinguish different scenarios.

[0016] The aforementioned natural language programming method, wherein splitting the training data into a plurality of sub-training data comprises: identifying location words, and / or time words, and / or chapter separator words in the corpus, wherein the chapter separator words are used to describe the beginning or end of a chapter, section, episode, or act; and dividing the content of the corpus into a plurality of parts according to the location words, and / or the time words, and / or the chapter separator words, wherein each part serves as a sub-training data.

[0017] The aforementioned natural language programming method, before inputting the training data into the language model for training, and / or before inputting the target scene into a pre-trained language model, further includes: judging the style type of the corpus used as training data and / or the target scene, and recording the judgment result in the style type field for model training or model inference according to the style type field; and / or, before inputting the training data into the language model for training, further includes: judging the style type of the corpus, and selecting the corpus that conforms to the target style as training data.

[0018] In the aforementioned natural language programming method, the training process of the language model includes: inputting part of the content in the training data into the current language model, using the current language model to infer the context of the part of the content in the training data, and comparing the inferred context with the actual context, and adjusting the parameters of the current language model according to the comparison result in a back-propagation manner; using the language model to generate text corresponding to the result of the character performing the target task includes: inputting the target scene into a pre-trained language model, and using the pre-trained language model to infer the context of the target scene.

[0019] The aforementioned natural language programming method further includes: presetting a vocabulary; before inputting part of the content in the training data into the current language model, the training process of the language model includes: extracting text and performing word segmentation processing on the original corpus used for training to segment the text of the original corpus into multiple word units, determining the corresponding identifier of each word unit of the original corpus in the vocabulary to obtain the word segmentation identifier vector of the original corpus, and using the word segmentation identifier vector of the original corpus as training data to train the language model; the training process of the language model specifically includes: the current language model outputs a corresponding subsequent context probability vector based on the word segmentation identifier vector of the partial content in the training data; subtracting the subsequent context probability vector corresponding to the partial content output by the model from the word segmentation identifier vector of the actual subsequent context of the partial content to obtain an error vector; and, based on the error vector, The parameters of the current language model are weighted; the inputting the target scene into the pre-trained language model includes: extracting text and performing word segmentation processing on the target scene to segment the text of the target scene into multiple word units, and determining the corresponding identifier of each word unit of the target scene in the vocabulary to obtain the word segmentation identifier vector of the target scene, and sending the word segmentation identifier vector of the target scene to the pre-trained language model; the using the language model to infer the context of the target scene includes: the pre-trained language model outputs a corresponding context probability vector according to the word segmentation identifier vector of the target scene; and, determining the context of the target scene according to the context probability vector; wherein each dimension of the context probability vector corresponds to a word unit in the vocabulary, and the value of each dimension of the context probability vector represents the probability of the corresponding word unit in the vocabulary inferred by the language model being selected as the context.

[0020] The aforementioned natural language programming method, wherein the use of the current language model to infer the context of the partial content in the training data, and / or the use of the pre-trained language model to infer the context of the target scene, includes: inferring the context one character at a time or one word at a time.

[0021] The aforementioned natural language programming method, wherein the current language model outputs a corresponding context probability vector based on the word segmentation identification vector of part of the training data, and / or the pre-trained language model outputs a corresponding context probability vector based on the word segmentation identification vector of the target scene, comprises: smoothing the probability of each word unit in the vocabulary inferred by the language model being selected as the context to obtain a smoothed context probability vector; determining the context of the target scene based on the context probability vector comprises: determining a plurality of word units with the highest probability in the vocabulary based on the smoothed context probability vector, and determining the context based on the plurality of word units with the highest probability.

[0022] In the aforementioned natural language programming method, the training process of the language model also includes: before the word segmenter processing is performed, the corpus used as training data is translated into the corpus of the target language, so that the language model is trained using the translated corpus; the inputting of the target scene into the pre-trained language model also includes: before the word segmenter processing is performed, the received target scene is translated into a target scene described in the natural language of the target language, so that the language model processes the translated target scene; the method also includes: translating the programming result text into a text corresponding to the original language of the target scene, and then displaying it to the user.

[0023] In the aforementioned natural language programming method, the language model is based on a Transformer architecture, and the architecture is provided with a memory vector.

[0024] In the aforementioned natural language programming method, the target scene described in natural language received by the user includes: using a front-end interface to receive the target scene input by the user; wherein the front-end interface includes a natural input interface, or a template matching input interface, the natural input interface includes a plain text box for receiving user input, and the template matching input interface includes template text, a control for receiving user input, an execution button, and a control for displaying output results; the target scene is input into a pre-trained language model, including: using a back-end service to send the target scene received by the front-end interface to the language model; the method also includes: using a back-end service to return the programming result text output by the language model to the front-end page for display by the front-end page.

[0025] The purpose of the present invention is also achieved by the following technical solutions. A natural language programming device proposed in the present disclosure includes: a front-end interface module for receiving a scene described in natural language input by a user, wherein the scene includes a character and a target task; a back-end service module for inputting the scene described in natural language into a pre-trained language model; and a language model processing module for using the language model to generate text corresponding to the result of the character performing the target task as a programming result text.

[0026] Optionally, the aforementioned natural language programming device also includes a module for executing the steps of any one of the aforementioned natural language programming methods.

[0027] The purpose of the present invention can be further achieved by adopting the following technical measures.

[0028] The aforementioned natural language programming device also includes a language model training module for training the language model, wherein in the process of training the language model, the training data input into the language model includes large-scale natural language description scenes, and the scenes include characters and tasks.

[0029] In the aforementioned natural language programming device, the language model training module is specifically used for: in the process of training the language model, the training data input into the language model includes the corpus of the target genre; wherein the corpus of the target genre includes novel corpus, story corpus and / or news corpus.

[0030] The aforementioned natural language programming device also includes a training data splitting module, which is used to split the training data into multiple sub-training data before inputting the training data into the language model for training, so as to distinguish different scenarios.

[0031] The aforementioned natural language programming device, wherein the training data splitting module is specifically used to: identify location words, and / or time words, and / or chapter separator words in the corpus, wherein the chapter separator words are used to describe the beginning or end of a chapter, section, episode, or act; and divide the content of the corpus into multiple parts based on the location words, and / or the time words, and / or the chapter separator words, each part being a sub-training data.

[0032] The aforementioned natural language programming device also includes a style field generation module and / or a style screening module. The style field generation module is used to judge the style type of the corpus used as training data and / or the target scene before inputting the training data into the language model for training and / or before inputting the target scene into the pre-trained language model, and record the judgment result in the style type field for model training or model inference according to the style type field. The style screening module is used to judge the style type of the corpus before inputting the training data into the language model for training, and select the corpus that meets the target style as training data.

[0033] In the aforementioned natural language programming device, the language model training module is specifically used to: input part of the content in the training data into the current language model, use the current language model to infer the context of the part of the content in the training data, and compare the inferred context with the actual context, and adjust the parameters of the current language model according to the comparison result in a back-propagation manner. The language model processing module is specifically used to: input the target scenario into a pre-trained language model, and use the pre-trained language model to infer the context of the target scenario.

[0034] The aforementioned natural language programming device also includes a vocabulary setting module, a training data preprocessing module, and a user input preprocessing module. Among them, the vocabulary setting module is used to pre-set the vocabulary. The training data preprocessing module is used to: before the part of the content in the training data is input into the current language model, extract the text and perform word segmentation processing on the original corpus used for training, so as to segment the text of the original corpus into multiple word units, determine the corresponding identifier of each word unit of the original corpus in the vocabulary to obtain the word segmentation identification vector of the original corpus, and use the word segmentation identification vector of the original corpus as training data to train the language model. The user input preprocessing module is used to: extract the text and perform word segmentation processing on the target scene, so as to segment the text of the target scene into multiple word units, determine the corresponding identifier of each word unit of the target scene in the vocabulary to obtain the word segmentation identification vector of the target scene, and send the word segmentation identification vector of the target scene to the pre-trained language model. The language model training module is specifically used for: the current language model outputs a corresponding context probability vector according to the word segmentation identification vector of part of the content in the training data; subtracts the context probability vector corresponding to the part of the content output by the model from the word segmentation identification vector of the actual context of the part of the content to obtain an error vector; and, weights the parameters of the current language model according to the error vector. The language model processing module is specifically used for: the pre-trained language model outputs a corresponding context probability vector according to the word segmentation identification vector of the target scene; and, determines the context of the target scene according to the context probability vector. Wherein, each dimension of the context probability vector corresponds to a word unit in the vocabulary, and the value of each dimension of the context probability vector represents the probability that the corresponding word unit in the vocabulary inferred by the language model is selected as the context.

[0035] The aforementioned natural language programming device, the language model training module, and / or the language model processing module are specifically used to infer the subsequent context one character at a time or one word at a time.

[0036] In the aforementioned natural language programming device, the language model training module is specifically used for: in the process of training the language model, smoothing the probability of each word unit in the vocabulary inferred by the language model being selected as the subsequent context to obtain a smoothed subsequent context probability vector. The language model processing module is specifically used for: in the process of generating text corresponding to the result of the character performing the target task using the language model, smoothing the probability of each word unit in the vocabulary inferred by the language model being selected as the subsequent context to obtain a smoothed subsequent context probability vector; determining multiple word units with the highest probability in the vocabulary according to the smoothed subsequent context probability vector, and determining the subsequent context according to the multiple word units with the highest probability.

[0037] In the aforementioned natural language programming device, the training data preprocessing module is also used to: translate the corpus used as training data into the corpus of the target language before the word segmentation processing is performed, so as to train the language model with the translated corpus. The user input preprocessing module is also used to: translate the received target scene into the target scene described in the natural language of the target language before the word segmentation processing is performed, so that the language model processes the translated target scene. The device also includes a result display module, which is used to: translate the programming result text into a text corresponding to the original language of the target scene, and then display it to the user.

[0038] In the aforementioned natural language programming device, the language model is based on a Transformer architecture, and the architecture is provided with a memory vector.

[0039] In the aforementioned natural language programming device, the front-end interface module is specifically used to: receive the target scene input by the user using the front-end interface; wherein the front-end interface includes a natural input interface or a template matching input interface, the natural input interface includes a plain text box for receiving user input, and the template matching input interface includes template text, controls for receiving user input, an execution button, and controls for displaying output results. The back-end service module is specifically used to: send the target scene received by the front-end interface to the language model using the back-end service. The device also includes a result display module, which is used to: return the programming result text output by the language model to the front-end page using the back-end service for display by the front-end page.

[0040] The purpose of the present invention is also achieved by the following technical solution. A natural language programming device proposed in accordance with the present disclosure includes: a memory for storing non-temporary computer-readable instructions; and a processor for running the computer-readable instructions, so that the processor implements any of the above-mentioned natural language programming methods when executing.

[0041] The object of the present invention is also achieved by the following technical solution: A computer-readable storage medium proposed in the present disclosure is used to store non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are executed by a computer, the computer executes any one of the above-mentioned natural language programming methods.

[0042] Compared with the prior art, the present invention has obvious advantages and beneficial effects. By means of the above technical solution, the natural language programming method, device, equipment and storage medium proposed by the present invention have at least the following advantages and beneficial effects:

[0043] 1. The present invention creates a scene through natural language, drives the characters in the scene to complete tasks, and thus achieves the desired results, converts programming into scene description, thereby realizing programming using natural language;

[0044] 2. Even if the program or the natural language description input by the user has certain errors, the programming method of the present invention can still be run to obtain the output result, and the programming result will not be unable to be obtained due to a small error;

[0045] 3. The present invention utilizes natural language tasks and can support programming in the language of any country through translation, thereby having multi-lingual capabilities.

[0046] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites a preferred embodiment and describes it in detail with the accompanying drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flowchart of a natural language programming method according to an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the processing performed by the back-end service of the natural language programming system provided by one embodiment of the present invention;

[0049] Figure 3 is a schematic diagram of a language model training process provided by an embodiment of the present invention;

[0050] Figure 4 It is a schematic diagram of a natural language programming device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the natural language programming method, device, equipment and storage medium proposed in the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0052] The present invention relates to Programming in Nature Language (PNL for short).

[0053] Figure 1 FIG. 1 is a schematic flow chart of an embodiment of the natural language programming method of the present invention. In some embodiments of the present invention, please refer to Figure 1 The natural language programming method of the present invention mainly includes the following steps:

[0054] Step S11, receiving a scene (also called a scenario, story) described in natural language input by a user. The scene input by the user may be referred to as a target scene. Optionally, the scene described by natural language includes characters and target tasks. The characters in the target scene may be referred to as target characters. Optionally, the scene described in natural language is a piece of text input by the user. It should be noted that the aforementioned "scene described in natural language" is equivalent to the input of the programmer in programming, but it is different from the standardized programming code input by the programmer in the existing programming method, but is a text described in natural language.

[0055] Step S12, inputting the target scene into a pre-trained language model.

[0056] Step S13, using the language model to generate text corresponding to the target scenario. Specifically, using the language model to generate text corresponding to the result of the character performing the target task as the programming result text.

[0057] The language model (LM) is also called a text generation model, which is used to create based on the input text, that is, to input the previous text and use the computer to give the following text or fill in the blanks. The present invention does not limit the specific type of the language model selected, for example, it can be a Bert language model or a GPT language model.

[0058] Optionally, after the aforementioned step S13, the natural language programming method of the present invention further includes displaying the programming result text to the user.

[0059] By using the natural language programming method of the present invention, functions such as "printing text", translation, recipe generation, writing poems according to situations, writing Spring Festival couplets, naming, writing stories, providing business advice, etc. can be realized.

[0060] As a specific example of printing text, the target scenario input by the user may be: "In class, the teacher said: 'Read "Ask the Heaven and Earth Where is Good" to me twice.' The student began to repeat it mechanically." Using the device to implement the natural language programming method proposed in the present invention may obtain the following programming result text: "'Ask the Heaven and Earth Where is Good, Ask the Heaven and Earth Where is Good.' Student, what are you doing? The teacher found that Wang, who was sitting at the back, had a small notebook in her hand and was recording something quickly."

[0061] As a specific example of recipe generation, the target scenario input by the user may be: "'How to make shredded pork with fish flavor?' the boy asked the professor curiously. The professor thought for a long time and said slowly". Using the device to implement the natural language programming method proposed in the present invention may obtain the following programming result text: "'The method of making shredded pork with fish flavor is actually not complicated. It is just to chop the tofu and meat into pieces and put them into the pot to stew. Then put the minced meat into the pot and stir-fry. Finally, lift the lid and put it into the pot. Add a little water and you can eat.' 'Is that all?' 'Yes, that's fine, but some seasoning is needed.' 'Seasoning? What seasoning?' 'Oil, vinegar, pepper, ginger, onion, pepper, sugar, eggs, egg liquid.'"

[0062] In some embodiments of the present invention, the overall structure of the present invention includes: a front-end interface, a back-end service, and the aforementioned language model. In order to use natural language programming, a language model is first required. The user inputs the content to the language model through the back-end service through the front-end interface, thereby obtaining the results required by natural language programming.

[0063] About the front-end interface

[0064] In some embodiments of the present invention, the aforementioned step S11 specifically includes: using a front-end interface to receive text input by a user, the text including a scene described in a natural language. Optionally, the front-end interface may use a Vue front-end framework.

[0065] Optionally, the front-end interface can adopt two modes: natural input interface and template matching input interface, so that users can perform natural input or template input.

[0066] As a specific example of a natural input interface, the natural input interface includes a plain text box for receiving user input. For example:

[0067] "How to make fish-flavored pork shreds?" the student asked the professor curiously. The professor thought for a long time and slowly said:

[0068] "The method of making fish-flavored shredded pork is not complicated. Simply chop the tofu and meat into small pieces and put them into the pot to stew. Then put the minced meat into the pot and stir-fry. Finally, lift the lid and put it in the pot. Add a little water and you can eat."

[0069] In the above example, the first paragraph is the user input, and the second paragraph is the result of the machine calculation. In this case, the user constructs a scenario: the student asks a question to get the answer to "How to make fish-flavored pork shreds". The user does not need to provide additional information.

[0070] As a specific example of template matching, the template matching input interface includes template text, a control for receiving user input, an execution button, and a control for displaying output results. For example, in an environment similar to the above-mentioned natural input interface example, the interface can be:

[0071] Template: "[Input]" the student asked the professor curiously. The professor thought for a long time and slowly said: "[Output]"

[0072] Input: How to make fish-flavored shredded pork

[0073] Execute button (click once)

[0074] Output: The method of making fish-flavored pork shreds is actually not complicated. Simply chop the tofu and meat into small pieces and stew them in a pot...

[0075] Please note that both [Input] and [Output] can be multiple.

[0076] About backend services

[0077] In some embodiments of the present invention, the aforementioned step S12 specifically includes: utilizing a backend service to send the scene described in natural language received by the front-end interface to a language model.

[0078] In some embodiments of the present invention, the method proposed by the present invention further includes: using a backend service to return the programming result text output by the language model to a front-end page for display on the front-end page.

[0079] Optionally, the backend service can use the Flask backend framework, which is responsible for receiving the user input returned by the frontend page, then inputting it into the model, and then returning the output of the model to the frontend page.

[0080] Figure 2 The figure shows a model inference process of an embodiment of the present invention, that is, a process of performing natural language programming using a pre-trained language model after pre-processing the user input.

[0081] like Figure 2 As shown, in some embodiments of the present invention, the backend service is specifically used to perform the following processing: extract text, clean text, perform Tokenizer (or word segmenter) processing on the scene based on natural language description input by the user, return the text Token id, and then send it to the language model.

[0082] Among them, text cleaning specifically includes: removing blanks, abnormal characters, character unification (full-width or half-width), etc. Tokenizer processing specifically includes: using BertTokenizer (a text processing technology used in the Bert language model), Chinese will be cut into each character, and English will be cut into each word, to obtain a list of text tokens (Token represents the unit after text segmentation). Then based on the token list, it is converted into the corresponding identifier (or id) in the vocabulary (also called word table, each line in the table contains one word, word or character, and the line number is the index number of the word), so as to obtain the Token id (Token is based on the index number in the vocabulary).

[0083] It should be noted that in the description of this specification, no distinction is made between words, phrases, characters, and characters in the corpus and the text input by the user. In fact, a single character can also be regarded as a word, and both characters and phrases can be processed accordingly using the method or device exemplified in the present invention. Therefore, the words and characters in the vocabulary and character table can be collectively referred to as units or collectively referred to as word units.

[0084] About language models

[0085] In some embodiments of the present invention, in the process of training a language model, the training data input to the language model includes large-scale natural language descriptions of scenes, and the scenes include content related to characters and tasks. The scenes described in natural language in the training data may be referred to as training scenes. Optionally, the corpus used for training includes a variety of characters, a variety of tasks, and execution results corresponding to the tasks.

[0086] In some embodiments of the present invention, in the process of training a language model, the training data input to the language model includes a corpus of a target genre. Optionally, the corpus of the target genre includes a novel corpus, a story corpus, and / or a news corpus.

[0087] Figure 3 The model training process of an embodiment of the present invention is shown, which obtains corpus used as training data, performs data cleaning and other preprocessing on the training corpus to obtain training data, and then uses the training data to train a language model (such as a model of a Transformer architecture), and finally obtains a trained language model.

[0088] like Figure 3As shown, in some embodiments of the present invention, a language model is used for creation based on input text. The language model can adopt a convolutional neural network (CNN), a recurrent neural network (RNN), and a long short-term memory neural network (LSTM). Specifically, the language model can adopt but is not limited to: Transformer architecture. The architecture of the model in some examples of the present invention is provided with memory (also called memory vector). Memory is a vector after compression of previous text knowledge, which mainly gives the model the ability to remember long texts, allowing the model to remember texts that appeared in distant places before. It is a model hyperparameter, the longer the better, but the longer the machine, the higher the requirements. Optionally, the corpus of this language model is mainly a large-scale novel corpus, which is input into the Transformer architecture for training to obtain a text generation model. It should be noted that the present invention does not limit the language model used, and the aforementioned Transformer architecture can be adopted, or models of other architectures can be adopted.

[0089] In some optional embodiments of the present invention, before the step of inputting the training data into the language model for training in the training phase, and / or before the step of inputting the target scene described in the natural language input by the user into the pre-trained language model in the inference phase, it also includes: judging the style type of the corpus used as training data and / or the target scene, and recording the judgment result in the style type field (for example, the field identifying the style type can be placed at the head of the text data) for model training or model inference according to the style type field. By judging and recording the style type of the corpus, the computer can better understand it, thereby achieving a more accurate effect.

[0090] Please note that Figure 3 Data cleaning in Figure 2 The "text cleaning" in is different.

[0091] The data cleaning of the examples of the present invention is applicable to the model training stage, and a large amount of raw data is screened and converted to obtain training data. For example, in some embodiments of the present invention, data cleaning includes screening the style type of the corpus: before inputting the training data into the language model for training in the training stage, it also includes: judging the style type of the corpus, and selecting the corpus that conforms to the target style as training data. For example, only the corpus of novels, stories and news is selected as training data, and the corpus of poetry, prose and other categories is not used. It should be noted that data cleaning can be achieved in a variety of ways, such as by judging whether the data conforms to the preset target rules, and pre-training the model used for data cleaning and using the data cleaning model to screen and convert the raw data.

[0092] As an optional embodiment of the present invention, data cleaning includes selecting corpus of novel, story and news genre types as training data, and judging the genre type of the corpus by judging whether the original corpus data contains preset target words, for example, whether it belongs to a novel, a story or news. Optionally, the target words include characteristic words of the novel category such as "chapter" and "episode" and characteristic words of the news category such as "report in a certain month" and "report on a certain day".

[0093] The text cleaning of the example of the present invention is applicable to the model inference stage, which belongs to text preprocessing and is used to convert the received user input text into a text capable of or suitable for the natural language programming method of the present invention. As an optional example, text cleaning is performed before the aforementioned step S12, including removing target characters, removing abnormal characters, and / or character unification, etc. Among them, removing the target characters specifically includes: deleting the target characters from the scene described in the received natural language in text form, for example, deleting the space characters and / or line breaks in the received user input text. Removing abnormal characters includes deleting special symbols that are not related to the natural language description. Character unification includes unifying characters into full-width characters or into half-width characters.

[0094] In some optional embodiments of the present invention, before inputting the training data into the language model for training, the training data is also split into multiple sub-training data to distinguish different scenes. Generally speaking, the results of splitting the training data include but are not limited to: each sub-training data corresponds to a scene. It should be noted that in fact, in some examples, a sub-training data may also include multiple scenes, and it is only necessary to divide different scenes into different sub-training data as much as possible to improve the accuracy of the trained language model. The method proposed in the present invention can make the trained language model more accurate and enable the computer to better understand the scene by dividing the content of the original corpus into scenes when training the language model and then inputting it into the speech model for training.

[0095] It should be noted that the aforementioned splitting of the training data into multiple sub-training data can be performed using preset rules or pre-trained models. As an optional embodiment, the aforementioned splitting of the training data into multiple sub-training data includes: identifying location words, and / or time words, and / or chapter separator words in the corpus; according to the location words, and / or time words, and / or chapter separator words, the content of the corpus is divided into multiple parts, each part being a sub-training data. Among them, the aforementioned location words include words used to describe the location and words used to describe the change of location. The aforementioned time words include words used to describe the time period or time point, and words used to describe the change of time. The aforementioned chapter separator words are used to describe the beginning or end of chapters, sections, chapters, and scenes, including but not limited to: the sequence numbers of chapters, chapters, and scenes, such as the first chapter, the second chapter, the prologue, the final chapter, the first chapter, the second chapter, the first act, the second act, etc., and common phrases used to separate chapters, chapters, and scenes, such as, and listen to the next chapter decomposition, etc. The method proposed in the present invention enables the computer to better understand the scene and improve the accuracy of the trained language model by splitting the content of different chapters, different locations, and different times into different training data for training.

[0096] The natural language programming method of the present invention also has multi-language capabilities. In some optional embodiments of the present invention, the training process of the language model also includes: before the word segmenter is processed, the corpus used as training data is translated into the corpus of the target language, so as to train the language model with the translated corpus. In some optional embodiments of the present invention, before the aforementioned step S12 is performed, the natural language programming method of the present invention also includes: before the word segmenter is processed, the target scene described in the natural language input received by the user is translated into the target scene described in the natural language of the target language. Generally speaking, the target language is a language consistent with the language used in the model training stage, such as Chinese, English, etc. It should be noted that multiple target languages ​​can be preset, and in this step, a target language corresponding to the language type of the user input text is determined, and the text is translated into the target language; specifically, it can be that multiple target languages ​​of different language systems are pre-set, and the text input by the user is translated into the target language of the corresponding language system, for example, if the user inputs French or German, it is translated into English, and if the user inputs Thai or Burmese, it is translated into Chinese. By translating the natural language description scene input by the user into the target language, it is convenient to use the language model to process the natural language description scene, and it is convenient to obtain accurate programming results. It should be noted that this step can be regarded as a step in text cleaning (or text preprocessing), or it can also be regarded as an independent step performed before or after text cleaning. In addition, the method may also include: translating the programming result text into a text corresponding to the original language of the target scene described in natural language, and then displaying it to the user.

[0097] In some embodiments of the present invention, the training process of the language model includes: inputting part of the content in the training data as the preceding text into the current language model, using the current language model to infer the following text of part of the content in the training data, and comparing the inferred following text with the actual following text, and then adjusting the parameters of the current language model according to the comparison results by back propagation. The parameters of the language model are adjusted by gradient descent during training. If the inferred following text is consistent with the actual following text or the error is small, the weight of the parameters of the current model is increased; if it is inconsistent or the error is large, the weight is reduced.

[0098] In some embodiments of the present invention, the aforementioned step S13 includes: inputting the target scene as a preceding context into a pre-trained language model, and using the pre-trained language model to infer the following context of the target scene.

[0099] In some embodiments of the present invention, specific strategies adopted by the language model include: vocabulary sampling (also called vocabulary sample, character list sampling); and / or, vocabulary probability smoothing (or called character list probability smoothing); and / or, a longer memory vector (memory), for example, a memory of length 1280.

[0100] Among them, when the language model is used to process the previous text (including training corpus or user input text), the subsequent text of the previous text can be inferred character by character or word by word. That is, first infer which unit in the vocabulary the first character (or word) after the previous text is, and then infer which unit in the vocabulary the second character (or word) after the previous text is, and so on to infer all the subsequent texts. It should be noted that the inferred first character can also be input into the language model as the previous text to infer the second character after the previous text. When inferring each character, the vocabulary sampling method can be used.

[0101] It should be noted that most languages ​​have a limited number of characters (or words), and the number of commonly used characters (or commonly used words) in all languages ​​is limited. For example, there are about 5,000 commonly used characters in Chinese and about 30,000 to 40,000 commonly used words in English. Therefore, the method shown in the present invention may include: setting corresponding identifiers (abbreviated as IDs) for all words (or all commonly used words) in advance to form a vocabulary (or word table).

[0102] The foregoing vocabulary sampling refers to: inputting the foregoing text into a language model, and the output of the language model is a multi-dimensional vector (for example, when using Chinese, it can be a 5000-dimensional vector). Each dimension of this vector corresponds to a character in the character table (or vocabulary). The value of each dimension of this vector represents the probability that the language model infers that the corresponding character in the character table (or vocabulary) is selected as the next character of the foregoing text. For the sake of convenience, the output multi-dimensional vector of the language model is called the vocabulary probability (or called the character table probability, or called the subsequent text probability vector). In some examples, the sum of the probabilities of all words in the vocabulary probability is 1; in other examples, the probability of each word in the vocabulary probability is independent and is a number between 0 and 1, and the sum of the probabilities of all words in the vocabulary probability is not necessarily 1. Then, according to the vocabulary probability and in accordance with a certain strategy, the next word of the foregoing text is determined. The process of training the language model and the process of generating text using the language model can both include vocabulary sampling.

[0103] It should be noted that during vocabulary sampling, after obtaining the vocabulary probability, various strategies can be adopted to determine the next word of the foregoing text. As an optional embodiment, select the word with the highest probability in the vocabulary as the next word of the foregoing text. As another optional embodiment, select one of the words with the highest probability in the vocabulary as the next word of the foregoing text. For example, one can be randomly selected from the words with the highest probability, or multiple generated result texts can be output simultaneously as alternative options or for the user to choose by themselves. As yet another optional embodiment, divide the [0, 1] interval into multiple sub-intervals according to the probabilities of each word in the vocabulary probability. Each sub-interval corresponds to a word in the vocabulary, and the probability of the word corresponds to the size of the sub-interval. Then generate a random number between 0 and 1, determine the sub-interval into which the random number falls as the target sub-interval, and select the word corresponding to the target sub-interval as the next word of the foregoing text. For example, there are two words A and B in the vocabulary, the weight of A is 0.9, and the weight of B is 0.1. Thus, the [0, 1] interval can be divided into two sub-intervals [0, 0.9] and [0.9, 1]. By generating a random number between 0 and 1 and determining the size of this random number, if the random number is greater than 0.9, select A as the subsequent text, and if the random number is between 0.9 and 1, select B as the subsequent text.

[0104] As a specific example of the vocabulary sampling of the present invention, the foregoing text (training corpus or text input by the user) input into the language model is "The weather is very good now". When using the vocabulary sampling method and adopting the strategy of selecting the character with the highest probability in the character table, the result after processing by the language model may be that the character "ran" in the character table has the highest probability as the next character of the foregoing text. Then select "ran" as the next character of the foregoing text, and then continue to judge which character in the character table has the highest probability as the next character of the foregoing text. Finally, based on the foregoing text "The weather is very good now", the text "Then it rained" may be generated.

[0105] In some embodiments of the present invention, a vocabulary is pre-set. And, before the aforementioned inputting of part of the content in the training data into the current language model, the training process of the language model includes: extracting text and tokenizer processing of the original corpus for training to segment the text of the original corpus into multiple word units; determining the identifier corresponding to each word unit of the original corpus in the vocabulary according to the vocabulary to obtain the word segmentation identifier vector of the original corpus, and using the word segmentation identifier vector of the original corpus as training data to train the language model. In fact, the corpus content data is converted into training data in the form of vectors through the word segmenter and the vocabulary. The training process of the language model in the aforementioned embodiment specifically includes: inputting the word segmentation identifier vector of part of the content in the training data as the previous text into the current language model, and the current language model outputs the corresponding subsequent text probability vector according to the word segmentation identifier vector of part of the content in the training data; subtracting the subsequent text probability vector corresponding to the part of the content in the training data output by the model from the actual subsequent text word segmentation identifier vector of part of the content in the training data to obtain an error vector; and, weighting the parameters of the current language model according to the error vector. After that, the next round of training is continued using the speech model with adjusted weights. Each dimension of the context probability vector corresponds to a word unit in the vocabulary, and the value of each dimension of the context probability vector represents the probability of the corresponding word unit in the vocabulary inferred by the language model being selected as the context.

[0106] It should be noted that the present invention does not limit the method of weighting the parameters of the current language model using the error vector. As an optional example, determine the norm of the error vector (the specific norm type is also not limited, for example, it can be a 1-norm, a 2-norm, or an ∞-norm), considering that generally speaking, the larger the norm of the error vector, the greater the difference between the context inferred by the current language model and the actual context, and calculate the weights of the parameters of the current language model according to the inverse of the norm of the error vector. As another optional example, when inferring the context word by word, each dimension of the context probability vector output by the model is a decimal of 0-1; in the word segmentation identification vector of the actual context, the one-dimensional value corresponding to the word of the actual context is 1, and the values ​​of the other dimensions are all 0. Therefore, the weight can be calculated using only one dimension of the error vector that corresponds to the actual words in the following context, and the weight can be used to weight the parameters of the current language model; alternatively, a first weight can be calculated based on one dimension of the error vector that corresponds to the actual words in the following context, and a second weight can be calculated based on other dimensions in the error vector, and the first weight and the second weight can be combined to weight the model parameters.

[0107] In some embodiments of the present invention, a vocabulary is pre-set. In addition, the target scene described in natural language in the aforementioned embodiment is input into a pre-trained language model, including: extracting text and word segmentation processing of the target scene to segment the text of the target scene into multiple word units, and determining the corresponding identifier of each word unit of the target scene in the vocabulary according to the vocabulary, so as to obtain the word segmentation identifier vector of the target scene, and sending the word segmentation identifier vector of the target scene to the pre-trained language model. In fact, the target scene described in natural language is digitized into a target scene in vector form through the word segmentation and the vocabulary. In the aforementioned embodiment, the use of the language model to infer the context of the target scene described in natural language includes: the pre-trained language model outputs the corresponding context probability vector according to the word segmentation identifier vector of the target scene; and, determining the context of the target scene according to the context probability vector. Among them, each dimension of the context probability vector corresponds to a word unit in the vocabulary, and the value of each dimension of the context probability vector represents the probability that the corresponding word unit in the vocabulary inferred by the language model is selected as the context.

[0108] In the vocabulary sampling process described above, the model is used to generate the vocabulary probability (also called word table probability, or called the subsequent text probability vector), that is, the model is used to calculate the probability of each word unit in the vocabulary (also called word table) being the next word (also called word) of the previous text. Sometimes, the probability of the word generated by the model in Top1 will be relatively large, for example, the probability of the word with the highest probability of being the next word of the previous text in the vocabulary may reach 0.99. But this makes the certainty too high every time the following text is generated. In some embodiments of the present invention, a vocabulary probability smoothing technology can be used: by pre-adjusting the model parameters so that the probabilities of multiple words with the highest probability are at a comparable level when the model generates the vocabulary probability, or after the model generates the vocabulary probability, the vocabulary probability is smoothed using a smoothing function so that the probabilities of multiple words with the highest probability are at a comparable level, that is, the probabilities of the words ranked in TopK (indicates the first K, it should be noted that the value of K is not limited) are all at a similar level, so that they can be selected in TopK without focusing on selecting a certain word.

[0109] In some embodiments of the present invention, the aforementioned current language model outputs a corresponding subsequent context probability vector according to the word segmentation identification vector of part of the content in the training data, specifically including: smoothing the probability of each word unit in the vocabulary inferred by the language model being selected as the subsequent context to obtain a smoothed subsequent context probability vector. In the training process of the language model, the smoothed subsequent context inference probability vector is used for subsequent processing such as determining the error vector and weighting the language model.

[0110] In some embodiments of the present invention, the aforementioned pre-trained language model outputs a corresponding context probability vector based on the word segmentation identification vector of the target scene, specifically including: smoothing the probability of each word unit in the vocabulary inferred by the language model being selected as the context to obtain a smoothed context probability vector. The aforementioned determination of the context of the target scene based on the context probability vector includes: determining multiple word units with the highest probability in the vocabulary based on the smoothed context probability vector, and determining the context based on the multiple word units with the highest probability.

[0111] The aforementioned smoothing process of the vocabulary probability using a smoothing function includes but is not limited to: taking the square root of the original vocabulary probability generated by the model, and using the square rooted vocabulary probability for the subsequent text generation process. Taking a word in the vocabulary as an example, if the weight of word A is x, take the square root of the weight x, and use the obtained x^0.5 for subsequent processing. For example, in one example, there are two words A and B in the vocabulary, the weight of A is 0.9, and the weight of B is 0.1. Through the smoothing operation of taking the square root of the weights, the original probabilities of A and B of 0.9 and 0.1 can be changed to 0.94 and 0.31, thereby reducing the weight ratio of A and B, which originally had a larger weight ratio, so that the probabilities of different words are at a similar level.

[0112] It should be noted that after using the vocabulary probability smoothing technology, a variety of methods can be used to determine the final generated text based on the multiple word units with the highest probability in the vocabulary. For example, one can be randomly selected from the K words with the probability of being in the TopK in the vocabulary, or the word with the highest probability can be selected from the TopK, or multiple generated result texts can be output at the same time as alternative options or for users to choose from.

[0113] The present invention can obtain more diverse returns by utilizing vocabulary probability smoothing, thereby being able to obtain multiple different intermediate results or alternative results of natural language programming. Utilizing multiple intermediate results or alternative result texts to obtain programming result text can improve the diversity of programming results.

[0114] In some embodiments of the present invention, the aforementioned step S13 of using the language model to generate text corresponding to the target scene includes: while performing the aforementioned generation of the text corresponding to the result of the character performing the target task, the voice model can also be used to obtain the explanation text, that is, the model can be required to give the reason for the output. In fact, the process of the aforementioned steps S12 and S13 can be performed by taking the target scene input by the user and the programming result text determined by the aforementioned method as input, and the output can be used as the explanation text. For example, the target scene described by the natural language input by the user includes "Can tomatoes be eaten?" Then, using the natural language programming method of the present invention, while obtaining the result "Yes", the explanation "Because tomatoes are a very common vegetable, the lycopene contained in tomatoes can improve the body's immunity" can also be obtained, and this explanation can also be obtained by the language model.

[0115] In some embodiments of the present invention, the template matching of the language model can be used to complete the acquisition of programming results. In a specific implementation of the present invention, the target scene described by the natural language input by the user is "writing Spring Festival couplets", then the language template can be:

[0116] "Today we are going to write a couplet. The theme is <Input 1>. The first line of this couplet is <Input

[0117] Output 1>, the next line is <output 2>"

[0118] When the user enters a specific requirement, for example <input 1> = "family harmony", this content is fed into the language model, and the language model can output the following text:

[0119] "Today we are going to write a couplet with the theme of 'family harmony'. The first line of this couplet is

[0120] 'May you be blessed as vast as the East Sea and live as long as the Southern Mountains', and the second line is 'May you be harmonious and blessed for thousands of years'".

[0121] Then we get the result of executing this program: <output 1> = "blessings as vast as the east sea, and longevity as great as the southern mountains", <output 2> = "harmony and happiness, and eternal blessings".

[0122] It should be noted that different symbols can be used to implement the natural language programming method. For example, in the above-mentioned embodiment of writing Spring Festival couplets, the above-mentioned angle brackets <> can be changed to square brackets without affecting the implementation of template matching.

[0123] It should be noted that the natural language programming method of the present invention can be run locally or on the server. As an optional embodiment, the system or device corresponding to the natural language programming method of the present invention includes a client device and a server device, and the aforementioned step S11 is implemented using a client such as a smart terminal, and the aforementioned steps S12 and S13 are implemented using a server such as a server, and data interaction is achieved through communication between the client and the server. As another optional embodiment, the system or device corresponding to the natural language programming method of the present invention is an integrated device for simultaneously implementing all steps of the natural language programming method.

[0124] The natural language programming method proposed in the present invention has many differences from the prior art, as follows.

[0125] Compared with the existing programming methods, the compiler corresponding to the natural language programming method proposed by the present invention does not need to strictly follow the grammatical analysis, but analyzes through the language model, and uses natural language to construct stories and scenes to drive characters to achieve tasks. Programmers only need to describe the task, and do not need to indicate how to complete it, and can even let the language model explain it. In addition, the quality of user programming is converted from the original programming ability to narrative ability. The clearer the description of the target scene, the better the output result.

[0126] Compared with few-shots learning, the natural language programming method proposed in the present invention does not require the construction of examples, and the present invention is scenario-driven so that parameters can be controlled more accurately.

[0127] Compared with the existing natural language understanding methods and natural language generation methods, the current natural language understanding tasks and language generation are designed for specific types of tasks, such as text classification, summary extraction, etc.; the natural language programming method proposed in the present invention is different from them in that it relies on expanding the language model and can be adapted to various programming situations. It is not limited to processing a specific task but can complete a variety of functions, and has strong general and generalization capabilities.

[0128] By using the natural language programming method proposed in the present invention, a scene is created through natural language, and the characters in the scene are driven to complete the task, so that the desired result can be achieved. At the same time, if there are certain errors in the program or language, the method of the present invention can still run and will not go wrong due to a small error. At the same time, because the present invention is a natural language task, it can support the language of any country through translation, rather than a specific one or several languages ​​such as English and Chinese.

[0129] An embodiment of the present invention further provides a natural language programming device, which includes: a front-end interface module, a back-end service module, and a language model processing module.

[0130] The front-end interface module is used to receive a target scene (scenario, story) described in natural language input by a user, wherein the target scene described in natural language includes characters and target tasks.

[0131] The backend service module is used to input the target scene into a pre-trained language model.

[0132] The language model processing module is used to: use a pre-trained language model to generate text corresponding to a target scenario, specifically, use the language model to generate text corresponding to the result of a character performing a target task as programming result text.

[0133] In addition, the various natural language programming devices shown in the embodiments of the present invention include modules and units for executing the methods corresponding to the aforementioned embodiments, and their detailed descriptions and technical effects can be referred to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0134] It should be noted that the natural language programming device of the present invention can be an independent device; or it can also include multiple sub-devices that cooperate with each other, each of which includes one or more modules of the front-end interface module, the back-end service module, and the language model processing module. As an optional embodiment, the natural language programming device of the present invention includes a client device and a server device, the client device includes the aforementioned front-end interface module, the server device includes the aforementioned back-end service module and the language model processing module, and data interaction is achieved through communication between the client and the server.

[0135] Figure 4 1 is a schematic block diagram illustrating a natural language programming device according to an embodiment of the present invention. Figure 4 As shown, the natural language programming device 100 according to an embodiment of the present disclosure includes a memory 101 and a processor 102.

[0136] The memory 101 is used to store non-temporary computer-readable instructions. Specifically, the memory 101 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0137] The processor 102 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the natural language programming device 100 to perform desired functions. In one embodiment of the present disclosure, the processor 102 is used to execute the computer-readable instructions stored in the memory 101, so that the natural language programming device 100 performs all or part of the steps of the natural language programming method of each embodiment of the present disclosure.

[0138] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present invention.

[0139] For detailed description and technical effects of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0140] An embodiment of the present invention further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on a device, the device executes the above-mentioned related method steps to implement the natural language programming method in the above-mentioned embodiment.

[0141] An embodiment of the present invention further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the natural language programming method in the above-mentioned embodiment.

[0142] In addition, an embodiment of the present invention also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the natural language programming method in the above-mentioned method embodiments.

[0143] Among them, the device, computer storage medium, computer program product or chip provided by the present invention are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0144] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A natural language programming method, characterized in that: The method comprises the following steps: Receiving a target scene described in natural language input by a user, wherein the target scene includes characters and target tasks; Inputting the target scene into a pre-trained language model; Using the language model, driving the character in the target scene to achieve the target task, so as to generate text corresponding to the result of the character performing the target task as programming result text; The step of using the language model to generate text corresponding to the result of the character performing the target task includes: inputting the target scene into a pre-trained language model, and using the pre-trained language model to infer the subsequent text of the target scene using a preset strategy; The preset strategy includes vocabulary sampling and / or vocabulary probability smoothing and / or memory vector, wherein when the vocabulary sampling is adopted, the context of inferring the target scene includes: Preset a vocabulary, obtain vocabulary probability, and divide the preset probability interval into multiple subintervals according to the probability of each word in the vocabulary probability, wherein each subinterval corresponds to a corresponding word in the vocabulary, and the probability of the word corresponds to the size of the subinterval; Generate a random number in the preset probability interval, take the subinterval into which the random number falls as a target subinterval, and take the word corresponding to the target subinterval as the next word of the previous text; The method of obtaining the vocabulary probability includes: inputting the previous text into the language model, obtaining a multidimensional vector output by the language model, and using the multidimensional vector output by the language model as the vocabulary probability, wherein each dimension in the multidimensional vector corresponds to a word in the vocabulary, and the value of each dimension of the multidimensional vector represents the probability that the language model infers that the corresponding word in the vocabulary is selected as the next word of the previous text.

2. The natural language programming method according to claim 1, characterized in that: In the process of training the language model, the training data input to the language model includes large-scale natural language described scenes, and the scenes include characters and tasks.

3. The natural language programming method according to claim 2, characterized in that: In the process of training the language model, the training data input into the language model includes corpus of a target genre; wherein the corpus of the target genre includes novel corpus, story corpus and / or news corpus.

4. The natural language programming method according to claim 3, characterized in that: Before inputting the training data into the language model for training, the method further includes splitting the training data into a plurality of sub-training data to distinguish different scenarios.

5. The natural language programming method according to claim 4, characterized in that: The step of splitting the training data into a plurality of sub-training data comprises: Identify location words, and / or time words, and / or chapter separator words in the corpus, wherein the chapter separator words are used to describe the beginning or end of a chapter, section, episode, or act; According to the location words, and / or the time words, and / or the chapter separator words, the content of the corpus is divided into multiple parts, and each part serves as a sub-training data.

6. The natural language programming method according to claim 3, characterized in that: Before inputting the training data into the language model for training, and / or before inputting the target scene into a pre-trained language model, it also includes: judging the style type of the corpus used as training data and / or the target scene, and recording the judgment result in the style type field for model training or model inference according to the style type field; and / or, Before inputting the training data into the language model for training, the method further includes: determining the style type of the corpus, and selecting corpus that meets the target style as training data.

7. The natural language programming method according to claim 2, characterized in that: The training process of the language model includes: inputting part of the content in the training data into the current language model, using the current language model to infer the context of the part of the content in the training data, and comparing the inferred context with the actual context, and adjusting the parameters of the current language model according to the comparison results in a back-propagation manner.

8. The natural language programming method according to claim 7, characterized in that: The method further comprises presetting a vocabulary; Before inputting part of the content in the training data into the current language model, the training process of the language model includes: extracting text and performing word segmentation processing on the original corpus used for training to segment the text of the original corpus into multiple word units, determining the corresponding identifier of each word unit of the original corpus in the vocabulary to obtain the word segmentation identifier vector of the original corpus, and using the word segmentation identifier vector of the original corpus as training data to train the language model; The training process of the language model specifically includes: the current language model outputs a corresponding subsequent context probability vector according to the word segmentation identification vector of the partial content in the training data; subtracts the subsequent context probability vector corresponding to the partial content output by the model from the word segmentation identification vector of the actual subsequent context of the partial content to obtain an error vector; and, weighting the parameters of the current language model according to the error vector; The inputting the target scene into the pre-trained language model comprises: extracting text and performing word segmentation processing on the target scene to segment the text of the target scene into a plurality of word units, and determining the corresponding identifier of each word unit of the target scene in the vocabulary to obtain a word segmentation identifier vector of the target scene, and sending the word segmentation identifier vector of the target scene to the pre-trained language model; The inferring the context of the target scene by using the language model comprises: the pre-trained language model outputs a corresponding context probability vector according to the word segmentation identification vector of the target scene; and, determining the context of the target scene according to the context probability vector; Among them, each dimension of the subsequent context probability vector corresponds to a word unit in the vocabulary, and the value of each dimension of the subsequent context probability vector represents the probability of the corresponding word unit in the vocabulary inferred by the language model being selected as the subsequent context.

9. The natural language programming method according to claim 8, characterized in that: The using the current language model to infer the context of the part of the content in the training data, and / or the using the pre-trained language model to infer the context of the target scene, includes: Infer the following text word by word or phrase by phrase.

10. The natural language programming method according to claim 8, characterized in that: The current language model outputs a corresponding context probability vector according to the word segmentation identification vector of part of the content in the training data, and / or the pre-trained language model outputs a corresponding context probability vector according to the word segmentation identification vector of the target scene, including: smoothing the probability of each word unit in the vocabulary inferred by the language model being selected as the context to obtain a smoothed context probability vector; Determining the context of the target scene according to the context probability vector includes: determining a plurality of word units with the highest probabilities in a vocabulary according to the smoothed context probability vector, and determining the context according to the plurality of word units with the highest probabilities.

11. The natural language programming method according to claim 8, characterized in that: The training process of the language model further includes: before the word segmentation process, translating the corpus used as training data into corpus of the target language, so as to train the language model using the translated corpus; The inputting the target scene into the pre-trained language model further includes: before the word segmentation processing, translating the received target scene into a target scene described in a natural language of a target language, so that the language model processes the translated target scene; The method further includes: translating the programming result text into a text corresponding to the original language of the target scenario, and then displaying it to the user.

12. The natural language programming method according to claim 1, characterized in that: The language model is based on a Transformer architecture, which is provided with a memory vector.

13. The natural language programming method according to claim 1, characterized in that: The target scene described in natural language receiving user input includes: the target scene receiving user input using a front-end interface; wherein the front-end interface includes a natural input interface or a template matching input interface, the natural input interface includes a plain text box for receiving user input, and the template matching input interface includes a template text, a control for receiving user input, an execution button, and a control for displaying output results; The inputting the target scene into the pre-trained language model comprises: using a backend service to send the target scene received by the front-end interface to the language model; The method also includes: utilizing a backend service to return the programming result text output by the language model to the front-end interface for display on the front-end interface.

14. A natural language programming device, characterized in that: The device comprises: A front-end interface module is used to receive a target scene described in natural language input by a user, wherein the target scene includes characters and target tasks; A backend service module, used for inputting the target scene into a pre-trained language model; A language model processing module, used to use the language model to drive the character in the target scene to achieve the target task, so as to generate text corresponding to the result of the character performing the target task as programming result text; The step of using the language model to generate text corresponding to the result of the character performing the target task includes: inputting the target scene into a pre-trained language model, and using the pre-trained language model to infer the subsequent text of the target scene using a preset strategy; The preset strategy includes vocabulary sampling and / or vocabulary probability smoothing and / or memory vector, wherein when the vocabulary sampling is adopted, the context of inferring the target scene includes: Preset a vocabulary, obtain vocabulary probability, and divide the preset probability interval into multiple subintervals according to the probability of each word in the vocabulary probability, wherein each subinterval corresponds to a corresponding word in the vocabulary, and the probability of the word corresponds to the size of the subinterval; Generate a random number in the preset probability interval, take the subinterval into which the random number falls as a target subinterval, and take the word corresponding to the target subinterval as the next word of the previous text; The method of obtaining the vocabulary probability includes: inputting the previous text into the language model, obtaining a multidimensional vector output by the language model, and using the multidimensional vector output by the language model as the vocabulary probability, wherein each dimension of the multidimensional vector corresponds to a word in the vocabulary, and the value of each dimension of the multidimensional vector represents the probability that the language model infers that the corresponding word in the vocabulary is selected as the next word of the previous text.

15. The natural language programming device according to claim 14, further comprising a module for executing the method according to any one of claims 2 to 13.

16. A natural language programming device, comprising: a memory for storing non-transitory computer-readable instructions; as well as A processor, configured to run the computer-readable instructions, so that when the computer-readable instructions are executed by the processor, the natural language programming method according to any one of claims 1 to 13 is implemented.

17. A computer storage medium, characterized in that: The method comprises computer instructions, which, when executed on a device, cause the device to execute the natural language programming method according to any one of claims 1 to 13.

Images