Text data generation method, computer device and storage medium

Abstract

The embodiment of the present application provides a text data generation method, a computer device and a storage medium, the text data generation method comprising: obtaining a third text without grammatical defects; inputting the third text into a generation model, generating a first generated text corresponding to the third text, and a target probability for indicating that the first generated text has grammatical defects, the generation model is trained based on the first text without grammatical defects and the second text with grammatical defects; according to the target probability corresponding to the first generated text, determining the first generated text as the target text. The generation model learns the grammatical error rules corresponding to the second text, and the trained generation model can give full play to the excellent characteristics of diverse generation results, and can automatically construct a large number of texts containing grammatical defects.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular, to a method for generating text data, a computer device, and a storage medium. Background Art

[0002] The goal of the grammar error detection task is to detect possible grammar errors in the text. These grammar errors can be classified into categories such as "misspelled words", "incomplete components", "redundant components", "improper collocations", "inappropriate word usage", and "improper word order". The grammar error detection task requires using text with marked grammar errors for model training. However, the current scale of Chinese corpora with grammar error annotations is small, the acquisition difficulty is large, and the annotation cost is high.

[0003] Due to the lack of sufficient annotated data, the grammar error detection model is prone to overfitting on a small-scale training set, which seriously restricts the improvement of the effect of the grammar error detection model. On the other hand, the small scale of the annotated data also means that there are few grammar error situations in the data, and even an extreme situation may occur where the vast majority of errors in the data are the misuse of "de", "děi", and "dì". This results in the model being able to train on only a few error types in the data and making it difficult to learn sufficiently generalized knowledge. Summary of the Invention

[0004] Embodiments of this application provide a method for generating text data, a computer device, and a storage medium, which can automatically construct a large number of high-quality texts containing grammar defects, so as to better optimize the effect of the grammar error detection model using these texts.

[0005] In a first aspect, this application provides a method for generating text data, including:

[0006] Obtain a third text without grammar defects;

[0007] Input the third text into a generation model to generate a first generated text corresponding to the third text and a target probability indicating that the first generated text has grammar defects. The generation model is trained based on a first text without grammar defects and a second text with grammar defects;

[0008] Determine the first generated text as the target text according to the target probability corresponding to the first generated text.

[0009] In a second aspect, this application provides a computer device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of the above method when executing the computer program.

[0010] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. If the computer program is executed by a processor, the steps of the above method are implemented.

[0011] The present application discloses a text data generation method, a computer device and a storage medium, including: obtaining a third text without grammatical defects; inputting the third text into a generation model, generating a first generated text corresponding to the third text, and a target probability for indicating that the first generated text has grammatical defects, the generation model is trained based on the first text without grammatical defects and the second text with grammatical defects; according to the target probability corresponding to the first generated text, determining the first generated text as the target text. The generation model trained based on the first text and the second text can give play to the excellent characteristics of diverse generation results, and can automatically construct a large number of texts containing grammatical defects, so that the effect of the grammatical error detection model can be improved by expanding the corpus scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A schematic diagram of generating a sentence containing grammatical errors in the prior art;

[0014] Figure 2 A flowchart of a method for generating text data according to an embodiment of the present application is shown;

[0015] Figure 3 A schematic diagram of an application scenario of a method for generating text data in one embodiment;

[0016] Figure 4 is a schematic diagram of a method for generating text data in one implementation;

[0017] Figure 5 A schematic diagram of a structure of a generated model in one implementation manner;

[0018] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0021] The embodiments of the present application provide a text data generation method, a computer device, and a storage medium to realize automatic construction of a text containing grammatical defects / errors.

[0022] See also Figure 1 At present, sentences with grammatical errors (defects) are usually constructed based on random rules to expand the grammar error detection corpus. The specific steps of a typical technical solution are as follows:

[0023] Step 1: Collect a large amount of text from the Internet or other sources. To ensure that the collected text has no or few grammatical errors, the text needs to be pre-screened. The typical method is to divide all the collected text into sentences, and then use a language model that has been trained on a large-scale corpus to score each sentence. If the score is greater than a certain threshold, it is considered that the sentence is likely to have grammatical errors and will be discarded. In this way, a large number of correct sentences can be screened.

[0024] Step 2: Randomly select the starting position and the ending position in the original sentence to determine the specific position of the grammatical error segment to be constructed.

[0025] Step three: randomly select one of the six error categories: "typo", "incomplete component", "redundant component", "inappropriate collocation", "inappropriate word use" and "inappropriate word order".

[0026] Step 4: modify some of the words selected in step 2 according to the error types selected in step 3. For example, "typos", "inappropriate collocations", and "inappropriate word usage" require the original text to be randomly replaced with other words. "Incomplete components" require the original text to be removed. "Redundant components" require the original text to be randomly added with redundant words. "Inappropriate word order" requires the original text to be swapped. In short, this step performs the third level of randomness: randomly selecting the modified content. Finally, the construction of the sentence with grammatical errors is completed.

[0027] For example: The original sentence is "Leaves fall from the tree." First, use the trained language model to score this sentence and confirm that it is a correct sentence. Randomly select the 4th to 5th characters, that is, "from the tree", which means the program needs to modify "from the tree". Next, randomly select the error type of "fragmentary components", that is, the two characters "from the tree" need to be removed, and the constructed sentence is "Leaves fall from.", which has the grammar error of "fragmentary components"; or randomly select the error type of "redundant components", and randomly add words. After supplementation, the sentence "Leaves fall from the tree from the tree." with the grammar error of "redundant components" is obtained. Repeat this way to construct a large number of correct and incorrect sentence pairs.

[0028] The method of constructing data based on random rules can directly extract the modification position, modification method, and modification result of each grammar error in the constructed sentence. Combining with the original sentence can obtain correct and incorrect parallel sentence pairs, which shows that it is feasible to automatically construct a large amount of data. However, the existing methods have at least one of the following problems:

[0029] First, the triple randomness of randomly selecting the position, randomly selecting the error type, and randomly selecting the modification result determines that the distribution of grammar errors in the constructed data is random and cannot be close to the distribution of real grammar errors. It is possible that the constructed grammar errors will not appear in the real scenario at all, and such data will mislead the downstream model.

[0030] Second, there will also be such a problem when randomly selecting the modification result: the constructed error text may not really contain grammar errors. For example, "Leaves fall from the tree." may be randomly modified to "Leaves fall from the tree and drop.", and be marked as "improper collocation". However, this modified sentence恰恰 uses the rhetorical device of personification and is a sentence that conforms to grammar rules and has a good expression effect. Similar problems introduce noise into the model training and are likely to cause misjudgment of the model.

[0031] Based on this, the inventors of this application improve the text data generation method to generate a target text based on a generation model trained with a first text without grammar defects and a second text with grammar defects. The model can learn information closer to real grammar errors, so that the sentences constructed by the model have a high accuracy of containing grammar defects and a high diversity of construction results.

[0032] The text data generation method of the embodiments of this application can be applied to a terminal or a server. The terminal can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, etc.; the server can be an independent server or a server cluster. However, for the sake of easy understanding, the following embodiments will introduce the method applied to the server in detail.

[0033] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] See also Figure 2 , Figure 2 It is a flowchart of a text data generation method provided in an embodiment of the present application.

[0035] like Figure 3 As shown, the server obtains text without grammatical defects from the terminal, generates text with grammatical defects according to the text data generation method, sends the text with grammatical defects to the terminal, and / or trains a grammatical error detection model based on the text with grammatical defects.

[0036] like Figure 2 As shown, the text data generating method includes the following steps S110 to S130.

[0037] Step S110: Acquire a third text without grammatical defects.

[0038] In some optional embodiments, the text without grammatical defects is text stored locally by a device for implementing the text data generation method, text obtained by the device from a network, text obtained by the device from an input device connected to it, text obtained by the device from other electronic devices, text converted by the device based on voice information, etc.

[0039] Exemplarily, a third text is obtained from a verified book or collection, and the third text includes, for example, a sentence, but is certainly not limited thereto. The third text may be the same as the first text or may be different.

[0040] See also Figure 4 , a third text (correct sentence) without grammatical defects is screened out from a large-scale corpus, and the trained generation model can generate sentences containing grammatical errors based on the third text.

[0041] Step S120: input the third text into the trained generative model to generate a first generated text corresponding to the third text and a target probability for indicating that the first generated text has grammatical defects, wherein the generative model is trained based on the first text without grammatical defects and the second text with grammatical defects.

[0042] In some embodiments, the step of inputting the third text into a trained generative model to generate a first generated text corresponding to the third text and a target probability for indicating that the first generated text has a grammatical defect includes: inputting the third text into an encoder of the generative model to obtain a semantic vector; and a decoder of the generative model generating a first generated text and a semantic representation of a terminator of the first generated text according to the semantic vector; and performing a fully connected transformation on the semantic representation of the terminator of the first generated text to obtain a target probability for indicating that the first generated text has a grammatical defect.

[0043] For example, see Figure 5 When the decoder of the generation model generates the sentence terminator "", the semantic representation of the terminator can obtain a number γ between 0 and 1 after a full connection transformation such as a full connection layer. The number γ is used to indicate the probability of a grammatical error in the generation result, that is, the target probability for indicating that the first generated text has a grammatical defect. Optionally, the larger the target probability, the more confident the model is that it has successfully generated a text with grammatical errors.

[0044] Step S130: Determine that the first generated text is a target text according to the target probability corresponding to the first generated text.

[0045] Given a correct sentence, the trained generative model generates text that may contain grammatical errors. Then, by screening the identifier γ, the first generated text with a high probability of containing grammatical errors can be selected for downstream grammar error detection model training.

[0046] Exemplarily, when the target probability corresponding to the first generated text is greater than or equal to a preset threshold, the first generated text is determined to be the target text, and for example, the first generated text can be stored as training data for a grammatical error detection model. If the target probability γ is greater than a certain threshold, it can be determined that the first generated text successfully generated by the model has grammatical defects, and the first generated text can be stored as training data for a grammatical error detection model.

[0047] Exemplarily, after comparing the first generated text with the original text, i.e., the third text, grammatical errors in the first generated text can be obtained. A grammatical error detection model can be trained based on the first generated text and the corresponding grammatical errors. For example, the grammatical error detection model can be used for essay review.

[0048] The generative model trained based on the first text and the second text can give full play to the excellent characteristics of diverse generation results, and can automatically construct a large number of texts containing grammatical defects, thereby improving the effect of the grammatical error detection model by expanding the corpus size.

[0049] In some implementations, the second text is generated according to the first text based on a preset defect text construction rule.

[0050] Exemplarily, the first text and the corresponding second text have substantially the same content, and can be referred to as a text pair. For example, a first text without grammatical defects and a second text with grammatical defects constructed based on the first text can be referred to as a text pair; a second text with grammatical defects and a first text without grammatical defects adjusted based on the second text can be referred to as a text pair. A generative model can be trained based on the text pair so that the generative model learns the rules of grammatical defects.

[0051] In some optional embodiments, the process of generating the second text includes the following steps S101 to S102.

[0052] Step S101: Acquire a first text without grammatical defects.

[0053] The first text without grammatical defects is text stored locally by the device for implementing the text data generation method, text obtained by the device from the network, text obtained by the device from an input device connected to it, text obtained by the device from other electronic devices, text converted by the device based on voice information, etc.

[0054] Exemplarily, a first text is obtained from a verified book or collection of articles, and the first text includes, for example, a sentence, but is certainly not limited thereto.

[0055] Step S102: Based on a preset defective text construction rule, a second text with grammatical defects is generated from the first text.

[0056] Specifically, the defective text construction rules are reasonable construction rules summarized and abstracted through expert observation of real error data (defective text).

[0057] For example, we can first collect a certain amount of text that actually contains grammatical errors, and through analysis and summary, customize corresponding defective text construction rules for grammatical errors of the type "typos", "incomplete components", "redundant components", "inappropriate collocation", "inappropriate word use", "inappropriate word order", etc., so that we can construct texts with grammatical errors more accurately and reasonably.

[0058] Based on the defective text construction rules summarized by experts, a second text containing grammatical errors can be constructed from the correct first text. The second text constructed based on the defective text construction rules is closer to the real grammatical errors, for example, it can be closer to the distribution law of real errors.

[0059] In some embodiments, based on preset defective text construction rules, generating a second text with grammatical defects according to the first text includes: determining a target defect corresponding to the first text and a probability of occurrence of the target defect based on preset defective text construction rules; and constructing a second text corresponding to the first text according to the target defect based on the probability of occurrence of the target defect.

[0060] Exemplarily, based on preset defect text construction rules, the location and type of defects that can be constructed on the first text are determined according to the characters / words / components in the first text; it is also possible to determine whether to construct the defect on the first text to obtain the second text according to the probability of occurrence corresponding to the defect described in the defect text construction rules.

[0061] It should be noted that the location of the defect that can be constructed on the first text is not limited to one location, and the type of defect is not limited to one type. The defects on the second text are also not limited to one location and one type. Exemplarily, when there are multiple defects that can be constructed on the first text, one or more defects can be selected to be constructed on the first text according to the probability of occurrence corresponding to each defect, and the second text can be obtained. For example, when there are multiple defects that can be constructed on the first text, one or more defects can be selected to be constructed on the first text according to the probability of occurrence corresponding to each defect, and the count of defects actually constructed when generating the second text based on other first texts, and the second text can be obtained. The defects in the constructed multiple second texts can be closer to the distribution pattern of real errors.

[0062] Exemplarily, the method of determining a target defect corresponding to the first text and the probability of occurrence of the target defect based on a preset defect text construction rule includes: based on a preset character / word defect construction table, determining in the first text the characters / words identical to those in the character / word defect construction table to obtain the construction position of the target defect; and determining the probability of occurrence of the target defect according to the probability of occurrence corresponding to the identical characters / words in the character / word defect construction table.

[0063] Exemplarily, based on the probability of occurrence of the target defect, constructing the second text corresponding to the first text according to the target defect includes: based on the probability of occurrence of the target defect, deleting the character / word at the construction position, or replacing the character / word at the construction position with a defective character / word, or adding a defective character / word at the construction position; wherein the defective character / word is a character / word determined in the character / word defect construction table and corresponding to the same character / word.

[0064] For example, first, count the error frequencies of common Chinese characters in real grammar error texts to form an error probability table for common characters. Also, collect a table of easily confused characters for common Chinese characters. For example, characters with the same pronunciation are called "homophonic" characters, characters with similar pronunciations are called "near-homophonic" characters, and characters with similar shapes are called "near-shaped" characters. When the first text is "The leaves fall from the tree.", it can be determined that the character "飘" (piāo) in the first text has easily confused characters in the error probability table, and the probability of the character "飘" being confused is relatively high. Then, it can be determined that the character "飘" in the first text is the construction position of the target defect. By looking up the table, the easily confused characters of the character "飘" in the error probability table include "漂" (piāo), "缥" (piǎo), "彪" (biāo), etc. The defective character corresponding to the character "飘", such as "漂", can be determined according to the preset probability in the error probability table or randomly from the easily confused characters. Replace the character "飘" in the first text with "漂" to obtain the second text corresponding to the first text, such as "The leaves float from the tree.", which is a sentence containing a misspelled character.

[0065] For example, by deleting the character / word at the construction position or replacing the character / word at the construction position with a defective character / word, a second text with "internal word missing" can be constructed.

[0066] For example, by replacing the character / word at the construction position with a defective character / word, a second text with "internal word order reversed" can be constructed. For example, in the sentence "A leaf falls from the tree.", according to error probability statistics, there is a certain possibility that "飘落" (piāo luò) is misused as "落飘" (luò piāo). Therefore, the sentence with incorrect word order "A leaf floats down from the tree." can be constructed.

[0067] For example, by summarizing, it is found that common defects of "improper word usage" include four types: "different from a single character of the original word", "the same pronunciation as the original word but different characters", "self-created words", and "hyponyms / hypernyms". A word formation table for each character can be collected and established, and an error probability table for each word's improper word usage error can be statistically obtained for data construction of improper word usage. For a correct sentence, according to the probability of each word in the original text making an improper word usage error, it is decided whether to perform a replacement for improper word usage. Further, select one from the four types and replace it with a candidate word of the corresponding type. For example, in the sentence "A leaf falls from the tree.", "飘落" (piāo luò) may make an improper word usage error of the type "different from a single character of the original word" and be misused as "飘荡" (piāo dàng). Therefore, a second text that is similar to the original text but contains an improper word usage error, such as "A leaf floats in the air from the tree.", can be constructed.

[0068] For example, by summarizing the mistakes that students actually make, a word list of common auxiliary words and redundant synonyms is constructed. The auxiliary words mainly include "了" (le), "着" (zhe), etc., and the synonyms include some common synonym pairs such as "大约" (dàyuē) - "左右" (zuǒyòu), "也许" (yěxǔ) - "可能" (kěnéng), etc. When the first text includes "大约三个小时" (dàyuē sān gè xiǎoshí), it can be determined that "大约" (dàyuē) in the first text has a synonym in the word list of redundant synonyms, such as "左右" (zuǒyòu), and there is a certain probability of this synonym redundancy. Then, the synonym "左右" (zuǒyòu) is added at the corresponding position of the first text, and through the "synonym redundancy" method, a sentence containing the error of redundant components, "大约三个小时左右" (dàyuē sān gè xiǎoshí zuǒyòu), is constructed. Such a construction method utilizes the prior knowledge of experts and makes the constructed data more similar to the real data. Of course, it is not limited to this. For example, the second text of defect types such as "single-character redundancy", "internal redundancy of multi-character words", "overall redundancy of multi-character words", "overall redundancy of fragments", and "redundancy of auxiliary words" can also be constructed. Among them, single-character words refer to words with only one character, multi-character words refer to words containing two or more Chinese characters, and fragments specifically refer to combinations of consecutive multiple words.

[0069] For example, the character / word defect construction table includes a common component collocation table. For example, "verb-object collocation" includes "升-国旗" (shēng - guóqí), "唱-国歌" (chàng - guógē), etc., and can form several common collocation word pairs, such as "玩电脑" (wán diànnǎo) and "看电视" (kàn diànshì). By identifying whether there are common collocation word pairs in the original sentence of the first text and using them as candidates for constructing grammar defects; according to the probability of this defect, any one of the collocation word pairs can be replaced, so as to obtain a second text containing the error of improper collocation. For example, "看-电视" (kàn - diànshì) in the first text "我们周一晚上一般都会看电视。" (Wǒmen zhōuyī wǎnshang yībān dōu huì kàn diànshì.) is a fixed collocation. This construction method will replace it with "玩-电视" (wán - diànshì), and obtain a sentence containing the error of improper collocation, "我们周一晚上一般都会玩电视。" (Wǒmen zhōuyī wǎnshang yībān dōu huì wán diànshì.) Optionally, improper component collocation includes at least one of the following: "improper conjunction collocation", "improper quantifier collocation", "improper verb-object collocation", "improper attributive-center collocation", "improper adverbial-center collocation".

[0070] Exemplarily, determining the target defect corresponding to the first text and the occurrence probability of the target defect based on the preset defect text construction rules includes: performing component analysis on the first text to determine the components and component order in the first text; based on the preset component defect construction rules, according to the components and component order in the first text, determining the component defect type corresponding to the first text, the corresponding construction position, and the occurrence probability of the component defect type.

[0071] Exemplarily, based on the occurrence probability of the target defect, constructing the second text corresponding to the first text according to the target defect includes: deleting components, adding components, or adjusting the sorting of components at the construction position of the first text based on the component defect type and the corresponding occurrence probability.

[0072] By performing part-of-speech and syntactic analysis on the first text, the components and their sorting in the first text are determined. For example, the part-of-speech of each word in the original text and the syntactic analysis result of the whole sentence are obtained through an open-source analysis tool.

[0073] For example, according to the components and their sorting in the first text, all possible types of component deficiencies that can be constructed in the original sentence of the first text are traversed, and one is randomly selected according to a preset occurrence probability or randomly, and the corresponding component segment is deleted, and finally the second text of the component deficiency type is obtained.

[0074] For example, the types of component deficiencies may include at least one of the following: "internal word deletion", "preposition deletion", "auxiliary word deletion", "causative verb deletion", "measure word deletion", "subject deletion", "predicate deletion", "object deletion".

[0075] For example, the first text is "A leaf floats down from the tree.", where "piece" is a measure word, and a "measure word deletion" defect can be constructed, and "leaf" is the subject, and a "subject deletion" defect can be constructed. According to the occurrence probability of each defect type, the second text "A leaf floats down from the tree." is finally constructed. This construction method avoids to a certain extent the risk that the original content is still correct after randomly deleting the original content.

[0076] For example, by adding components at the construction position of the first text, component redundancy can be obtained, such as the second text with "auxiliary word redundancy".

[0077] For example, by adjusting the sorting of components at the construction position of the first text, the second text with "improper word order", such as the second text with "components in the sentence being reversed", can be constructed.

[0078] In the embodiments of the present application, through the observation and summary of real data, and design optimization according to artificial prior knowledge, the defect text construction rules are constructed. Optionally, first determine the positions in the original text that need to be modified, and then determine the content that should be modified at these positions; for example, it includes construction schemes for various specific grammar errors for the six major error types of "typo", "component deficiency", "component redundancy", "mismatch", "improper word usage", and "improper word order".

[0079] Compared with the grammatical errors constructed by the construction method based on random rules, the second text constructed based on the defective text construction rule is closer to the real errors, such as the distribution pattern of the errors.

[0080] However, the number and diversity of construction rules summarized by experts are necessarily limited, so the types of incorrect sentences obtained by construction are also limited, and the construction results are not diverse and rich enough. For example, the scheme based on rules and statistics has the problem that the construction results are limited by the vocabulary and rules, and the diversity of the constructed data is not enough.

[0081] The embodiment of the present application also provides a pseudo-data automatic generation model, i.e., a generation model; first, a second text with grammatical defects is constructed using expert construction rules, and then the generation model is trained with these data to allow the generation model to learn the grammatical error rules summarized manually; the trained generation model can give play to the excellent characteristics of diverse generation results. By constructing defective texts based on this model, it can be further ensured that the constructed sentences do contain grammatical defects, thereby improving the diversity of construction results. Finally, a large number of texts containing grammatical defects are automatically constructed, so that the effect of the grammatical error detection model can be improved by expanding the corpus size.

[0082] In some implementations, a generation model is trained based on the first text and a second text corresponding to the first text.

[0083] In some embodiments, the generative model is a model based on a seq2seq network (Sequence to Sequence, end-to-end neural network). The seq2seq network model allows the length of the constructed defective text to be different from the length of the first text, so the defect types that can be constructed by the model are more diverse.

[0084] like Figure 5 As shown, the generative model includes an encoder and a decoder. The encoder is used to generate a semantic vector of the text based on the input text, and the semantic vector includes, for example, the implicit representation of each word in the text in the context. The decoder is used to output the predicted text based on the semantic vector output by the encoder. The generative model is trained based on the first text and the second text corresponding to the first text, so that the generative model can generate a text with grammatical defects based on a text without grammatical defects. For example, the encoder includes a multi-layer neural network, including at least one of the following: a transformer network, an LSTM network, and a CNN network; the decoder includes a multi-layer neural network, including at least one of the following: a transformer network, an LSTM network, and a CNN network.

[0085] In some embodiments, the training process of the generative model, such as training the generative model based on the first text and the second text corresponding to the first text, includes: inputting the first text into the encoder of the generative model to obtain a semantic vector; and the decoder of the generative model generates a second generated text based on the semantic vector; and adjusting the parameters of the encoder and / or the decoder based on the difference between the second text corresponding to the first text and the second generated text corresponding to the first text.

[0086] Exemplarily, the input granularity of the generative model is at the word level, and the first text containing n words can be represented as X=(#CLS, c1, c2, ..., c n ), where c i represents the i-th (1≤i≤n) word in the first text, #CLS is the sentence start identifier (marked as c0), that is, c i (0≤i≤n) represents any word in sentence X.

[0087] Exemplarily, the encoder includes a word embedding layer, which is used to embed each word c in the sentence X through word embedding processing. i , converted to vector representation; for example, the position embedding and word embedding can be used to map the text and the position of the text in the sentence into the vector space respectively, and the position embedding vector e can be obtained. position (c i ), word embedding vector e char (c i ). The position embedding vector e position (c i ) means c i The position information in the sentence, and the word embedding vector e char (c i ) means c i The context-independent semantic information can be combined to obtain the initial representation of each word. i :

[0088] e i =e position (c i )+e char (c i )

[0089] By modeling the entire sentence through the multi-layer neural network in the encoder, we can get the context-dependent representation vector h of each word i :

[0090] h i =encoder(e i , X)

[0091] It is understandable that the initial vector e of each wordi Through the processing of the encoder, the interaction with the context occurs, and finally the implicit representation h of each word in the current context can be obtained. i ; The implicit representation corresponding to the entire sentence is H = (h1, h2, ..., h n ), which can be called a semantic vector.

[0092] The semantic vector obtained by the encoder must be processed by the decoder to obtain the generated result. The process can be expressed as follows:

[0093] y t = decoder(y t-1 , H)

[0094] Optionally, the decoder adopts an autoregressive generation method, that is, given the semantic representation of each word obtained by the encoder and the previous word y generated by the decoder t-1 , generate the current word y t . Among them, y0 is a special symbol " <s> ", which can be called the start symbol, the termination condition of the generated text is to generate the sentence terminator"< / s> ", and obtain the model generation result Y = (y1, y2, ..., y m ), which includes the text generated by the model based on the input text.

[0095] Exemplarily, the initial weights of the model when the model is not trained can be random weights or public pre-trained weights. Generally speaking, pre-trained weights will bring better results to the model.

[0096] Input the first text into the generative model to obtain a corresponding second generated text; adjust the parameters of the encoder and / or the decoder according to the difference between the second text and the corresponding second generated text. Exemplarily, determine the vector representation of the second text and the vector representation of the corresponding second generated text, determine the loss value of the generative model according to the vector representations of the two based on a preset loss function, and adjust the parameters of the encoder and / or the decoder according to the loss value so that the generative model learns information that is closer to real grammatical errors.

[0097] Optionally, the training process of the generative model also includes: obtaining a fourth text with grammatical defects, and a fifth text obtained by eliminating the defects of the fourth text; inputting the fifth text into the encoder of the generative model to obtain a semantic vector; and the decoder of the generative model generating a third generated text according to the semantic vector; and fine-tuning the generative model according to the difference between the fifth text corresponding to the fourth text and the third generated text corresponding to the fourth text.

[0098] See also Figure 4Based on the data obtained by constructing the expert summary rules, the generative model is preliminarily trained, and the model is preliminarily learned to generate the target through pseudo data (the first text and the second text constructed based on the defective text construction rules); then the generative model can be fine-tuned according to the small-scale real grammatical error text collected to obtain the final generative model, which has higher accuracy and generates grammatically defective texts that are closer to real grammatical errors.

[0099] The text data generation method provided in the embodiment of the present application obtains a generation model, including: obtaining a third text without grammatical defects; inputting the third text into the generation model to generate a first generated text corresponding to the third text, and a target probability for indicating that the first generated text has grammatical defects, the generation model is obtained by training based on the first text without grammatical defects and the second text with grammatical defects; according to the target probability corresponding to the first generated text, the first generated text is determined to be the target text. The generation model trained based on the first text and the second text can give play to the excellent characteristics of diverse generation results, and can automatically construct a large number of texts containing grammatical defects, so that the effect of the grammatical error detection model can be improved by expanding the corpus size.

[0100] Exemplarily, the second text is generated based on a preset defective text construction rule from the first text without grammatical defects, and the generation model can learn information that is closer to real grammatical errors. The defective text construction rule can be obtained by manual summary.

[0101] Exemplarily, in order to automatically construct data containing grammatical errors on a large scale, the embodiments of the present application start from two levels: rules and models. On the one hand, the rules of data construction are refined, the construction rules of data are optimized according to artificial prior knowledge design, and the generation model is trained based on the data constructed according to the construction rules; on the other hand, based on the generation model, the model method is used to automatically generate pseudo data containing grammatical errors.

[0102] The generative model is trained on rule-constructed text so that it can generate incorrect text given correct text, allowing the model to automatically learn the distribution of grammatical errors from the data, giving full play to the advantages of the model's strong ability to learn semantics and the fact that the prediction results do not rely on manually formulated vocabulary lists, making the generated data more flexible and diverse.

[0103] For example, the generative model can learn both expert knowledge and the actual error distribution, and can also give play to its own diversity. Therefore, the large amount of data generated is of relatively high quality, which can improve the training effect of downstream models.

[0104] It should be noted that the grammatical error detection task requires a large amount of diverse training data to achieve a better error detection effect. Therefore, automatically constructing text containing grammatical errors has become an explorable direction. The construction of grammatical error pseudo data is a difficult task. The constructed pseudo data needs to meet the following conditions: First, the quantity is large and the types are rich. Only by constructing data of sufficient scale can the model learn sufficiently generalized error patterns; second, it needs to be as close to the distribution of real grammatical errors as possible. If the constructed grammatical errors are too different from the real errors, it will interfere with the model's learning error detection ability. These conditions bring certain challenges to the construction of high-quality pseudo data. In order to construct large-scale data that contains rich error types and is close to the real error situations of students, the embodiment of the present application designs a set of grammatical error detection data construction methods that combine rules and models.

[0105] The method of the embodiment of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0106] Exemplarily, the above method can be implemented in the form of a computer program. The computer program can be Figure 6 Runs on the computer device shown.

[0107] See also Figure 6 , Figure 6 1 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device may be a server or a terminal.

[0108] See also Figure 6 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0109] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute the steps of the above method.

[0110] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0111] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute the steps of any method.

[0112] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that the structure of the computer device is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0114] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0115] Obtaining a first text without grammatical defects;

[0116] Based on a preset defective text construction rule, generating a second text with grammatical defects from the first text;

[0117] Training a generation model according to the first text and a second text corresponding to the first text;

[0118] Obtaining a third text free of grammatical defects;

[0119] Inputting the third text into the trained generative model to generate a first generated text corresponding to the third text and a target probability for indicating that the first generated text has grammatical defects;

[0120] According to the target probability corresponding to the first generated text, the first generated text is determined to be the target text.

[0121] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or some parts of the embodiments, such as:

[0122] A computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement the steps of the method provided in an embodiment of the present application.

[0123] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for generating text data, characterized in that: include: Obtaining a third text free of grammatical defects; Inputting the third text into a generative model to generate a first generated text corresponding to the third text and a target probability for indicating that the first generated text has a grammatical defect, wherein the generative model is trained based on the first text without grammatical defects and the second text with grammatical defects; Determining, according to a target probability corresponding to the first generated text, that the first generated text is a target text; The second text includes text generated by the following steps: performing component analysis on the first text to determine components and component ordering in the first text; determining component defect types and corresponding construction positions corresponding to the first text, as well as occurrence probabilities of the component defect types, based on preset component defect construction rules and components and component ordering in the first text; Based on the component defect type and the corresponding occurrence probability, components are deleted, components are added, or the order of components is adjusted at the construction position of the first text.

2. The text data generating method according to claim 1, characterized in that: The second text also includes a text generated by the following steps: based on a preset character / word defect construction table, determining in the first text the same character / word as in the character / word defect construction table to obtain a construction position of a target defect; Determining the probability of occurrence of the target defect according to the probability of occurrence corresponding to the same character / word in the character / word defect construction table; Based on the occurrence probability of the target defect, the word / phrase at the structural position is deleted, or the word / phrase at the structural position is replaced with the defective word / phrase, or the defective word / phrase is added to the structural position; The defective character / word is a character / word determined in the character / word defect construction table and corresponding to the same character / word.

3. The text data generating method according to any one of claims 1 to 2, characterized in that: The training process of the generative model includes: Inputting the first text into the encoder of the generative model to obtain a semantic vector; and the decoder of the generative model generates a second generated text according to the semantic vector; Adjust parameters of the encoder and / or the decoder according to a difference between a second text corresponding to the first text and a second generated text corresponding to the first text.

4. The text data generating method according to claim 3, characterized in that: The training process of the generative model further includes: Acquire a fourth text having grammatical defects, and a fifth text obtained by eliminating the defects of the fourth text; Inputting the fifth text into the encoder of the generative model to obtain a semantic vector; and the decoder of the generative model generates a third generated text according to the semantic vector; The generation model is fine-tuned according to the difference between a fifth text corresponding to the fourth text and a third generated text corresponding to the fourth text.

5. The text data generating method according to any one of claims 1 to 2, characterized in that: The step of inputting the third text into the trained generative model to generate a first generated text corresponding to the third text and a target probability for indicating that the first generated text has grammatical defects includes: Inputting the third text into the encoder of the generative model to obtain a semantic vector; and the decoder of the generative model generates a semantic representation of the first generated text and the terminator of the first generated text according to the semantic vector; A fully connected transformation is performed on the semantic representation of the terminator of the first generated text to obtain a target probability for indicating that the first generated text has a grammatical defect.

6. The text data generating method according to any one of claims 1 to 2, characterized in that: The generation model is a model based on a seq2seq network.

7. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement: The steps of the text data generating method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that: If the computer program is executed by a processor, it realizes: The steps of the text data generating method according to any one of claims 1 to 6.

Images