A text data enhancement method based on grammatical error correction of government texts

Abstract

The invention discloses a text data enhancement method based on error correction of government text, comprising constructing a dictionary of proper nouns and professional terms, recording the positions and types of proper nouns and professional terms; analyzing syntactic structure and stylistic features according to the dictionary of proper nouns and professional terms, and constructing error generation rules; and also comprising the following steps: preprocessing text data to be processed to obtain preprocessed text; back-translating the preprocessed text: first translating the preprocessed text into a text of a target language, and then translating the text of the target language back to the original language; comparing the texts before and after the back-translation, screening out new texts with semantics not destroyed but with error as enhanced data; using the error generation rules to generate negative samples based on the enhanced data as unprocessed negative sample expansion; calculating the edit distance of the unprocessed negative samples, retaining the texts with edit distances less than a threshold value as the negative samples finally generated.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and in particular relates to a text data enhancement method based on linguistic error correction of government texts. Background Art

[0002] Data augmentation is a branch of natural language processing that generates more diverse training data by performing various operations on existing texts, such as synonym replacement, random insertion, deletion, and swapping, thereby improving the robustness and generalization ability of the model.

[0003] At present, there is a problem of missing a large amount of high-quality data in government documents. Low-quality government data will significantly hinder the further development and utilization of the value of government data elements, and directly affect the role of government data elements in the circulation of all-factor data.

[0004] In view of this, it is necessary to design a text data enhancement method based on linguistic error correction of government texts to solve the above problems.

[0005] Chinese patent CN112836047A discloses a method for electronic medical record text data enhancement based on sentence semantic replacement. This method only extends the replacement from the word level to the sentence level, and the data enhancement effect is limited. The generated enhanced text may still be insufficiently different from the original text.

[0006] Chinese patent CN115048940A discloses a Chinese financial text data enhancement method based on entity word attribute features and back translation. This method introduces entity word layered processing and back translation technology to enhance financial text, but the technical innovation is insufficient, it relies on the performance of the translation model, and there is a problem of accuracy in identifying entity words. Summary of the invention

[0007] In order to solve the problem that the current government text corpus lacks negative samples, the present invention provides a text data enhancement method based on linguistic error correction of government text.

[0008] The technical solution of the present invention is as follows:

[0009] A text data enhancement method based on error correction of government texts includes constructing a dictionary of proper nouns and professional terms to record the positions and types of proper nouns and professional terms; analyzing the syntactic structure and stylistic features according to the dictionary of proper nouns and professional terms to construct error generation rules. It also includes the following steps:

[0010] S10, preprocessing the text data to be processed to obtain a preprocessed text, where the text to be processed includes one or more sentences;

[0011] S20, back-translating the preprocessed text: first translating the preprocessed text into a text in a target language, and then translating the text in the target language back into the original language;

[0012] S30, compare the texts before and after back translation, and select new texts with grammatical errors but without semantic damage as enhanced data;

[0013] S40, using the error generation rule, generating negative samples based on the enhanced data as the unprocessed negative sample expansion;

[0014] S50, calculating the edit distance of the unscreened negative samples, and retaining the texts with edit distances less than a threshold as the final generated negative samples.

[0015] Furthermore, S10 specifically includes:

[0016] S101, clean the text to be processed through HarvestTest to remove special characters, extra spaces and other invalid information;

[0017] S102, using the JioNLP toolkit to parse the demonstrative pronouns in the text to be processed into the objects they refer to, i.e., entities.

[0018] Furthermore, S20 specifically includes:

[0019] S201, extracting entities mentioned in the preprocessed text using the Extract tool in the JioNLP toolkit;

[0020] S202, using the BackTrans model, back-translate the entities extracted from each text in turn: translate the entities in each text into the target language in turn, and then translate the translated text back to the original language. After all entities of each text are translated, the back-translated sentence text is obtained.

[0021] Furthermore, S30 specifically includes: using the SentenceBert model to analyze the texts before and after the back-translation, that is, the semantic similarity between the pre-processed text and the back-translated text, and screening out the back-translated texts whose semantic similarity is greater than a threshold.

[0022] Furthermore, S30 specifically includes the following steps: S301, using a word segmenter to encode the input preprocessed text and the back-translated text, and then inputting the encoded text into the SentenceBert model to obtain an embedding vector, that is, a semantic vector;

[0023] S302, calculating cosine similarity based on the semantic vectors of the original text to be processed and the back-translated text, using cosine similarity to represent semantic similarity, and normalizing the cosine similarity to the interval [-1, 1], where the closer the value is to -1, the greater the semantic difference is, and the closer the value is to 1, the higher the semantic similarity is;

[0024] S303, screening out back-translated texts with a semantic similarity greater than 0.5.

[0025] Furthermore, S50 specifically includes:

[0026] S501, calculating the Levenshtein distance between the unscreened negative sample and the corresponding enhanced data, i.e., the edit distance;

[0027] S502, retain the texts with Levenshtein distance less than or equal to 2;

[0028] S503, summarizing all texts to generate a set of negative samples of government texts.

[0029] Furthermore, the error generation rules have four types of error errors: omission error, redundancy error, substitution error, and word order error. Omission error: Deleting some necessary words (generally referring to words or phrases) in the text intentionally, so that the sentence lacks complete meaning or logical coherence. The deleted words can be subjects, predicates, objects, attributes, adverbs, or complements. Redundancy error: Adding unnecessary words to the text makes the sentence cumbersome or repetitive. Redundant words can be repeated synonyms, too many modifiers, or meaningless redundancies. Substitution error: Replacing some words or phrases in the text with inappropriate or wrong words, causing the meaning of the sentence to deviate or cause logical errors. The replaced words can be synonyms, irrelevant words, or similar words. Word order error: Adjusting the order of words or phrases in the text makes the sentence structure unreasonable or the meaning ambiguous. The adjusted words can be subjects, predicates, objects, attributes, adverbs, or complements.

[0030] Furthermore, S40 specifically includes: selecting the error types of the error generation rules one by one, using the Transformer model to generate four negative samples of "missing error, redundant error, replacement error, word order error" based on the new text enhancement data in the previous step as unprocessed negative sample expansion.

[0031] The beneficial effects of the present invention are as follows: The method of the present invention combines natural language processing and deep learning to form a method for generating negative samples of government texts, which effectively solves the problem of lack of negative samples in government corpora. This solution can generate high-quality negative samples and enrich the government text corpus by back-translating government texts and performing semantic similarity analysis. At the same time, the present invention designs special grammatical error generation rules, and the generated negative samples are more authentic and representative, thereby improving the diversity of text data. Through edit distance calculation, negative samples that are semantically close to the original text are screened out to ensure the validity and accuracy of the generated data. These enhanced data can not only improve the training effect of the government text error correction model, but can also be widely used in the fields of government text processing and natural language processing, and improve the overall performance of text error correction and text generation technology.

[0032] This patent uses the method of extracting entities, model training, and grammatical error generation rules to further increase the diversity of the generated text. In addition, by combining back-translation technology and model training, and then through screening, on the basis of accurate recognition of entity words, enhanced text with better quality can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a graph of speech error generation rules for a text data enhancement method based on speech error correction of government texts according to an embodiment of the present invention.

[0034] Figure 2 The present invention is a flowchart of a method for text data enhancement based on linguistic error correction of government texts according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] To facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings.

[0036] The method of this embodiment specifically includes the following steps:

[0037] Based on the characteristics of government texts, such as many proper nouns, professional terms or semi-professional terms, and journalistic features, a dictionary of proper nouns and professional terms is constructed to record the locations and types of proper nouns and professional terms.

[0038] Analyze the syntactic structure and stylistic features of government texts according to the dictionary, extract features, and construct grammatical error generation rules.

[0039] like Figure 1 As shown, the error generation rules include four types of error: missing error, redundancy error, substitution error, and word order error.

[0040] Omission error: Deliberately deleting certain necessary words or phrases in a sentence, making the sentence lack complete meaning or logical coherence. The deleted words can be subjects, predicates, objects, attributives, adverbials or complements.

[0041] Redundancy error: adding unnecessary words or phrases to a sentence, making the sentence appear cumbersome or repetitive. Redundant words can be repeated synonyms, too many modifiers or meaningless elaboration.

[0042] Substitution error: replacing certain words or phrases in a sentence with inappropriate or incorrect words, causing the meaning of the sentence to deviate or become logically incorrect. The replaced words can be synonyms, irrelevant words, or similar words.

[0043] Word order error: The order of words or phrases in a sentence is adjusted to make the sentence structure unreasonable or the meaning ambiguous. The adjusted words can be the subject, predicate, object, attributive, adverbial or complement.

[0044] like Figure 2 As shown, in step S10, the input government text (or sentence) is preprocessed to clean the corpus, remove invalid information such as special characters and redundant spaces, and parse the demonstrative pronouns in the government text into the objects they refer to in turn.

[0045] S101, firstly cleans the government text through HarvestTest to remove special characters, extra spaces and other invalid information;

[0046] S102, using the JioNLP toolkit, the demonstrative pronouns in the government text are parsed into the objects they refer to in turn.

[0047] In step S20, entities are extracted from all pre-processed government documents processed in S10 and back-translated:

[0048] S201, extract the entities mentioned in the government text using the Extract tool in the JioNLP toolkit. Extract the information corresponding to the entity and return its position in the government text;

[0049] S202, using the BackTrans model, back-translate the entities extracted from each text in turn, translate the entities into the target language, and then translate the translated text back to the original language. After all entities of each text are translated, a back-translated sentence text is obtained.

[0050] In step S30, the texts before and after the back translation are compared, and the texts with grammatical errors but without overall semantic loss are selected as enhanced data.

[0051] S301, using the pre-trained model SentenceBert to analyze the semantic similarity between the original text to be processed and the back-translated text. SentenceBert is a pre-trained model based on the Transformer architecture. Specifically: use a word segmenter to encode the input government text, and then input the encoded government text into the model to obtain an embedding vector (semantic vector). Subsequently, these embedding vectors are average-pooled to obtain a more stable feature representation, and finally the similarity between the two embedding vectors is calculated using the cosine similarity function;

[0052] S302, semantic similarity is expressed by cosine similarity, and the specific formula is:

[0053] ,

[0054] The cosine similarity is normalized to the interval [-1, 1]. The closer the value is to -1, the greater the semantic difference is, and the closer the value is to 1, the higher the semantic similarity is.

[0055] S303, screening out back-translated texts with a semantic similarity greater than 0.5.

[0056] In step S40, the error generation rules are input into the model, the error types of the error generation rules are selected one by one, and the enhanced data are imported to generate unscreened negative samples.

[0057] S401, inputting the grammatical error generation rules into the Transformer model;

[0058] S402, using the Transformer model to generate four types of negative samples of "missing error, redundant error, replacement error, and word order error" based on the new text enhancement data in the previous step as unprocessed negative sample expansion.

[0059] In step S50, the unscreened negative samples are introduced into the edit distance calculation model, and the sentences with edit distances less than the threshold are retained as the final generated negative samples.

[0060] S501, calculating the Levenshtein distance between the generated text and the original text , the calculation formula is as follows:

[0061] ,

[0062] Among them, string a is the back-translated text, string b is the unscreened negative sample, i is the i-th character of string a, and j is the j-th character of string b; yes The indicator function is 1 when , and 0 otherwise. yes Before characters and Before Levenshtein distance between characters;

[0063] S502, retaining texts with a Levenshtein distance less than or equal to 2; the retained texts are summarized as a set of negative samples of government texts.

[0064] The method of this embodiment has good flexibility and scalability, and can adapt to different corpus requirements by adjusting the grammatical error generation rules, and obtain a negative sample set with good quality under different government text input conditions.

[0065] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A text data enhancement method based on grammatical error correction of government texts, characterized in that: It includes constructing a dictionary of proper nouns and professional terms, recording the positions and types of proper nouns and professional terms; analyzing the syntactic structure and stylistic features according to the dictionary of proper nouns and professional terms, and constructing grammatical error generation rules; and also includes the following steps: S10, preprocessing the text data to be processed to obtain preprocessed text; S20, back-translating the preprocessed text: first translating the preprocessed text into a text in a target language, and then translating the text in the target language back into the original language; S30, compare the texts before and after back translation, and select new texts with grammatical errors but without semantic damage as enhanced data; S40, using the error generation rules, generates text with errors based on the enhanced data as unprocessed negative sample expansion; S50, calculating the edit distance of the unprocessed negative samples, and retaining the texts with edit distances less than a threshold as the final generated negative samples; The rules for generating grammatical errors include: missing errors, redundant errors, substitution errors, and word order errors; Deletion error: Deletion of words in the text. The deleted words are subjects, predicates, objects, attributes, adverbials or complements. Redundancy error: adding words to the text, such as repeated synonyms, excessive modifiers, or meaningless redundancy; Substitution error: replacing a word with an incorrect word in the text. The replaced word is a synonym, an unrelated word, or a similar word. Word order error: Adjust the order of words in the text. The adjusted words are subject, predicate, object, attributive, adverbial or complement.

2. The text data enhancement method based on linguistic error correction of government text according to claim 1 is characterized in that: S10 specifically includes: S101, clean the text to be processed through HarvestTest to remove special characters and redundant blank information; S102, using the JioNLP toolkit to parse the demonstrative pronouns in the text to be processed into the objects they refer to, i.e., entities.

3. The text data enhancement method based on error correction of government text according to claim 1 is characterized in that: S20 specifically includes: S201, extracting entities mentioned in the preprocessed text using the Extract tool in the JioNLP toolkit; S202, using the BackTrans model, back-translate the entities extracted from each text in turn: translate the entities in each text into the target language in turn, and then translate the translated text back to the original language. After all entities of each text are translated, the back-translated sentence text is obtained.

4. The text data enhancement method based on linguistic error correction of government text according to claim 1 is characterized in that: S30 specifically includes: The SentenceBert model is used to analyze the semantic similarity of the text before and after back-translation, and the back-translated texts with semantic similarity greater than the threshold are screened out.

5. The text data enhancement method based on error correction of government text according to claim 4 is characterized in that: S30 specifically includes: S301, using a word segmenter to encode the text before and after back translation, and then inputting the encoded text into the SentenceBert model to obtain a semantic vector; S302, calculating the cosine similarity according to the semantic vectors of the sentences before and after the back translation, and standardizing the cosine similarity to the interval [-1, 1]; S303, screening out back-translated texts with a semantic similarity greater than 0.

5.

6. The text data enhancement method based on error correction of government text according to claim 1 is characterized in that: S50 specifically includes: S501, calculating the Levenshtein distance between the unscreened negative sample and the corresponding enhanced data, i.e., the edit distance; S502, retain the texts with Levenshtein distance less than or equal to 2; S503, the retained texts are summarized as a set of negative samples of government texts.

7. The text data enhancement method based on linguistic error correction of government text according to claim 1 is characterized in that: S40 specifically includes: The error types of the error generation rules are selected one by one, and the Transformer model is used to generate four unscreened negative samples corresponding to the enhanced data: missing errors, redundancy errors, substitution errors, and word order errors.

Images