Text data enhancement method and device

Abstract

An enhancement method and device for text data, the method comprising: obtaining text corpus for a target task, the text corpus comprising annotated corpus; determining text to be replaced in first annotated corpus according to the target task; selecting a plurality of candidate texts similar to the text to be replaced using a first semantic feature model, selecting a target text most similar to the text to be replaced from the plurality of candidate texts using a second semantic feature model, replacing the text to be replaced in the first annotated corpus with the target text, and obtaining intermediate corpus; and performing back-translation on the intermediate corpus to obtain second annotated corpus, wherein the label information of the second annotated corpus is the same as the label information of the first annotated corpus. The present application can improve the quality of newly added annotated corpus, provide support for training subsequent natural language processing task models based on annotated corpus, improve the quality of trained models, and improve the inference effect of the models.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, specifically to a method and apparatus for enhancing text data. Background Technology

[0002] Natural language processing tasks such as text classification and named entity recognition often employ supervised learning for model training. The quality of supervised learning depends on labeled data, but labeled data is costly to obtain, scarce, and often suffers from imbalanced samples. Using text data augmentation techniques to expand existing labeled corpora can effectively improve the training performance of models.

[0003] Current text augmentation techniques mainly include Easy Data Augmentation (EDA) (such as random replacement, random insertion, random deletion, and random swapping), back-translation (using machine translation to translate a piece of text from a first language into a second language and then back into the first language), and constructing new data using neural network models (e.g., using generative adversarial networks to generate new text data). Directly using neural network models to construct new data introduces more uncontrollable noise, resulting in poor performance. A combination of EDA and back-translation typically involves segmenting sentences in the text, randomly selecting a certain proportion (proportional to sentence length) of words for synonym replacement, and then back-translating to obtain the augmented data.

[0004] Since randomly selecting a certain proportion of words for synonym replacement has very limited effect on improving the training model, current solutions often select specific words for replacement, such as keywords with high Term Frequency-Inverse Document Frequency (TF-IDF) weights, entity words with type labels, and subject-verb-object pairs in syntactic structures. Furthermore, regarding synonym replacement, named entity recognition tasks can directly use words of the same type from the entity database for replacement, while text classification tasks often employ semantic models based on global vectors for word representation (GloVe), word-to-vector (word2vec), or bidirectional encoder representation from Transformers (BERT) to select text with similar vector values ​​for replacement. Summary of the Invention

[0005] At least one embodiment of this application provides a text data augmentation method and apparatus for improving the quality of new annotated corpora obtained through text data augmentation, thereby providing support for improving model training performance.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a method for enhancing text data, including:

[0008] Obtain text corpus for the target task, the text corpus including annotated corpus;

[0009] Based on the target task, determine the text to be replaced in the first annotated corpus;

[0010] The first semantic feature model is used to select multiple candidate texts that are similar to the text to be replaced. The second semantic feature model is used to select the target text that is most similar to the text to be replaced from the multiple candidate texts. The text to be replaced in the first annotated corpus is replaced with the target text to obtain intermediate corpus.

[0011] The intermediate corpus is back-translated to obtain a second annotated corpus, wherein the label information of the second annotated corpus is the same as that of the first annotated corpus.

[0012] Optionally, determining the text to be replaced in the first annotated corpus according to the target task includes:

[0013] When the target task is a text classification task, a topic model is built based on the text corpus;

[0014] For the first labeled corpus, the topic model is used to obtain the first topic type with the highest probability corresponding to the first labeled corpus and multiple keywords in the first labeled corpus that correspond to the first topic type, wherein the keywords are the text to be replaced.

[0015] Optionally, the probability that the keyword belongs to the first topic type is greater than a preset threshold.

[0016] Optionally, the topic model is an LSI model or an LDA model.

[0017] Optionally, the text corpus also includes unlabeled corpus; the step of building a topic model based on the text corpus includes:

[0018] The text corpus is preprocessed to obtain preprocessed corpus, wherein the preprocessing operations include word segmentation, stop word removal, and filtering of illegal characters;

[0019] Based on the preprocessed corpus, a topic model is established, wherein the number of topics in the topic model is set according to the number of target categories of the text classification task, and the number of word frequencies is set according to the word frequency distribution of all words in the labeled corpus in the unlabeled corpus.

[0020] Optionally, determining the text to be replaced in the first annotated corpus according to the target task includes:

[0021] When the target task is named entity recognition, the text corpus is preprocessed to obtain preprocessed corpus;

[0022] Based on the named entities in the preprocessed annotated corpus, an entity dictionary is constructed, which includes vocabulary under each named entity type.

[0023] For the first annotated corpus, a portion of the first words corresponding to named entities are randomly selected from the first annotated corpus, and each first word in the first annotated corpus is replaced with a second word under the same named entity type in the entity dictionary to obtain the updated first annotated corpus. The context words of the second words in the updated first annotated corpus are the text to be replaced.

[0024] Optionally, replacing the text to be replaced in the first annotated corpus with the target text includes: replacing the context words of the second word in the updated first annotated corpus with the corresponding target text to obtain the intermediate corpus;

[0025] The second annotated corpus includes a third word corresponding to the second word, and the category label of the third word is the same as the category label of the first word.

[0026] Optionally, the number n named entities in the first annotated corpus e When the value is 1, the number of named entities in the partial group is 1; when the number of named entities in the first annotated corpus is greater than 1, the number of named entities in the partial group is greater than or equal to n. e / 2.

[0027] Optionally, the first semantic feature model is obtained by unsupervised training on data from a preset corpus, or by unsupervised training on unlabeled corpus.

[0028] Optionally, the semantic representation capability of the first semantic feature model is lower than that of the second semantic feature model.

[0029] Optionally, the first semantic feature model is a GloVe model or a word2vec model, and the second semantic feature model is a SimBert model, a SimCSE model, or a RosemerSim model.

[0030] Secondly, embodiments of this application provide a text data enhancement device, comprising:

[0031] The acquisition module is used to acquire text corpus for the target task, the text corpus including labeled corpus;

[0032] The determination module is used to determine the text to be replaced in the first labeled corpus;

[0033] The replacement module is used to select multiple candidate texts that are similar to the text to be replaced using a first semantic feature model, select the target text that is most similar to the text to be replaced from the multiple candidate texts using a second semantic feature model, and replace the text to be replaced in the first annotated corpus with the target text to obtain intermediate corpus;

[0034] The back-translation module is used to perform a back-translation operation on the intermediate corpus to obtain a second annotated corpus, wherein the label information of the second annotated corpus is the same as the label information of the first annotated corpus.

[0035] Optionally, the determining module is further configured to:

[0036] When the target task is a text classification task, a topic model is built based on the text corpus;

[0037] For the first labeled corpus, the topic model is used to obtain the first topic type with the highest probability corresponding to the first labeled corpus and multiple keywords in the first labeled corpus that correspond to the first topic type, wherein the keywords are the text to be replaced.

[0038] Optionally, the probability that the keyword belongs to the first topic type is greater than a preset threshold.

[0039] Optionally, the topic model is an LSI model or an LDA model.

[0040] Optionally, the text corpus also includes unannotated corpus; the determining module is further configured to:

[0041] The text corpus is preprocessed to obtain preprocessed corpus, wherein the preprocessing operations include word segmentation, stop word removal, and filtering of illegal characters;

[0042] Based on the preprocessed corpus, a topic model is established, wherein the number of topics in the topic model is set according to the number of target categories of the text classification task, and the number of word frequencies is set according to the word frequency distribution of all words in the labeled corpus in the unlabeled corpus.

[0043] Optionally, the determining module is further configured to:

[0044] When the target task is named entity recognition, the text corpus is preprocessed to obtain preprocessed corpus;

[0045] An entity dictionary is constructed based on the named entities in the preprocessed annotated corpus. The entity dictionary includes vocabulary for each named entity type.

[0046] For the first annotated corpus, a portion of the first words corresponding to named entities are randomly selected from the first annotated corpus, and each first word in the first annotated corpus is replaced with a second word under the same named entity type in the entity dictionary to obtain the updated first annotated corpus. The context words of the second words in the updated first annotated corpus are the text to be replaced.

[0047] Optionally, the replacement module is further configured to: replace the context words of the second word in the updated first annotated corpus with the corresponding target text to obtain the intermediate corpus;

[0048] The second annotated corpus includes a third word corresponding to the second word, and the category label of the third word is the same as the category label of the first word.

[0049] Optionally, the number n named entities in the first annotated corpus e When the value is 1, the number of named entities in the partial group is 1; when the number of named entities in the first annotated corpus is greater than 1, the number of named entities in the partial group is greater than or equal to n. e / 2.

[0050] Optionally, the first semantic feature model is obtained by unsupervised training on data from a preset corpus, or by unsupervised training on unlabeled corpus.

[0051] Optionally, the semantic representation capability of the first semantic feature model is lower than that of the second semantic feature model.

[0052] Optionally, the first semantic feature model is a GloVe model or a word2vec model, and the second semantic feature model is a SimBert model, a SimCSE model, or a RosemerSim model.

[0053] Thirdly, embodiments of this application provide a text data enhancement apparatus, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method described in the first aspect.

[0054] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the method described above.

[0055] Compared with the prior art, the text data enhancement method and apparatus provided in this application, for specific natural language processing tasks, selects the text to be replaced in the labeled corpus, and improves the accuracy of semantic similarity of the target text by combining the first semantic feature model and the second semantic feature model during replacement. In addition, it improves the problem of the target text being similar to the subsequent natural language processing task model, thereby improving the quality of the newly added labeled corpus, and thus providing support for training the subsequent natural language processing task model based on the labeled corpus. It can improve the quality of the trained model and improve the inference effect of the model. Attached Figure Description

[0056] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0057] Figure 1 A flowchart is provided for a text data enhancement method according to an embodiment of this application;

[0058] Figure 2 This is a schematic diagram of a text data enhancement device according to an embodiment of this application;

[0059] Figure 3 This is another structural schematic diagram of a text data enhancement device according to an embodiment of this application;

[0060] Figure 4 This is another structural schematic diagram of a text data enhancement device according to an embodiment of this application. Detailed Implementation

[0061] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0062] The terms “first,” “second,” etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The terms “and / or” in the specification and claims indicate at least one of the connected objects.

[0063] Current text data augmentation solutions suffer from two main problems: various natural language processing tasks (such as text classification and named entity recognition) have their own characteristics, and existing solutions do not differentiate between specific tasks when selecting texts to be replaced from labeled corpora; on the other hand, the GloVe (or word2vec) network model is relatively simple and has limited semantic representation capabilities, resulting in low text similarity accuracy. While the BERT model has strong representation capabilities, it is similar to the models used in subsequent natural language processing tasks (text classification and named entity recognition), which can lead to severe overfitting and significantly reduce the model's inference performance.

[0064] To address at least one of the above problems, embodiments of this application provide a text data augmentation method that can improve the quality of the new annotated corpus obtained from text data augmentation, thereby providing support for improving the training effect of subsequent natural language processing task models.

[0065] Please refer to Figure 1 This application provides a method for enhancing text data, comprising:

[0066] Step 101: Obtain text corpus for the target task, the text corpus including labeled corpus.

[0067] Here, the target task is a natural language processing task, specifically a text classification task, named entity recognition task, etc. The text corpus is the data used to train the model performing the natural language processing task, and typically includes labeled corpus. Labeled corpus refers to corpus with labeled information; the specific label information varies depending on the natural language processing task. Since obtaining labeled corpus is costly, this embodiment performs text augmentation on the labeled corpus to obtain new labeled corpus, thereby expanding the quantity of labeled corpus and improving the training effect of the natural language processing task model. Additionally, the text corpus in this embodiment may also include unlabeled corpus, i.e., corpus without labeled information.

[0068] Step 102: Based on the target task, determine the text to be replaced in the first annotated corpus.

[0069] Here, in this embodiment of the application, when obtaining new annotated corpus based on a certain annotated corpus (referred to as the first annotated corpus for ease of description), the text to be replaced in the first annotated corpus is determined according to the specific natural language processing task. The text to be replaced is the part of the annotated corpus that needs to be replaced. Through replacement processing, a new corpus can be generated based on the annotated corpus. This new corpus still uses the label information of the original annotated corpus, thereby obtaining a new annotated corpus. In addition, there can be one or more texts to be replaced in the first annotated corpus.

[0070] Step 103: Use the first semantic feature model to select multiple candidate texts that are similar to the text to be replaced, use the second semantic feature model to select the target text that is most similar to the text to be replaced from the multiple candidate texts, and replace the text to be replaced in the first annotated corpus with the target text to obtain intermediate corpus.

[0071] To address the issue that when selecting target text for replacement text, models with lower semantic representation capabilities result in lower similarity accuracy, while models with higher semantic representation capabilities are prone to overfitting, this embodiment first uses a first semantic feature model to select multiple candidate texts, and then uses a second semantic feature model to select a target text from these candidate texts. The semantic representation capability of the first semantic feature model is lower than that of the second semantic feature model. Through this method, a corresponding target text is selected for each text to be replaced in the first annotated corpus. Then, the text to be replaced in the first annotated corpus is replaced with the corresponding target text, resulting in an intermediate corpus. This method avoids the aforementioned problems and improves the quality of the selected target text.

[0072] In this embodiment of the application, the first semantic feature model may be obtained by unsupervised training on data from a preset corpus (for example, Chinese encyclopedia data), or, when the text corpus for the target task includes unlabeled corpus, the unlabeled corpus may be subjected to unsupervised training to obtain the first semantic feature model.

[0073] Specifically, the first semantic feature model is a GloVe model or a word2vec model, etc., and the second semantic feature model is a SimBert model, a SimCSE model, or a RoformerSim model. Among them, the SimBert model is a BERT model that integrates generation and retrieval, and the SimCSE model is a Simple Contrastive Learning of Sentence Embeddings (SimCSE) model, etc.

[0074] Step 104: Perform a back-translation operation on the intermediate corpus to obtain a second annotated corpus, wherein the label information of the second annotated corpus is the same as the label information of the first annotated corpus.

[0075] Here, for the intermediate corpus obtained in step 103, this embodiment of the application uses a back-translation operation to obtain the second labeled corpus. For example, assuming the language of the intermediate corpus is a first language (such as Chinese), the back-translation operation first translates the intermediate corpus of the first language into the intermediate corpus of the second language (different from the first language, such as English), and then translates the intermediate corpus of the second language back into the intermediate corpus of the first language, thereby obtaining the second labeled corpus. The second labeled corpus uses the tag information of the first labeled corpus.

[0076] Through the above steps, this embodiment of the application selects the text to be replaced in the labeled corpus for a specific natural language processing task. During the replacement, it improves the accuracy of the semantic similarity of the target text by combining the first semantic feature model and the second semantic feature model. In addition, it improves the problem of the target text being similar to the subsequent natural language processing task model, thereby improving the quality of the newly added labeled corpus. This provides support for training the subsequent natural language processing task model based on the labeled corpus, improves the quality of the trained model, and enhances the inference performance of the model.

[0077] The following examples, using text classification and named entity recognition tasks, illustrate how the embodiments of this application determine the text to be replaced in the first annotated corpus.

[0078] 1. Text classification task

[0079] When the target task is a text classification task, in step 102 of this application embodiment, a topic model is first established based on the text corpus. Then, for the first labeled corpus, the topic model is used to obtain the first topic type with the highest probability corresponding to the first labeled corpus and multiple keywords in the first labeled corpus corresponding to the first topic type, wherein the keywords are the text to be replaced. In addition, to ensure the quality of the selected keywords, this application embodiment selects multiple keywords with a probability greater than a preset threshold belonging to the first topic type from the keywords in the first labeled corpus provided by the topic model that correspond to the first topic type, as the text to be replaced.

[0080] In this embodiment, the text corpus may further include unlabeled corpus. When building a topic model based on the text corpus, the text corpus can be preprocessed to obtain preprocessed corpus. The preprocessing operations include word segmentation, stop word removal, and filtering of illegal characters. Then, based on the preprocessed corpus, a topic model is built. The number of topics in the topic model is set according to the number of target categories in the text classification task. For example, the number of topics is greater than the number of target categories in the text classification task and less than the product of the number of target categories in the text classification task and a preset ratio coefficient, where the preset coefficient is greater than 1 and less than 2. The number of word frequencies is set according to the word frequency distribution of all words in the labeled corpus in the unlabeled corpus. For example, the median of the word frequency distribution of all words in the unlabeled corpus is taken. Here, the topic model can be a Latent Semantic Indexing (LSI) model or a Latent Dirichlet Allocation (LDA) model, etc.

[0081] Then, in step 103 above, the text to be replaced in the first annotated corpus is replaced to obtain intermediate corpus, and in step 104 above, the second annotated corpus is obtained through back translation operation, wherein the category label of the second annotated corpus is the same as the category label of the first annotated corpus.

[0082] Below is an example of a text data augmentation method using the LDA model as the topic model:

[0083] Step 1.1: Perform text preprocessing operations such as word segmentation, stop word removal, and illegal character filtering on the labeled and unlabeled corpora for the text classification task to obtain the preprocessed corpora.

[0084] Step 1.2: Assuming the number of target categories in the text classification task is k, then build an LDA model for all preprocessed corpora (labeled and unlabeled), with the number of topics set to k. t (k+1<=k t <=3*k / 2), the number of word frequencies is set to k. f (k f (This represents the median of the word frequency distribution of all words in the labeled corpus in the unlabeled corpus).

[0085] Step 1.3: Obtain the topic type t with the highest probability corresponding to a certain labeled corpus using the LDA model. Then, obtain the keywords corresponding to topic type t in the labeled corpus using the LDA model, and select the first m keywords according to the probability of each keyword belonging to topic type t. Among them, if the probability of a keyword belonging to topic type t is less than the threshold p, the keyword is considered to belong to topic type t. t Then ignore the keyword to ensure that the probability of the selected keyword belonging to topic type t is greater than or equal to less than the threshold p. t Here, m and p t All are preset values.

[0086] Step 1.4: Use the GloVe model to perform unsupervised training on the Chinese encyclopedia data (if there is enough corresponding unlabeled corpus, the unlabeled corpus can be used to train the GloVe model). Use the GloVe model to obtain N closest texts for each keyword selected in Step 1.3, resulting in a total of m*N texts. Here, N is also a preset value.

[0087] Step 1.5: For each keyword in the annotated corpus, the SimBert model is used to select the closest cosine distance from N closest texts as the target text, and the corresponding keyword is replaced (if the nearest cosine distance is greater than the preset threshold s, the keyword is ignored, that is, the keyword is not replaced), thus obtaining the replaced corpus.

[0088] Step 1.6: Finally, back-translate the replaced corpus to obtain a new annotated corpus. The category labels of the new annotated corpus are consistent with those of the original annotated corpus before the replacement.

[0089] In the above examples, topic models (LSI, LDA, etc.) are used to extract keywords for text classification tasks. Then, shallow semantic feature models (glove, word2vec, etc.) and deep semantic feature models (SimBert, SimCSE, RoformerSim, etc.) are combined to replace similar texts. The replacement results are then back-translated to generate enhanced labeled corpora.

[0090] Through the above process, embodiments of this application can tailor the training model performance for different natural language processing tasks by employing different data augmentation schemes. For example, for text classification tasks, LDA models are used to obtain keywords, allowing for better utilization of text features at the topic level. Furthermore, embodiments of this application use methods different from those used in subsequent natural language processing task model training to obtain similar text, thereby reducing overfitting while maintaining text similarity accuracy.

[0091] 2. Named Entity Recognition Task

[0092] When the target task is named entity recognition, in step 102 of this application embodiment, the text corpus is first preprocessed to obtain preprocessed corpus. Then, based on the named entities in the preprocessed annotated corpus, an entity dictionary is constructed, which includes words under each named entity type. Then, for a certain annotated corpus (for ease of description, referred to as the first annotated corpus), a portion of the first words corresponding to named entities are randomly selected from the first annotated corpus, and each first word in the first annotated corpus is replaced with a second word under the same named entity type in the entity dictionary, thereby obtaining an updated first annotated corpus. The context words of the second word in the updated first annotated corpus are the text to be replaced. Here, the context words can be the previous word adjacent to the text to be replaced in the updated first annotated corpus, and / or the next word adjacent to the text to be replaced in the updated first annotated corpus. When both the previous word and the next word are used as the text to be replaced, step 103 is executed for the previous word and the next word respectively, so as to select the corresponding target text for the previous word and the next word respectively and replace it.

[0093] Here, to ensure the quantity of text to be replaced, the number of named entities n in the first annotated corpus in this embodiment of the application is [not specified]. e When the value is 1, the number of named entities in the partial group is 1; when the number of named entities in the first annotated corpus is greater than 1, the number of named entities in the partial group is greater than or equal to n. e / 2.

[0094] Then, in step 103 above, the text to be replaced (the context words of the named entities) in the updated first annotated corpus is replaced to obtain intermediate corpus, and in step 104 above, the intermediate corpus is back-translated to obtain second annotated corpus, wherein the second annotated corpus includes a third word corresponding to the second word, and the category label of the third word is the same as the category label of the first word.

[0095] The following example, using named entity recognition as an example, provides an example of a text data augmentation method:

[0096] Step 2.1: Perform text preprocessing operations such as word segmentation, stop word removal, and illegal character filtering on the labeled and unlabeled corpora of the named entity recognition task, and construct an entity dictionary based on the labeled corpora.

[0097] Step 2.2: Assume that a certain labeled corpus contains n e There are n entities, if n e If the value is ≥2, then a random selection will be made from the labeled corpus. If ne is 1, then this 1 entity is replaced with another word of the same type in the entity dictionary; if there is no entity, then no replacement is performed.

[0098] Step 2.3: Use the GloVe model to perform unsupervised training on the Chinese encyclopedia data (if there is enough corresponding unlabeled corpus, it can be used to replace the Chinese encyclopedia data for training). Use the GloVe model to obtain the M closest texts of the context words of entity words in the labeled corpus. Here, M is a preset value.

[0099] Step 2.4: Using the SimBert model, select the text with the closest cosine distance from the N nearest texts and replace the context words (if the cosine distance to the nearest text is greater than a preset threshold, then ignore the context word, i.e., do not replace the context word), thus obtaining the replaced corpus.

[0100] Step 2.5: Finally, back-translate the replaced corpus to obtain a new annotated corpus. This new annotated corpus includes words corresponding to the entities replaced in Step 2.2, and the type labels of these words are consistent with the type labels of the entities replaced in Step 2.2.

[0101] In the above examples, for the named entity recognition task, the entity words are randomly replaced with words of the same type. The entity context is replaced with similar text by combining shallow semantic feature models (glove, word2vec, etc.) and deep semantic feature models (SimBert, SimCSE, RoformerSim, etc.). The replacement results are back-translated to generate an enhanced labeled corpus.

[0102] As can be seen from the above, in the examples of the embodiments of this application, keywords are selected by the LDA model for text classification tasks, and entity words with entity labels are selected as keywords for named entity recognition tasks. In addition, the target text is obtained by combining the GloVe and SimBert models, which reduces the overfitting problem while ensuring the accuracy of text similarity.

[0103] The various methods of the embodiments of this application have been described above. Apparatus for implementing the above methods will now be provided.

[0104] Please refer to Figure 2 This application also provides a text data enhancement device, including:

[0105] The acquisition module 201 is used to acquire text corpus for the target task, the text corpus including labeled corpus;

[0106] Module 202 is used to determine the text to be replaced in the first annotated corpus;

[0107] The replacement module 203 is used to select multiple candidate texts that are similar to the text to be replaced using a first semantic feature model, select the target text that is most similar to the text to be replaced from the multiple candidate texts using a second semantic feature model, and replace the text to be replaced in the first annotated corpus with the target text to obtain intermediate corpus;

[0108] The back-translation module 204 is used to perform a back-translation operation on the intermediate corpus to obtain a second annotated corpus, wherein the label information of the second annotated corpus is the same as the label information of the first annotated corpus.

[0109] Through the above modules, the embodiments of this application can improve the quality of the new labeled corpus obtained by text data augmentation, and provide support for improving the model training effect.

[0110] Optionally, the determining module 202 is further configured to:

[0111] When the target task is a text classification task, a topic model is built based on the text corpus;

[0112] For the first labeled corpus, the topic model is used to obtain the first topic type with the highest probability corresponding to the first labeled corpus and multiple keywords in the first labeled corpus that correspond to the first topic type, wherein the keywords are the text to be replaced.

[0113] Optionally, the probability that the keyword belongs to the first topic type is greater than a preset threshold.

[0114] Optionally, the topic model is an LSI model or an LDA model.

[0115] Optionally, the text corpus also includes unannotated corpus; the determining module 202 is further configured to:

[0116] The text corpus is preprocessed to obtain preprocessed corpus, wherein the preprocessing operations include word segmentation, stop word removal, and filtering of illegal characters;

[0117] Based on the preprocessed corpus, a topic model is established, wherein the number of topics in the topic model is set according to the number of target categories of the text classification task, and the number of word frequencies is set according to the word frequency distribution of all words in the labeled corpus in the unlabeled corpus.

[0118] Optionally, the determining module 202 is further configured to:

[0119] When the target task is named entity recognition, the text corpus is preprocessed to obtain preprocessed corpus;

[0120] An entity dictionary is constructed based on the named entities in the preprocessed annotated corpus. The entity dictionary includes vocabulary for each named entity type.

[0121] For the first annotated corpus, a portion of the first words corresponding to named entities are randomly selected from the first annotated corpus, and each first word in the first annotated corpus is replaced with a second word under the same named entity type in the entity dictionary to obtain the updated first annotated corpus. The context words of the second words in the updated first annotated corpus are the text to be replaced.

[0122] Optionally, the replacement module 203 is further configured to: replace the context words of the second word in the updated first annotated corpus with the corresponding target text to obtain the intermediate corpus;

[0123] The second annotated corpus includes a third word corresponding to the second word, and the category label of the third word is the same as the category label of the first word.

[0124] Optionally, the number n named entities in the first annotated corpus e When the value is 1, the number of named entities in the partial group is 1; when the number of named entities in the first annotated corpus is greater than 1, the number of named entities in the partial group is greater than or equal to n. e / 2.

[0125] Optionally, the first semantic feature model is obtained by unsupervised training on data from a preset corpus, or by unsupervised training on unlabeled corpus.

[0126] Optionally, the semantic representation capability of the first semantic feature model is lower than that of the second semantic feature model.

[0127] Optionally, the first semantic feature model is a GloVe model or a word2vec model, and the second semantic feature model is a SimBert model, a SimCSE model, or a RosemerSim model.

[0128] It should be noted that the device in this embodiment corresponds to the method described above, and the implementation methods in each of the above embodiments are applicable to the device embodiment and can achieve the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail here.

[0129] Please refer to Figure 3 This application also provides a text data enhancement device 300, including: a transceiver 301 and a processor 302;

[0130] The transceiver 301 is used to acquire text corpus for the target task, the text corpus including labeled corpus;

[0131] The processor 302 is configured to: determine the text to be replaced in the first labeled corpus according to the target task; select multiple candidate texts that are similar to the text to be replaced using a first semantic feature model; select the target text that is most similar to the text to be replaced from the multiple candidate texts using a second semantic feature model; replace the text to be replaced in the first labeled corpus with the target text to obtain intermediate corpus; and perform a back-translation operation on the intermediate corpus to obtain a second labeled corpus, wherein the label information of the second labeled corpus is the same as the label information of the first labeled corpus.

[0132] It should be noted that the device in this embodiment corresponds to the method described above, and the implementation methods in each of the above embodiments are applicable to the device embodiment and can achieve the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail here.

[0133] Please refer to Figure 4 This application also provides a text data enhancement device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the computer program is executed by the processor 401, it implements the various processes of the above-described text data enhancement method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0134] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described text data enhancement method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a text data enhancement device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0137] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for enhancing text data, characterized in that, include: Obtain text corpus for the target task, the text corpus including annotated corpus; Based on the target task, determine the text to be replaced in the first annotated corpus; The first semantic feature model is used to select multiple candidate texts that are similar to the text to be replaced. The second semantic feature model is used to select the target text that is most similar to the text to be replaced from the multiple candidate texts. The text to be replaced in the first annotated corpus is replaced with the target text to obtain intermediate corpus. The intermediate corpus is back-translated to obtain a second annotated corpus, wherein the label information of the second annotated corpus is the same as the label information of the first annotated corpus; The semantic representation ability of the first semantic feature model is lower than that of the second semantic feature model.

2. The method according to claim 1, characterized in that, The step of determining the text to be replaced in the first annotated corpus according to the target task includes: When the target task is a text classification task, a topic model is built based on the text corpus; For the first labeled corpus, the topic model is used to obtain the first topic type with the highest probability corresponding to the first labeled corpus and multiple keywords in the first labeled corpus that correspond to the first topic type, wherein the keywords are the text to be replaced.

3. The method according to claim 2, characterized in that, The probability that the keyword belongs to the first topic type is greater than a preset threshold.

4. The method according to claim 2, characterized in that, The topic model is either an LSI model or an LDA model.

5. The method according to claim 2, characterized in that, The text corpus also includes unannotated corpus; the step of building a topic model based on the text corpus includes: The text corpus is preprocessed to obtain preprocessed corpus, wherein the preprocessing includes word segmentation, stop word removal, and filtering of illegal characters; Based on the preprocessed corpus, a topic model is established, wherein the number of topics in the topic model is set according to the number of target categories of the text classification task, and the number of word frequencies is set according to the word frequency distribution of all words in the labeled corpus in the unlabeled corpus.

6. The method according to claim 1, characterized in that, The step of determining the text to be replaced in the first annotated corpus according to the target task includes: When the target task is named entity recognition, the text corpus is preprocessed to obtain preprocessed corpus; Based on the named entities in the preprocessed annotated corpus, an entity dictionary is constructed, which includes vocabulary under each named entity type. For the first annotated corpus, a portion of the first words corresponding to named entities are randomly selected from the first annotated corpus, and each first word in the first annotated corpus is replaced with a second word under the same named entity type in the entity dictionary to obtain the updated first annotated corpus. The context words of the second words in the updated first annotated corpus are the text to be replaced.

7. The method according to claim 6, characterized in that, Replacing the text to be replaced in the first annotated corpus with the target text includes: replacing the context words of the second word in the updated first annotated corpus with the corresponding target text to obtain the intermediate corpus; The second annotated corpus includes a third word corresponding to the second word, and the category label of the third word is the same as the category label of the first word.

8. The method according to claim 7, characterized in that, The number of named entities n in the first annotated corpus e When the value is 1, the number of named entities in the partial group is 1; when the number of named entities in the first annotated corpus is greater than 1, the number of named entities in the partial group is greater than or equal to n. e / 2.

9. The method according to any one of claims 1 to 8, characterized in that, The first semantic feature model is obtained by unsupervised training on data from a pre-defined corpus, or by unsupervised training on unlabeled corpus.

10. The method according to claim 9, characterized in that, The first semantic feature model is the GloVe model or the word2vec model, and the second semantic feature model is the SimBert model, the SimCSE model, or the RosemerSim model.

11. A text data enhancement device, characterized in that, include: The acquisition module is used to acquire text corpus for the target task, the text corpus including labeled corpus; The determination module is used to determine the text to be replaced in the first labeled corpus; The replacement module is used to select multiple candidate texts that are similar to the text to be replaced using a first semantic feature model, select the target text that is most similar to the text to be replaced from the multiple candidate texts using a second semantic feature model, and replace the text to be replaced in the first annotated corpus with the target text to obtain intermediate corpus; The back-translation module is used to perform a back-translation operation on the intermediate corpus to obtain a second labeled corpus, wherein the label information of the second labeled corpus is the same as the label information of the first labeled corpus; The semantic representation ability of the first semantic feature model is lower than that of the second semantic feature model.

12. The apparatus according to claim 11, characterized in that, The determining module is further configured to: When the target task is a text classification task, a topic model is built based on the text corpus; For the first labeled corpus, the topic model is used to obtain the first topic type with the highest probability corresponding to the first labeled corpus and multiple keywords in the first labeled corpus that correspond to the first topic type, wherein the keywords are the text to be replaced.

13. The apparatus according to claim 12, characterized in that, The probability that the keyword belongs to the first topic type is greater than a preset threshold.

14. The apparatus according to claim 12, characterized in that, The topic model is either an LSI model or an LDA model.

15. The apparatus according to claim 12, characterized in that, The text corpus also includes unannotated corpus; the determining module is further configured to: The text corpus is preprocessed to obtain preprocessed corpus, wherein the preprocessing includes word segmentation, stop word removal, and filtering of illegal characters; Based on the preprocessed corpus, a topic model is established, wherein the number of topics in the topic model is set according to the number of target categories of the text classification task, and the number of word frequencies is set according to the word frequency distribution of all words in the labeled corpus in the unlabeled corpus.

16. The apparatus according to claim 11, characterized in that, The determining module is further configured to: When the target task is named entity recognition, the text corpus is preprocessed to obtain preprocessed corpus; An entity dictionary is constructed based on the named entities in the preprocessed annotated corpus. The entity dictionary includes vocabulary for each named entity type. For the first annotated corpus, a portion of the first words corresponding to named entities are randomly selected from the first annotated corpus, and each first word in the first annotated corpus is replaced with a second word under the same named entity type in the entity dictionary to obtain the updated first annotated corpus. The context words of the second words in the updated first annotated corpus are the text to be replaced.

17. The apparatus according to claim 16, characterized in that, The replacement module is further configured to: replace the context words of the second word in the updated first annotated corpus with the corresponding target text to obtain the intermediate corpus; The second annotated corpus includes a third word corresponding to the second word, and the category label of the third word is the same as the category label of the first word.

18. The apparatus according to claim 17, characterized in that, The number of named entities n in the first annotated corpus e When the value is 1, the number of named entities in the partial group is 1; when the number of named entities in the first annotated corpus is greater than 1, the number of named entities in the partial group is greater than or equal to n. e / 2.

19. The apparatus according to any one of claims 11 to 18, characterized in that, The first semantic feature model is obtained by unsupervised training on data from a pre-defined corpus, or by unsupervised training on unlabeled corpus.

20. The apparatus according to any one of claims 11 to 18, characterized in that, The first semantic feature model is the GloVe model or the word2vec model, and the second semantic feature model is the SimBert model, the SimCSE model, or the RosemerSim model.

21. A text data enhancement device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 10.

Images