Text data enhancement method, device and apparatus and computer readable storage medium

Abstract

The invention relates to a text data enhancement method, device and apparatus and a computer readable storage medium, and the method comprises the following steps: dividing training data into a training set and a verification set according to a preset proportion, grouping the training set according to the polarity of sentence aspect words in the training set, and counting the number of polarity sentences in each group; according to a preset enhancement rate, generating polar sentences with similar or equal sentence numbers in each group; and combining the enhanced polar sentences with the corresponding parent sentences, disorganizing, sorting and storing. According to the text data enhancement method provided by the invention, the training data of the preset scale is classified, and the classified sentences are enhanced according to the polarity of the aspect words in the sentences, so that the quantity of the training data is increased on the premise of ensuring that the syntactic structure is not changed and reducing the introduction of noise interference, and thus the accuracy of natural language processing is improved.

Description

technical field [0001] The invention relates to the field of computer processing of natural language, in particular to a text data enhancement method, device, equipment and computer-readable storage medium. Background technique [0002] In supervised text data classification tasks, deep learning models are trained on data with associated class labels. Models tend to perform better if there are sufficient, representative samples of each class label in the data. In natural language processing, the main challenge is that there are few labeled sample data, especially for low-resource language data, such as certain dialects or minority languages. Manually collecting and labeling additional data is a time-consuming and inefficiently challenging task. Therefore, in such text data classification tasks, text data enhancement methods are commonly used. [0003] Existing text data enhancement methods are mainly aimed at sentences, through methods such as synonym replacement, back tr...

AI Technical Summary

Problems solved by technology

However, this method is often limited by the calling frequency of the translation interface, and cannot guarantee the diversity of translation result clauses

The generative model method mainly uses pre-trained language models such as Bert to predict and generate words that are replaced by MASK in sentences, combined with sequence labels, and uses the model to complete the words that are masked to obtain new clauses. This method ensures that the enhancer The diversity of sentence expressions, but often requires more hyperparameter optimization and a high-configuration operating environment to ensure the accuracy of predictions

Images