Small sample text classification method fusing data augmentation and curriculum learning

Abstract

The application discloses a small sample text classification method fusing data expansion and course learning, first expands data of a labeled sample set by using a strategy, then divides the data set according to a change degree of the expanded sample, then constructs a neural network classifier and trains a classification model, and finally classifies new samples by using the trained model. The learning method is improved and reconstructed, on one hand, data expansion is performed by using multiple strategies, and on the other hand, samples are divided into grades according to change degrees from small to large, and in the learning process, a course learning strategy is used to learn gradually from easy to difficult, and the classification accuracy is significantly improved.

Description

Technical Field

[0001] This invention relates to a small-sample text classification method that integrates data augmentation and curriculum learning. Background Technology

[0002] Text classification is a common and crucial downstream task in the field of natural language processing, with specific scenarios including sentiment classification, intent classification, and news topic classification. These tasks typically require a large number of labeled samples for learning; however, in real-world business scenarios, it is often difficult to obtain large-scale labeled real text data. Sometimes, there are many categories but only a few samples (a few or a dozen) for each category, and the imbalance between the number of samples in different categories is very common. In such cases, traditional text classification methods fail to learn valuable features or overfit to the limited samples available, either failing completely or performing poorly in some categories. Therefore, building a small-sample text classification model is extremely challenging.

[0003] Few-shot learning is a task that humans excel at, but it is a relatively new field for machine learning. Few-shot learning is an application of meta-learning in supervised learning. Specifically, during the training phase, C classes are selected from the dataset, with K samples from each class (a total of C*K samples) to construct a meta-task, which serves as the support set input for the model to learn the model's generalization ability under class changes. During the testing phase, when faced with these C new samples, the model can complete class prediction without modification. However, due to learning limitations (C is too small) and the characteristics of the samples, the few-shot learning method, which has achieved remarkable results in computer vision, cannot meet the practical needs when directly applied to text classification tasks. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a small-sample text classification method that integrates data augmentation and course learning.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A small-sample text classification method that integrates data augmentation and curriculum learning is characterized by the following steps:

[0007] 1) Use strategies to augment the labeled sample set;

[0008] 2) Divide the dataset according to the degree of change in the expanded samples;

[0009] 3) Construct a neural network classifier and train the classification model;

[0010] 4) Use the trained model to classify new samples.

[0011] Furthermore, in the aforementioned small-sample text classification method that integrates data augmentation and course learning, S1) data augmentation is performed on the labeled sample set using a strategy, wherein the strategy is one of the following data augmentation strategies or a combination thereof:

[0012] (a) Word equivalence replacement: Select one or more non-stop words from the original sentence and randomly select their synonyms or related concepts to replace them;

[0013] (ii) Random word replacement: Select one or more words from the original sentence and replace them with words randomly selected from the vocabulary list;

[0014] (iii) Random word swapping: Randomly select two words from the original sentence and swap their positions, and perform this once or more;

[0015] (iv) Random insertion: Select one or more non-stop words from the original sentence, randomly select their synonyms or related terms, and insert them into random positions in the sentence;

[0016] (v) Random deletion: randomly select words from the sentence at a certain ratio and delete them;

[0017] (vi) Back-and-forth translation: the original sentence is translated into another language and then back into the original language. Depending on the needs, more than two rounds of translation steps may be required to obtain another expression of the original sentence.

[0018] (vii) Sentence transformation: active sentences become passive sentences, and direct expressions become indirect / euphemistic expressions.

[0019] Furthermore, the aforementioned small-sample text classification method that integrates data augmentation and course learning generates several times or even ten times more variant samples with differences based on the original samples through the individual or combined use of strategies; after data augmentation processing, multiple variants of different forms are obtained.

[0020] Furthermore, in the aforementioned small-sample text classification method that integrates data augmentation and course learning, S2) divides the dataset according to the degree of change of the augmented samples. After obtaining the original sample variants in step S1), to achieve better learning results, the sample variants are grouped according to the degree of change and the machine learning progress is arranged in sequence. The steps are as follows:

[0021] S21) Calculate the edit distance between the sample variant and the original sample;

[0022] Edit distance, also known as Levenshtein distance, is a quantitative metric that measures the degree of difference between two strings. It refers to the minimum number of edit operations required to transform one string into another. The basic edit operations allowed include inserting a character, deleting a character, and replacing one character with another.

[0023] S22) Calculate the change coefficient of the sample variant relative to the original sample;

[0024] Under the same edit distance change, longer sentences can tolerate more character changes while maintaining semantic consistency, while shorter sentences are more prone to semantic changes; in order to better measure the degree of change of the sample relative to the original sentence, the sentence length needs to be taken into consideration; introduce the change coefficient g, which can be calculated by formula (1);

[0025] g(Sen0, Sen1)=Lev(Sen0, Sen1) / Len(sen0) Formula (1)

[0026] Where Lev(Sen0, Sen1) is the edit distance between the two samples, and Len(sen0) is the sentence length of Sen0, which is the number of words in the sentence;

[0027] S23) The sample variants are divided into several levels according to the change coefficients and their distribution;

[0028] When classifying levels, if some ranges have few samples, these isolated samples can be ignored, and finally these out-of-range samples can be merged into the neighboring levels.

[0029] S24) Perform class equalization on sample variants at each level;

[0030] The strategy is to perform class balancing on the sample variants at each level so that the number of samples in each class is roughly equal. If there are too many samples in some classes at certain levels, some can be randomly removed for later use. If there are too few samples in some classes, step S1) can be repeated to supplement them appropriately. Finally, K variant sample sets with different change levels can be obtained for model learning.

[0031] S25) Divide the sample set into a training set, a validation set, and a test set;

[0032] Before model training, the data is segmented according to function, and samples are randomly selected from each category in proportion to form the training set, validation set and test set.

[0033] Furthermore, in the aforementioned method for small-sample text classification that integrates data augmentation and course learning, S3) constructs a neural network classifier and trains the classification model. The classifier uses a multilayer perceptron neural network or a neural network with convolutional layers, and is constructed according to the following structure:

[0034] Input layer: This is the vector representation of the sample. We select BERT and Roberta pre-trained language models to obtain the vectors of the sample sentences.

[0035] Intermediate layer: Uses a hidden layer with 200 hidden units and a dropout layer with a probability p = 0.4;

[0036] Final linear layer: has 40 hidden units.

[0037] Furthermore, the aforementioned method for small-sample text classification that integrates data augmentation and course learning includes the following steps for training the classification model:

[0038] S31) Perform model training once on the original samples;

[0039] The training process can adopt the triplet loss model, that is, given a triplet, anchor point m, positive sample z, negative sample f, the total loss of the network is calculated by formula (2):

[0040]

[0041] Where d(m,z) and d(m,f) represent the distance between the input codes of two samples, using cosine distance; α is a forced discrimination interval assigned to the positive and negative sample pairs; and the total loss Loss is the sum of the losses for each triple.

[0042] The triplet sampling scheme employs two strategies: random sampling and hard sample mining. Hard sample mining involves selecting samples where d(m,z)+α>d(m,f). One sample is chosen from the dataset as an anchor point, another sample from the same class is selected as a positive sample, and a negative sample is randomly selected from all samples in other classes to construct a triplet as a learning unit. Through the constraints of the loss function, the network continuously updates its parameters to make each anchor point as close as possible to the positive sample and as far away as possible from the negative sample. This results in a dataset where samples from different classes maintain a certain distance from each other, while samples of the same class are clustered together in close proximity.

[0043] S32) Continue training the model on the variant sample set with the lowest change level;

[0044] After performing step S31) on the training set consisting of the original samples, the model is trained again using the variant sample set with the lowest change level to optimize the network parameters; the word changes in this part of the sample set relative to the original samples are equivalent to introducing noise, which increases the difficulty of model classification.

[0045] S33) Continue training the model on a variant sample set with a higher change level until all training sets have been used up;

[0046] After step S32), the model is trained again using a variant sample set with higher change levels to optimize the network parameters. As the change of the sample set relative to the original sample becomes larger and larger, the difficulty of classification gradually increases, which helps to improve the generalization ability of the model. After learning variant samples at all levels, the model training process is completed.

[0047] Furthermore, in the aforementioned small-sample text classification method that integrates data augmentation and course learning, S4) uses the trained model to classify new samples. Through the gradual learning process in step S3), the model fully learns the common features of samples within the same category and the main distinguishing features of samples from different categories. When the model receives a new sample to be predicted, the classifier first finds the training sample that is closest to the current sample and returns the category label corresponding to that training sample.

[0048] Compared with the prior art, the present invention has significant advantages and beneficial effects, specifically reflected in the following aspects:

[0049] This invention employs multiple strategies to augment the classification samples. It introduces a certain degree of noise into the original real samples to alleviate the scarcity of real samples in practical business scenarios. Then, it classifies the augmented samples according to the degree of change they represent compared to the original samples, and trains the model sequentially according to this change level. Mimicking the human learning process, it gradually increases the learning difficulty by adding data noise, thereby improving the model's generalization ability. During the classifier's learning phase, various pre-trained models can be used for sentence embedding, and various network structures can also be employed. The limited number of labeled samples is significantly augmented, and through learning from easy to difficult, samples of different categories are more easily distinguished. This invention has broad application prospects in the field of text classification in natural language processing.

[0050] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing specific embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 : A schematic diagram of the process of this invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0054] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, directional and ordinal terms are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0055] To address the difficulties in small-sample text classification in real-world scenarios, this invention improves and reconstructs the learning method. On one hand, it employs multiple strategies to expand the data; on the other hand, it categorizes samples according to their degree of change from small to large. Furthermore, it utilizes a course-based learning strategy, gradually teaching from easy to difficult. Ultimately, this method achieves a significant improvement in classification accuracy compared to previous methods, better meeting business needs.

[0056] like Figure 1 As shown, the small sample text classification method that integrates data augmentation and course learning has the following specific steps:

[0057] S1) Use strategies to augment the labeled sample set;

[0058] The strategy is the following data augmentation strategies or a combination thereof:

[0059] (a) Word equivalence replacement: Select one or more non-stop words from the original sentence and randomly select their synonyms or related concepts to replace them;

[0060] (ii) Random word replacement: Select one or more words from the original sentence and replace them with words randomly selected from the vocabulary list;

[0061] (iii) Random word swapping: Randomly select two words from the original sentence and swap their positions, and perform this once or more;

[0062] (iv) Random insertion: Select one or more non-stop words from the original sentence, randomly select their synonyms or related terms, and insert them into random positions in the sentence;

[0063] (v) Random deletion: randomly select words from the sentence at a certain ratio and delete them, such as 5% or 10%;

[0064] (vi) Back-and-forth translation: the original sentence is translated into another language and then back into the original language. Depending on the needs, more than two rounds of translation steps may be required to obtain another expression of the original sentence.

[0065] (vii) Sentence transformation: active sentences are changed into passive sentences, and direct expressions are changed into indirect / euphemistic expressions;

[0066] The implementation of the above data augmentation strategies may require varying degrees of manual assistance. By using these strategies individually or in combination, several or even ten times more variant samples with certain differences can be generated based on the original samples. For example, a sample Sen0: "Where should I go to apply for a social security card for my child in Xiamen?" can be augmented to obtain multiple variants in different forms, such as:

[0067] Sen1: My child wants to get a medical insurance card in Xiamen. Where should I go to apply for it?

[0068] Sen2: My child is currently in Xiamen and wants to apply for a medical insurance card and a social security card. Where should we go to apply for them?

[0069] Sen3: My child wants to apply for a social security card in Fuzhou. Where should we go to apply for it?

[0070] Sen4: My child wants to apply for a social security card. Where in Xiamen should we go to apply for it?

[0071] Sen5: My friend's child wants to apply for a social security card in Quanzhou. Where should they go to apply for it?

[0072] ……wait;

[0073] When augmenting data, the smaller the average number of samples per class in the original sample, the greater the expansion factor required for each sample, and vice versa. When the average number of samples per class is less than 30, each sample needs to be expanded to a dozen to several dozen. When each class has an average of several hundred or more samples, each sample needs to be expanded to 4 to 8.

[0074] S2) Divide the dataset according to the degree of change in the expanded samples;

[0075] After obtaining the original sample variants in step S1), in order to achieve better learning results, the sample variants are grouped according to the degree of change and the machine learning progress is arranged in sequence, as follows:

[0076] S21) Calculate the edit distance between the sample variant and the original sample.

[0077] Edit distance, also known as Levenshtein distance, is a quantitative metric that measures the degree of difference between two strings. It refers to the minimum number of edit operations required to transform one string into another. The basic edit operations allowed include inserting a character, deleting a character, and replacing one character with another.

[0078] For example, for the sample Sen0 in step S1), by replacing 'friend' with 'child', 'each' with 'a', and 'society' with 'medicine', it can be transformed into sample Sen1, with an edit distance Lev(Sen0, Sen1) of 4; similarly, the edit distances of Sen2, Sen3, Sen4, and Sen5 relative to Sen0 are 5, 5, 8, and 10, respectively.

[0079] S22) Calculate the coefficient of change of the sample variant relative to the original sample.

[0080] Under the same edit distance change, longer sentences can tolerate more character changes while maintaining semantic consistency, while shorter sentences are more prone to semantic changes; in order to better measure the degree of change of the sample relative to the original sentence, the sentence length needs to be taken into consideration; introduce the change coefficient g, which can be calculated by formula (1);

[0081] g(Sen0, Sen1)=Lev(Sen0, Sen1) / Len(sen0) Formula (1)

[0082] Where Lev(Sen0, Sen1) is the edit distance between the two samples, and Len(sen0) is the sentence length of Sen0, which is the number of words in the sentence (excluding punctuation, symbols, etc.).

[0083] For example, the change coefficient g(Sen0, Sen1) of the aforementioned variant Sen1 relative to the sample Sen0 is approximately 0.167 (4 / 24); similarly, the change coefficients of Sen2, Sen3, Sen4, and Sen5 relative to the sample Sen0 are approximately 0.208, 0.208, 0.333, and 0.417, respectively.

[0084] S23) The sample variants are divided into several levels according to the change coefficients and their distribution;

[0085] Typically, a level can be defined as 0.1, and samples with a change coefficient not exceeding 0.5 can be divided into 5 levels. When dividing the levels, if some ranges have few samples, these isolated samples can be ignored, and finally, these out-of-range samples can be merged into the neighboring levels.

[0086] For example, according to the typical classification method, Sen1 is classified as the second class, Sen2 and Sen3 are classified as the third class, Sen4 is classified as the fourth class, and Sen5 is classified as the fifth class.

[0087] Since variant samples are still given the original sample's category label during data augmentation, this is based on the premise that small changes will not affect the semantics of the original sentence; in order to make the premise hold as much as possible, the degree of change of the variant should not be too large; the change coefficient should not exceed 0.5;

[0088] S24) Perform class equalization on sample variants at each level.

[0089] By employing appropriate strategies to perform class balancing on the sample variants at each level, the number of samples in each class can be made roughly equal. For example, if there are too many samples in some classes at certain levels, some samples can be randomly removed for backup. If there are too few samples in some classes, step S1) can be repeated to supplement them appropriately. In the end, K variant sample sets with different change levels can be obtained for model learning.

[0090] S25) Divide the sample set into a training set, a validation set, and a test set;

[0091] Before model training, the data is split according to function, and there is no fixed value for the specific proportion; samples can be randomly selected from each category to form the training set, validation set and test set at a ratio of 70%, 10% and 20% respectively.

[0092] S3) Construct a neural network classifier and train the classification model;

[0093] The classifier is constructed using a multilayer perceptron neural network, or a neural network with special units such as convolutional layers. For example, a neural network can be built with the following structure:

[0094] Input layer: This is the vector representation of the sample. Pre-trained language models such as BERT and Roberta are selected to obtain the vectors of the sample sentences.

[0095] Intermediate layer: Use a hidden layer with 200 hidden units (using tanh as the activation function) and a dropout layer with a probability p = 0.4;

[0096] Final linear layer: has 40 hidden units;

[0097] Training a classification model involves the following steps:

[0098] S31) Perform model training once on the original samples;

[0099] The training process can adopt the triplet loss model, that is, given a triplet, anchor point m, positive sample z, negative sample f, the total loss of the network is calculated by formula (2):

[0100]

[0101] Where d(m,z) and d(m,f) represent the distance between the input codes of two samples, using cosine distance; α is a forced discrimination interval assigned to the positive and negative sample pairs, which can be set to 0.4; the total loss Loss is the sum of the losses of each triple;

[0102] The triplet sampling scheme employs two strategies: random sampling and hard sample mining. Hard sample mining involves selecting samples where d(m,z)+α>d(m,f). One sample is chosen from the dataset as an anchor point, another sample from the same class is selected as a positive sample, and a negative sample is randomly selected from all samples in other classes to construct a triplet as a learning unit. Through the constraints of the loss function, the network continuously updates its parameters to make each anchor point as close as possible to the positive sample and as far away as possible from the negative sample. This results in a dataset where samples from different classes maintain a certain distance from each other, while samples of the same class are clustered together in close proximity.

[0103] S32) Continue training the model on the variant sample set with the lowest change level;

[0104] After performing step S31) on the training set consisting of the original samples, the model is trained again using the variant sample set with the lowest change level (first level) to optimize the network parameters; the word changes in this part of the sample set relative to the original samples are equivalent to introducing noise, which slightly increases the difficulty of model classification.

[0105] S33) Continue training the model on a variant sample set with a higher change level until all training sets have been used up;

[0106] After step S32), the model is trained again using a variant sample set with higher change levels to optimize network parameters. As the change of the sample set relative to the original sample becomes larger and larger, the difficulty of classification gradually increases, which helps to improve the generalization ability of the model. After learning variant samples at all levels, the model training process is completed.

[0107] S4) Use the trained model to classify new samples;

[0108] Through the gradual learning process in step S3), the model fully learns the common features of samples within the same category and the main distinguishing features of samples from different categories. When the model receives a new sample to be predicted, the classifier first finds the training sample that is closest to the current sample and returns the category label corresponding to that training sample. The quality of the category labeling of the training set samples is crucial to the accuracy of the classification model's prediction.

[0109] In summary, this invention employs multiple strategies to augment the classification samples. It introduces a certain degree of noise into the original real samples to alleviate the problem of scarce real samples in practical business scenarios. Then, it classifies the augmented samples according to the degree of change relative to the original samples and trains the model sequentially according to the degree of change. Mimicking the human learning process, it gradually increases the learning difficulty by adding data noise, thereby improving the model's generalization ability. During the classifier's learning phase, various pre-trained models can be used for sentence embedding, as well as various network structures, such as triplet networks. The limited number of labeled samples is significantly augmented, and through learning from easy to difficult, samples of different categories are more easily distinguished. This invention has broad application prospects in the field of text classification in natural language processing.

[0110] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of protection of the invention. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0111] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0112] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A small sample text classification method fusing data augmentation and curriculum learning, characterized in that: Comprising the following steps: S1) data augmentation on the labeled sample set with strategies; S2) dividing the data set according to the change degree of the augmented samples; After obtaining the original sample variants through step S1), in order to achieve better learning effect, the sample variants are grouped according to the change degree and the machine learning schedule is arranged in sequence, and the steps are as follows: S21) calculating the edit distance of the sample variant and the original sample; The edit distance, also known as Levenshtein distance, is a quantitative indicator for measuring the difference between two strings, which refers to at least how many editing operations are needed to change one string into another string; The basic editing operations allowed include inserting a character, deleting a character, and replacing a character with another character; S22) calculating the change coefficient of the sample variant relative to the original sample; In the case of equal edit distance change, longer sentences can tolerate more character changes while maintaining semantic consistency, while shorter sentences are more likely to change semantics; In order to better measure the change degree of the sample relative to the original sentence, the sentence length needs to be considered; Introduce the change coefficient g, which can be calculated by formula (1); g(Sen0, Sen1) = Lev(Sen0, Sen1) / Len(sen0) Formula (1) Where Lev(Sen0, Sen1) is the edit distance of the two samples, and Len(sen0) is the sentence length of Sen0, which contains the number of words in the sentence; S23) dividing the sample variants into several levels according to the change coefficient and its distribution; When dividing the levels, if some range samples are sparse, these isolated samples can be ignored, and finally these out-of-range samples are merged into the adjacent levels; S24) class balancing in sample variants in each level; Through the strategy, the sample variants in each level are balanced in class, so that the number of samples in each class is roughly the same; If there are too many samples in some classes, a part of the standby can be randomly removed; If the number of samples in some classes is too small, repeat step S1) to appropriately supplement, and finally obtain K variant sample sets of different change levels for model learning; S25) dividing the sample set into training set, validation set and test set; Before model training, the data is divided according to the function, and the training set, validation set and test set can be randomly selected from each category according to the proportion; S3) constructing a neural network classifier and training a classification model; S4) using the trained model to classify new samples.

2. The method of claim 1, wherein the method of fusing data augmentation and curriculum learning for small sample text classification is characterized by: S1) data augmentation on the labeled sample set with strategies, the strategies are the following data augmentation strategies or combinations thereof: (1) word equivalent replacement, selecting one or more non-stop words from the original sentence, and randomly selecting the same synonym or superordinate concept to replace; (2) random word replacement, selecting one or more words from the original sentence and replacing them with randomly selected words from the vocabulary; (3) random word exchange, randomly selecting two words in the original sentence and exchanging their positions, which can be executed once or more; (4) random insertion, selecting one or more non-stop words from the original sentence, and randomly selecting the same synonym or superordinate concept to insert into a random position in the sentence; (five) random deletion, delete any word from the sentence at a certain ratio; (six) back-and-forth translation, translate the original sentence into another language, and then translate it back into the original language. If necessary, the original sentence can be translated into another language more than twice to obtain another expression of the original sentence; (seven) sentence transformation, active sentence to passive sentence, direct expression to indirect / euphemistic expression. 3.The method of claim 2, wherein: Through the use of strategies alone or in combination, several or dozens of times more variant samples with differences are generated based on the original sample; After data expansion processing, multiple different forms of variants are obtained.

4. The method of claim 1, wherein the method of fusing data augmentation and curriculum learning for small sample text classification is characterized by: S3) Construct a neural network classifier and train a classification model. The classifier is constructed using a multilayer perceptron neural network or a neural network with convolutional layers, and the neural network is built as follows: Input layer: is the vector representation of the sample, select BERT, Roberta pre-training language model to obtain the vector of the sample sentence; Intermediate layer: use an implicit layer with 200 hidden units and a dropout layer with a probability p = 0.4; Final linear layer: has 40 hidden units.

5. The method of claim 1 or 4, wherein the method of small sample text classification fusing data augmentation and curriculum learning is characterized in that: Training the classification model includes the following steps: S31) Perform one iteration of model training on the original sample; The training process can use a triple loss mode, that is, given a triple, anchor m, positive sample z, and negative sample f, the total loss of the network is calculated by formula (2): Where d(m,z), d(m,f) represents the distance between the input encoding of the two samples, and the cosine distance is used; α is a distinguishing interval forced to be given to positive and negative samples; The total loss Loss is the sum of the loss of each triple; The sampling scheme of the triple adopts two strategies: random sampling and difficult sample mining. Difficult sample mining is to select samples that satisfy d(m,z) + α > d(m,f). Select one sample as the anchor point from the data set, select another as the positive sample in the same category, and randomly select one as the negative sample from all samples in other categories to construct a triple as a learning unit; Through the constraint of the loss function, the network learns to continuously update the parameters so that the distance between each anchor point and the positive sample is as close as possible, while the anchor point and the negative sample are as far apart as possible, so that the sample distribution of the data set presents the characteristics that the samples of each category maintain a certain distance, while the samples of the same category are close to each other; S32) Continue training the model on the lowest change level variant sample set; After step S31) on the training set composed of original samples, continue to train the model using the variant sample set with the lowest change level to optimize the network parameters; This part of the sample set, relative to the word changes of the original sample, is equivalent to introducing noise, increasing the difficulty of model classification; S33) Continue training the model on the higher change level variant sample set until all training sets are used up; After step S32), continue to train the model using the variant sample set with a higher change level to optimize the network parameters; As the change of the sample set relative to the original sample becomes larger and larger, the difficulty of classification gradually increases, which helps to improve the generalization ability of the model. After learning all the variant samples at all levels, the model training process is completed.

6. The method of claim 1, wherein the method of fusing data augmentation and curriculum learning for small sample text classification is characterized by: S4) using the trained model to classify new samples, after the step S3) of the sequential learning process, the model has learned the common features of the samples in the same class and the main distinguishing features of the samples in different classes; when the model classifies a new sample, the classifier first finds the training sample closest to the current sample, and returns the class label corresponding to the training sample.

Images