A text sentiment classification method and system

Abstract

The present invention discloses a text sentiment classification method and system. The text sentiment classification method includes the following steps: obtaining the text to be sentiment classified; based on the obtained text to be sentiment classified, using a pre-trained sentiment classification model to perform text sentiment classification to obtain a sentiment classification result; wherein, the sentiment classification model includes: a feature extractor, which is used to input the text and perform feature extraction, and output a feature vector; a linear classifier, which is used to input the feature vector output by the feature extractor and perform classification, and output a sentiment classification result. In the text sentiment classification method disclosed by the present invention, a new model training method is provided to improve the overall performance of the model. Text sentiment classification based on the above-mentioned trained sentiment classification model can have a high accuracy rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and particularly relates to a text sentiment classification method and system. Background Art

[0002] Text sentiment classification is the most common and important task type in the field of natural language processing. Currently, most existing text sentiment classification methods are implemented based on trained classification models.

[0003] When implementing text sentiment classification based on a trained classification model, the capacity of training samples and labels have a great impact on the final classification result of the model. Currently, when training an emotion classification model, semi-supervised or supervised training methods are generally used; however, using the semi-supervised training method cannot obtain a model with good performance; when using the supervised training method to obtain good model performance, the labeled data is very limited (specifically explanatory, manual annotation consumes a large amount of manpower and time, and it is generally difficult to directly obtain labeled samples during the actual model training process). In summary, the accuracy of existing text sentiment classification based on trained classification models is low, and there is an urgent need for a new text sentiment classification method. Summary of the Invention

[0004] The purpose of the present invention is to provide a text sentiment classification method and system to solve one or more of the above technical problems. In the text sentiment classification method disclosed by the present invention, a new model training method is provided to improve the overall performance of the model, and text sentiment classification based on the trained sentiment classification model can have a high accuracy.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A text sentiment classification method provided in the first aspect of the present invention includes the following steps:

[0007] Obtain the text to be sentiment classified;

[0008] Based on the obtained text to be sentiment classified, use the pre-trained sentiment classification model to perform text sentiment classification and obtain the sentiment classification result;

[0009] Among them, the sentiment classification model includes:

[0010] A feature extractor for inputting text, extracting features, and outputting a feature vector;

[0011] A linear classifier for inputting the feature vector output by the feature extractor, performing classification, and outputting the sentiment classification result;

[0012] Among them, the steps for obtaining the pre-trained sentiment classification model include:

[0013] Obtain a first training sample set; wherein, the training samples in the first training sample set include labeled training samples and unlabeled training samples with pseudo-labels;

[0014] When training the model based on the training samples in the first training sample set, input the text of the selected training sample into a pre-trained feature extractor to obtain a feature vector; input the obtained feature vector into a linear classifier, and update the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample. After reaching the preset convergence condition, obtain the pre-trained sentiment classification model;

[0015] The steps for obtaining the pre-trained feature extractor include:

[0016] Obtain a second training sample set; wherein, the training samples in the second training sample set include a first preset number of labeled training samples and a second preset number of unlabeled training samples;

[0017] Perform data augmentation on all training samples in the second training sample set in two different ways, and each training sample obtains two homologous augmented data; in each training batch, calculate the contrast loss of the two homologous augmented data, and update the parameters of the feature extractor based on the contrast loss, so that the feature vectors of homologous texts are close to each other and the feature vectors of non-homologous texts are far from each other. After reaching the preset training requirements, obtain the trained feature extractor;

[0018] Based on the first preset number of labeled training samples, train the trained feature extractor and the pre-acquired label classifier to obtain the trained feature extractor and label classifier.

[0019] A further improvement of the method provided by the present invention is that in the step of updating the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample, the expression of the loss function used when updating the parameters is

[0020]

[0021] wherein, is the probability of being predicted as the positive class.

[0022] A further improvement of the method provided by the present invention is that the step of obtaining the first training sample set specifically includes:

[0023] Input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain the pseudo-labels of the unlabeled training samples;

[0024] Based on all the unlabeled training samples that have obtained pseudo-labels, select the unlabeled training samples whose confidence levels meet the preset conditions; add the selected unlabeled training samples to the labeled training samples to obtain the first training sample set.

[0025] A further improvement of the method provided by the present invention lies in that the step of respectively inputting the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier to obtain pseudo-labels of the unlabeled training samples specifically includes:

[0026] Respectively input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier to obtain prediction results;

[0027] When the prediction result corresponding to an unlabeled training sample belongs to a certain class with a probability exceeding the preset threshold, assign a pseudo-label to the unlabeled training sample.

[0028] A further improvement of the method provided by the present invention lies in that the step of, based on all the unlabeled training samples that have obtained pseudo-labels, selecting the unlabeled training samples whose confidence levels meet the preset conditions specifically includes:

[0029] Based on all the unlabeled training samples that have obtained pseudo-labels, select the threshold τ p 、τ n 、k p 、k n by optimizing the loss function, and select the unlabeled training samples that meet the threshold requirements;

[0030] Among them, the expression of the optimized loss function is,

[0031] In the formula, for sample i, represents the pseudo-label of whether sample i belongs to class c; represents the probability that the original output of the model belongs to class c; is used to indicate whether the sample is assigned a pseudo-label; τ p 、τ n are the thresholds for becoming positive and negative example samples; is used to represent the uncertainty of the model prediction result ; k p 、k n are the uncertainty thresholds; is the probability of being predicted as the positive class; C represents the total number of classes.

[0032] A text sentiment classification system provided by the second aspect of the present invention includes:

[0033] A text acquisition module, configured to acquire the text to be sentiment classified;

[0034] An emotion classification result acquisition module, which is used to perform text emotion classification on the acquired text to be emotion classified by using a pre-trained emotion classification model, and obtain an emotion classification result;

[0035] Among them, the acquisition steps of the pre-trained emotion classification model include:

[0036] Obtain a first training sample set; among them, the training samples in the first training sample set include labeled training samples and unlabeled training samples with pseudo-labels;

[0037] When performing model training based on the training samples in the first training sample set, input the text of the selected training sample into a pre-trained feature extractor to obtain a feature vector; input the obtained feature vector into a linear classifier, and update the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample. After reaching the preset convergence condition, obtain the pre-trained emotion classification model;

[0038] The acquisition steps of the pre-trained feature extractor include:

[0039] Obtain a second training sample set; among them, the training samples in the second training sample set include a first preset number of labeled training samples and a second preset number of unlabeled training samples;

[0040] Perform data augmentation on all training samples in the second training sample set in two different ways, and each training sample obtains two homologous augmented data; in each training batch, calculate the contrast loss of the two homologous augmented data, and update the parameters of the feature extractor based on the contrast loss, so that the feature vectors of homologous texts are close to each other and the feature vectors of non-homologous texts are far from each other. After reaching the preset training requirements, obtain the trained feature extractor;

[0041] Based on the first preset number of labeled training samples, train the trained feature extractor and the pre-acquired label classifier to obtain a trained feature extractor and label classifier.

[0042] A further improvement of the system of the present invention lies in that in the updating of the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample, the expression of the loss function used for updating the parameters is

[0043]

[0044] In the formula, is the probability of being predicted as the positive class.

[0045] A further improvement of the system of the present invention lies in that the steps of obtaining the first training sample set specifically include:

[0046] Input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain the pseudo-labels of the unlabeled training samples;

[0047] Based on all the unlabeled training samples with obtained pseudo-labels, select the unlabeled training samples whose confidence levels meet the preset conditions; add the selected unlabeled training samples to the labeled training samples to obtain the first training sample set.

[0048] A further improvement of the system of the present invention is that the step of inputting the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain the pseudo-labels of the unlabeled training samples specifically includes:

[0049] Input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain prediction results;

[0050] When the prediction result corresponding to an unlabeled training sample belongs to a certain class with a probability exceeding the preset threshold, assign a pseudo-label to the unlabeled training sample.

[0051] A further improvement of the system of the present invention is that the step of selecting the unlabeled training samples whose confidence levels meet the preset conditions based on all the unlabeled training samples with obtained pseudo-labels specifically includes:

[0052] Based on all the unlabeled training samples with obtained pseudo-labels, select the threshold τ p 、τ n 、k p 、k n by optimizing the loss function, and select the unlabeled training samples that meet the threshold requirements;

[0053] Among them, the expression of the optimized loss function is,

[0054] In the formula, for sample i, represents the pseudo-label of whether sample i belongs to class c; represents the probability that the original output of the model belongs to class c; is used to indicate whether the sample is assigned a pseudo-label; τ p 、τ n are the thresholds for becoming positive and negative example samples; is used to represent the uncertainty of the model prediction result ; k p 、k n are the uncertainty thresholds; is the probability of being predicted as the positive class; C represents the total number of classes.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] In the text sentiment classification method disclosed by the present invention, a contrastive learning and UPS (Uncertainty-Aware Pseudo-Label Selection Framework) strategy are combined to train a text sentiment classification model, enabling the model to additionally learn from unlabeled samples and augmented data of samples, which can improve the overall performance of the model. Using the sentiment classification model trained by the above method for text sentiment classification can achieve a high accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for the description of the embodiments or the prior art; obviously, the drawings in the following description are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0058] Figure 1 is a flowchart of a text sentiment classification method according to an embodiment of the present invention;

[0059] Figure 2 is a schematic diagram of the architecture of a text sentiment classification model according to an embodiment of the present invention;

[0060] Figure 3 is a flowchart of the training process of a sentiment classification model according to an embodiment of the present invention;

[0061] Figure 4 is a schematic diagram of the contrastive learning encoding part according to an embodiment of the present invention;

[0062] Figure 5 is a flowchart of a text sentiment classification system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0064] It should be noted that the terms "first", "second", etc. in the specification, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0065] The present invention will be further described in detail below with reference to the accompanying drawings:

[0066] Embodiment 1

[0067] Please refer to Figure 1 , a text sentiment classification method provided by an embodiment of the present invention includes the following steps:

[0068] Obtain the text to be sentiment classified;

[0069] Input the text into the sentiment classification model, and the sentiment classification model outputs a sentiment classification result; wherein, the sentiment classification model includes a pre-trained feature extractor and a linear classifier; the feature extractor is used to extract features from the input text and output a feature vector; the linear classifier is used to classify based on the input feature vector and output a sentiment classification result.

[0070] Specifically, the steps for obtaining the pre-trained feature extractor in the embodiment of the present invention include:

[0071] (1) Obtain a training sample set; wherein, the training sample set includes a preset small amount of labeled sample data and a large amount (exemplarily, in the millions) of unlabeled sample data; further exemplarily optional, the ratio of the total amount of labeled sample data to the total amount of unlabeled sample data is 1:(100 - 1000);

[0072] (2) Perform data augmentation on all training sample data (including labeled and unlabeled data) in two different ways, and each text obtains two homologous augmented data; in each training batch, calculate the contrast loss of the two homologous augmented data, and update the parameters of the initially trained feature extractor based on the contrast loss to obtain a pre-trained feature extractor;

[0073] (3) Based on the labeled sample data, fine-tune and train the feature extractor and the classifier to obtain a trained feature extractor and classifier;

[0074] (4) Input the large amount of unlabeled sample data into the initially trained feature extractor respectively to obtain the pseudo-labels of each unlabeled sample data;

[0075] (5) Among the unlabeled sample data with pseudo-labels, select the unlabeled sample data whose confidence meets the preset conditions; add the selected unlabeled sample data to the labeled data pair classifier for training.

[0076] In step (2) of the present invention, the calculation expression of the contrast loss is

[0077]

[0078]

[0079] In the formula, for the feature vector set v, v -i : = v\{v i}, S SimCLR is the similarity between feature vectors calculated by the inner product, and the superscripts (1) and (2) represent the feature vectors generated by the feature extractor after two data-augmented samples.

[0080] In the embodiment of the present invention, by optimizing the contrast loss function, the feature vectors of homologous texts are made to approach each other, and the feature vectors of different-source texts are made to move away from each other, maximizing the ability of the encoder to capture the effective information of the text and providing a good basic model for subsequent discrimination.

[0081] Step (4) of the present invention specifically includes: inputting the unlabeled sample data into the initially trained feature extractor to obtain a prediction result; when the probability that the prediction result belongs to a certain class exceeds a preset threshold, assign a pseudo-label to the unlabeled sample data.

[0082] In step (5) of the present invention, the step of selecting the unlabeled sample data whose confidence meets the preset conditions among the unlabeled sample data with pseudo-labels specifically includes: by optimizing the loss function, selecting the most suitable thresholds (τ p , τ n , k p , k n ) and assigning pseudo-labels to the unlabeled samples that meet the conditions for subsequent training.

[0083] The specific loss function is as follows:

[0084]

[0085]

[0086] In the formula, for sample i, The pseudo-label indicating whether sample i belongs to class c, which is 1 if it does, and 0 if it doesn't; Indicates the probability that the original output of the model belongs to class c; Indicates whether the sample is labeled with a pseudo-label (for subsequent training); τ p 、τ n Is the threshold for becoming positive and negative example samples; u(p) is used to represent the uncertainty of the model prediction result p (exemplarily, it can be calculated by modeling the estimation through dropout NN); k p 、k n Is the uncertainty threshold.

[0087] In the embodiments of the present invention, the steps for obtaining the pre-trained linear classifier specifically include:

[0088] Obtain a sample set; each sample includes a text feature vector and a classification label; the classification label includes a pre-annotated label and a pseudo-label with a confidence level higher than a preset threshold;

[0089] Input the text feature vector of each sample into the linear classifier respectively, and update the parameters of the linear classifier based on the corresponding classification label to improve the accuracy and stability of the model output probability;

[0090] Among them, the expression of the loss function used during the update is,

[0091]

[0092] In the formula, Is the probability of being predicted as the positive class.

[0093] In the embodiments of the present invention, for the specifically improved text classification, based on the large amount of augmented data provided by contrastive learning, the requirement for the quantity of the initial training data is reduced; the pre-trained contrastive loss function can guide the discriminator's encoding part in extracting effective features of the text, thereby improving the accuracy and stability of the judgment, and further obtaining good encoder parameters after pre-training. In addition, based on the UPS model, not only can useful information be obtained from unlabeled data, thus greatly reducing the training cost, but also because the uncertainty of the deep network is considered, the confidence level of the pseudo-labeled data can be further improved, reducing the noise brought by incorrect pseudo-labels to the model, and playing a role in correcting the model.

[0094] Embodiment 2

[0095] Please refer to Figures 2 to 4 , in the embodiments of the present invention, taking simple binary classification as an example, to determine whether the text expresses positive or negative emotions, the feature extraction of the text is based on network structures such as the latest Transformer encoder, BiLSTM, and TextCNN, etc.; Figure 2Hyperparameters such as the dimension d of the word vector, the number of convolutional kernels, etc. can be adjusted according to the actual data.

[0096] Exemplarily explained in the embodiments of the present invention, the TextCNN network structure includes: an input layer, a convolutional layer, and a pooling layer; a vector representation of the text sentence is obtained, and then a classifier (such as SVM, logistic regression, etc.) is added, and the final classification result is obtained through training. Among them, the input layer: the input layer is an n×k matrix, where n is the number of words in a sentence, and k is the dimension of the word vector corresponding to each word. That is to say, each row of the input layer is a k-dimensional word vector corresponding to a word. In addition, a padding operation is performed on the original sentence to make the vector lengths consistent. Here, use to represent the k-dimensional word embedding of the i-th word in the sentence, and the word vector is expressed by the trained word embedding. Convolutional layer: Different from computer vision, in NLP, the input layer is a word matrix composed of word vectors, and the width of the convolutional kernel is the same as the width of the word matrix. The convolutional kernel only moves in the height direction, which means that the minimum unit of each convolutional kernel sliding is a word. Pooling layer: The network uses 1-Max pooling to select the largest feature from the feature vectors generated by each sliding window, and then splices these features together to form a new feature vector.

[0097] Please refer to Figure 3 , in the specific embodiments of the present invention, the specific training method is as follows:

[0098] (1) Data acquisition and preprocessing, including: using keyword retrieval (for example, positive words such as happy and delighted are directly labeled as a certain amount of positive samples that can be determined as positive emotional expressions, and similarly, angry and sad are directly labeled as negative samples), and at the same time, samples that cannot directly judge the emotional expression are used as unlabeled samples; in this way, a small amount of positive and negative samples and a large amount of unlabeled samples are obtained.

[0099] (2) Contrastive learning training encoding part, including: In this stage, contrastive learning is introduced, and through data augmentation, the encoding part of the classifier (that is, before entering the fully connected layer) is trained to improve its ability to extract text content features and provide a good parameter basis for the downstream discrimination task.

[0100] Please refer to Figure 4 , the specific implementation of the embodiments of the present invention includes: performing data augmentation on all sample sets X in two different ways to obtain two independent augmented matrices t1(X) and t2(X); where t1(x i ), t2(x i ) represent different augmented data of the same text, and t2(x j ) represents the elements in the t2 augmented matrix.

[0101] Augment the text and input it into text-CNN respectively. Denote the feature vector encoded by the discriminator as v( Then they respectively correspond to t1(x i ), t2(x i ), t2(x j ).

[0102] Use S SimCLR to calculate the distance between feature vectors in the embedding space by inner product and the contrast loss function, which is expressed as:

[0103]

[0104]

[0105] In the formula, v -i : = v\{v i}.

[0106] By optimizing the contrast loss function, make the feature vectors of homologous texts approach each other, and the feature vectors of non-homologous texts move away from each other, maximizing the encoder's ability to capture effective information of the text and providing a good basic model for subsequent discrimination.

[0107] (3) Overall model pre-training, including: based on the good encoder parameters in step (2), use the labeled dataset to fine-tune the overall classification model. By optimizing the binary cross-entropy loss function, improve the accuracy and stability of the model output probability, where is the probability of being predicted as the positive class.

[0108] (4) Introduce the UPS concept to screen pseudo-label samples: The principle is that, based on the output result of the discriminator (the probability that the text belongs to a certain class), set a threshold. When the probability of predicting to belong to a certain class exceeds this threshold, consider attaching a pseudo-label to the sample and using it for subsequent training.

[0109] At the same time, also consider the uncertainty relationship between network calibration and individual sample data output, and add it to the loss function as another indicator for screening pseudo-label samples.

[0110] The specific loss function is as follows:

[0111]

[0112]

[0113] Indicates whether the sample is a pseudo-label belonging to class c. If so, it is 1; if not, it is 0;

[0114] Indicates the probability that the original output of the model belongs to class c;

[0115] Indicates whether the sample is labeled with a pseudo-label (for subsequent training);

[0116] τ p and τ n are the thresholds for becoming positive and negative example samples;

[0117] u(p) is used to represent the uncertainty of the model prediction result p (calculated by modeling the valuation through dropout NN);

[0118] k p and k n are the uncertainty thresholds.

[0119] By optimizing the loss function, select the most appropriate thresholds (τ p and τ n and k p and k n ), and label the unlabeled samples that meet the conditions with pseudo-labels for subsequent training.

[0120] In a further preferred embodiment of the present invention, it further includes step (5), substituting the samples with existing labels and the samples with pseudo-labels obtained from (4) into the model pre-trained in (3) for discriminative training; then repeat steps (4) and (5) until the maximum number of iterations is reached. In each iteration of the embodiment of the present invention, the result obtained from the pre-training in step (3) is substituted instead of step (4), aiming to avoid the continuous propagation of errors caused by mislabeled samples in the iterative training.

[0121] In the embodiment of the present invention, on the standard NewsGroup20 dataset, for a text classifier with TextCNN as the backbone network, after adopting the method of the embodiment of the present invention, the F1 value can be improved by 3%.

[0122] Embodiment Three

[0123] The following is the device embodiment of the present invention, which can be used to execute the method embodiment of the present invention. For the details not disclosed in the device embodiment, please refer to the method embodiment of the present invention.

[0124] Please refer to Figure 5 , in another embodiment of the present invention, a text sentiment classification system is provided, including:

[0125] A text acquisition module, configured to acquire the text to be sentiment classified;

[0126] A sentiment classification result acquisition module, configured to perform text sentiment classification on the acquired text to be sentiment classified by using a pre-trained sentiment classification model, and obtain a sentiment classification result;

[0127] Among them, the sentiment classification model includes:

[0128] A feature extractor, which is used to input text, extract features, and output a feature vector;

[0129] A linear classifier, which is used to input the feature vector output by the feature extractor, perform classification, and output a sentiment classification result.

[0130] In the embodiment of the present invention, the obtaining steps of the pre-trained sentiment classification model include:

[0131] Obtain a first training sample set; among them, the training samples in the first training sample set include labeled training samples and unlabeled training samples with pseudo-labels;

[0132] When training the model based on the training samples in the first training sample set, input the text of the selected training sample into the pre-trained feature extractor to obtain a feature vector; input the obtained feature vector into the linear classifier, and update the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample. After reaching the preset convergence condition, obtain the pre-trained sentiment classification model;

[0133] The obtaining steps of the pre-trained feature extractor include:

[0134] Obtain a second training sample set; among them, the training samples in the second training sample set include a first preset number of labeled training samples and a second preset number of unlabeled training samples;

[0135] Perform data augmentation on all training samples in the second training sample set in two different ways, and each training sample obtains two homologous augmented data; in each training batch, calculate the contrast loss of the two homologous augmented data, and update the parameters of the feature extractor based on the contrast loss, so that the feature vectors of homologous texts are close to each other and the feature vectors of non-homologous texts are far from each other. After reaching the preset training requirements, obtain the trained feature extractor;

[0136] Based on the first preset number of labeled training samples, train the trained feature extractor and the pre-obtained label classifier to obtain the trained feature extractor and label classifier;

[0137] The steps of obtaining the first training sample set specifically include:

[0138] Input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain pseudo-labels of the unlabeled training samples;

[0139] Based on all the unlabeled training samples that have obtained pseudo-labels, select the unlabeled training samples whose confidence levels meet the preset conditions; add the selected unlabeled training samples to the labeled training samples to obtain the first training sample set.

[0140] Embodiment 4

[0141] In another embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program. The computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiments of the present invention can be used for the operation of the text sentiment classification method.

[0142] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and this storage space stores the operating system of the terminal. And, one or more instructions suitable for being loaded and executed by the processor are also stored in this storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the text sentiment classification method in the above embodiments.

[0143] Embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0144] Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0145] The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0146] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, system, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0148] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one process Figure 1 one process or a plurality of processes and / or blocks Figure 1 the functions specified in one block or a plurality of blocks.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process Figure 1 one process or a plurality of processes and / or blocks Figure 1 the functions specified in one block or a plurality of blocks.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation manners of the present invention can still be modified or equivalently replaced, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A text sentiment classification method, characterized in that, Including the following steps: Obtain the text to be sentiment classified; Based on the obtained text to be sentiment classified, use a pre-trained sentiment classification model to perform text sentiment classification and obtain a sentiment classification result; Among them, the sentiment classification model includes: A feature extractor for inputting text, extracting features, and outputting a feature vector; A linear classifier for inputting the feature vector output by the feature extractor, performing classification, and outputting a sentiment classification result; Among them, the steps for obtaining the pre-trained sentiment classification model include: Obtain a first training sample set; among them, the training samples in the first training sample set include labeled training samples and unlabeled training samples with pseudo-labels; When performing model training based on the training samples in the first training sample set, input the text of the selected training sample into the pre-trained feature extractor to obtain a feature vector; input the obtained feature vector into the linear classifier, and update the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample. After reaching the preset convergence condition, obtain the pre-trained sentiment classification model; The steps for obtaining the pre-trained feature extractor include: Obtain a second training sample set; among them, the training samples in the second training sample set include a first preset number of labeled training samples and a second preset number of unlabeled training samples; Perform data augmentation on all training samples in the second training sample set in two different ways, and each training sample obtains two homologous augmented data; in each training batch, calculate the contrast loss of the two homologous augmented data, and update the parameters of the feature extractor based on the contrast loss, so that the feature vectors of homologous texts are close to each other and the feature vectors of non-homologous texts are far from each other. After reaching the preset training requirements, obtain the trained feature extractor; Based on the first preset number of labeled training samples, train the trained feature extractor and the pre-obtained label classifier to obtain a trained feature extractor and label classifier.

2. The text sentiment classification method according to claim 1, characterized in that In the step of updating the parameters of the linear classifier based on the label or pseudo-label corresponding to the selected training sample, the expression of the loss function used for updating the parameters is Wherein, is the probability predicted to be the positive class.

3. A text sentiment classification method according to claim 1, characterized in that The steps for obtaining the first training sample set specifically include: Input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain pseudo-labels of the unlabeled training samples; Based on all unlabeled training samples with obtained pseudo-labels, select unlabeled training samples whose confidence levels meet the preset conditions; add the selected unlabeled training samples to the labeled training samples to obtain the first training sample set.

4. A text sentiment classification method according to claim 3, characterized in that, The step of inputting the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain pseudo-labels of the unlabeled training samples specifically includes: Input the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain prediction results; When the prediction result corresponding to an unlabeled training sample belongs to a certain class with a probability exceeding the preset threshold, assign a pseudo-label to the unlabeled training sample.

5. A text sentiment classification method according to claim 3, characterized in that, The step of selecting unlabeled training samples with confidence levels meeting a preset condition based on all unlabeled training samples with pseudo-labels specifically includes: Based on all unlabeled training samples with pseudo-labels, select the threshold τ by optimizing the loss function p , τ n , k p , k n , and select the unlabeled training samples that meet the threshold requirements; Among them, the optimized loss function expression is where, for sample i, denotes the pseudo-label indicating whether sample i belongs to class c; denotes the probability that the original output of the model belongs to class c; is used to indicate whether the sample is assigned a pseudo-label; τ p and τ n are the thresholds for becoming positive and negative example samples; is used to represent the uncertainty of the model prediction result; k p and k n are the uncertainty thresholds; is the probability of predicting as the positive class; C represents the total number of classes.

6. A text sentiment classification system, characterized in that, Including: A text acquisition module for acquiring the text to be sentiment classified; A sentiment classification result acquisition module for performing text sentiment classification on the acquired text to be sentiment classified by using a pre-trained sentiment classification model to obtain a sentiment classification result; Among them, the acquisition steps of the pre-trained sentiment classification model include: Acquiring a first training sample set; among them, the training samples in the first training sample set include labeled training samples and unlabeled training samples with pseudo-labels; When performing model training based on the training samples in the first training sample set, inputting the text of the selected training samples into a pre-trained feature extractor to obtain feature vectors; inputting the obtained feature vectors into a linear classifier, and updating the parameters of the linear classifier based on the labels or pseudo-labels corresponding to the selected training samples, and obtaining the pre-trained sentiment classification model after reaching a preset convergence condition; The acquisition steps of the pre-trained feature extractor include: Acquiring a second training sample set; among them, the training samples in the second training sample set include a first preset number of labeled training samples and a second preset number of unlabeled training samples; Performing data augmentation on all training samples in the second training sample set in two different ways, and each training sample obtains two homologous augmented data; in each training batch, calculating the contrast loss of the two homologous augmented data, and updating the parameters of the feature extractor based on the contrast loss, so that the feature vectors of homologous texts are close to each other and the feature vectors of non-homologous texts are far from each other, and obtaining the trained feature extractor after reaching the preset training requirements; Based on the first preset number of labeled training samples, training the trained feature extractor and the pre-acquired label classifier to obtain a trained feature extractor and label classifier.

7. A text sentiment classification system according to claim 6, characterized in that, In the updating of the parameters of the linear classifier based on the labels or pseudo-labels corresponding to the selected training samples, the expression of the loss function used when updating the parameters is wherein, is the probability predicted as the positive class.

8. A text sentiment classification system according to claim 6, characterized in that, The step of acquiring the first training sample set specifically includes: Inputting the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain pseudo-labels of the unlabeled training samples; Selecting unlabeled training samples with confidence levels meeting a preset condition based on all unlabeled training samples with pseudo-labels; adding the selected unlabeled training samples to the labeled training samples to obtain the first training sample set.

9. A text sentiment classification system according to claim 8, characterized in that, The step of inputting the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain pseudo-labels of the unlabeled training samples specifically includes: Inputting the texts in the second preset number of unlabeled training samples into the trained feature extractor and label classifier respectively to obtain prediction results; When the prediction result corresponding to an unlabeled training sample belongs to a certain class with a probability exceeding a preset threshold, attaching a pseudo-label to the unlabeled training sample.

10. A text sentiment classification system according to claim 8, characterized in that, The step of selecting unlabeled training samples with confidence satisfying a preset condition based on all unlabeled training samples that have obtained pseudo-labels specifically includes: Based on all unlabeled training samples with pseudo-labels, select the threshold τ by optimizing the loss function p , τ n , k p , k n , and select the unlabeled training samples that meet the threshold requirements; Among them, the expression of the optimized loss function is where, for sample i, denotes the pseudo-label indicating whether sample i belongs to class c; denotes the probability that the original output of the model belongs to class c; is used to indicate whether the sample is assigned a pseudo-label; τ p and τ n are the thresholds for becoming positive and negative example samples; is used to represent the uncertainty of the model prediction result; k p and k n are the uncertainty thresholds; is the probability of predicting as the positive class; C represents the total number of classes.

Images