Sentiment analysis method, device, computer equipment and storage medium

Abstract

The present invention relates to the technical field of natural language processing, and in particular to a sentiment analysis method, apparatus, computer equipment and storage medium. A teacher model is trained based on a predicted sentiment probability distribution set of a source language set and a target language set, and a predicted sentiment probability distribution set of a target language unlabeled sentence set output by the trained teacher model and a preset student model is obtained as training data for the student model. The student model is trained using a knowledge distillation method, thereby improving the accuracy of the student model in performing cross-language aspect-level sentiment classification tasks, thereby achieving the extraction of aspect words based on the target language and the confirmation of the sentiment polarity of the aspect words, and improving the accuracy of sentiment analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a sentiment analysis method, apparatus, computer equipment and storage medium. Background Art

[0002] Aspect-based sentiment analysis (ABSA) involves identifying specific aspect terms in sentences and their sentiment polarity. Based on large annotated corpora, ABSA has been successfully applied to English texts, but real-world social media interactions often involve multiple languages, highlighting the need for cross-lingual sentiment analysis.

[0003] In cross-lingual aspect-level sentiment analysis, the sentiment of each aspect word is classified according to a set of predefined sentiment polarities, i.e., positive, neutral, or negative, and the task aims to train the model on a labeled dataset of the source language and then predict it on an unlabeled dataset of the target language. Since it is very expensive to obtain a large amount of annotated training data for low-resource languages, early cross-lingual sentiment analysis works relied solely on annotated data from different source languages ​​to learn sentiment classification in the target language. These models usually rely on bilingual dictionaries, pre-trained cross-lingual word embeddings, or machine translation to bridge the gap between the source and target languages.

[0004] Current technical solutions usually use multilingual pre-trained models to fine-tune the aligned translation data and align the cross-language aspect word consistency to bridge the gap between the source language and the target language. However, although there are related works in the above technical solutions to align the source language and the target language at the sentence level, in practical applications, relying solely on sentence-level alignment may introduce contextual noise, resulting in a reduced emphasis on aspect words, making it difficult to perform accurate sentiment analysis on aspect words. Summary of the invention

[0005] Based on this, the present invention provides a sentiment analysis method, device, computer equipment and storage medium. Based on the predicted sentiment probability distribution set of the source language set and the target language set, the teacher model is trained to obtain the predicted sentiment probability distribution set of the target language unlabeled sentence set output by the trained teacher model and the preset student model as the training data of the student model. The student model is trained using the knowledge distillation method, which improves the accuracy of the student model in performing cross-language aspect-level sentiment classification tasks, thereby realizing the extraction of aspect words based on the target language and the confirmation of the sentiment polarity of aspect words, and improving the accuracy of sentiment analysis. The technical method is as follows:

[0006] In a first aspect, an embodiment of the present application provides a sentiment analysis method, comprising the following steps:

[0007] Obtaining a source language set, a target language set, and a target language unlabeled sentence set, wherein the source language set includes a source language sentence set and a source language sentence set replaced by aspect words; the target language set includes a target language sentence set and a target language sentence set replaced by aspect words;

[0008] Inputting the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing to obtain a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set;

[0009] Inputting the sentence encoding representation sets of the source language set and the target language set into a preset teacher model to perform emotion probability distribution calculation to obtain the predicted emotion probability distribution sets of the source language set and the target language set; training the preset teacher model according to the predicted emotion probability distribution sets of the source language set and the target language set to obtain a target teacher model;

[0010] Inputting the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, obtaining the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; training the student model according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model, and obtaining the target student model;

[0011] A sentence to be analyzed is obtained, the sentence to be analyzed is input into the encoding model for encoding processing, a sentence encoding representation of the sentence to be analyzed is obtained, the sentence encoding representation of the sentence to be analyzed is input into the target student model, and an aspect word sentiment analysis result of the sentence to be analyzed is obtained.

[0012] In a second aspect, an embodiment of the present application provides a sentiment analysis device, comprising:

[0013] A data acquisition module, used to obtain a source language set, a target language set and a target language unlabeled sentence set, wherein the source language set includes a source language sentence set and a source language sentence set replaced by aspect words; the target language set includes a target language sentence set and a target language sentence set replaced by aspect words;

[0014] A sentence encoding module, used for inputting the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing, and obtaining a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set;

[0015] A teacher model training module is used to input the sentence encoding representation sets of the source language set and the target language set into a preset teacher model to perform emotion probability distribution calculation, and obtain the predicted emotion probability distribution sets of the source language set and the target language set; according to the predicted emotion probability distribution sets of the source language set and the target language set, the preset teacher model is trained to obtain a target teacher model;

[0016] A student model training module is used to input the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, and obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model, the student model is trained to obtain a target student model;

[0017] The sentiment analysis module is used to obtain the sentence to be analyzed, input the sentence to be analyzed into the encoding model for encoding processing, obtain the sentence encoding representation of the sentence to be analyzed, input the sentence encoding representation of the sentence to be analyzed into the target student model, and obtain the aspect word sentiment analysis result of the sentence to be analyzed.

[0018] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the sentiment analysis method described in the first aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sentiment analysis method described in the first aspect are implemented.

[0020] In this embodiment, a sentiment analysis method, apparatus, computer device and storage medium are provided. A teacher model is trained based on a predicted sentiment probability distribution set of a source language set and a target language set, and a predicted sentiment probability distribution set of a target language unlabeled sentence set output by the trained teacher model and a preset student model is obtained as training data for the student model. The student model is trained using a knowledge distillation method, thereby improving the accuracy of the student model in performing cross-language aspect-level sentiment classification tasks, thereby achieving the extraction of aspect words based on the target language and the confirmation of the sentiment polarity of the aspect words, thereby improving the accuracy of sentiment analysis.

[0021] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a sentiment analysis method provided by one embodiment of the present application;

[0023] Figure 2 A schematic diagram of the process of S6 in the sentiment analysis method provided in another embodiment of the present application;

[0024] Figure 3 A schematic diagram of the process of S3 in the sentiment analysis method provided in one embodiment of the present application;

[0025] Figure 4 A schematic diagram of the process of S4 in the sentiment analysis method provided in one embodiment of the present application;

[0026] Figure 5 A schematic diagram of the process of S4 in the sentiment analysis method provided in one embodiment of the present application;

[0027] Figure 6 A schematic diagram of the process of S5 in the sentiment analysis method provided in one embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of a sentiment analysis device provided in one embodiment of the present application;

[0029] Figure 8 A schematic diagram of the structure of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] The terms used in this application are for the purpose of describing the target embodiments only and are not intended to limit the application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0032] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0033] See also Figure 1 , Figure 1 The flowchart of the sentiment analysis method provided in the first embodiment of the present application includes the following steps:

[0034] S1: Get the source language set, the target language set, and the target language unlabeled sentence set.

[0035] The executor of the sentiment analysis method of the present application is an analysis device of the sentiment analysis method (hereinafter referred to as the analysis device).

[0036] In an optional embodiment, the analysis device may be a computer device, a server, or a server cluster formed by a combination of multiple computer devices.

[0037] The analysis device obtains a training data set and a target language unlabeled sentence set, wherein the training data set includes a source language set and a target language set.

[0038] Specifically, the source language set includes a source language sentence set and an aspect word replacement source language sentence set; the target language set includes a target language sentence set and an aspect word replacement target language sentence set. The source language sentence set includes a plurality of source language sentences; the target language sentence set includes target language sentences; the aspect word replacement source language sentence set includes a plurality of source language sentences in which aspect words are replaced by aspect words in corresponding target language sentences; the aspect word replacement target language sentence set includes a plurality of target language sentences in which aspect words are replaced by aspect words in corresponding source language sentences.

[0039] For the source language sentence set and the target language sentence set, the analysis device obtains a plurality of one-to-one matched source language sentences and target language sentences based on a given input sentence and according to a preset parallel corpus, wherein the target language sentences are obtained by text translation conversion obtained by converting the source language sentences, and a plurality of words in the target language sentences are provided with corresponding tags, and the tags are serial numbers of the plurality of words in the sentences.

[0040] For the set of source language sentences replaced with aspect words and the set of target language sentences replaced with aspect words, the analysis device exchanges the aspect words in the one-to-one paired source language sentences and target language sentences to obtain source language sentences in which several aspect words are replaced with aspect words in corresponding target language sentences and target language sentences in which several aspect words are replaced with aspect words in corresponding source language sentences.

[0041] S2: Inputting the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing to obtain a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set.

[0042] The encoding model uses a multilingual pre-trained model mBERT (Bidirectional Encoder Representation from Transformers) or XLM (Cross-lingual Language Model) to convert word vectors of several words in a sentence into corresponding embedding vectors.

[0043] In this embodiment, the analysis device inputs the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing to obtain a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set, wherein the sentence encoding representation set includes sentence encoding representations of several sentences, and the sentence encoding representations include embedding vectors of several words.

[0044] Specifically, the analysis device inputs the sentences in the source language set, the target language set, and the target language unlabeled sentence set into the multilingual pre-trained model, maps each word in the sentence to a low-dimensional vector space, and obtains hidden layer vectors of several words in the sentence output by the multilingual pre-trained model by querying the pre-trained matrix as the embedding vector, and performs encoding processing to obtain the sentence encoding representation of the sentence, and obtains the sentence encoding representation set of the source language set, the target language set, and the target language unlabeled sentence set.

[0045] In an optional embodiment, the step of: training the encoding model is further included; see Figure 2 , Figure 2 The flowchart of S6 in the sentiment analysis method provided in another embodiment of the present application includes steps S61-S62, which are as follows:

[0046] S61: performing binary classification probability distribution calculation on the sentence encoding representation sets of the source language set and the target language set to obtain the binary classification probability distribution sets of the source language set and the target language set.

[0047] In this embodiment, the analysis device uses a language identifier, which is a binary classifier with a sigmoid layer on top, so that the language identification score is always between 0 and 1, and is intended to determine the probability that the input sentence is from the source language or the target language based on the hidden features captured by the feature extractor, namely the embedding vector.

[0048] The analysis device performs binary classification probability distribution calculation on the sentence encoding representation set of the source language set and the target language set according to a preset binary classification probability distribution calculation algorithm to obtain the binary classification probability distribution set of the source language set and the target language set, wherein the binary classification probability distribution set includes the binary classification probability distribution of a plurality of sentences, the binary classification probability distribution data includes the binary classification probability distribution vectors of a plurality of words, and the binary classification probability distribution calculation algorithm is:

[0049]

[0050] In the formula, For the sentence i The binary probability distribution word vector of the word, is the activation function, is the regularization function, W is the weight parameter, For the i The embedding vectors of words, b is the bias parameter.

[0051] S62: Calculate the bulldozer distance based on the binary probability distribution set of the source language set and the target language set, and train the encoding model using the duality approximate minimization adversarial learning method based on the calculated bulldozer distance to obtain a target encoding model.

[0052] In this embodiment, the analysis device performs a Wasserstein distance calculation based on the binary probability distribution set of the source language set and the target language set, and trains the encoding model using a duality approximate minimization adversarial learning method based on the calculated Wasserstein distance to obtain a target encoding model, as follows:

[0053]

[0054] In the formula, is the bulldozer distance, To obtain the maximum value function, are the parameters of the language discriminator, is the bulldozer distance calculation function, is the processing function of the language identifier, is the first sentence in the source language seti The embedding vectors of words, is the first sentence in the target language set i The embedding vectors of the words.

[0055] In addition to the source language sentence set and the target language sentence set, the source language sentence set after aspect word replacement and the target language sentence set after aspect word replacement are also used as training data to stimulate the language discriminator to recognize the target language sentences after aspect word replacement as the source language sentences and the source language sentences after aspect word replacement as the target language sentences, induce local perturbations of aspect word changes in sentences to encourage the language discriminator to recognize the common features of the language, and stimulate the feature extractor to capture the invariant features of the language through back propagation, and train the encoding model through bulldozer distance calculation and duality approximate minimization adversarial learning method to improve the accuracy and robustness of sentence encoding by the encoding model and improve sentence-level alignment.

[0056] S3: Input the sentence encoding representation sets of the source language set and the target language set into a preset teacher model to perform emotion probability distribution calculation to obtain the predicted emotion probability distribution sets of the source language set and the target language set; train the preset teacher model according to the predicted emotion probability distribution sets of the source language set and the target language set to obtain a target teacher model.

[0057] The teacher model includes several sub-teacher models, each of which includes a sentiment classifier. The sentiment classifier can calculate the sentiment probability distribution based on the sentence encoding representation, and the consistency training module is used for model training.

[0058] In this embodiment, the analysis device inputs the sentence encoding representation sets of the source language set and the target language set into several sub-teacher models in the preset teacher model, performs emotion probability distribution calculation based on the emotion classifiers of the several sub-teacher models, obtains the predicted emotion probability distribution subsets of the source language set and the target language set output by the several sub-teacher models, and respectively combines the predicted emotion probability distribution subsets of the source language set and the target language set to obtain the predicted emotion probability distribution set of the source language set and the target language set, wherein the predicted emotion probability distribution set includes predicted emotion probability distribution sentence vectors of several sentences, and the predicted emotion probability distribution sentence vectors of the sentences include predicted emotion probability distribution word vectors of several words.

[0059] The analysis device trains a preset teacher model according to the predicted emotion probability distribution set of the source language set and the target language set to obtain a target teacher model.

[0060] See also Figure 3 , Figure 3The flowchart of S3 in the sentiment analysis method provided in one embodiment of the present application includes steps S31 to S33, which are as follows:

[0061] S31: Obtain the true emotion probability distribution set of the source language set and the target language set; perform cross entropy calculation based on the predicted emotion probability distribution set and the true emotion probability distribution set of the source language set and the target language set to obtain a cross entropy loss value.

[0062] In this embodiment, the analysis device obtains the real emotion probability distribution set of the source language set and the target language set, wherein the real emotion probability distribution set includes the real emotion probability distribution sentence vectors of several sentences, and the real emotion probability distribution sentence vectors of the sentences include the real emotion probability distribution word vectors of several words.

[0063] The analysis device performs cross entropy calculation according to the predicted emotion probability distribution set of the source language set and the target language set, the true emotion probability distribution set and a preset cross entropy calculation algorithm to obtain a cross entropy loss value, wherein the cross entropy calculation algorithm is:

[0064]

[0065] In the formula, is the cross entropy loss value, is the training data set, L is the sentence length, For the sentence i The true probability distribution of words is the word vector, For the sentence i The predicted probability distribution word vector of each word, For the i The embedding vectors of words, Compute the function for the first probability distribution.

[0066] S32: performing a divergence loss calculation based on the predicted emotion probability distribution sets of the source language set and the target language set to obtain a divergence loss value.

[0067] In this embodiment, the analysis device performs divergence loss calculation according to the predicted emotion probability distribution set of the source language set and the target language set according to a preset divergence loss calculation algorithm to obtain a divergence loss value, wherein the divergence loss calculation algorithm is:

[0068]

[0069] In the formula, is the divergence loss value, m is the total number of aspect word pairs, is a sentence in the source language set, is a sentence in the target language set, for Middle i The word vector of the aspect word, for Middle i The word vector of the aspect word, is the divergence function, Compute the function for the second probability distribution, for Middle i The predicted probability distribution word vector of each aspect word, for Middle i The predicted probability distribution word vector of each aspect word.

[0070] We enhance the language discriminator’s ability to capture invariant features by introducing aspect word perturbations and develop more robust representations across languages, and use bidirectional KL Divergence is used to calculate the divergence between the probability distributions of aspect words at the span level and minimize their consistency loss for model training. This encourages the model to capture the feature representations of aspect words with the same sentiment in different languages ​​and improve the accuracy of the model’s sentiment analysis.

[0071] S33: According to the cross entropy loss value and the divergence loss value, the teacher model is trained to obtain a target teacher model.

[0072] In this embodiment, the analysis device obtains a total loss value according to the cross entropy loss value, the divergence loss value and the preset total loss calculation algorithm, trains the teacher model according to the total loss value, obtains the target teacher model, and adopts a multi-objective optimization method to perform consistency training on several sub-teacher models in the teacher model, so that the teacher model can better learn more fine-grained entity-level semantic features and enhance the adaptability of the model to the target language, wherein the total loss value is:

[0073]

[0074] In the formula, is the total loss value, U is the sentence in the training dataset, is a hyperparameter.

[0075] S4: Input the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, and obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; train the student model according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model to obtain the target student model.

[0076] In this embodiment, the analysis device inputs the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, and obtains the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model according to the emotion classifiers in the target teacher model and the preset student model, wherein the predicted emotion probability distribution set of the target language unlabeled sentence set includes predicted emotion probability distribution sentence vectors of several target language unlabeled sentences, and the predicted emotion probability distribution sentence vectors of the target language unlabeled sentences include predicted emotion probability word vectors of several unlabeled words.

[0077] The analysis device trains the student model according to the target teacher model and the predicted emotion probability distribution set of the target language unlabeled sentence set output by the student model to obtain a target student model.

[0078] See also Figure 4 , Figure 4 The flowchart of S4 in the sentiment analysis method provided in one embodiment of the present application includes steps S41 to S42, which are as follows:

[0079] S41: Input the sentence encoding representation sets of the target language unlabeled sentence set into several of the sub-teacher models respectively, perform sentiment probability distribution calculation on the sentence encoding representations of several unlabeled sentences in the sentence encoding representation set, and obtain the predicted sentiment probability distribution sentence vectors of several unlabeled sentences output by several of the sub-teacher models.

[0080] In this embodiment, the analysis device inputs the sentence encoding representation set of the unlabeled sentence set of the target language into several of the sub-teacher models respectively, performs sentiment probability distribution calculation on the sentence encoding representations of several unlabeled sentences in the sentence encoding representation set, and obtains the predicted sentiment probability distribution sentence vectors of several unlabeled sentences output by several of the sub-teacher models.

[0081] S42: Combine the predicted emotion probability distribution sentence vectors of several unlabeled sentences output by the same sub-teacher model to obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by several sub-teacher models; perform multi-teacher distillation on the predicted emotion probability distribution set of the target language unlabeled sentence set output by several sub-teacher models to obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model.

[0082] In this embodiment, the analysis device combines the predicted emotion probability distribution sentence vectors of several unlabeled sentences output by the same sub-teacher model to obtain a predicted emotion probability distribution set of the target language unlabeled sentence sets output by several sub-teacher models.

[0083] The analysis device performs multi-teacher distillation on the predicted emotion probability distribution set of the target language unlabeled sentence set output by the plurality of sub-teacher models according to a preset multi-teacher distillation algorithm, obtains the predicted emotion probability distribution sentence vectors of the plurality of sentences in the target language unlabeled sentence set, and constructs the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model, wherein the multi-teacher distillation algorithm is:

[0084]

[0085] In the formula, The predicted sentiment probability distribution set of the target language unlabeled sentence set output by the target teacher model, K is the number of sub-teacher models, For the k The weight parameters of the sub-teacher model, For the k The predicted sentiment probability distribution set of the target language unlabeled sentence set output by the sub-teacher model.

[0086] See also Figure 5 , Figure 5 The flowchart of S4 in the sentiment analysis method provided in one embodiment of the present application includes step S43, which is as follows:

[0087] S43: According to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model, a mean square error loss calculation method is adopted to obtain a mean square error loss value, and according to the mean square error loss value, the student model is trained to obtain a target student model.

[0088] In this embodiment, the analysis device adopts a mean square error loss calculation method, obtains a mean square error loss value according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model and a preset mean square error loss calculation algorithm, and performs sentence-level adversarial training and gradient update on the student model according to the mean square error loss value to obtain a target student model, wherein the mean square error loss calculation algorithm is:

[0089]

[0090] In the formula, is the mean square error loss value, is a set of unlabeled sentences in the target language, N is a sentence in the target language unlabeled sentence set, The first sentence in the predicted sentiment probability distribution set of the target language unlabeled sentence set output by the target teacher model i The predicted sentiment probability distribution word vector of each word, The predicted sentiment probability distribution of the target language unlabeled sentence set output by the student-teacher model is the sentence i The predicted sentiment probability distribution word vector of each word, is the mean square error function.

[0091] Based on the predicted emotion probability distribution set of the source language set and the target language set, the teacher model is trained to obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the trained teacher model and the preset student model. The obtained set is used as the training data of the student model. The student model is trained using the knowledge distillation method, which improves the accuracy of the student model in performing cross-language aspect-level sentiment classification tasks.

[0092] S5: Obtain a sentence to be analyzed, input the sentence to be analyzed into the encoding model for encoding processing, obtain a sentence encoding representation of the sentence to be analyzed, input the sentence encoding representation of the sentence to be analyzed into the target student model, and obtain an aspect word sentiment analysis result of the sentence to be analyzed.

[0093] In this embodiment, the analysis device obtains a sentence to be analyzed, inputs the sentence to be analyzed into the encoding model for encoding processing, and obtains a sentence encoding representation of the sentence to be analyzed. For a specific embodiment, please refer to step S2, which will not be repeated here.

[0094] The analysis device inputs the sentence encoding representation of the sentence to be analyzed into the target student model to obtain the aspect word sentiment analysis result of the sentence to be analyzed.

[0095] See also Figure 6 , Figure 6 The flowchart of S5 in the sentiment analysis method provided in one embodiment of the present application includes steps S51 to S52, which are as follows:

[0096] S51: Calculate the emotion probability distribution of the sentence encoding representation of the sentence to be analyzed to obtain a predicted emotion probability distribution sentence vector of the sentence to be analyzed.

[0097] In this embodiment, the analysis device performs sentiment probability distribution calculation on the sentence encoding representation of the sentence to be analyzed according to a preset fully connected layer, and obtains a predicted sentiment probability distribution sentence vector of the sentence to be analyzed.

[0098] S52: According to the predicted emotion probability vectors corresponding to the emotion polarities of several dimensions in the predicted emotion probability distribution word vectors of several aspect words in the predicted emotion probability distribution sentence vector of the sentence to be analyzed, the emotion polarity corresponding to the dimension with the largest probability is obtained as the aspect word emotion analysis result of the several aspect words, thereby obtaining the aspect word emotion analysis result of the sentence to be analyzed.

[0099] In this embodiment, the analysis device obtains the sentiment polarity corresponding to the dimension with the largest probability based on the predicted sentiment probability vector corresponding to the sentiment polarity of several dimensions in the predicted sentiment probability distribution word vector of several aspect words in the predicted sentiment probability distribution sentence vector of the sentence to be analyzed, and uses it as the aspect word sentiment analysis result of the several aspect words to obtain the aspect word sentiment analysis result of the sentence to be analyzed, thereby extracting the aspect words based on the target language of the sentence to be analyzed and the sentiment polarity of the aspect words to achieve sentiment classification of the aspect words.

[0100] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a sentiment analysis device provided in one embodiment of the present application. The device can implement all or part of the sentiment analysis method through software, hardware, or a combination of both. The device 7 includes:

[0101] The data acquisition module 71 is used to obtain a source language set, a target language set and a target language unlabeled sentence set, wherein the source language set includes a source language sentence set and a source language sentence set replaced by aspect words; the target language set includes a target language sentence set and a target language sentence set replaced by aspect words;

[0102] A sentence encoding module 72 is used to input the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing, and obtain a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set;

[0103] The teacher model training module 73 is used to input the sentence encoding representation sets of the source language set and the target language set into the preset teacher model to perform emotion probability distribution calculation, and obtain the predicted emotion probability distribution sets of the source language set and the target language set; according to the predicted emotion probability distribution sets of the source language set and the target language set, the preset teacher model is trained to obtain the target teacher model;

[0104] The student model training module 74 is used to input the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, and obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; train the student model according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model to obtain a target student model;

[0105] The sentiment analysis module 75 is used to obtain the sentence to be analyzed, input the sentence to be analyzed into the encoding model for encoding processing, obtain the sentence encoding representation of the sentence to be analyzed, input the sentence encoding representation of the sentence to be analyzed into the target student model, and obtain the aspect word sentiment analysis result of the sentence to be analyzed.

[0106] In an embodiment of the present application, a source language set, a target language set and a target language unlabeled sentence set are obtained through a data acquisition module, wherein the source language set includes a source language sentence set and a source language sentence set replaced by aspect words; the target language set includes a target language sentence set and a target language sentence set replaced by aspect words; through a sentence encoding module, the source language set, the target language set and the target language unlabeled sentence set are input into a preset encoding model for encoding processing to obtain a sentence encoding representation set of the source language set, the target language set and the target language unlabeled sentence set; through a teacher model training module, the sentence encoding representation set of the source language set and the target language set is input into a preset teacher model for emotion probability distribution calculation to obtain a predicted emotion probability distribution set of the source language set and the target language set; according to the predicted emotion probability distribution of the source language set and the target language set, the predicted emotion probability distribution set of the target language set and the target language set is obtained. Probability distribution set, train the preset teacher model to obtain the target teacher model; through the student model training module, input the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, and obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model, train the student model to obtain the target student model; through the sentiment analysis module, obtain the sentence to be analyzed, input the sentence to be analyzed into the encoding model for encoding processing, obtain the sentence encoding representation of the sentence to be analyzed, input the sentence encoding representation of the sentence to be analyzed into the target student model, and obtain the aspect word sentiment analysis result of the sentence to be analyzed. Based on the predicted sentiment probability distribution sets of the source language set and the target language set, the teacher model is trained to obtain the predicted sentiment probability distribution sets of the target language unlabeled sentence sets output by the trained teacher model and the preset student model. The trained teacher model is used as the training data of the student model. The knowledge distillation method is used to train the student model, which improves the accuracy of the student model in performing cross-language aspect-level sentiment classification tasks, thereby realizing the extraction of aspect words based on the target language and the confirmation of the sentiment polarity of aspect words, thereby improving the accuracy of sentiment analysis.

[0107] Please refer to Figure 8 , Figure 8 The computer device 8 is a schematic diagram of a structure of a computer device provided in an embodiment of the present application. The computer device 8 includes: a processor 81, a memory 82, and a computer program 83 stored in the memory 82 and executable on the processor 81; the computer device may store multiple instructions, which are suitable for being loaded and executed by the processor 81. Figures 1 to 6 The method steps of the embodiment shown in the figure can be found in the specific implementation process. Figures 1 to 6 The specific description of the illustrated embodiment will not be repeated here.

[0108] Among them, the processor 81 may include one or more processing cores. The processor 81 uses various interfaces and lines to connect various parts in the server, and executes various functions and processes data of the emotion analysis device 7 by running or executing instructions, programs, code sets or instruction sets stored in the memory 82, and calling the data in the memory 82. Optionally, the processor 81 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programble Logic Array, PLA). The processor 81 can integrate one or a combination of a central processing unit 81 (Central Processing Unit, CPU), an image processor 81 (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display screen; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 81, and it can be implemented by a single chip.

[0109] The memory 82 may include a random access memory 82 (RAM) or a read-only memory 82 (Read-Only Memory). Optionally, the memory 82 includes a non-transitory computer-readable storage medium. The memory 82 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 82 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 82 may also be optionally at least one storage device located away from the aforementioned processor 81.

[0110] The present application also provides a storage medium, which can store multiple instructions. The instructions are suitable for the processor to load and execute the method steps of the first to third embodiments shown. The specific execution process can be seen in the Figures 1 to 6 The specific description of the illustrated embodiment will not be repeated here.

[0111] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0112] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0113] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the target application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each target application, but such implementation should not be considered to be beyond the scope of the present invention.

[0114] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0117] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0118] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications to the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims and equivalent technologies of the present invention, the present invention is also intended to include these changes and modifications.

Claims

1. A sentiment analysis method, characterized in that: The following steps are involved: Obtaining a source language set, a target language set, and a target language unlabeled sentence set, wherein the source language set includes a source language sentence set and a source language sentence set replaced by aspect words; the target language set includes a target language sentence set and a target language sentence set replaced by aspect words; Inputting the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing to obtain a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set; Inputting the sentence encoding representation sets of the source language set and the target language set into a preset teacher model to perform sentiment probability distribution calculation, and obtaining the predicted sentiment probability distribution sets of the source language set and the target language set, wherein the predicted sentiment probability distribution sets include predicted sentiment probability distribution sentence vectors of a plurality of sentences, and the predicted sentiment probability distribution sentence vectors of the sentences include predicted sentiment probability distribution word vectors of a plurality of words; Obtaining a true emotion probability distribution set of the source language set and the target language set; performing cross entropy calculation based on the predicted emotion probability distribution set and the true emotion probability distribution set of the source language set and the target language set to obtain a cross entropy loss value; Respectively extracting aspect words of corresponding sentences in the source language set and the target language set to construct a source language aspect word set and a target language aspect word set; performing divergence loss calculation based on the source language aspect word set, the target language aspect word set, and the predicted emotion probability distribution set of the source language set and the target language set to obtain a divergence loss value; Obtaining a total loss value according to the cross entropy loss value and the divergence loss value, and training a preset teacher model according to the total loss value to obtain a target teacher model; Inputting the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, obtaining the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; training the student model according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model, and obtaining the target student model; Obtaining a sentence to be analyzed, inputting the sentence to be analyzed into the encoding model for encoding processing, obtaining a sentence encoding representation of the sentence to be analyzed, inputting the sentence encoding representation of the sentence to be analyzed into the target student model, and obtaining an aspect word sentiment analysis result of the sentence to be analyzed; Training the encoding model includes: using a language identifier, and performing binary classification probability distribution calculation on the sentence encoding representation sets of the source language set and the target language set according to a preset binary classification probability distribution calculation algorithm to obtain a binary classification probability distribution set of the source language set and the target language set, wherein the binary classification probability distribution set includes binary classification probability distributions of a plurality of sentences, and the binary classification probability distribution includes binary classification probability distribution vectors of a plurality of words; The bulldozer distance is calculated according to the binary probability distribution set of the source language set and the target language set. According to the calculated bulldozer distance, the encoding model is trained by adopting the duality approximate minimization adversarial learning method to obtain the target encoding model.

2. The sentiment analysis method according to claim 1, characterized in that: The target teacher model includes a plurality of sub-teacher models; The sentence encoding representation set of the target language unlabeled sentence set is input into the target teacher model and the preset student model respectively, and the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model is obtained, including the steps of: Inputting the sentence encoding representation sets of the target language unlabeled sentence set into the plurality of sub-teacher models respectively, performing sentiment probability distribution calculation on the sentence encoding representations of the plurality of unlabeled sentences in the sentence encoding representation set, and obtaining the predicted sentiment probability distribution sentence vectors of the plurality of unlabeled sentences output by the plurality of sub-teacher models; The predicted emotion probability distribution sentence vectors of several unlabeled sentences output by the same sub-teacher model are combined to obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by several sub-teacher models; multi-teacher distillation is performed on the predicted emotion probability distribution set of the target language unlabeled sentence set output by several sub-teacher models to obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model.

3. The sentiment analysis method according to claim 2, characterized in that: The method of training the student model according to the target teacher model and the predicted emotion probability distribution set of the target language unlabeled sentence set output by the student model to obtain the target student model comprises the steps of: According to the target teacher model and the predicted emotion probability distribution set of the target language unlabeled sentence set output by the student model, a mean square error loss calculation method is adopted to obtain a mean square error loss value, and according to the mean square error loss value, the student model is trained to obtain a target student model.

4. The sentiment analysis method according to claim 3, characterized in that: The step of inputting the sentence encoding representation of the sentence to be analyzed into the target student model to obtain the aspect word sentiment analysis result of the sentence to be analyzed comprises the following steps: Performing sentiment probability distribution calculation on the sentence encoding representation of the sentence to be analyzed to obtain a predicted sentiment probability distribution sentence vector of the sentence to be analyzed; According to the predicted emotion probability vectors corresponding to the emotion polarities of several dimensions in the predicted emotion probability distribution word vectors of several aspect words in the predicted emotion probability distribution sentence vector of the sentence to be analyzed, the emotion polarity corresponding to the dimension with the largest probability is obtained as the aspect word emotion analysis result of the several aspect words, and the aspect word emotion analysis result of the sentence to be analyzed is obtained.

5. A device applied to the sentiment analysis method according to any one of claims 1 to 4, characterized in that: include: A data acquisition module, used to obtain a source language set, a target language set and a target language unlabeled sentence set, wherein the source language set includes a source language sentence set and a source language sentence set replaced by aspect words; the target language set includes a target language sentence set and a target language sentence set replaced by aspect words; A sentence encoding module, used for inputting the source language set, the target language set and the target language unmarked sentence set into a preset encoding model for encoding processing, and obtaining a sentence encoding representation set of the source language set, the target language set and the target language unmarked sentence set; A teacher model training module is used to input the sentence encoding representation sets of the source language set and the target language set into a preset teacher model to perform emotion probability distribution calculation, and obtain the predicted emotion probability distribution sets of the source language set and the target language set; according to the predicted emotion probability distribution sets of the source language set and the target language set, the preset teacher model is trained to obtain a target teacher model; A student model training module is used to input the sentence encoding representation set of the target language unlabeled sentence set into the target teacher model and the preset student model respectively, and obtain the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model; according to the predicted emotion probability distribution set of the target language unlabeled sentence set output by the target teacher model and the student model, the student model is trained to obtain a target student model; The sentiment analysis module is used to obtain the sentence to be analyzed, input the sentence to be analyzed into the encoding model for encoding processing, obtain the sentence encoding representation of the sentence to be analyzed, input the sentence encoding representation of the sentence to be analyzed into the target student model, and obtain the aspect word sentiment analysis result of the sentence to be analyzed.

6. A computer device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the sentiment analysis method according to any one of claims 1 to 4 when executing the computer program.

7. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sentiment analysis method according to any one of claims 1 to 4 are implemented.

Images