Aspect-level sentiment analysis method and device

Abstract

One or more embodiments of the present specification provide an aspect-level sentiment classification method and device, including: encoding comment sentences to generate sentence pair vectors and comment vectors; encoding predetermined aspect categories to generate aspect category vectors; inputting the comment vectors into a pre-trained comment feature extraction model to obtain comment feature vectors; generating dynamic aspect vectors based on the comment feature vectors and the aspect category vectors; generating comment vectors with semantic relationships based on the target entity vector, context vector and dynamic aspect vectors; inputting the sentence pair vectors and the comment vectors with semantic relationships into a sentiment classifier to obtain predicted sentiment polarity. The method of this embodiment can improve the accuracy of sentiment prediction.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of natural language processing technology, and in particular, to an aspect-level sentiment analysis method and device. Background Art

[0002] With the development of artificial intelligence technology, the emotions expressed by the speaker can be mined from texts containing sentiment words to complete sentiment analysis tasks, which has a wide range of application scenarios, such as product review analysis, film review analysis, hot event opinion analysis, etc. Aspect-level sentiment analysis tasks can mine fine-grained sentiment expressions and achieve more accurate and complete sentiment analysis. However, current aspect-level sentiment analysis methods have problems such as insufficient dynamic aspect representation and incomplete context feature learning. Summary of the invention

[0003] In view of this, an object of one or more embodiments of the present specification is to propose an aspect-level sentiment analysis method and device.

[0004] Based on the above purpose, one or more embodiments of this specification provide an aspect-level sentiment classification method, including:

[0005] Encoding the comment sentence to generate a sentence pair vector and a comment vector; wherein the vector corresponding to the comment sentence in the sentence pair vector includes a target entity vector and a context vector;

[0006] Encoding the predetermined aspect categories to generate aspect category vectors;

[0007] Inputting the comment vector into a pre-trained comment feature extraction model to obtain a comment feature vector;

[0008] generating a dynamic aspect vector according to the comment feature vector and the aspect category vector;

[0009] Generate a comment vector with semantic relationship according to the target entity vector, the context vector and the dynamic aspect vector;

[0010] The sentence pair vector and the comment vector having a semantic relationship are input into a sentiment classifier to obtain a predicted sentiment polarity.

[0011] Optionally, before encoding the comment statement, the step further includes:

[0012] For each word in the review sentence, retrieving a predetermined number of related concept words;

[0013] Constructing explanation sentences according to the related concept words;

[0014] An expanded comment sentence is generated based on the comment sentence and the explanation sentence.

[0015] Optionally, encoding the comment sentences to generate sentence pair vectors and comment vectors includes:

[0016] Using a preset encoding model to encode the expanded comment sentence to obtain the comment vector;

[0017] constructing at least one auxiliary sentence according to the target entity and aspect category in the comment sentence;

[0018] splicing the comment sentence and at least one auxiliary sentence into a sentence pair;

[0019] The encoding model is used to encode the sentence pair to obtain the sentence pair vector.

[0020] Optionally, generating a dynamic aspect vector according to the comment feature vector and the aspect category vector includes:

[0021] The comment feature vector and the aspect category vector are weightedly summed according to a preset dynamic proportional coefficient to obtain the dynamic aspect vector.

[0022] Optionally, generating a comment vector having a semantic relationship according to the target entity vector, the context vector and the dynamic aspect vector includes:

[0023] Generate a first comment vector in which the target entity and the context are mutually concerned according to the target entity vector and the context vector;

[0024] The comment vector having the semantic relationship is generated according to the first comment vector and the dynamic aspect vector.

[0025] Optionally, generating a first comment vector in which the target entity and the context pay attention to each other according to the target entity vector and the context vector includes:

[0026] Determine, according to the target entity vector and the context vector, a weighted target entity vector of the context-focused target entity and a weighted context vector of the target entity-focused context;

[0027] The weighted target entity vector and the weighted context vector are concatenated into the first comment vector.

[0028] Optionally, determining a weighted target entity vector of a context-focused target entity according to the target entity vector and the context vector includes:

[0029] Calculate a context average vector based on all word vectors in the context vector;

[0030] Calculating the first attention of the context to the target entity according to the context average vector and the target entity vector;

[0031] The weighted target entity vector is determined according to the target entity vector and the first attention.

[0032] Optionally, determining a weighted context vector of a target entity focus context according to the target entity vector and the context vector includes:

[0033] Calculate the target entity average vector based on all word vectors in the target entity vector;

[0034] Calculating a second attention of the target entity to the context according to the target entity average vector and the context vector;

[0035] The weighted context vector is determined based on the context vector and the second attention.

[0036] Optionally, generating the comment vector having the semantic relationship according to the first comment vector and the dynamic aspect vector includes:

[0037] Calculating aspect-level attention according to the first comment vector and the dynamic aspect vector;

[0038] The comment vector having a semantic relationship is determined according to the aspect-level attention and the first comment vector.

[0039] The embodiment of this specification also provides an aspect-level sentiment classification device, including:

[0040] A first encoding module is used to encode the comment sentence to generate a sentence pair vector and a comment vector; wherein the vector corresponding to the comment sentence in the sentence pair vector includes a target entity vector and a context vector;

[0041] A second encoding module is used to encode the predetermined aspect category to generate an aspect category vector;

[0042] A feature extraction module, used for inputting the comment vector into a pre-trained comment feature extraction model to obtain a comment feature vector;

[0043] A dynamic aspect module, used for generating a dynamic aspect vector according to the comment feature vector and the aspect category vector;

[0044] A semantic analysis module is used to generate a comment vector with semantic relations based on the target entity vector, the context vector and the dynamic aspect vector;

[0045] The prediction module is used to input the sentence pair vector and the comment vector with semantic relationship into a sentiment classifier to obtain predicted sentiment polarity.

[0046] From the above description, it can be seen that the aspect-level sentiment analysis method and device provided in one or more embodiments of this specification encodes the comment sentences to generate sentence pair vectors and comment vectors; encodes the predetermined aspect categories to generate aspect category vectors; inputs the comment vectors into the pre-trained comment feature extraction model to obtain the comment feature vectors; generates dynamic aspect vectors based on the comment feature vectors and the aspect category vectors; generates comment vectors with semantic relationships based on the target entity vector, the context vector and the dynamic aspect vectors; inputs the sentence pair vectors and the comment vectors with semantic relationships into the sentiment classifier to obtain predicted sentiment polarity. The method of this embodiment can improve the accuracy of sentiment prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one or more embodiments of the present specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a method flow of one or more embodiments of this specification;

[0049] Figure 2 A schematic diagram of generating dynamic aspect vectors for one or more embodiments of the present specification;

[0050] Figure 3 A schematic diagram of a hierarchical attention mechanism for one or more embodiments of the present specification;

[0051] Figure 4 A schematic diagram of the device structure of one or more embodiments of this specification;

[0052] Figure 5 A schematic diagram of the structure of an electronic device according to one or more embodiments of the present specification. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0054] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of this specification should be understood by people with ordinary skills in the field to which this disclosure belongs. The "first", "second" and similar words used in one or more embodiments of this specification do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0055] As described in the background technology section, in the relevant sentiment analysis method, the context of the comment sentence is integrated into the static aspect representation, so that the comment sentences with different semantics can guide the generation of different aspect vectors, thereby realizing the dynamic nature of the aspect. Although the correlation between the aspect representation and the context is considered, the context features are generally extracted using attention mechanisms, step functions, etc., which do not reflect the correlation between context words. There are problems such as incomplete context feature learning and insufficient comment feature extraction. The generated dynamic aspect vector is not highly correlated with the comment sentence, which affects the accuracy of the sentiment analysis results.

[0056] To solve the above problems, the present application provides an aspect-level sentiment analysis method, which incorporates effective comment features into dynamic aspect vectors. The generated dynamic aspect vectors improve the correlation between aspect categories and comments, and use a hierarchical attention mechanism to fully explore the semantic relationship between target entities, context and dynamic aspect vectors, thereby improving the accuracy of sentiment analysis results.

[0057] The technical solution of the present application is further described in detail below through specific embodiments.

[0058] like Figure 1 As shown, one or more embodiments of this specification provide an aspect-level sentiment analysis method, including:

[0059] S101: Encode the comment sentence to generate a sentence pair vector and a comment vector; wherein the vector corresponding to the comment sentence in the sentence pair vector includes a target entity vector and a context vector;

[0060] In this embodiment, for the comment sentences to be analyzed, encoding processing is first performed to convert the comment sentences into vector representations for subsequent processing. Among them, the comment sentences include target entities, aspect words, sentiment words and other contents. The target entity is the object of evaluation, the aspect words are the attributes of the target entity, and the sentiment words are the emotional expressions of the attributes of the target entity, which can be divided into positive sentiment words and negative sentiment words. For example, in the comment sentences of a hotel, the target entity that can be extracted is the hotel, and the aspect words are, for example, coffee, hamburgers, waiters, luggage check-in, TV, broadband and other attribute contents related to the hotel. The sentiment words are, for example, delicious coffee, good service, good experience, bad signal, etc., which express emotional tendencies.

[0061] In some methods, the BERT model is used to encode the input comment sentence to obtain a vector representation of the comment sentence, that is, the comment vector. Auxiliary sentences are constructed based on the target entity and aspect category in the comment sentence, and the BERT model is used to encode the input comment sentence and auxiliary sentences to obtain sentence pair vectors, and the sentence pair vectors are used to predict the relevance of sentence pairs.

[0062] S102: performing encoding processing on the predetermined aspect category to generate an aspect category vector;

[0063] Aspect categories are the categories to which aspect words belong. For example, among the aspect words listed above, coffee and hamburger belong to the food category, waiter and luggage check-in belong to the service category, and television and broadband belong to the facility category. In the sentiment analysis task, it is necessary to predict the sentiment polarity of the review sentence in the predetermined aspect category. Therefore, the predetermined aspect type is encoded and processed to generate an aspect category vector, so as to capture the sentiment tendency that is more relevant to the preset aspect category from the review sentence in the future.

[0064] In some embodiments, a BERT model is used to encode an input predetermined aspect category to obtain a vector representation of the predetermined aspect category, namely, an aspect category vector.

[0065] S103: inputting the comment vector into a pre-trained comment feature extraction model to obtain a comment feature vector;

[0066] In this embodiment, considering that the comment sentence includes a lot of sentiment words, but the sentiment words are not explained, the comment sentence is expanded based on the comment sentence, and the explanation sentence of the words is added. Based on the expanded comment sentence, the corresponding comment vector is generated, and the comment vector is input into the comment feature extraction model, and the model outputs the comment feature vector. By expanding the external knowledge of the comment sentence, the accuracy and completeness of the extracted comment features can be improved.

[0067] S104: Generate a dynamic aspect vector according to the comment feature vector and the aspect category vector;

[0068] In this embodiment, a dynamic aspect representation, i.e., a dynamic aspect vector, is obtained by fusing a static aspect category vector and a dynamic comment feature vector. To prevent the semantics of the fused dynamic aspect vector from deviating greatly from the original aspect category, the comment feature vector and the aspect category vector are added according to a certain weight to generate a dynamic aspect vector.

[0069] S105: Generate a comment vector with semantic relationship according to the target entity vector, the context vector and the dynamic aspect vector;

[0070] In this embodiment, through the hierarchical attention mechanism, a semantic relationship between the target entity vector and the context vector is established at the interactive attention layer between the target entity and the context, and a semantic relationship between the aspect category vector and the comment vector is established at the general attention layer between the aspect category and the comment sentence, thereby generating a comment vector with a semantic relationship.

[0071] S106: Input the sentence pair vector and the comment vector with semantic relationship into the sentiment classifier to obtain the predicted sentiment polarity.

[0072] In this embodiment, sentiment classification is performed based on sentence pair vectors and comment vectors with semantic relationships. The classification vector with classification information in the sentence pair vector and the comment vector with semantic relationships are concatenated and input into a sentiment classifier, which outputs a predicted sentiment polarity, thereby obtaining the sentiment polarity of the comment sentence in a predetermined aspect category.

[0073] The aspect-level sentiment analysis method provided in this embodiment includes encoding the comment sentences to generate sentence pair vectors and comment vectors, encoding the predetermined aspect categories to generate aspect category vectors, inputting the comment vectors into a pre-trained comment feature extraction model to obtain comment feature vectors, generating dynamic aspect vectors based on the comment feature vectors and the aspect category vectors, generating comment vectors with semantic relationships based on the target entity vector, the context vector and the dynamic aspect vectors, inputting the sentence pair vectors and the comment vectors with semantic relationships into a sentiment classifier to obtain predicted sentiment polarity. In the method of this embodiment, the expanded comment sentences can extract more accurate and effective comment features, and the dynamic aspect vectors that integrate the comment features are more relevant to the comments, so that the sentiment prediction of the comment sentences in the predetermined aspect categories is more accurate.

[0074] The aspect-level sentiment analysis method of the embodiment of this specification is described below in conjunction with specific embodiments.

[0075] Combination Figure 1-3 As shown, given a comment sentence S consisting of L words, the sequence form S = {w1, w2, ..., w L}, the M words in the comment sentence S are pre-marked as the sequence form of the target entity currently analyzed The remaining N words in the comment sentence S except the target entity constitute the context Where L = M + N. The aspect-based sentiment analysis task is to predict the sentiment polarity y of the comment statement S in aspect category a, where aspect category a∈A, A is a predefined aspect category set, and y∈{Negative, Positive, None}; the target entity-oriented aspect-based sentiment analysis task is to predict the sentiment polarity y of the comment statement S in aspect a and the corresponding target entity t.

[0076] In some embodiments, before encoding the comment statement, the comment statement is first expanded, and the expansion method includes:

[0077] For each word in the review sentence, a predetermined number of related concept words are retrieved;

[0078] Construct explanation sentences based on relevant concept words;

[0079] An expanded comment sentence is generated according to the comment sentence and the explanation sentence.

[0080] In this embodiment, the feature extraction and sentiment analysis of the comments are performed based on the original comment sentences, which may result in inaccurate and incomplete feature extraction and inaccurate sentiment analysis. Therefore, before sentiment analysis is performed on the comment sentences, the comment sentences are first expanded. Specifically, for each word in the comment sentence, the existing knowledge base is searched to obtain related concept words that are similar to the word, and corresponding explanation sentences are constructed based on the related concept words. Then, based on the comment sentence and the explanation sentence, an expanded comment sentence is generated, and sentiment analysis is subsequently performed based on the expanded comment sentence.

[0081] In some implementations, the SenticNet sentiment dictionary is used as the common sense knowledge base. For each word in the comment sentence, the five most relevant concept words are retrieved from the SenticNet sentiment dictionary. Based on the five retrieved concept words, construct the corresponding explanation sentence in the format of For example, in the comment sentence “If you want to go out drinking where the cool kids go head to Locationl.”, five related concept words related to the word “cool” are retrieved from the SenticNet sentiment dictionary, namely “great”, “awesome”, “amazing”, “terrific” and “utterly-delightful”, and the constructed explanation sentence is “it is great, awesome, amazing, terrific and utterly-delightful”. By expanding and adding word explanations in the comment sentence, the emotional tendency of the word can be enhanced and the accuracy of emotional prediction can be improved.

[0082] In some embodiments, encoding the comment sentences to generate sentence pair vectors and comment vectors includes:

[0083] Encode the expanded comment sentence using a preset encoding model to obtain a comment vector;

[0084] constructing at least one auxiliary sentence according to the target entity and aspect category in the comment sentence;

[0085] splicing the comment sentence and at least one auxiliary sentence into a sentence pair;

[0086] The encoding model is used to encode the sentence pairs to obtain sentence pair vectors.

[0087] In this embodiment, encoding processing is performed based on the expanded comment sentence. On the one hand, the expanded comment sentence is input into the encoding model, and the encoding model outputs the corresponding comment vector. On the other hand, an auxiliary sentence is constructed based on the original comment sentence, and the comment sentence and the auxiliary sentence are concatenated into a sentence pair and input into the encoding model, and the encoding model outputs the corresponding sentence pair vector.

[0088] In some implementations, the expanded comment statement S' is input into the BERT model, and the BERT model encodes the input data to obtain the comment vector of the expanded comment statement S' The expanded comment sentence consists of L` words. Since aspect category a consists of multiple words, aspect category a is input into the BERT model, and the model outputs the word vector of each word in the corresponding aspect category. Aspect category a is a phrase consisting of d words, where d∈{1, 2, ..., K}, K is an integer. After encoding to obtain the word vectors of each word, the average embedding of multiple word vectors is taken as the aspect category vector.

[0089] Based on the target entities and aspect categories contained in the comment statement S, auxiliary sentences are constructed. The method is as follows: if the comment statement S includes n1 target entities and n2 aspect categories, n1×n2 auxiliary sentences can be constructed. The constructed auxiliary sentence S is recorded as add It consists of p words, and the sequence form is The original comment sentence S and the auxiliary sentence S ada The concatenated sentence pairs are formed by starting with [CLS] and using [SEP] as the separator between single sentences. [CLS] is the key vector for the sentence pair classification task. The concatenated sentence pairs are input into the BERT model, which encodes the input data and obtains the sentence pair vector.

[0090] Among them, the vector corresponding to the comment sentence S in the sentence pair vector includes the target entity vector and context vector

[0091] After analysis, there are two characteristics of sentiment analysis tasks. The first is the uniqueness of the task. For feature extractors that have not been trained for sentiment classification tasks, they cannot accurately and effectively extract the features of comment sentences. Second, comment sentences contain a wealth of sentiment words, but do not contain explanations of sentiment words, and feature extractors do not have prior sentiment knowledge of these sentiment words. Based on this, in this embodiment, on the one hand, a common sense knowledge base is selected to expand the comment sentences, and on the other hand, a supervised learning method is used to pre-train the comment feature extraction model, and comment feature extraction is performed based on the expanded comment sentences to improve the accuracy of extracted features.

[0092] In some embodiments, a CNN (Convolutional Neural Networks) network model and a classifier are supervisedly trained using document-level comment data samples to obtain a comment feature extraction model.

[0093] Let the comment vector corresponding to the document-level comment data sample be X doc , let the initialization parameter of the CNN network model be θ1, and the document comment feature vector v output by the model doc , expressed as:

[0094] v doc =CNN(X doc ;θ1) (1)

[0095] CNN() means that multiple convolution kernels are used in the CNN network model, and convolution is performed with a step size of 1. For each convolution kernel, a convolution operation is performed on the document comment vector. doc The i-th word window in When the jth convolution kernel of size λ is used for convolution operation, the eigenvalues ​​that can be generated are:

[0096]

[0097]

[0098] in, represents the concatenation of vectors, w j represents the convolution weight, b j Represents the convolution bias, i∈(1, L doc -λ+1), j∈(1,z).

[0099] The jth convolution kernel is in the comment vector X doc The features obtained are:

[0100]

[0101] The features extracted by multiple convolution kernels are concatenated to output the feature vector of the document-level comment data sample:

[0102]

[0103] The document comment feature vector v doc Input the sentiment classifier, the sentiment classifier outputs the predicted sentiment classification probability of the document comment, and selects the maximum probability as the predicted sentiment polarity Among them, the sentiment classifier adopts the softmax classifier, which is expressed as:

[0104] p doc =softmax(W·v doc +b) (6)

[0105]

[0106] Among them, W represents the weight of the sentiment classifier and b represents the bias.

[0107] The CNN network model and sentiment classifier are trained according to the real sentiment polarity of the document-level comment data samples and the prediction results output by the sentiment classifier. After training, the parameter θ2 of the comment feature extraction model is obtained.

[0108] The expanded comment vector is input into the comment feature extraction model, and the comment feature extraction model outputs the extracted comment feature vector vS` , expressed as:

[0109] v S` =CNN(X S` ;θ2) (8)

[0110] Among them, X S` is the expanded review vector.

[0111] In some embodiments, generating a dynamic aspect vector according to the comment feature vector and the aspect category vector includes:

[0112] According to the preset dynamic proportional coefficient, the comment feature vector and the aspect category vector are weighted and summed to obtain the dynamic aspect vector.

[0113] In this embodiment, the comment feature extraction model is used to extract the comment feature vector v S` Afterwards, according to the dynamic comment feature vector v S` and the static aspect category vector X a , generate a dynamic aspect representation, i.e., a dynamic aspect vector v a , expressed as:

[0114] v a =X a +δ·v S` (9)

[0115] Among them, δ is the dynamic scaling factor set. By transforming the comment feature vector v S` and aspect category vector X a Weighting according to a certain dynamic proportional coefficient can avoid a large deviation between the fused dynamic aspect vector and the original aspect category.

[0116] In some embodiments, generating a comment vector with a semantic relationship according to the target entity vector, the context vector, and the dynamic aspect vector includes:

[0117] According to the target entity vector and the context vector, a first comment vector in which the target entity and the context are mutually concerned is generated;

[0118] A comment vector having a semantic relationship is generated according to the first comment vector and the dynamic aspect vector.

[0119] In this embodiment, a two-layer attention mechanism is used to generate comment vectors with semantic relationships. One layer is an interactive attention layer where the target entity and the context pay attention to each other, and the interactive attention layer outputs the first comment vector where the target entity and the context pay attention to each other; the second layer is a general attention layer for dynamic aspect representation and comments, and the general attention layer outputs a comment vector with semantic relationships, fully analyzing the semantic relationship between the target entity and the context, and the semantic relationship between the dynamic aspect representation and the comments, so as to improve the accuracy of sentiment prediction.

[0120] In some embodiments, generating a first comment vector in which the target entity and the context pay attention to each other according to the target entity vector and the context vector includes:

[0121] Determine, according to the target entity vector and the context vector, a weighted target entity vector of the context focusing on the target entity and a weighted context vector of the target entity focusing on the context;

[0122] The weighted target entity vector and the weighted context vector are concatenated into the first comment vector.

[0123] In this embodiment, in the interactive attention layer between the target entity and the context, the weighted target entity vector of the context paying attention to the target entity and the weighted context vector of the target entity paying attention to the context are determined respectively. After obtaining the vector of their mutual attention, the weighted target entity vector and the weighted context vector are concatenated as the first comment vector output by the interactive attention layer.

[0124] For the interactive attention layer between the target entity and the context, in actual comment sentences, some context words are more closely related to the target entity, and the emotional expression of these context words is more obvious, that is, the target entity has different correlations with different context words, and the target entity has different levels of attention to different context words. On the other hand, when the target entity is a more complex phrase, such as "Universal Studio Hollywood", the word "Studio" is highly correlated with the words describing movies in the context, while the word "Universal" is not highly correlated with the context, that is, different words of the same target entity have different correlations with the context, and the context words have different levels of attention to different words of the target entity. Therefore, this embodiment proposes an interactive attention mechanism between the target entity and the context, which pays attention to the target entity vector and the context vector to each other, and obtains the first comment vector in which the two pay attention to each other.

[0125] In some embodiments, determining a weighted target entity vector of a context-focused target entity according to the target entity vector and the context vector includes:

[0126] Calculate the context average vector based on all word vectors in the context vector;

[0127] Calculate the first attention of the context to the target entity based on the context average vector and the target entity vector;

[0128] According to the target entity vector and the first attention, a weighted target entity vector is determined.

[0129] In this embodiment, for the weighted target entity vector of the context-focused target entity, first, based on all the word vectors in the context vector, the average vector of all word vectors is calculated as the context average vector, and then based on the context average vector and the target entity vector, the first attention of the context to the target entity is calculated, and then based on the target entity vector and the first attention, the weighted target entity vector is calculated.

[0130] Specifically, for the first attention of the context to the target entity, the context vector h c Perform pooling operation to obtain the context average vector It is expressed as:

[0131]

[0132] Afterwards, the first attention α is calculated based on the context average vector and the target entity vector, that is, the context is used to guide the target entity to calculate the first attention. The calculation method is:

[0133]

[0134]

[0135] Among them, score() is the alignment score of general dot product attention, W α The weight matrix representing the attention of the context to the target entity, α i Represents the context average vector For the i-th target entity word vector The attention value, α and α i The following relations are satisfied:

[0136] α i ∈α, α=[α1, α2,...α M ] (13)

[0137] Then, according to the first attention α and the target entity vector, the weighted target entity vector v is calculated target , expressed as:

[0138]

[0139] in, d hid represents the latent vector dimension, Represents the weighted vector of the Mth target entity word.

[0140] In some embodiments, determining a weighted context vector of a target entity focus context according to the target entity vector and the context vector includes:

[0141] Calculate the average vector of the target entity based on all word vectors in the target entity vector;

[0142] Calculate the second attention of the target entity to the context based on the target entity average vector and the context vector;

[0143] According to the context vector and the second attention, a weighted context vector is determined.

[0144] In this embodiment, for the weighted context vector of the target entity's attention context, first, based on all the word vectors in the target entity vector, the average vector of all word vectors is calculated as the target entity average vector, and then based on the target entity average vector and the context vector, the second attention of the target entity to the context is calculated, and then based on the context vector and the second attention, the weighted context vector is calculated.

[0145] Specifically, for the second attention of the target entity to the context, first the target entity vector h t Perform pooling operation to obtain the average vector of the target entity It is expressed as:

[0146]

[0147] Then, according to the target entity average vector and the context vector h c Calculate the second attention β, that is, the target entity guides the context to calculate the second attention, expressed as:

[0148]

[0149]

[0150] Among them, score() is the alignment score of general dot product attention, W β represents the weight matrix of the target entity’s attention to the context, β j Represents the average vector of the target entity For the jth context word vector The attention value, β and β j The following relations are satisfied:

[0151] β j ∈β,β=[β1,β2,...β N ] (18)

[0152] Then, according to the second attention β and the context vector, the weighted context vector v is calculated context , expressed as:

[0153]

[0154] in, d hid represents the latent vector dimension, Represents the weighted vector of the Nth context word.

[0155] Finally, the weighted target entity vector v target and the weighted context vector v context The first comment vector v output by the interactive attention layer of the target entity and the context is obtained by concatenating them. review , expressed as:

[0156] v review =[v target ;v context ] (20)

[0157] In some embodiments, generating a comment vector having a semantic relationship according to the first comment vector and the dynamic aspect vector includes:

[0158] Calculate aspect-level attention based on the first comment vector and the dynamic aspect vector;

[0159] According to the aspect-level attention and the first comment vector, the comment vectors with semantic relationship are determined.

[0160] In this embodiment, based on the first comment vector output by the interactive attention layer, the general attention layer calculates the aspect-level attention according to the first comment vector and the dynamic aspect vector, and then calculates a comment vector with a semantic relationship based on the aspect-level attention and the first comment vector.

[0161] As for the general attention layer between aspect categories and comment sentences, since specific aspect words generally appear in comment sentences but aspect categories do not, this embodiment uses aspect-level attention to calculate the attention of aspect categories to comment sentences.

[0162] Specifically, according to the first comment vector v output by the interactive attention layer review and the dynamic aspect vector v a , calculate the aspect-level attention γ, expressed as:

[0163]

[0164]

[0165] Among them, γ k Represents the dynamic aspect vector va For the kth word vector in the first comment vector The attention value, γ and γ k The following relations are satisfied:

[0166] γ k ∈γ, γ=[γ1, γ2,...γ M+N ] (twenty three)

[0167] Afterwards, according to the aspect-level attention γ and the first review vector v review , calculate the comment vector with semantic relationship Comment Vector It is the comment vector output by the general attention layer, that is, the comment vector processed by the hierarchical attention mechanism.

[0168] In this embodiment, after obtaining the sentence pair vector and the comment vector with a semantic relationship, the sentence pair vector and the comment vector with a semantic relationship can be used to obtain the sentence pair vector and the comment vector with a semantic relationship. Perform sentiment prediction. The classification vector h in the sentence pair vector H [CLS] and comment vector After splicing, they are input into the sentiment classifier, which outputs the probability values ​​of predicting the three sentiment polarities, and then selects the sentiment polarity with the largest probability value as the predicted sentiment polarity. It is expressed as:

[0169]

[0170]

[0171] One or more embodiments of the present specification provide an aspect-level sentiment analysis method, which expands external knowledge of the comment sentence to be analyzed and enhances the sentiment tendency of the sentence by adding word explanations; performs supervised learning of similar tasks on the comment feature extraction model and extracts features based on the expanded comment vector, which is conducive to fully understanding the semantic relationship of the comment sentence and improving the accuracy and completeness of the extracted features; fuses the dynamic comment feature vector and the static aspect category vector to obtain a dynamic aspect vector, which can improve the correlation between the aspect category and the comment sentence and improve the accuracy of sentiment prediction; uses the hierarchical attention mechanism to fully explore the semantic relationship between the target entity, context and dynamic aspect vector, and improves the accuracy of sentiment prediction.

[0172] It should be noted that the method of one or more embodiments of this specification can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of this specification, and the multiple devices will interact with each other to complete the described method.

[0173] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] like Figure 4 As shown, the embodiment of this specification also provides an aspect-level sentiment classification device, including:

[0175] A first encoding module is used to encode the comment sentence to generate a sentence pair vector and a comment vector; wherein the vector corresponding to the comment sentence in the sentence pair vector includes a target entity vector and a context vector;

[0176] A second encoding module is used to encode the predetermined aspect category to generate an aspect category vector;

[0177] A feature extraction module, used for inputting the comment vector into a pre-trained comment feature extraction model to obtain a comment feature vector;

[0178] A dynamic aspect module, used for generating a dynamic aspect vector according to the comment feature vector and the aspect category vector;

[0179] A semantic analysis module is used to generate a comment vector with semantic relations based on the target entity vector, the context vector and the dynamic aspect vector;

[0180] The prediction module is used to input the sentence pair vector and the comment vector with semantic relationship into the sentiment classifier to obtain the predicted sentiment polarity.

[0181] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing one or more embodiments of this specification, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0182] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0183] Figure 5 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0184] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0185] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0186] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0187] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0188] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0189] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0190] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0191] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0192] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. In line with the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present specification as described above, which are not provided in detail for the sake of simplicity.

[0193] In addition, to simplify the description and discussion, and in order not to obscure one or more embodiments of the present specification, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device may be shown in the form of a block diagram so as to avoid obscuring one or more embodiments of the present specification, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present specification will be implemented (i.e., these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that one or more embodiments of the present specification may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0194] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0195] One or more embodiments of this specification are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification should be included in the scope of protection of this disclosure.

Claims

1. A method for aspect-level sentiment classification, characterized in that: include: For each word in the review sentence, a predetermined number of related concept words are retrieved; Constructing explanation sentences according to the related concept words; generating an expanded comment sentence according to the comment sentence and the explanation sentence; The comment sentence is encoded to generate a sentence pair vector and a comment vector, including: encoding the expanded comment sentence using a preset encoding model to obtain the comment vector; constructing at least one auxiliary sentence according to the target entity and aspect category in the comment sentence; splicing the comment sentence and the at least one auxiliary sentence into a sentence pair; encoding the sentence pair using the encoding model to obtain the sentence pair vector; wherein the vector corresponding to the comment sentence in the sentence pair vector includes a target entity vector and a context vector; Encoding the predetermined aspect category to generate an aspect category vector, including: encoding the predetermined aspect category using the encoding model to obtain word vectors of each word in the predetermined aspect category, and taking the average embedding of the word vectors of each word as the aspect category vector; Inputting the comment vector into a pre-trained comment feature extraction model to obtain a comment feature vector; wherein the comment feature extraction model is obtained by using document-level comment data samples to perform supervised training on a CNN network model and a classifier; Generating a dynamic aspect vector according to the comment feature vector and the aspect category vector, comprising: performing weighted summation of the comment feature vector and the aspect category vector according to a preset dynamic proportional coefficient to obtain the dynamic aspect vector; Generating a comment vector with a semantic relationship according to the target entity vector, the context vector and the dynamic aspect vector, including: generating a first comment vector in which the target entity and the context pay attention to each other according to the target entity vector and the context vector; generating the comment vector with a semantic relationship according to the first comment vector and the dynamic aspect vector; The sentence pair vector and the comment vector having a semantic relationship are input into a sentiment classifier to obtain a predicted sentiment polarity.

2. The method according to claim 1, characterized in that Generating a first comment vector in which the target entity and the context pay attention to each other according to the target entity vector and the context vector, comprising: Determine, according to the target entity vector and the context vector, a weighted target entity vector of the context-focused target entity and a weighted context vector of the target entity-focused context; The weighted target entity vector and the weighted context vector are concatenated into the first comment vector.

3. The method according to claim 2, characterized in that Determining a weighted target entity vector of a context-focused target entity according to the target entity vector and the context vector includes: Calculate a context average vector based on all word vectors in the context vector; Calculating the first attention of the context to the target entity according to the context average vector and the target entity vector; The weighted target entity vector is determined according to the target entity vector and the first attention.

4. The method according to claim 2, characterized in that: Determining a weighted context vector of a target entity focus context according to the target entity vector and the context vector includes: Calculate the target entity average vector based on all word vectors in the target entity vector; Calculating a second attention of the target entity to the context according to the target entity average vector and the context vector; The weighted context vector is determined based on the context vector and the second attention.

5. The method according to claim 1, characterized in that Generating the comment vector having the semantic relationship according to the first comment vector and the dynamic aspect vector includes: Calculating aspect-level attention according to the first comment vector and the dynamic aspect vector; The comment vector having a semantic relationship is determined according to the aspect-level attention and the first comment vector.

6. An aspect-level sentiment classification device, characterized in that: For implementing the method according to claim 1, the device comprises: A first encoding module is used to encode the comment sentence to generate a sentence pair vector and a comment vector; wherein the vector corresponding to the comment sentence in the sentence pair vector includes a target entity vector and a context vector; A second encoding module is used to encode the predetermined aspect category to generate an aspect category vector; A feature extraction module, used for inputting the comment vector into a pre-trained comment feature extraction model to obtain a comment feature vector; A dynamic aspect module, used for generating a dynamic aspect vector according to the comment feature vector and the aspect category vector; A semantic analysis module is used to generate a comment vector with semantic relations based on the target entity vector, the context vector and the dynamic aspect vector; The prediction module is used to input the sentence pair vector and the comment vector with semantic relationship into a sentiment classifier to obtain predicted sentiment polarity.

Images