Implicit sentiment analysis model method based on knowledge enhancement and double attention mechanism
By adopting a model of knowledge enhancement and dual attention mechanism in implicit sentiment analysis, the problem of difficulty in capturing hidden emotional characteristics in the existing technology is solved, and higher analysis accuracy and better application value are achieved.
Patent Information
- Application Number
- CN202411975736.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-02
AI Technical Summary
Existing implicit emotion analysis methods are difficult to effectively capture hidden emotional characteristics, resulting in low analysis accuracy.
The implicit sentiment analysis model based on knowledge enhancement and dual attention mechanism is adopted, word vector embedding is performed through pre-trained language models, combined with external knowledge base enhancement features, and a context-aware attention layer and co-attention layer are designed to dynamically adjust the attention of the target sentence to the context, and promote feature interaction and fusion.
It significantly improves the accuracy of implicit sentiment analysis, can more effectively capture the hidden emotional characteristics in the text, and has good practical application value and promotion prospects.
Smart Images
Figure CN119918535A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of the Internet and social network, and relates to natural language processing sentiment analysis, specifically to an implicit sentiment analysis model method, device, storage medium and electronic device based on knowledge enhancement and dual attention mechanism. Background Art
[0002] In recent years, with the development of the Internet and social networks, text sentiment analysis has become one of the hottest research directions in the field of natural language processing. Comprehensive and accurate understanding of the emotions expressed in text can be applied to many scenarios such as stock market forecasting, customer feedback tracking, opinion mining, etc., generating extensive social benefits.
[0003] Text sentiment analysis is the process of analyzing, processing, summarizing and reasoning about subjective texts with emotional connotations. From the perspective of text language expression, it can be divided into explicit sentiment analysis and implicit sentiment analysis. Among them, implicit sentiment analysis is defined as "a language fragment (sentence, clause or phrase) that does not contain explicit sentiment words but expresses subjective emotions."
[0004] At present, explicit sentiment analysis, as a basic research content in this field, has achieved many fruitful results, while implicit sentiment analysis research is still in its infancy. In daily expression, people's emotions reflected in their experience of objective things and their behaviors are rich and abstract. In addition to using explicit sentiment words to express emotions, they also use objective statements or rhetoric to implicitly express their emotions. Through the study of implicit sentiment analysis, the performance of text sentiment analysis can be improved more comprehensively and accurately, which can play a positive role in promoting research in text representation learning, natural language understanding, user modeling, knowledge embedding, etc., and can also further promote the rapid development of applications and industries related to text sentiment analysis.
[0005] At present, the existing implicit sentiment analysis methods mainly focus on encoding text semantic information, syntactic information and context information, and identify implicit sentiment by integrating multi-level information. However, due to the lack of information provided by the implicit sentiment text itself, only modeling the implicit sentiment text cannot effectively capture the hidden sentiment features. Summary of the invention
[0006] In view of the defect that the prior art fails to fully make up for the lack of effective information in implicit sentiment text, the present invention proposes an implicit sentiment analysis model method, device, storage medium and electronic device based on knowledge enhancement and dual attention mechanism to improve the accuracy of implicit sentiment analysis.
[0007] The present invention is achieved by adopting the following technical solutions:
[0008] An implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism includes the following steps:
[0009] Step 1: Use the pre-trained language model to embed the implicit sentiment target sentence and its context sentences into word vectors to obtain implicit sentiment text semantic features and implicit sentiment text context features;
[0010] Step 2: Use the external knowledge base to enhance the implicit sentiment text semantic features and implicit sentiment text context features obtained in step 1 to obtain a richer semantic feature representation;
[0011] Step 3: Construct a dual attention mechanism layer, including a context-aware attention layer and a co-attention layer:
[0012] The context-aware attention layer is used to capture the semantic association between the implicit emotional target sentence and its context, and dynamically adjust the attention of the target sentence to the context;
[0013] The co-attention layer is used to promote the interaction and fusion between the implicit emotional target sentence and the contextual features, and to extract effective information from the context to enrich the output of the context-aware attention layer;
[0014] Step 4: Process the features of step 2 in step 3 and perform a pooling operation;
[0015] Step 5: Input the pooled features into the fully connected layer, and use the Softmax function to predict the probability distribution of sentiment polarity and perform label smoothing regularization.
[0016] The method of the present invention first adopts implicit sentiment text representation, uses the BERT / RoBERTa pre-trained language model to embed the implicit sentiment target sentence and its context sentences into word vectors, and obtains implicit sentiment text semantic features and implicit sentiment text context features. Then, knowledge enhancement is performed, and the implicit sentiment text features are enhanced using an external semantic source knowledge base, and the semantic source vector is fused with the implicit sentiment text features to obtain a richer semantic representation. Then, a dual attention mechanism is designed to construct a Figure 2The context-aware attention layer and co-attention layer enhance the model's ability to capture and process the context information of implicit sentiment text; the context-aware attention layer uses Intra-MHA to perform self-attention mechanism on implicit sentiment text to capture the semantic association within the text; Inter-MHA is used to perform interactive attention mechanism on implicit sentiment text and context text to capture the semantic association between texts; the features of MHA output are transformed using point-by-point convolutional transform (PCT) to further extract semantic information. The co-attention layer uses a nonlinear network to transform the output of the context-aware attention layer to obtain the encoder vector; the correlation matrix is calculated to obtain the attention score of each word pair of interactive implicit sentiment text features in the output of the context-aware attention layer, and the attention score of each word pair of self-implicit sentiment text features in the output of the context-aware attention layer. The corrected context vector is calculated based on the attention score and concatenated with the output of the context-aware attention layer; the corrected vector is calculated using the attention mechanism to obtain the final vector that integrates the context and implicit sentiment text information; and the Bi-LSTM is used to fuse and obtain the final semantic representation. Finally, sentiment classification is performed, and the final representation is obtained using average pooling, which is then concatenated as the final representation. The final representation is mapped to the target C class space using a fully connected layer, and the probability distribution of sentiment tendency is obtained through the Softmax activation function.
[0017] The technical solution provided by the present invention has the following advantages compared with the prior art:
[0018] The implicit sentiment analysis model (KEDAM) based on knowledge enhancement and dual attention mechanism proposed in this invention is reasonably designed. By selecting the BERT / RoBERTa pre-trained language model for word vector embedding, and combining the semantic original knowledge base to enhance the implicit sentiment sentences and context features, the semantic understanding ability of the model is improved. At the same time, the dual attention mechanism of the context-aware attention layer and the co-attention layer is used to enhance the model's ability to capture contextual information. The experimental results on the public dataset SMP2019-ECISA show that the model has a good effect in implicit sentiment analysis tasks, has good practical application value and good promotion prospects, and provides new ideas and methods for research in the field of implicit sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 Represents the implicit sentiment analysis model framework based on knowledge enhancement and dual attention mechanism.
[0022] Figure 2 Representation of the structural framework of the context-aware attention layer and the co-attention layer. DETAILED DESCRIPTION
[0023] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the scheme of the present invention will be further described below. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.
[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present invention, rather than all of the embodiments.
[0025] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0026] The embodiment of the present invention provides an implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism, which aims to solve the problem that implicit sentiment text lacks rich knowledge and common sense support and ignores context when expressing emotions. First, the implicit sentiment text representation layer of the model method uses the pre-trained language model BERT / RoBERTa to embed word vectors for implicit sentiment target sentences and their contexts, and enhances the implicit sentiment sentence features and implicit sentiment context features in combination with the semantic original knowledge base. Then, the dual attention mechanism layer dynamically adjusts the target sentence's attention to the context through the context-aware attention layer and the co-attention layer, and promotes the interaction and fusion of the target sentence and context features. Finally, the sentiment classification layer predicts the probability distribution of sentiment polarity through the Softmax function.
[0027] 1. Implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism, such as Figure 1 As shown, the specific steps include:
[0028] Step 1: Use the pre-trained language model BERT / RoBERTa to embed the implicit sentiment target sentence and its context sentences into word vectors to obtain the implicit sentiment text semantic features and implicit sentiment text context features.
[0029]
[0030] Step 2: Use an external knowledge base (such as a semantic knowledge base) to enhance the implicit sentiment text semantic features and implicit sentiment text context features obtained in step 1 to obtain a richer semantic feature representation.
[0031] Step 2.1: Select HowNet as the source of the semantic primitive knowledge base, filter out unimportant low-frequency semantic primitives, and finally use 1,335 semantic primitives and use their pre-trained vectors as external knowledge for knowledge enhancement.
[0032] Step 2.2: The pre-trained vector is integrated with the implicit sentiment text semantic features and the implicit sentiment text context features into richer implicit sentiment text semantic features and implicit sentiment text context features.
[0033]
[0034]
[0035] in, is the context sequence, is a sequence containing implicit emotions, is the implicit sentiment text semantic feature, is the implicit sentiment text context feature, is the fused implicit sentiment text semantic feature, is the fused implicit sentiment text context feature, is the semantic vector corresponding to the word, d h is the size of the hidden state.
[0036] Step 3: Construct a dual attention mechanism layer, such as Figure 2 As shown, it includes a context-aware attention layer and a co-attention layer.
[0037] The context-aware attention layer is used to capture the semantic association between the implicit emotional target sentence and its context, and dynamically adjust the attention of the target sentence to the context. This layer includes the multi-head attention mechanism (MHA) and the point-by-point convolutional transform (PCT). The co-attention layer is used to promote the interaction and fusion between the implicit emotional target sentence and the context features, and to extract effective information from the context to enrich the output of the context-aware attention layer.
[0038] Step 3.1: In the context-aware attention layer, MHA includes Intra-MHA and Inter-MHA.
[0039] Intra-MHA: Perform internal attention mechanism calculation on implicit sentiment target sentences to obtain self-implicit sentiment text feature representation.
[0040] Inter-MHA: An interactive attention mechanism is performed on the implicit sentiment target sentence and contextual features to obtain an interactive implicit sentiment text feature representation.
[0041]
[0042] o h =Attention h (k,q)
[0043] Attention(k,q)=softmax(f s (k,q))k
[0044] f s (k i ,q j )=tanh([k i ;q j ]·W att )
[0045]
[0046]
[0047] Here, the attention function maps a key sequence k = {k1, k2, ..., k n} and query sequence q={q1,q2,…,q m} to the output sequence o; f s Representation learning, an alignment function that represents the semantic correlation between and; is the learnable weight; ";" is the vector concatenation, is the output of the h-th head attention, h∈[1,n head ]; σ represents ELU activation; * is the convolution operator; c intra is the implicit sentiment text feature; t inter It is the interactive implicit sentiment text feature.
[0048] In step 3.2, in the context-aware attention layer, PCT transforms the outputs of Intra-MHA and Inter-MHA to obtain the output of the context-aware attention layer.
[0049]
[0050] H oatt-intra = PCT(c intra )
[0051] H oatt-inter = PCT(t inter )
[0052] in, and are the learnable weights of the two convolution kernels; and is the bias of the two convolution kernels.
[0053] In step 3.3, in the co-attention layer, the output of the context-aware attention layer and the context features are first converted through a nonlinear network to obtain the encoder vector.
[0054] H oatt-intra =tanh(W c H oatt-intra +b c )
[0055] H oatt-inter =tanh(W t H oatt-inter +b t )
[0056] Among them, W c , W t , b c , b t is a trainable parameter.
[0057] Step 3.4, calculate the correlation matrix based on the encoder vector, and calculate the attention score of each word pair's interactive implicit sentiment text feature and the attention score of each word pair's self-implicit sentiment text feature in the output of the context-aware attention layer based on the correlation matrix.
[0058]
[0059] Step 3.5: Use the attention score to correct the output of the context-aware attention layer to obtain the corrected context vector.
[0060]
[0061] Step 3.6: Concatenate the corrected context vector with the output of the context-aware attention layer, and obtain the corrected vector through attention calculation.
[0062]
[0063] Among them, C intra It is a co-dependency representation of contextual and self-implicit sentiment text features.
[0064] Step 3.7: Use Bi-LSTM to fuse the corrected context vector and the implicit sentiment text feature to obtain the final vector that fuses the context and implicit sentiment text information.
[0065]
[0066] Step 4: The fused implicit sentiment text semantic features obtained in step 2 and the fused implicit sentiment text context features The input is processed in the dual attention mechanism layer of step 3 and pooled.
[0067] Step 5: Input the pooled features into the fully connected layer, predict the sentiment polarity probability distribution through the Softmax function, and perform label smoothing regularization.
[0068]
[0069]
[0070] in, is the predicted sentiment polarity distribution, and is a learnable parameter, v′ is the smoothed label distribution, u is the predicted distribution, G is the cross entropy function, v is the actual label distribution, e is the prior distribution on the label, and ε is the smoothing parameter.
[0071] 2. Comparative Experiment
[0072] To verify the effectiveness of the model, several baseline models are introduced to conduct comparative experiments on the public dataset SMP2019-ECISA. The comparison results are as follows:
[0073] Model Precision Recall F1-Macro DPCNN 0.743 0.708 0.725 Lu Jingqian and others - - 0.794 HERCULES 0.604 0.605 0.604 MPOA(TENCE) - - 0.733 C-MPOA(TENCE) - - 0.746 CnS 0.795 0.76 0.769 TGR 0.793 0.784 0.788 KC-ISA - - 0.817 EBA 0.823 0.801 0.803 RBLA 0.838 0.828 0.817 RBLA-GCN - - 0.818 ContextBert(BERT) - - 0.827 KEDAM(BERT) 0.839 0.829 0.834 KEDAM(RoBERTa) 0.838 0.828 0.832
[0074] (1) DPCNN continuously deepens the network by stacking convolution and negative sampling layers to extract dependencies in long-distance text.
[0075] (2) Lu Jingqian et al. combined the GCN model of implicit sentiment text sentences, parts of speech and dependency relationships to perform implicit sentiment text classification.
[0076] (3) HERCULES expands external knowledge and combines heterogeneous user knowledge and target sentences to propose a personalized implicit sentiment classification model.
[0077] (4)MPOA(TENCE) is a BiLSTM model with multi-level orthogonal attention. Compared with the traditional single attention model, multi-level attention can identify the differences between words and sentiment tendencies.
[0078] (5) C-MPOA (TENCE) is an implicit sentiment sentence recognition method based on contextual information. A multi-polar orthogonal attention representation model is used to embed implicit sentiment sentence representation. Secondly, by integrating the context and modeling the context information, a multi-polar attention layer is established to mine the key information in the context.
[0079] (6) CsGCN proposes a context-specific heterogeneous graph convolutional network framework that can combine all contextual representations and has a complete context that more comprehensively reflects the sentence information.
[0080] (7) TGR proposes a Chinese implicit sentiment analysis model based on text graph representation, which makes full use of external knowledge and context to enhance implicit sentiment text and realize semantic interaction at the word level.
[0081] (8) KC-ISA uses the Coattention layer and BiAffine module to fuse context, target sentence and external knowledge to obtain a more complete semantic representation.
[0082] (9) EBA can better capture the semantic and contextual information of implicit emotional sentences and effectively improve the ability of implicit emotion recognition.
[0083] (10) RBLA uses the RoBERTa pre-trained model to capture the semantic features of words in implicit sentiment sentences, and then uses a bidirectional long short-term memory network to learn the positive and negative semantic information of sentences to capture the dependency relationship between sentences and realize the extraction of deep-level features of text.
[0084] (11)RBLA-GCN is a complementary model of RBLA combined with GCN.
[0085] (12) ContextBert (BERT) adopts implicit sentiment query attention to find contextual information related to the target sentence to enhance implicit sentiment analysis.
[0086] (13) KEDAM (BERT) is a model of the present invention that uses the pre-trained model BERT to obtain the target sentence and its context embedding.
[0087] (14) KEDAM (RoBERTa) is a model of the present invention that uses the pre-trained model RoBERTa to obtain the target sentence and its context embedding.
[0088] Through comparison, it can be seen that the model described in the present invention is good at learning implicit emotions and has achieved the best results in implicit sentiment analysis. The overall performance of the models based on CNN and RNN is poor because these models only roughly process the target words. Compared with DPCNN, KEDAM's F1-Macro is improved by 10.9%, which shows that KEDAM can better extract the features of implicit emotional target sentences for sentiment classification. KEDAM uses BERT and RoBERTa to obtain word embeddings for implicit emotional target sentences and their context sentences, which is 7.5% and 2.5% higher than BERT and RoBERTa on SMP2019-ECISA, respectively. For models that integrate contextual information, KEDAM's F1-Macro is improved compared with C-MPOA (TENCE), CsGCN, KC-ISA, EBA and ContextBert (BERT), indicating that the model described in the present invention effectively utilizes contextual features. For the use of external knowledge and pre-trained models, KEDAM's F1-Macro is 4.6% and 1.7% higher than TGR and RBLA, respectively, indicating that the sentences in the implicit sentiment analysis dataset contain limited effective information, and adding external knowledge and pre-trained models can effectively improve the model's classification ability. In summary, the model described in the present invention shows effectiveness in implicit sentiment analysis tasks.
[0089] 3. Ablation Experiment
[0090] In order to verify the role of each module in KEDAM, the present invention designs four groups of ablation experiments to analyze the pre-trained models BERT and RoBERTa respectively, while keeping the corresponding parameters of the model unchanged. The analysis results are shown in the following table:
[0091]
[0092] (1) KEDAM w / o CAA means removing the context-aware attention layer and using the co-attention mechanism layer.
[0093] (2) KEDAM w / o CoA means removing the co-attention mechanism layer and using the context-aware attention layer.
[0094] (3) KEDAM w / o DA means removing the dual attention mechanism layer and directly using the pre-trained model BERT / RoBERTa embedding representation for implicit sentiment classification.
[0095] (4) KEDAM w / o SK represents the fusion of the semantically removed knowledge base with implicit sentiment text and its context embedding.
[0096] (5) KEDAM is the model of the present invention.
[0097] From the above experimental results, it can be seen that: from the experimental results based on the BERT pre-trained model, the experimental results of KEDAM w / o CAA and KEDAM w / o CoA are not much different, indicating that the model effects of using only the context-aware attention layer and only the co-attention layer are not much different. However, when the context-aware attention layer and the co-attention layer are removed at the same time, that is, the dual attention mechanism layer is removed, the model effects are quite different, indicating that the dual attention layer of KEDAM has shown excellent ability in processing the context of implicit sentiment text. The final results of KEDAM w / o SK are very different from those of the KEDAM model, indicating that the semantic original knowledge base has limited performance in improving the semantic knowledge of implicit sentiment text. The experimental results based on the RoBERTa pre-trained model follow similar rules to the experimental results based on the BERT pre-trained model.
[0098] 4. The implicit sentiment analysis model device based on knowledge enhancement and dual attention mechanism provided by the embodiment of the present invention includes:
[0099] A preprocessing module, used to collect and preprocess implicit sentiment text and its context text;
[0100] The embedding module is used to input the preprocessed implicit sentiment text and its context text into the pre-trained language model (such as BERT / RoBERTa) for word vector embedding to obtain the semantic features of the implicit sentiment text and its context features;
[0101] The knowledge enhancement module is used to fuse the semantic vectors in the semantic knowledge base with the implicit sentiment text semantic features and its context features to obtain richer implicit sentiment text semantic features and its context features;
[0102] Dual attention mechanism module, including: context-aware attention layer and co-attention layer;
[0103] The context-aware attention layer is used to dynamically adjust the attention of implicit sentiment text to the context, including the multi-head attention mechanism (MHA) and point-wise convolutional transform (PCT);
[0104] The co-attention layer is used to promote the interaction and fusion between implicit sentiment text and contextual features, including nonlinear network, correlation matrix calculation, attention score calculation, corrected context vector calculation, corrected vector calculation, and final vector calculation that integrates context and implicit sentiment text information;
[0105] The sentiment classification module is used to average pool the final vector that integrates the context and implicit sentiment text information, and predict the sentiment polarity probability distribution through a fully connected layer and a softmax function.
[0106] 5. A storage medium provided by an embodiment of the present invention stores a computer program of the implicit sentiment analysis model based on knowledge enhancement and dual attention mechanism, and the computer program executes the steps of the method of the present invention.
[0107] 6. An electronic device provided by an embodiment of the present invention includes one or more memories and processors, wherein the memories store one or more computer programs, and when the one or more processors execute the computer programs, the steps of the implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism are implemented.
[0108] The above is only a specific implementation of the present invention, which enables those skilled in the art to understand or implement the present invention. Although detailed descriptions are given with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments, and they should all be covered by the protection scope of the claims.
Claims
1. An implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism, characterized by: The steps include: Step 1: Use the pre-trained language model to embed the implicit sentiment target sentence and its context sentences into word vectors to obtain implicit sentiment text semantic features and implicit sentiment text context features; Step 2: Use an external knowledge base to enhance the implicit sentiment text semantic features and implicit sentiment text context features obtained in step 1 to obtain a richer semantic feature representation; Step 3: Construct a dual attention mechanism layer, including a context-aware attention layer and a co-attention layer: The context-aware attention layer is used to capture the semantic association between the implicit emotional target sentence and its context, and dynamically adjust the attention of the target sentence to the context; The co-attention layer is used to promote the interaction and fusion between the implicit emotional target sentence and the contextual features, and to extract effective information from the context to enrich the output of the context-aware attention layer; Step 4 processes the features of step 2 in step 3 and performs a pooling operation; Step 5: Input the pooled features into the fully connected layer, and use the Softmax function to predict the probability distribution of sentiment polarity and perform label smoothing regularization.
2. The implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism according to claim 1 is characterized in that: In step 1, the pre-trained language model uses BERT / RoBERTa.
3. The implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism according to claim 1 is characterized in that: In step 2, the external knowledge base adopts a semantic source knowledge base.
4. The implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism according to claim 1 is characterized in that: In step 3, the context-aware attention layer includes a multi-head attention mechanism MHA and a point-by-point convolutional transform PCT.
5. The implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism according to claim 3 is characterized in that: Step 2 is as follows: Step 2.1, select HowNet as the source of the sememe knowledge base, filter out unimportant low-frequency semes, and finally use 1335 semes and use their pre-trained vectors as external knowledge for knowledge enhancement; Step 2.2, fusing the pre-trained vector with the implicit sentiment text semantic features and the implicit sentiment text context features into richer implicit sentiment text semantic features and implicit sentiment text context features; in, is the implicit sentiment text semantic feature, is the implicit sentiment text context feature, is the fused implicit sentiment text semantic feature, is the fused implicit sentiment text context feature, is the semantic vector corresponding to the word, d h is the size of the hidden state.
6. The implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism according to claim 4 is characterized in that: Step 3 is as follows: Step 3.1: In the context-aware attention layer, MHA includes Intra-MHA and Inter-MHA. The Intra-MHA performs internal attention mechanism calculation on the implicit sentiment target sentence to obtain a self-implicit sentiment text feature representation; The Inter-MHA performs interactive attention mechanism calculation on the implicit emotional target sentence and the contextual features to obtain interactive implicit emotional text feature representation; o h =Attention h (k,q) Attention(k,q)=softmax(f s (k,q))k f s (k i ,q j )=tanh([k i ;q j ]·W att ) Here, the attention function maps a key sequence k = {k1, k2, ..., k n } and query sequence q={q1,q2,…,q m } to the output sequence o; f s Representation learning, an alignment function that represents the semantic correlation between and; is the learnable weight; ";" is the vector concatenation, is the output of the h-th head attention, h∈[1,n head ]; σ represents ELU activation; * is the convolution operator; c intra is the implicit sentiment text feature; t inter It is the interactive implicit sentiment text feature; Step 3.2: In the context-aware attention layer, PCT transforms the outputs of Intra-MHA and Inter-MHA to obtain the output of the context-aware attention layer. H oatt-intra =PCT(c intra ) H oatt-inter =PCT(t inter ) in, and are the learnable weights of the two convolution kernels; and is the bias of the two convolution kernels; Step 3.3: In the co-attention layer, the output of the context-aware attention layer and the context features are first converted through a nonlinear network to obtain an encoder vector. H oatt-intra =tanh(W c H oatt-intra +b c ) H oatt-inter =tanh(W t H oatt-inter +b t ) Among them, W c , W t , b c , b t is a trainable parameter; Step 3.4, calculate the correlation matrix based on the encoder vector, and calculate the attention score of each word pair of interactive implicit sentiment text features and the attention score of each word pair of self-implicit sentiment text features in the output of the context-aware attention layer based on the correlation matrix; Step 3.5: Use the attention score to correct the output of the context-aware attention layer to obtain the corrected context vector; Step 3.6: Concatenate the corrected context vector with the output of the context-aware attention layer, and obtain the corrected vector through attention calculation; Among them, C intra It is a co-dependency representation of contextual and self-implicit sentiment text features; Step 3.7, using Bi-LSTM to fuse the corrected context vector and the implicit sentiment text feature to obtain the final vector that fuses the context and implicit sentiment text information; 7. The implicit sentiment analysis model method based on knowledge enhancement and dual attention mechanism according to claim 1 is characterized in that: Step 5 is as follows: in, is the predicted sentiment polarity distribution, and is a learnable parameter, v' is the smoothed label distribution, u is the predicted distribution, G is the cross entropy function, v is the actual label distribution, e is the prior distribution on the label, and ε is the smoothing parameter.
8. An implicit sentiment analysis model device based on knowledge enhancement and dual attention mechanism, characterized by: include: A preprocessing module, used for collecting and preprocessing implicit sentiment text and its context text; The embedding module is used to input the preprocessed implicit sentiment text and its context text into the pre-trained language model for word vector embedding to obtain the semantic features of the implicit sentiment text and its context features; The knowledge enhancement module is used to fuse the semantic vectors in the semantic knowledge base with the implicit sentiment text semantic features and its context features to obtain richer implicit sentiment text semantic features and its context features; Dual attention mechanism module, including context-aware attention layer and co-attention layer; The context-aware attention layer is used to dynamically adjust the attention of implicit sentiment text to the context, including the multi-head attention mechanism MHA and point-by-point convolutional transform PCT; The co-attention layer is used to promote the interaction and fusion between implicit sentiment text and contextual features, including nonlinear network, correlation matrix calculation, attention score calculation, corrected context vector calculation, corrected vector calculation, and final vector calculation that integrates context and implicit sentiment text information; The sentiment classification module is used to average pool the final vector that integrates the context and implicit sentiment text information, and predict the sentiment polarity probability distribution through a fully connected layer and a softmax function.
9. A storage medium, characterized in that: A computer program for storing an implicit sentiment analysis model based on knowledge enhancement and dual attention mechanism, wherein the computer program executes the implicit sentiment analysis model method as described in any one of claims 1 to 7.
10. An electronic device, characterized in that: The method comprises one or more memories and processors, wherein the memories store one or more computer programs, and when the one or more processors execute the computer programs, the implicit sentiment analysis model method as described in any one of claims 1 to 7 is implemented.