A feature interaction and association fusion-based sentiment analysis method and system

Abstract

The application belongs to the technical field of sentiment analysis, and particularly relates to a sentiment analysis method and system based on feature interaction and association fusion, which comprises the following steps: inputting obtained text data into a BERT model to obtain multi-layer representation vectors; inputting all intermediate layer representation vectors into a multi-layer feature interaction module to obtain a first association feature fusion vector; inputting the first association feature fusion vector and the final layer representation vector into a dynamic association fusion module to obtain a classification text feature vector; performing feature discarding on the classification text feature vector based on a dynamic mask adaptive discarding method, and then obtaining a sentiment category of the to-be-identified text through a full connection layer and an activation function layer. The application performs feature fusion on the multi-layer representation vectors, can fully utilize semantic representations of different levels output by the model, thereby obtaining more rich and comprehensive feature information, and is helpful to improve the understanding ability of the model to text semantics and the accuracy of sentiment analysis.

Description

Technical Field

[0001] This invention belongs to the field of sentiment analysis technology, and in particular relates to a sentiment analysis method and system based on feature interaction and association fusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Sentiment analysis is an important task in the field of natural language processing, aiming to identify and analyze the author's emotional stance from text, typically including positive, negative, or neutral sentiment. With the rapid increase in social media, online comments, and other textual data, the need for effective sentiment analysis methods is becoming increasingly urgent.

[0004] In recent years, with the rapid development of the hydrogen energy industry chain, sentiment analysis technology can be deeply integrated with the user needs of the hydrogen energy industry chain to achieve more accurate service matching and personalized experiences. For example, in services targeting government departments, hydrogen energy companies, and end users, sentiment analysis can be used to analyze user feedback on hydrogen energy policies, products, and services, thereby building accurate user profiles. This can not only provide decision support for government departments but also provide market insights for hydrogen energy companies, while offering customized service experiences for end users.

[0005] In the early stages of sentiment analysis, researchers primarily relied on manually generated rules and traditional machine learning methods. These methods, based on building sentiment dictionaries, designing rules, and performing feature engineering, achieved satisfactory performance for simpler contexts. However, with the rapid growth of social media and large-scale text data, these traditional methods gradually revealed their limitations in handling complex contexts. The rise of deep learning technology has brought significant progress to sentiment analysis. The introduction of neural networks, especially architectures such as convolutional neural networks and recurrent neural networks, has enabled models to better capture local and long-range dependencies. These deep learning methods perform well in some tasks, but still face challenges in context understanding, particularly in handling complex sentence structures, high ambiguity, and long-range dependencies, where their performance is relatively limited. The limitation of traditional machine learning and deep learning methods lies in their relatively rigid handling of contextual information, making it difficult to adapt to multi-layered structures and rich contexts. This makes it difficult for traditional methods and early deep learning methods to capture key information in texts containing multiple semantic layers and complex structures, thus limiting the accuracy and generalization ability of sentiment analysis.

[0006] In recent years, with the development of deep learning technology, the BERT model, as a powerful pre-trained language model, has achieved remarkable results in natural language processing tasks. Existing sentiment classification methods based on the BERT model include... Figure 1 As shown, compared to traditional models based on recurrent neural networks or convolutional neural networks, the BERT model adopts a Transformer structure and achieves bidirectional encoding of text through a self-attention mechanism, enabling it to better understand the semantic and contextual information of the text.

[0007] However, existing sentiment classification methods based on the BERT model still have problems:

[0008] 1. Existing research using the BERT model for sentiment analysis typically only uses the classifier label vectors from the last encoder layer for classification. This approach relies solely on features extracted from the last layer, ignoring information from other layers. This results in incomplete classification features, making it difficult to handle large-scale classification problems and hindering more specific classification of ambiguous or unclear sentences.

[0009] 2. Existing research methods tend to consider only the linear relationships between features, failing to delve into the complex interactions between features at different levels. This approach limits the model's full utilization of multi-level information and fails to fully explore the richness of multi-level information.

[0010] 3. In the feature integration process, existing methods often neglect the correlation information between feature vectors. This correlation information can include semantic relationships between different parts of the text, contextual information, and mutual influence between features. Fully mining and integrating this correlation information can improve the model's ability to understand text semantics and sentiment.

[0011] 4. Existing dropout methods lack flexibility and cannot effectively adapt to changes in different data distributions and scenarios, resulting in unstable model performance when processing different data, which limits the model's generalization ability and robustness. Summary of the Invention

[0012] To overcome the shortcomings of the prior art, the present invention provides a sentiment analysis method and system based on feature interaction and association fusion.

[0013] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0014] The first aspect of this invention provides a sentiment analysis method based on the fusion of feature interaction and association, comprising:

[0015] Obtain the text data to be recognized;

[0016] The acquired text data is input into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model;

[0017] All intermediate layer representation vectors are input into the multi-layer feature interaction module. By weighting and superimposing the representation vectors of different levels, contextual information is integrated. The importance of each dimension of features in the representation vectors of different levels is dynamically adjusted through a gating mechanism. The weighted superimposed features are fused with the output of the gating mechanism and then weighted and averaged with the initial input intermediate layer representation vectors to obtain the first associated feature fusion vector.

[0018] The first associated feature fusion vector and the final layer CLS representation vector are input into the dynamic association fusion module to obtain the classified text feature vector;

[0019] An adaptive discarding method based on dynamic masking dynamically discards features from the feature vector of classified text, and then obtains the sentiment category of the text to be identified through a fully connected layer and an activation function layer.

[0020] A second aspect of the present invention provides a sentiment analysis system based on feature interaction and association fusion, comprising:

[0021] The data acquisition module is configured to acquire the text data to be recognized.

[0022] The vector representation module is configured to input the acquired text data into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model.

[0023] The first feature fusion module is configured to: input all intermediate layer representation vectors into the multi-layer feature interaction module; integrate context information by weighting and superimposing representation vectors at different levels; dynamically adjust the importance of each dimension of features in the representation vectors at different levels through a gating mechanism; fuse the weighted superimposed features with the output of the gating mechanism; and then perform a weighted average with the initial input intermediate layer representation vectors to obtain the first associated feature fusion vector.

[0024] The second feature fusion module is configured to input the first associated feature fusion vector and the final layer CLS representation vector into the dynamic associated fusion module to obtain the classified text feature vector;

[0025] The text sentiment classification module is configured to dynamically discard features from the feature vector of the classified text using an adaptive discarding method based on dynamic masking, and then obtain the sentiment category of the text to be identified through a fully connected layer and an activation function layer.

[0026] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a sentiment analysis method based on feature interaction and association fusion as described in the first aspect of the present invention.

[0027] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a sentiment analysis method based on feature interaction and association fusion as described in the first aspect of the present invention.

[0028] The above one or more technical solutions have the following beneficial effects:

[0029] (1) This invention uses a dual-branch method to extract the CLS vectors of the last layer encoder as well as the CLS vectors of the intermediate layers encoders. These features not only include the output of the last layer but also cover the semantic representations of the intermediate layers. This invention integrates the output features of multiple layers encoders, which can make full use of the semantic representations of different layers of the BERT model, thereby obtaining richer and more comprehensive feature information, which helps to improve the model's ability to understand text semantics and the accuracy of sentiment analysis.

[0030] (2) This invention proposes a multi-layer feature interaction module (MFI) to realize complex interactions between features at different levels; it integrates contextual information and dynamically adjusts the importance of features by using weighted superposition and gating mechanisms; this enables the model to better understand the semantic structure and emotional expression of the text, and improves the accuracy and robustness of sentiment analysis.

[0031] (3) This invention proposes a bilinear correlation fusion module RFM, which designs a concatenated vector processed by transpose product, linear transformation and activation function, dynamically evaluates the correlation between correlation fusion feature vectors X and Y, and performs a weighted summation of the correlation vector and the output vector to generate a fusion feature vector with richer information and semantic expressive power.

[0032] (4) This invention proposes an adaptive discarding method DMAD, which dynamically discards input feature vectors by designing a random binary mask matrix, effectively improving the model's adaptability to different data distributions and scenarios, and enhancing the model's generalization ability and robustness.

[0033] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0034] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0035] Figure 1 This is a flowchart of an existing sentiment classification method based on the BERT model.

[0036] Figure 2 This is an overall flowchart of a sentiment analysis method based on feature interaction and association fusion, which is the first embodiment.

[0037] Figure 3 This is a detailed step diagram of a sentiment analysis method based on feature interaction and association fusion, as shown in the first embodiment.

[0038] Figure 4 This is a model architecture diagram for the first embodiment.

[0039] Figure 5 This is a flowchart of the multi-layer feature interaction process in the first embodiment. Detailed Implementation

[0040] Example 1

[0041] like Figure 2 , Figure 3 As shown, this embodiment discloses a sentiment analysis method based on feature interaction and association fusion, including:

[0042] Step 1: Obtain the text data to be recognized;

[0043] Step 2: Input the acquired text data into the BERT pre-trained model to obtain the multi-layer classification token (CLS) representation vector, including the representation vectors output by the intermediate layers and the final layer encoder of the BERT pre-trained model;

[0044] Step 3: Input all intermediate layer representation vectors into the multi-layer feature interaction module to capture semantic information at different levels of the input sequence, and dynamically adjust the importance of each dimension of features in the intermediate layer representation vector through a gating mechanism; after concatenating the semantic information, the output of the gating mechanism and the initial input intermediate layer CLS representation vector, the associated feature fusion vector (first associated feature fusion vector) is obtained.

[0045] Step 4: Input the associated feature fusion vector and the final layer representation vector into the dynamic association fusion module to obtain the classified text feature vector;

[0046] Step 5: The adaptive discarding method based on dynamic masking dynamically discards features from the feature vector of the classified text, and then passes it through a fully connected layer and an activation function layer to obtain the sentiment category of the text to be identified.

[0047] In step 1, after obtaining the text data to be recognized, the obtained text data is preprocessed, mainly including word segmentation, removal of stop words, addition of special markers, etc., and then converted into the input format required by the BERT model.

[0048] In step 2, the preprocessed text data is input into the BERT pre-trained model to extract features and obtain the feature vector output by BERT; including the CLS representation vectors output by all intermediate layer encoders of the BERT pre-trained model, and the CLS representation vector output by the final layer encoder; where all intermediate layer CLS representation vectors refer to the outputs of all encoder layers of the BERT pre-trained model except the last layer encoder, that is, the outputs of the first layer to the penultimate layer.

[0049] like Figure 4 As shown, step 2 specifically includes:

[0050] Step 201: After preprocessing the text content, input it into the BERT model to extract features:

[0051] BERT_Input

[0052] =Embed(Text,Position_Embeddings,Segment_Embeddings)

[0053] Here, Text represents the preprocessed representation of the text content, Position_Embeddings represents the embedded representation of the position vector, and Segment_Embeddings represents the embedded representation of the segmentation information.

[0054] Step 202: Obtain the feature vectors output by BERT, including the final layer CLS representation vector and all intermediate layer CLS representation vectors.

[0055] Final_CLS = BERT_Output Last_Layer

[0056] Inter_CLS i =BERT_OutputLayer i

[0057] Among them, BERT_Output Last_LayerThis represents the output of the last layer of the BERT model, where i represents the intermediate layer number of the BERT model, from the first layer to the second to last. BERT_OutputLayer i This represents the output of the i-th layer of the BERT model.

[0058] In step 3, the CLS representation vectors output by all intermediate layer encoders generated in step 2 are input into the designed multi-layer feature interaction module MFI. Through complex multi-layer feature interaction, the semantic information of different levels of the input sequence is captured, richer semantic representation is obtained, and the first associated feature fusion vector X is constructed.

[0059] The multi-layer feature interaction module includes: an input layer, a gating mechanism layer, a feature interaction layer, a weighted average layer, and an output layer connected in sequence.

[0060] Input layer: Used to input multiple CLS representation vectors obtained from the intermediate layers of BERT into the next layer.

[0061] Gating mechanism layer: Used to adjust the importance of features in the CLS representation vector through gating mechanism.

[0062] Feature interaction layer: used to weight and superimpose representation vectors from different levels to integrate contextual information.

[0063] Weighted average layer: used to perform a weighted average of the interaction results of multiple features and add it to the initial vector.

[0064] Output layer: Used to output the first associated fusion feature vector generated.

[0065] like Figure 5 As shown, step 3 specifically includes:

[0066] Step 301: Input the obtained CLS feature vectors from all intermediate layers and format them. Key steps in formatting include dimension alignment, data type conversion, normalization, and concatenation. These steps ensure that the input vectors are compatible in shape, type, and scale. Then, concatenate them with their context information and dynamically control the importance of different intermediate layer features through a gating mechanism. This can be expressed as the following formula:

[0067] Input layer:

[0068] S i =Inter_CLS i ;

[0069] C = S i-1 ;

[0070] Gating mechanism layer:

[0071]

[0072] Among them, S i This represents the CLS representation vector of the current layer (the i-th layer); C = S i-1 This represents the CLS feature vector of the previous layer (i.e., the (i-1)th layer), which carries semantic information from the previous layer to help the current layer better understand the contextual relationships.

[0073] σ represents the sigmoid activation function; wg1, wg2, ..., wg n These are the learning parameters used in the gating mechanism, used to perform a linear transformation for each element of the input; [Si; C] represents the feature representation S of the current layer. i Concatenate with the context information C; Gate i The output of the gating mechanism is used to dynamically adjust the importance of each element in the concatenated vector. By strengthening or suppressing certain features through the gating mechanism, the model can ensure that it focuses on the most important parts.

[0074] Among them, the importance of each dimension of features in the intermediate layer CLS representation vector is dynamically adjusted through a gating mechanism, such as the importance of key features, secondary features, redundant features, context features, auxiliary features and other multi-dimensional features.

[0075] The gating mechanism processing steps are as follows:

[0076] (1) Input concatenation: The current layer's CLS representation vector S i It is concatenated with the context information C (the output of the previous layer) to form a new vector; the purpose is to combine the features of the current layer with the context information.

[0077] (2) Linear transformation: The concatenated vector is passed through a fully connected layer, that is, a linear transformation is performed using a set of weight matrices; the linear transformation is the weighted summation operation of the features.

[0078] (3) Activation Function: The result of the linear transformation is processed by a Sigmoid activation function σ, with the output value between 0 and 1. This value serves as the gate signal. i Used to determine input features S i The importance of each dimension.

[0079] Wherein, CLS represents the feature sentiment dimension of the vector, which includes:

[0080] S i [0] indicates the relevance of the "plot"; S i [1] indicates the relevance of the "outcome"; S i [2] Indicates a feeling of "wonderful"; S i[3] indicates the emotional tendency of "disappointment"; other dimensions may indicate some sentence structural features, grammatical features, etc.

[0081] (4) Output of the gating mechanism:

[0082] Assuming the weight vector Gate is obtained after the gating mechanism is applied... i It is [0.8, 0.3, 0.7, 0.9];

[0083] The model suggests that the relevance of "plot" (dimensional 1) and the emotion of "disappointment" (dimensional 4) are more important in the classification, while the relevance of "ending" (dimensional 2) has a lower weight.

[0084] (5) Adjusted CLS representation vector:

[0085] The output of the gating mechanism is then fused with the original input CLS representation vector to obtain the adjusted CLS representation vector, thus avoiding information loss. In other words, the adjusted CLS representation vector is obtained by element-wise multiplication of the original CLS representation vector with the gating weight vector. Si =S i ⊙Gate i

[0086] For example, suppose the original CLS vector S i =[0.5,0.6,0.7,0.9], after adjustment by the gating mechanism: Adjusted Si =[0.5×0.8,0.6×0.3,0.7×0.7,0.9×0.9] =[0.4,0.18,0.49,0.81]

[0087] This means that the dimension of "disappointment" is strengthened in the final classification decision, while the features related to "outcome" are weakened.

[0088] In summary, this invention achieves complex interactions between features at different levels by introducing a gating mechanism to dynamically adjust the importance of features at different intermediate layers. This enables the model to better understand the semantic structure and sentiment expression of text, resulting in more accurate service matching.

[0089] Step 302: Perform multi-layer feature complex interactions on the obtained input representation, context information, and gating mechanism output, specifically including:

[0090] (1) Linear transformation and activation:

[0091] First, the current layer's CLS representation vector (S) iThe information (C) and context information (Ws) are transformed by linear transformation matrices (Ws and Wc) and nonlinear transformation by ReLU activation function to learn complex nonlinear relationships.

[0092] (2) Feature Weighting and Integration: The feature representation after linear transformation is scaled and normalized using the Softmax function, and then combined with the current layer's CLS representation vector and the output of the gate mechanism. i Element-wise multiplication (⊙) is performed to dynamically adjust the importance of features, resulting in a new interactive feature vector Z. i .

[0093] (3) Feature concatenation and re-interaction: Finally, the original CLS representation vector (S i The interaction with the feature vector Z i The data is concatenated and then further processed using a linear transformation matrix Wh and a ReLU activation function to obtain the final output feature H. i .

[0094] The above feature interaction process helps the model better capture the complex relationships between features, which can ultimately be expressed as the following formula:

[0095]

[0096] H i =ReLU(W h ·[S i Z i] ])

[0097] Among them, S i CLS represents the vector of the current layer; C represents the context information; Wz, Ws, W c This represents the learning parameter matrix used for the linear transformation; represents the scaling factor; ReLU and Softmax represent activation functions; ⊙ represents the element-wise multiplication operation, which can dynamically adjust the importance of each dimension in the CLS feature vectors of different layers; Z i This represents the result obtained after the current layer feature Si has undergone multiple feature interactions, including the integration of contextual information and dynamic adjustment of feature importance; W h This represents the learning parameter matrix used for feature interaction, obtained through learning; [S] i Z i ] indicates a splicing operation; H i This represents the output of multi-layer feature interaction, which is the current layer feature S. i The feature Z obtained after feature interaction i The results obtained after further interaction can enable the model to better capture the complex relationships between features.

[0098] In summary, this invention effectively captures semantic information at different levels of the input sequence through multi-layer feature interaction, making feature extraction more comprehensive and helping to improve the model's ability to understand text semantics and the accuracy of sentiment analysis.

[0099] Step 303: Output H of multi-layer feature interaction i Perform a weighted average, and then add the initial intermediate layer representation S. i This yields the final first associated fusion feature vector X; the second associated fusion feature vector Y is the CLS representation vector output by the final layer encoder, which can be expressed by the following formula:

[0100] X = Σ i H i +S i

[0101] Y = Final_CLS

[0102] In step 4, the first associated feature fusion vector X and the final layer CLS representation vector Y are input into the dynamic association fusion module to obtain the classified text feature vector.

[0103] The dynamic correlation fusion module includes: a bilinear correlation calculation module, a weight calculation module, and a weighted summation and vector fusion module. Step 4 specifically includes:

[0104] Step 401: Input the correlation fusion feature vectors X and Y into the Dynamic Correlation Fusion (RFM) module. First, through bilinear correlation calculation, capture their dynamic relationship, mutual influence, and better fuse their information, which can be expressed as the following formula:

[0105] e = X T WY+σ(Linear(W1·[Concatenate(X,Y)]+b1))

[0106] e′=Y T W T X+σ(Linear(W1·[Concatenate(X,Y)]+b1))

[0107] Where X and Y represent the input association fusion feature vectors, X T and Y T Let W represent the transpose of X and Y, and W1 and W2 represent the weight matrices. T denoted as the transpose of W, concatenate represents the concatenation operation, linear represents the linear transformation, b1 represents the bias term, σ represents the ReLU activation function, and e and e′ represent the scores obtained from the bilinear association calculation, used to measure the association between X and Y.

[0108] Step 402: Calculate the weights by applying the softmax function, then apply them to X and Y for weighted summation to obtain the weighted X and Y vectors, which can be expressed as the following formula:

[0109]

[0110] Where e and e′ represent the scores calculated by bilinear association, exp() represents the exponential function, k is the index in e or e′, represents each element in the attention score, i represents the element index in the X vector, j represents the element index in the Y vector, and X weighted and Y weighted This represents the weighted X and Y vectors.

[0111] Step 403: Fuse the weighted vectors to generate the categorical text feature vector Z, which can be expressed by the following formula:

[0112] Z = [X] weighted ,Y weighted ]

[0113] In step 5, the classified text feature vector Z obtained in step 4 is processed by an adaptive discarding method based on dynamic masking (DMAD), which dynamically adjusts the discarding rate according to the features of the input data.

[0114] Step 5 specifically includes:

[0115] Step 501: Calculate the dropout rate p for each feature in the categorized text feature vector. i

[0116] (1) Feature strength calculation: First, for each feature vector Z :,i Calculate its L2 norm, i.e., ||Z :,i ||2, to measure the importance or intensity of a feature.

[0117] (2) Nonlinear transformation: This characteristic intensity is transformed by a linear transformation (using the parameter matrix W). p and bias vector b p The process is performed to obtain the linearly transformed result W. p ·Z :,i +b p .

[0118] (3) Normalization: The calculated value is normalized to its maximum value to scale the range of features.

[0119] (4) Dynamic adjustment term: A learning parameter α is introduced on the normalized value to control the degree of adjustment.

[0120] (5) ReLU activation: The result is activated by the ReLU function so that only the positive values ​​are retained.

[0121] (6) Dropout rate: Finally, the results are mapped to the [0,1] range using the Sigmoid activation function to obtain the dropout rate p for each feature. i .

[0122] Different input features will result in different dropout rates based on their feature strength and the model's learning outcomes. The dropout rate p i The calculation process can be expressed as the following formula:

[0123]

[0124] Where, p i W represents the discard rate of the i-th feature; p Let b represent the parameter matrix used for learning. p Z represents the bias vector; :,i The i-th column of the input feature vector Z represents the maximum value function; ReLU and sigmoid represent activation functions; ||Z :,i ||2 represents the L2 norm of the i-th column in the feature vector Z, and α represents the learning parameter used to adjust the size of the dynamic adjustment term;

[0125] The dynamic adjustment term refers to the learning parameter α and the feature strength α·||Z used when calculating the dropout rate. :,i The product of ||2. This term dynamically adjusts the dropout rate for each feature to better adapt to the characteristics of the input data. It automatically adjusts the dropout rate based on the strength of the input features, enhancing the model's flexibility and robustness under different feature distributions.

[0126] In summary, this invention dynamically evaluates the correlation between feature vectors through bilinear correlation calculation and then weights and sums the correlation information with the output vector to generate a more informative and semantically expressive fused feature vector. This feature fusion method helps improve the model's ability to understand text semantics and sentiment, further enhancing the accuracy of sentiment analysis.

[0127] Step 502: Generate a random binary mask matrix based on the discard rate of each feature, and use the random binary mask matrix to randomly discard the feature vectors of the classified text.

[0128] Specifically: based on the discard rate p for each feature i Generating a random binary mask matrix M to randomly discard input feature vector Z can be expressed as follows:

[0129]

[0130] Among them, M i,j p represents the element in the i-th row and j-th column of the mask matrix M. i Let r represent the discard rate of the i-th feature, where r is a random number in the range [0,1].

[0131] Specifically: based on the discard rate p calculated in step 501 i For each feature Z(:,i), a random binary mask matrix M is generated. Mask matrix M i,j Each element is randomly generated, satisfying the following rule: if a random number r is less than or equal to the discard rate p of that feature... i If the matrix element M(i,j) is 0, then M(i,j) = 0; otherwise, M(i,j) = 1.

[0132] The generated mask matrix M is multiplied element-wise with the input feature vector Z (i.e., Z ⊙ M). For elements of M(i,j) = 0, it means that the corresponding Z(i,j) features will be discarded (set to zero), while elements of M(i,j) = 1 retain the original feature values.

[0133] The obtained Z⊙M is the feature vector after random discarding. This step effectively reduces the impact of unimportant or redundant features, improving the model's generalization ability.

[0134] Step 503: Randomly discard elements from the input feature vector Z according to the random binary mask matrix M, and then input it into the fully connected layer for linear transformation. After passing through the softmax activation function, the final classification probability distribution is obtained.

[0135]

[0136] Where Z represents the input categorized text feature vector, M represents the random binary mask matrix, and ⊙ represents element-wise multiplication. This represents the vector of categorized text features after random discarding. `Linear()` represents a linear transformation, and `sigmoid` represents the activation function. This represents the final classification probability distribution.

[0137] The sentiment category of the text to be identified includes:

[0138] Positive: This indicates that the text conveys positive emotions, such as happiness, satisfaction, excitement, etc.

[0139] Negative: This indicates that the text conveys negative emotions, such as anger, sadness, or disappointment.

[0140] Neutral: This indicates that the text does not have a clear emotional bias, or the emotional expression is relatively bland.

[0141] In summary, this invention effectively improves the model's adaptability to different data distributions and scenarios by dynamically adjusting the dropout rate. This makes the model more stable when processing different data, enhancing its generalization ability and robustness.

[0142] Example 2

[0143] This embodiment discloses a sentiment analysis system based on feature interaction and association fusion, including:

[0144] The data acquisition module is configured to acquire the text data to be recognized.

[0145] The vector representation module is configured to input the acquired text data into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model.

[0146] The first feature fusion module is configured to: input all intermediate layer representation vectors into the multi-layer feature interaction module; integrate context information by weighting and superimposing representation vectors at different levels; dynamically adjust the importance of each dimension of features in the representation vectors at different levels through a gating mechanism; fuse the weighted superimposed features with the output of the gating mechanism; and then perform a weighted average with the initial input intermediate layer representation vectors to obtain the first associated feature fusion vector.

[0147] The second feature fusion module is configured to input the first associated feature fusion vector and the final layer CLS representation vector into the dynamic associated fusion module to obtain the classified text feature vector;

[0148] The text sentiment classification module is configured to dynamically discard features from the feature vector of the classified text using an adaptive discarding method based on dynamic masking, and then obtain the sentiment category of the text to be identified through a fully connected layer and an activation function layer.

[0149] Example 3

[0150] The purpose of this embodiment is to provide a computer-readable storage medium.

[0151] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a sentiment analysis method based on feature interaction and association fusion as described in Embodiment 1 of this disclosure.

[0152] Example 4

[0153] The purpose of this embodiment is to provide an electronic device.

[0154] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in a sentiment analysis method based on feature interaction and association fusion as described in Embodiment 1 of this disclosure.

[0155] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0156] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0157] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for sentiment analysis based on feature interaction and association fusion, characterized in that, include: Obtain the text data to be recognized; The acquired text data is input into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model; All intermediate layer representation vectors are input into the multi-layer feature interaction module. By weighting and superimposing the representation vectors of different levels, contextual information is integrated. The importance of each dimension of features in the representation vectors of different levels is dynamically adjusted through a gating mechanism. The weighted superimposed features are fused with the output of the gating mechanism and then weighted and averaged with the initial input intermediate layer representation vectors to obtain the first associated feature fusion vector. The first associated feature fusion vector and the final layer CLS representation vector are input into the dynamic association fusion module to obtain the classified text feature vector, including: The correlation scores of the first correlation feature fusion vector and the final layer CLS representation vector are calculated using the bilinear correlation calculation method. Using the correlation score, the elements in the first correlation feature fusion vector and the final CLS representation vector are weighted and fused respectively to obtain the weighted fused first correlation feature fusion vector and the final CLS representation vector. The first associated feature fusion vector after weighted fusion and the final CLS representation vector are concatenated to obtain the classified text feature vector. An adaptive discarding method based on dynamic masking dynamically discards features from the feature vector of classified text, and then obtains the sentiment category of the text to be identified through a fully connected layer and an activation function layer. The adaptive discarding method based on dynamic masking dynamically discards features from the feature vector of classified text, including: Calculate the discard rate of each feature in the feature vector of the classified text; A random binary mask matrix is ​​generated based on the discard rate of each feature, and the random binary mask matrix is ​​used to randomly discard the feature vectors of the classified text.

2. The sentiment analysis method based on feature interaction and association fusion as described in claim 1, characterized in that, The calculation of the correlation score between the first correlation feature fusion vector and the final layer CLS representation vector using the bilinear correlation calculation method includes: Calculate the transpose of the first associated feature fusion vector and the final layer CLS representation vector; Linear transformation and activation function are used to process the concatenated first associated feature fusion vector and the final layer CLS representation vector; The correlation score of the vector is obtained by fusing the transpose product with the concatenated vector after linear transformation and activation function processing.

3. The sentiment analysis method based on feature interaction and association fusion as described in claim 1, characterized in that, The calculation of the discard rate for each feature in the classified text feature vector includes: Calculate the L2 norm of each feature in the classified text feature vector to measure the importance or strength of the feature; After linear transformation and normalization, the L2 norm of each feature is used to control the adjustment intensity by introducing learning parameters; finally, the result is mapped to the range [0,1] through the activation function to obtain the discard rate of each feature.

4. The sentiment analysis method based on feature interaction and association fusion as described in claim 1, characterized in that, The sentiment category of the text to be identified includes: positive: indicating that the text conveys positive sentiment; negative: indicating that the text conveys negative sentiment; and neutral: indicating that the text has no obvious sentiment tendency or the sentiment expression is relatively bland.

5. The sentiment analysis method based on feature interaction and association fusion as described in claim 4, characterized in that, The positive emotions include happiness, satisfaction, and excitement; the negative emotions include anger, sadness, and disappointment.

6. A sentiment analysis system based on feature interaction and association fusion, characterized in that, include: The data acquisition module is configured to acquire the text data to be recognized. The vector representation module is configured to input the acquired text data into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model. The first feature fusion module is configured to: input all intermediate layer representation vectors into the multi-layer feature interaction module; integrate context information by weighting and superimposing representation vectors at different levels; dynamically adjust the importance of each dimension of features in the representation vectors at different levels through a gating mechanism; fuse the weighted superimposed features with the output of the gating mechanism; and then perform a weighted average with the initial input intermediate layer representation vectors to obtain the first associated feature fusion vector. The second feature fusion module is configured to input the first associated feature fusion vector and the final layer CLS representation vector into the dynamic association fusion module to obtain the classified text feature vector, including: calculating the association score of the first associated feature fusion vector and the final layer CLS representation vector through the bilinear association calculation method; Using the correlation score, the elements in the first correlation feature fusion vector and the final CLS representation vector are weighted and fused respectively to obtain the weighted fused first correlation feature fusion vector and the final CLS representation vector. The first associated feature fusion vector after weighted fusion and the final CLS representation vector are concatenated to obtain the classified text feature vector. The text sentiment classification module is configured to: dynamically discard features from the feature vector of the classified text using an adaptive discarding method based on dynamic masking, and then obtain the sentiment category of the text to be identified through a fully connected layer and an activation function layer; The adaptive discarding method based on dynamic masking dynamically discards features from the feature vector of classified text, including: Calculate the discard rate of each feature in the feature vector of the classified text; A random binary mask matrix is ​​generated based on the discard rate of each feature, and the random binary mask matrix is ​​used to randomly discard the feature vectors of the classified text.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the sentiment analysis method based on feature interaction and association fusion as described in any one of claims 1-5.

8. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the sentiment analysis method based on feature interaction and association fusion as described in any one of claims 1-5.

Images