Emotion analysis method and system based on feature interaction and association fusion

Abstract

The invention belongs to the technical field of sentiment analysis, and particularly relates to a sentiment analysis method and system based on feature interaction and association fusion, and the method comprises the steps: inputting obtained text data into a BERT model, and obtaining a multi-layer representation vector; inputting all the intermediate layer representation vectors into a multi-layer feature interaction module to obtain a first associated feature fusion vector; inputting the first association feature fusion vector and the final layer representation vector into a dynamic association fusion module to obtain a classified text feature vector; and carrying out feature discarding on the classified text feature vector by a self-adaptive discarding method based on a dynamic mask, and then obtaining an emotion category of a to-be-recognized text through a full connection layer and an activation function layer. According to the method, feature fusion is carried out on multiple layers of representation vectors, different layers of semantic representations output by the model can be fully utilized, so that richer and more comprehensive feature information is obtained, and the understanding ability of the model on text semantics and the accuracy of sentiment analysis are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of sentiment analysis, and in particular to a sentiment analysis method and system based on feature interaction and association fusion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Sentiment analysis is an important task in the field of natural language processing, which aims to identify and analyze the author's emotional tendencies from text, usually including positive, negative or neutral emotions. With the rapid increase of social media, online reviews and other text data, the demand for effective sentiment analysis methods is becoming increasingly urgent.

[0004] In recent years, with the rapid development of the hydrogen energy industry chain, sentiment analysis technology can be deeply integrated with the user needs of the hydrogen energy industry chain to achieve more accurate service matching and personalized experience. For example, in services for government departments, hydrogen energy companies and end users, sentiment analysis can be used to analyze user feedback on hydrogen energy policies, products and services, thereby building accurate user portraits. This can not only provide decision-making support for government departments, but also provide market insights for hydrogen energy companies, while providing customized service experiences for end users.

[0005] In the early field of sentiment analysis, researchers mainly relied on artificial rules and traditional machine learning methods. These methods are based on building sentiment lexicons, designing rules, and performing feature engineering, and have achieved satisfactory performance relative to some simple contexts. However, with the rapid growth of social media and large-scale text data, these traditional methods have gradually revealed their limitations in dealing with complex contexts. The rise of deep learning technology has brought significant progress to sentiment analysis. The introduction of neural networks, especially convolutional neural networks and recurrent neural networks, enables models to better capture local and long-distance dependencies. These deep learning methods perform well in some tasks, but there are still challenges in context understanding, especially when dealing with complex sentence structures, high ambiguity, and long-distance dependencies. The limitation of traditional machine learning methods and deep learning methods is that they still appear to be relatively rigid in processing context information and are difficult to adapt to multi-level structures and rich contexts. This makes it difficult for traditional methods and early deep learning methods to capture key information in texts containing multiple semantic levels and complex structures, resulting in limited accuracy and generalization of sentiment analysis.

[0006] In recent years, with the development of deep learning technology, the BERT model, as a powerful pre-trained language model, has achieved remarkable results in natural language processing tasks. Existing sentiment classification methods based on the BERT model include Figure 1 As shown in the figure, compared with traditional models based on recurrent neural networks or convolutional neural networks, the BERT model adopts the Transformer structure and realizes bidirectional encoding of text through the self-attention mechanism, enabling it to better understand the semantics and contextual information of the text.

[0007] However, the existing sentiment classification methods based on the BERT model still have problems:

[0008] 1. Existing studies usually only use the classifier tag vector of the last layer of encoder for classification when using the BERT model for sentiment analysis. This approach only relies on the features extracted from the last layer and ignores the information of other layers, resulting in incomplete classification features, making it difficult to handle large-scale classification problems, and not conducive to more specific classification of ambiguous or unclear sentences.

[0009] 2. Existing research methods tend to only consider the linear relationship between features, but fail to deeply explore the complex interactions between features at different levels. This approach limits the model's full utilization of multi-layer information and fails to fully explore the richness of multi-layer information.

[0010] 3. In the process of feature integration, existing methods often ignore the correlation information between feature vectors, which can include semantic relationships between different parts of the text, contextual information, and the mutual influence between features. Fully mining and integrating this correlation information can improve the model's ability to understand text semantics and emotions.

[0011] 4. Existing discarding methods lack flexibility and cannot effectively adapt to changes in different data distributions and scenarios, resulting in unstable performance of the model when processing different data, which limits the generalization ability and robustness of the model. Summary of the invention

[0012] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a sentiment analysis method and system based on feature interaction and association fusion.

[0013] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0014] The first aspect of the present invention provides a sentiment analysis method based on feature interaction and association fusion, comprising:

[0015] Obtaining text data to be recognized;

[0016] Input the acquired text data into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model;

[0017] All intermediate layer representation vectors are input into the multi-layer feature interaction module. The context information is integrated by weighted superposition of representation vectors at different levels. The importance of each dimension feature in the representation vectors at different levels is dynamically adjusted through the gating mechanism. The weighted superposition features are fused with the output of the gating mechanism, and then weighted averaged with the initial input intermediate layer representation vector to obtain the first associated feature fusion vector.

[0018] The first correlation feature fusion vector and the final layer CLS representation vector are input into the dynamic correlation fusion module to obtain the classified text feature vector;

[0019] The adaptive discarding method based on dynamic mask dynamically discards the features of the classified text feature vector, and then obtains the sentiment category of the text to be identified through the fully connected layer and the activation function layer.

[0020] A second aspect of the present invention provides a sentiment analysis system based on feature interaction and association fusion, comprising:

[0021] The data acquisition module is configured to: acquire text data to be recognized;

[0022] The vector representation module is configured to: input the acquired text data into the BERT model to obtain a multi-layer representation vector, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model;

[0023] The first feature fusion module is configured to: input all the intermediate layer representation vectors into the multi-layer feature interaction module, integrate the context information by weighted superposition of the representation vectors of different layers, and dynamically adjust the importance of the features of each dimension in the representation vectors of different layers through the gating mechanism; fuse the weighted superposition features with the output of the gating mechanism, and perform weighted averaging with the initially input intermediate layer representation vector to obtain a first associated feature fusion vector;

[0024] The second feature fusion module is configured to: input the first correlation feature fusion vector and the final layer CLS representation vector into the dynamic correlation fusion module to obtain a classified text feature vector;

[0025] The text sentiment classification module is configured as follows: a dynamic mask-based adaptive discarding method is used to dynamically discard the features of the classified text feature vector, and then the sentiment category of the text to be identified is obtained through a fully connected layer and an activation function layer.

[0026] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a sentiment analysis method based on feature interaction and association fusion as described in the first aspect of the present invention.

[0027] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the sentiment analysis method based on feature interaction and association fusion as described in the first aspect of the present invention are implemented.

[0028] One or more of the above technical solutions have the following beneficial effects:

[0029] (1) In addition to extracting the CLS vector of the last layer of encoder, the present invention also extracts the CLS vector of each intermediate layer of encoder by using a dual-branch method. These features not only include the output of the last layer, but also cover the semantic representation of the intermediate layer; the present invention integrates the output features of multiple layers of encoders, and can make full use of the semantic representation of different levels of the BERT model, thereby obtaining richer and more comprehensive feature information, which helps to improve the model's ability to understand text semantics and the accuracy of sentiment analysis.

[0030] (2) The present invention proposes a multi-layer feature interaction module MFI, which realizes the complex interaction between features at different levels; it uses weighted superposition and gating mechanism methods to integrate contextual information and dynamically adjust the importance of features; it enables the model to better understand the semantic structure and emotional expression of the text, thereby improving the accuracy and robustness of sentiment analysis.

[0031] (3) The present invention proposes a bilinear correlation fusion module RFM, designs a concatenated vector processed by transpose product, linear transformation and activation function, dynamically evaluates the correlation between the correlation fusion feature vectors X and Y, and performs weighted summation of the correlation vector and the output vector to generate a fusion feature vector with richer information and more semantic expression ability.

[0032] (4) The present invention proposes an adaptive discarding method DMAD, which dynamically discards input feature vectors by designing a random binary mask matrix, effectively improving the model's adaptability to different data distributions and scenarios, and enhancing the model's generalization ability and robustness.

[0033] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0035] Figure 1 This is a flowchart of the existing sentiment classification method based on the BERT model.

[0036] Figure 2 This is an overall flow chart of a sentiment analysis method based on feature interaction and association fusion in the first embodiment.

[0037] Figure 3 This is a specific step diagram of a sentiment analysis method based on feature interaction and association fusion in the first embodiment.

[0038] Figure 4 This is a model architecture diagram of the first embodiment.

[0039] Figure 5 This is a multi-layer feature interaction flowchart of the first embodiment. DETAILED DESCRIPTION

[0040] Embodiment 1

[0041] like Figure 2 , Figure 3 As shown, this embodiment discloses a sentiment analysis method based on feature interaction and association fusion, including:

[0042] Step 1: Obtain the text data to be recognized;

[0043] Step 2: Input the acquired text data into the BERT pre-trained model to obtain the multi-layer classification token (CLS) representation vector, including the representation vector output by the middle layer and final layer encoder of the BERT pre-trained model;

[0044] Step 3: Input all the intermediate layer representation vectors into the multi-layer feature interaction module to capture the semantic information of different levels of the input sequence, and dynamically adjust the importance of each dimension feature in the intermediate layer representation vector through the gating mechanism; after splicing the semantic information, the output of the gating mechanism and the initial input intermediate layer CLS representation vector, the associated feature fusion vector (the first associated feature fusion vector) is obtained;

[0045] Step 4: Input the associated feature fusion vector and the final layer representation vector into the dynamic associated fusion module to obtain the classified text feature vector;

[0046] Step 5: The feature vector of the classified text is dynamically discarded based on the adaptive discarding method of dynamic mask, and then the sentiment category of the text to be identified is obtained by passing through the fully connected layer and the activation function layer in sequence.

[0047] In step 1, after obtaining the text data to be recognized, the obtained text data is preprocessed, mainly including word segmentation, stop word removal, special tag addition, etc., and converted into the input format required by the BERT model.

[0048] In step 2, the preprocessed text data is input into the BERT pre-trained model to extract features and obtain the feature vector of BERT output; including the CLS representation vectors of all intermediate layer encoder outputs of the BERT pre-trained model, and the CLS representation vectors of the final layer encoder output; among which, all intermediate layer CLS representation vectors refer to the outputs of all encoder layers of the BERT pre-trained model except the last layer encoder, that is, the outputs from the first layer to the second to last layer.

[0049] like Figure 4 As shown, step 2 specifically includes:

[0050] Step 201: After preprocessing, the text content is input into the BERT model to extract features:

[0051] BERT_Input

[0052] =Embed(Text,Position_Embeddings,Segment_Embeddings)

[0053] Among them, Text represents the representation of the text content after preprocessing, Position_Embeddings represents the embedded representation of the position vector, and Segment_Embeddings represents the embedded representation of the segmentation information.

[0054] Step 202: Obtain the feature vector output by BERT, including the final layer CLS representation vector and all intermediate layer CLS representation vectors.

[0055] Final_CLS=BERT_Output Last_Layer

[0056] Inter_CLS i =BERT_OutputLayer i

[0057] Among them, BERT_Output Last_LayerRepresents the last layer output of the BERT model, i represents the intermediate layer number of the BERT model, from the first layer to the second to last layer, BERT_OutputLayer i Represents the i-th layer output of the BERT model.

[0058] In step 3, the CLS representation vectors output by all intermediate layer encoders generated in step 2 are input into the designed multi-layer feature interaction module MFI. Through the complex interaction of multi-layer features, the semantic information of different levels of the input sequence is captured, richer semantic representation is obtained, and the first associated feature fusion vector X is constructed.

[0059] Among them, the multi-layer feature interaction module includes: a sequentially connected input layer, a gating mechanism layer, a feature interaction layer, a weighted average layer, and an output layer.

[0060] Input layer: used to input multiple CLS representation vectors obtained from the BERT intermediate layer to the next layer.

[0061] Gating mechanism layer: used to adjust the importance of features in the CLS representation vector through a gating mechanism.

[0062] Feature interaction layer: used to weightedly superimpose representation vectors at different levels to integrate contextual information.

[0063] Weighted average layer: used to perform weighted average of multiple feature interaction results and add them to the initial vector.

[0064] Output layer: used to output the generated first associated fused feature vector.

[0065] like Figure 5 As shown, step 3 specifically includes:

[0066] Step 301: The CLS feature vectors of all intermediate layers are input and formatted. The key steps of formatting include dimension alignment, data type conversion, normalization and concatenation operations. These steps ensure that the input vectors are compatible in shape, type and scale. Then they are concatenated with their context information and the importance of different intermediate layer features is dynamically controlled through the gating mechanism, which can be expressed as the following formula:

[0067] Input Layer:

[0068] S i =Inter_CLS i ;

[0069] C=S i-1 ;

[0070] Gating mechanism layer:

[0071]

[0072] Among them, S i Represents the CLS representation vector of the current layer (i-th layer); C = S i-1 The CLS feature vector representing the previous layer (i.e., layer i-1) is used to carry the semantic information from the previous layer to help the current layer better understand the contextual relationship.

[0073] σ represents the sigmoid activation function; wg1, wg2, …, wg n It is a learning parameter for the gating mechanism, which is used to perform a linear transformation on each element of the corresponding input; [Si; C] represents the feature representation S of the current layer. i and context information C; Gate i Represents the output of the gating mechanism, which is used to dynamically adjust the importance of each element in the concatenated vector. The gating mechanism is used to strengthen or suppress certain features to ensure that the model focuses on the most important parts.

[0074] Among them, the importance of each dimensional feature in the intermediate layer CLS representation vector is dynamically adjusted through a gating mechanism, such as the importance of key features, secondary features, redundant features, contextual features, auxiliary features and other multi-dimensional features.

[0075] The gating mechanism processing steps are:

[0076] (1) Input concatenation: The CLS representation vector S of the current layer is i It is concatenated with the context information C (the output of the previous layer) into a new vector; the purpose is to combine the features of the current layer with the context information.

[0077] (2) Linear transformation: The concatenated vector passes through a fully connected layer, which uses a set of weight matrices to perform linear transformation; linear transformation is the weighted summation operation of the features.

[0078] (3) Activation function: The result of the linear transformation is processed by a Sigmoid activation function σ, and the output value is between 0 and 1. This value is used as the gate signal Gate i , used to determine the input feature S i The importance of each dimension.

[0079] Among them, the characteristic sentiment dimensions of the CLS representation vector include:

[0080] S i [0] indicates the relevance of the “plot”; S i [1] indicates the relevance of “outcome”; S i [2] Indicates the emotional tendency of "wonderful"; S i[3] Indicates the emotional tendency of "disappointment"; other dimensions may represent the structural features and grammatical features of some sentences.

[0081] (4) Output of the gating mechanism:

[0082] Assume that the weight vector Gate obtained after the gating mechanism is calculated i is [0.8, 0.3, 0.7, 0.9];

[0083] This means that the model considers the relevance of “plot” (dimension 1) and the emotion of “disappointment” (dimension 4) to be more important in the classification, while the relevance of “ending” (dimension 2) has a lower weight.

[0084] (5) The adjusted CLS representation vector:

[0085] The output of the gating mechanism is then fused with the original input CLS representation vector to obtain the adjusted CLS representation vector to avoid information loss. In other words, the adjusted CLS representation vector is obtained by multiplying the original CLS representation vector and the gating weight vector element by element, that is: Si =S i ⊙Gate i

[0086] For example, assuming the original CLS vector S i =[0.5,0.6,0.7,0.9], after adjustment by the gating mechanism: Adjusted Si =[0.5×0.8,0.6×0.3,0.7×0.7,0.9×0.9]=[0.4,0.18,0.49,0.81]

[0087] This means that the dimension of “disappointment” is strengthened in the final classification decision, while the features related to “outcome” are weakened.

[0088] In summary, the present invention dynamically adjusts the importance of different intermediate layer features by introducing a gating mechanism, thereby achieving complex interactions between features at different levels. This enables the model to better understand the semantic structure and emotional expression of the text, and achieve more accurate service matching.

[0089] Step 302: Perform multi-layer feature complex interaction on the acquired input representation, context information, and the output of the gating mechanism, specifically including:

[0090] (1) Linear transformation and activation:

[0091] First, the CLS representation vector (S i) and context information (C) are transformed by linear transformation matrices (Ws and Wc) respectively, and nonlinearly transformed by ReLU activation function to learn complex nonlinear relationships;

[0092] (2) Feature weighting and integration: The linearly transformed feature representation is scaled and normalized using the Softmax function, and then combined with the CLS representation vector of the current layer and the output of the gating mechanism (Gate i ) element-by-element multiplication (⊙) to dynamically adjust the importance of the features and obtain a new interactive feature vector Z i .

[0093] (3) Feature concatenation and reinteraction: Finally, the original CLS representation vector (S i ) and the eigenvector Z after interaction i After concatenation, the linear transformation matrix Wh and the ReLU activation function are further interactively processed to obtain the final output feature H i .

[0094] The above feature interaction process helps the model better capture the complex relationship between features, which can eventually be expressed as the following formula:

[0095]

[0096] H i =ReLU(W h ·[S i ; Z i] ])

[0097] Among them, S i represents the CLS representation vector of the current layer; C represents the context information; Wz, Ws, W c represents the learning parameter matrix for linear transformation; represents the scaling factor; ReLU and Softmax represent activation functions; ⊙ represents the element-by-element multiplication operation, which can dynamically adjust the importance of each dimension in the CLS feature vector of different layers; Z i It represents the result obtained after the interaction of multiple layers of features of the current layer Si, including the integration of context information and the dynamic adjustment of feature importance; W h represents the learning parameter matrix for feature interaction, obtained through learning; [S i ; Z i ] indicates splicing operation; H i Represents the output of multi-layer feature interaction, which is the current layer feature S i And the feature Z obtained after feature interaction i The results obtained after further interaction can enable the model to better capture the complex relationships between features.

[0098] In summary, the present invention effectively captures the semantic information of different levels of the input sequence through multi-layer feature interaction, making feature extraction more comprehensive, which helps to improve the model's ability to understand text semantics and the accuracy of sentiment analysis.

[0099] Step 303: Output H of multi-layer feature interaction i Perform weighted averaging and add the initial intermediate layer representation S i , and the final first associated fusion feature vector X is obtained; the second associated fusion feature vector Y is the CLS representation vector output by the final layer encoder, which can be expressed as the following formula:

[0100] X=Σ i H i +S i

[0101] Y=Final_CLS

[0102] In step 4, the first association feature fusion vector X and the final layer CLS representation vector Y are input into the dynamic association fusion module to obtain the classified text feature vector.

[0103] The dynamic association fusion module includes: a bilinear association calculation module, a weight calculation module, and a weighted summation and vector fusion module. Step 4 specifically includes:

[0104] Step 401: Input the associated fusion feature vectors X and Y into the dynamic associated fusion module RFM. First, through bilinear association calculation, their dynamic relationship and mutual influence are captured, and the information of the two is better integrated, which can be expressed as the following formula:

[0105] e=X T WY+σ(Linear(W1·[Concatenate(X,Y)]+b1))

[0106] e′=Y T W T X+σ(Linear(W1·[Concatenate(X,Y)]+b1))

[0107] Among them, X and Y represent the associated fusion feature vector of the input, X T and Y T represents the transpose of X and Y, W and W1 represent the weight matrix, W T represents the transpose of W, Concatenate represents the concatenation operation, Linear represents the linear transformation, b1 represents the bias term, σ represents the ReLU activation function, e and e′ represent the scores calculated by the bilinear association, which are used to measure the association between X and Y.

[0108] Step 402: Calculate the weights by applying the softmax function, and then apply them to X and Y for weighted summation to obtain the weighted X vector and Y vector, which can be expressed as the following formula:

[0109]

[0110] Among them, e and e′ represent the scores calculated by bilinear association, exp() represents the exponential function, k is the index in e or e′, representing each element in the attention score, i represents the element index in the X vector, j represents the element index in the Y vector, and X weighted and Y weighted Represents the weighted X and Y vectors.

[0111] Step 403: The weighted vectors are fused to generate a classified text feature vector Z, which can be expressed as the following formula:

[0112] Z=[X weighted ,Y weighted ]

[0113] In step 5, the classified text feature vector Z obtained in step 4 is subjected to a dynamic mask-based adaptive discarding method DMAD to dynamically adjust the discarding rate according to the characteristics of the input data.

[0114] Step 5 specifically includes:

[0115] Step 501: Calculate the discard rate p of each feature in the classified text feature vector i

[0116] (1) Feature strength calculation: First, for each feature vector Z :,i Calculate its L2 norm, i.e. ||Z :,i ||2, to measure the importance or strength of the feature.

[0117] (2) Nonlinear transformation: This feature intensity is transformed through a linear transformation (using the parameter matrix W p and the bias vector b p ) to obtain the result W after linear transformation p ·Z :,i +b p .

[0118] (3) Normalization: The calculated value is normalized with its maximum value to scale the range of the feature.

[0119] (4) Dynamic adjustment term: A learning parameter α is introduced on the normalized value to control the intensity of the adjustment.

[0120] (5) ReLU activation: The result is passed through the ReLU activation function so that only the positive value is retained.

[0121] (6) Discard rate: Finally, the result is mapped to the range [0, 1] through the Sigmoid activation function to obtain the discard rate p of each feature. i .

[0122] Different input features will have different discard rates according to their feature strength and the learning results of the model. The discard rate p i The calculation process can be expressed as the following formula:

[0123]

[0124] Among them, p i represents the discard rate of the i-th feature; W p represents the parameter matrix used for learning, b p represents the bias vector; Z :,i represents the i-th column in the input feature vector Z; max represents the maximum function; ReLU and sigmoid represent activation functions; ||Z :,i ||2 represents the L2 norm of the i-th column in the feature vector Z, and α represents a learning parameter used to adjust the size of the dynamic adjustment term;

[0125] The dynamic adjustment term refers to the learning parameter α and feature strength α·||Z used in calculating the dropout rate. :,i ||2. This term is used to dynamically adjust the dropout rate of each feature to better adapt it to the characteristics of the input data. It automatically adjusts the dropout rate according to the strength of the input feature, enhancing the flexibility and robustness of the model under different feature distributions.

[0126] In summary, the present invention dynamically evaluates the correlation between feature vectors through bilinear correlation calculation, and performs weighted summation of correlation information and output vector to generate a fused feature vector with more information richness and semantic expression ability. This feature fusion method helps to improve the model's ability to understand text semantics and emotions, and further improves the accuracy of sentiment analysis.

[0127] Step 502: Generate a random binary mask matrix according to the discard rate of each feature, and use the random binary mask matrix to perform a random discard operation on the classified text feature vector.

[0128] Specifically: According to the discard rate p of each feature i Generate a random binary mask matrix M for randomly discarding the input feature vector Z, which can be expressed as the following formula:

[0129]

[0130] Among them, M i,j represents the element in the i-th row and j-th column of the mask matrix M, p i represents the dropout rate of the i-th feature, and r is a random number in the range [0,1].

[0131] Specifically: According to the discard rate p calculated in step 501 i , generate a random binary mask matrix M for each feature Z(:,i). Mask matrix M i,j Each element of is randomly generated and satisfies the following rule: If a random number r is less than or equal to the drop rate p of the feature i , then the matrix element M(i,j)=0; otherwise M(i,j)=1.

[0132] The generated mask matrix M is multiplied element by element with the input feature vector Z (i.e., Z⊙M). For the elements of M(i,j)0, it means that the corresponding Z(i,j) features will be discarded (set to zero), while the elements of M(i,j)=1 retain the original feature values.

[0133] The obtained Z⊙M is the feature vector after random discarding This step effectively weakens the impact of unimportant or redundant features and improves the generalization ability of the model.

[0134] Step 503: Perform a random discard operation on the input feature vector Z according to the random binary mask matrix M, and input it into the fully connected layer for linear transformation, and then pass it through the softmax activation function to obtain the final classification probability distribution

[0135]

[0136] Among them, Z represents the input classification text feature vector, M represents the random binary mask matrix, ⊙ represents element-by-element multiplication, represents the classified text feature vector after random discarding, Linear() represents linear transformation, sigmoid represents activation function, Represents the final classification probability distribution.

[0137] Among them, the emotional categories of the text to be identified include:

[0138] Positive: Indicates that the text conveys positive emotions, such as happiness, satisfaction, excitement, etc.

[0139] Negative: Indicates that the text conveys negative emotions, such as anger, sadness, disappointment, etc.

[0140] Neutral: Indicates that the text has no obvious emotional tendency, or the emotional expression is relatively bland.

[0141] In summary, the present invention effectively improves the adaptability of the model to different data distributions and scenarios by dynamically adjusting the discard rate. This makes the model more stable when processing different data, and enhances the generalization ability and robustness of the model.

[0142] Embodiment 2

[0143] This embodiment discloses a sentiment analysis system based on feature interaction and association fusion, including:

[0144] The data acquisition module is configured to: acquire text data to be recognized;

[0145] The vector representation module is configured to: input the acquired text data into the BERT model to obtain a multi-layer representation vector, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model;

[0146] The first feature fusion module is configured to: input all the intermediate layer representation vectors into the multi-layer feature interaction module, integrate the context information by weighted superposition of the representation vectors of different layers, and dynamically adjust the importance of the features of each dimension in the representation vectors of different layers through the gating mechanism; fuse the weighted superposition features with the output of the gating mechanism, and perform weighted averaging with the initially input intermediate layer representation vector to obtain a first associated feature fusion vector;

[0147] The second feature fusion module is configured to: input the first correlation feature fusion vector and the final layer CLS representation vector into the dynamic correlation fusion module to obtain a classified text feature vector;

[0148] The text sentiment classification module is configured as follows: a dynamic mask-based adaptive discarding method is used to dynamically discard the features of the classified text feature vector, and then the sentiment category of the text to be identified is obtained through a fully connected layer and an activation function layer.

[0149] Embodiment 3

[0150] The purpose of this embodiment is to provide a computer-readable storage medium.

[0151] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in a sentiment analysis method based on feature interaction and association fusion as described in Example 1 of the present disclosure.

[0152] Embodiment 4

[0153] The purpose of this embodiment is to provide an electronic device.

[0154] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in a sentiment analysis method based on feature interaction and association fusion as described in Example 1 of the present disclosure are implemented.

[0155] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0156] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0157] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A sentiment analysis method based on feature interaction and association fusion, characterized in that: include: Obtaining text data to be recognized; Input the acquired text data into the BERT model to obtain multi-layer representation vectors, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model; All intermediate layer representation vectors are input into the multi-layer feature interaction module. The context information is integrated by weighted superposition of representation vectors at different levels. The importance of each dimension feature in the representation vectors at different levels is dynamically adjusted through the gating mechanism. The weighted superposition features are fused with the output of the gating mechanism, and then weighted averaged with the initial input intermediate layer representation vector to obtain the first associated feature fusion vector. The first correlation feature fusion vector and the final layer CLS representation vector are input into the dynamic correlation fusion module to obtain the classified text feature vector; The adaptive discarding method based on dynamic mask dynamically discards the features of the classified text feature vector, and then obtains the sentiment category of the text to be identified through the fully connected layer and the activation function layer.

2. A sentiment analysis method based on feature interaction and association fusion as claimed in claim 1, characterized in that: The first correlation feature fusion vector and the final layer CLS representation vector are input into the dynamic correlation fusion module to obtain the classified text feature vector, including: The correlation scores of the first correlation feature fusion vector and the final layer CLS representation vector are calculated by a bilinear correlation calculation method; Using the correlation score, weighted fusion is performed on the elements in the first correlation feature fusion vector and the final layer CLS representation vector respectively to obtain the weighted fusion first correlation feature fusion vector and the final layer CLS representation vector; The first associated feature fusion vector after weighted fusion and the final layer CLS representation vector are concatenated to obtain the classified text feature vector.

3. A sentiment analysis method based on feature interaction and association fusion as claimed in claim 2, characterized in that: The method of calculating the correlation score of the first correlation feature fusion vector and the final layer CLS representation vector by a bilinear correlation calculation method includes: Calculate the transposed product of the first associated feature fusion vector and the final layer CLS representation vector; The concatenated first correlation feature fusion vector and the final layer CLS representation vector are processed using linear transformation and activation function; The transposed product is fused with the concatenated vector after linear transformation and activation function processing to obtain the relevance score of the vector.

4. The sentiment analysis method based on feature interaction and association fusion as claimed in claim 1, characterized in that: The adaptive discarding method based on dynamic mask performs dynamic feature discarding on the classified text feature vector, including: Calculate the discard rate of each feature in the classification text feature vector; A random binary mask matrix is ​​generated according to the discard rate of each feature, and the classified text feature vector is randomly discarded using the random binary mask matrix.

5. The sentiment analysis method based on feature interaction and association fusion as claimed in claim 4, characterized in that: The calculation of the discard rate of each feature in the classified text feature vector includes: Calculating the L2 norm of each feature in the classified text feature vector to measure the importance or strength of the feature; After the L2 norm of each feature is linearly transformed and normalized, a learning parameter is introduced to control the adjustment strength; finally, the result is mapped to the range of [0,1] through an activation function to obtain the discard rate of each feature.

6. The sentiment analysis method based on feature interaction and association fusion as claimed in claim 1, characterized in that: The emotion category of the text to be identified includes: positive: indicating that the text conveys positive emotions; negative: indicating that the text conveys negative emotions; and neutral: indicating that the text has no obvious emotional tendency, or the emotional expression is relatively bland.

7. A sentiment analysis method based on feature interaction and association fusion as claimed in claim 6, characterized in that: The positive emotions include happiness, satisfaction and excitement; the negative emotions include anger, sadness and disappointment.

8. A sentiment analysis system based on feature interaction and association fusion, characterized in that: include: The data acquisition module is configured to: acquire text data to be recognized; The vector representation module is configured to: input the acquired text data into the BERT model to obtain a multi-layer representation vector, including the representation vectors output by all intermediate layers and the final layer encoder of the BERT model; The first feature fusion module is configured to: input all the intermediate layer representation vectors into the multi-layer feature interaction module, integrate the context information by weighted superposition of the representation vectors of different layers, and dynamically adjust the importance of the features of each dimension in the representation vectors of different layers through the gating mechanism; fuse the weighted superposition features with the output of the gating mechanism, and perform weighted averaging with the initially input intermediate layer representation vector to obtain a first associated feature fusion vector; The second feature fusion module is configured to: input the first correlation feature fusion vector and the final layer CLS representation vector into the dynamic correlation fusion module to obtain a classified text feature vector; The text sentiment classification module is configured as follows: a dynamic mask-based adaptive discarding method is used to dynamically discard the features of the classified text feature vector, and then the sentiment category of the text to be identified is obtained through a fully connected layer and an activation function layer.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the sentiment analysis method based on feature interaction and association fusion as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the sentiment analysis method based on feature interaction and association fusion as described in any one of claims 1 to 7 are implemented.

Images