Emotion recognition method and device based on SoftmaxOne agency attention mechanism and medium
Through the combination of SoftmaxOne agent attention mechanism and deep learning model, the problem of insufficient long-distance context information capture in dialogue emotion recognition in the prior art is solved, and the accuracy and effectiveness of emotion recognition are improved.
Patent Information
- Application Number
- CN202410891532.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-04
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art is difficult to effectively capture long-distance context information in dialogue emotion recognition, resulting in poor emotional recognition, especially when dealing with discourse lacking clear emotions.
Using the SoftmaxOne proxy attention mechanism method, the context-independent features are extracted through pre-training language model RoBERTa, combined with the dual-channel bidirectional LSTM and Transformer encoder, the SoftmaxOne proxy attention mechanism is used to selectively add long-distance attention information to obtain the optimized context Information features and emotion recognition classification through jump connections and multi-layer perceptrons.
Improved performance of dialogue emotion recognition, able to capture long-distance context information more effectively, reduce noise and redundancy, and improve the accuracy of emotion recognition, especially when dealing with discourses lacking clear emotions.
Smart Images

Figure CN118964609A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of speech emotion recognition in natural language processing, and in particular to an emotion recognition method, device and medium based on a SoftmaxOne proxy attention mechanism. Background Art
[0002] With the rapid development of natural language processing technology, emotion recognition in conversations has become a hot research direction. Due to the rapid development of online social media, there are a large amount of public conversation data on some communication platforms. In addition, emotion recognition in conversations can also promote the generative dialogue model's understanding of user emotions.
[0003] There are three main aspects of identifying emotions in conversations, including: 1. The speech itself and its contextual information; 2. Speaker information; 3. Emotions expressed in previous speech. Effectively obtaining the speech itself and its contextual information is the core of this task. Some existing dictionary-based methods and some deep learning-based methods often ignore factors specific to conversations. These methods usually focus on how to effectively obtain the speech itself and its contextual information, while ignoring factors specific to conversations, and do not pay attention to the contextual information contained in adjacent speech and the problem that emotional inertia will decrease over time, resulting in poor results of these methods.
[0004] Based on the above problems, the present invention introduces a proxy attention mechanism with a larger receptive field. By allowing the proxy attention mechanism to ignore irrelevant information and reduce the noise in the information, it can help the self-attention selectively focus on information and better obtain the contextual information in the feature sequence, thereby improving the shortcomings of the existing methods.
[0005] The patent document with the document number CN109285562B discloses a speech emotion recognition method based on the attention mechanism. The invention is based on the speech emotion recognition method of the attention mechanism, which can improve the performance of speech emotion recognition. The method is ingenious and novel, and has good application prospects. However, it also has the problem of being unable to capture long-distance contextual information and using the rich global information implicit in long-distance speech to help classify speech that lacks clear emotions. Summary of the invention
[0006] The purpose of the present invention is to provide an emotion recognition method, device and medium based on the SoftmaxOne proxy attention mechanism, which can capture long-distance contextual information and use the rich information implicit in long-distance speech to help classify speech that lacks clear emotions.
[0007] The embodiments of the present invention provide a method, device and medium for emotion recognition based on SoftmaxOne proxy attention mechanism.
[0008] Aspect 1: A sentiment recognition method based on SoftmaxOne proxy attention mechanism, including:
[0009] S1. Use the pre-trained language model RoBERTa to extract context-independent discourse-level feature vectors as initial text features;
[0010] S2. Use a dual-channel bidirectional LSTM model to contextually encode the initial text features and obtain the initial context information features:
[0011] S3, embedding the speaker information into the initial context information features, inputting them into the Transformer encoder for processing, and then performing layer normalization to obtain a coded normalized feature sequence;
[0012] S4, using the SoftmaxOne proxy attention mechanism, selectively add long-distance attention information to the feature sequence to obtain optimized context information features;
[0013] S5, jump-connecting the processed initial text features with the optimized context information features to obtain the final information features;
[0014] S6. Send the final information features into a multi-layer perceptron for sentiment recognition and classification.
[0015] The emotion recognition method based on the SoftmaxOne proxy attention mechanism according to claim 1 is characterized in that, in S1, a pre-trained language model RoBERTa is used to extract a context-independent speech level feature vector as an initial text feature, comprising the steps of:
[0016] S11. Add a special token [CLS] at the beginning of the utterance, and the output sequence is [CLS], x1, x2, …, xN;
[0017] S12, pass the [CLS] sequence through the model, and then use the activation of the last layer corresponding to the [CLS] token in a feed-forward network to classify it into the sentiment class;
[0018] S13 passes it the BPE tokenized utterance appended with the [CLS] sequence and extracts the activations from the last four layers corresponding to the [CLS] tokens;
[0019] S14. The four activation vectors are averaged and then passed through a linear layer to obtain a context-independent discourse-level feature vector with a dimension of 100 as the initial text feature u i .
[0020] Optionally, in S2, a dual-channel bidirectional LSTM model is used to contextually encode the initial text features to obtain initial context information features, including:
[0021] Use a dual-channel bidirectional LSTM model to train the initial text feature u i Perform context encoding to obtain the context information vector l i , the formula is:
[0022]
[0023] Among them, i represents the i-th speech sample; u i The context-independent text features in the representation, represents the contextual relationship of the input features obtained from the forward sequence, represents the contextual relationship of the input features obtained from the reverse sequence, and [·] represents the concatenation operation.
[0024] Optionally, in S3, speaker information is embedded into the initial context information features, inputted into the Transformer encoder for processing, and then layer normalization is performed to obtain a coded normalized feature sequence, including the steps of:
[0025] S31. Speaker information is embedded into the initial context information features. The formula is:
[0026] l si =w ps s i +l i
[0027] Where W ps represents the learnable weight parameter, S i is the speaker information, l si It is the feature matrix after the speaker information is embedded;
[0028] S32, the feature matrix l si Input the Transformer encoder for processing, then perform layer normalization to obtain the encoded normalized feature sequence T i , the formula is:
[0029] T i =LayerNorm(TE(l i ))
[0030] Among them, TE stands for Transformer encoder and LayerNorm stands for layer normalization.
[0031] Optionally, the S4 uses the SoftmaxOne proxy attention mechanism to selectively add long-distance attention information to the feature sequence to obtain optimized context information features; including:
[0032] The improved SoftmaxOne is obtained and added to the proxy attention mechanism to obtain the context information features in the optimized feature sequence. The formula is expressed as:
[0033]
[0034] Among them, C i represents the optimized context information feature of the output, B1 and B2 are biases, DWC(Vi) represents the deep convolution function, and Q i =W Q T i , K i =W K T i , V i =W V T i , A i =pool(Q i ) represents the proxy vector, W Q , W K , W V represents the learnable weight matrix, d k =d T / h,d T Indicates T i The number of dimensions.
[0035] Optionally, the improved SoftmaxOne is expressed as:
[0036]
[0037] Among them, x represents a K-dimensional vector, x j represents the jth element;
[0038] Optionally, in S5, the initial text feature is processed and then jump-connected with the optimized context information feature to obtain the final information feature, including:
[0039] The initial text feature u i The feature scale is changed through the fully connected layer and the feature b is obtained after layer normalization. i , the formula is:
[0040] bi=(LayerNorm(FC(ui))
[0041] Among them, FC represents the fully connected layer, u i It is the original text feature, LayerNorm represents layer normalization;
[0042] Using skip connection method, feature b i and optimized context information feature C i Add and normalize again through the layer to generate the final information feature, the formula is expressed as:
[0043] e i =(LayerNorm(b i +C i ))
[0044] Among them, C i is the output from the proxy attention mechanism, + represents matrix addition, and LayerNrom represents layer normalization.
[0045] Optionally, in S6, the final information features are sent to a multi-layer perceptron for emotion recognition and classification, including:
[0046] The final information feature e i It is sent to the multi-layer perceptron MLP for emotion recognition and classification. The formula is:
[0047] l i =sigmoid(W l e i +b l )
[0048] P i =Softmax(W smax l i +b smax )
[0049]
[0050] Among them, Sigmoid represents the activation function, W l The weight matrix, b l , b smax represents the offset parameter, P i represents the probability of each category predicted by the model, Softmax represents the normalization function, and W smax represents the weight matrix, Represents finding the i [k] The largest category k, k represents the sentiment classification, Represents the predicted sentiment label.
[0051] A second aspect: An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method provided in the first aspect when executing the program.
[0052] A third aspect: A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.
[0053] Beneficial effects of the present invention:
[0054] The present invention introduces a proxy attention mechanism by effectively modeling context and coping with obtaining long-distance and short-distance context information. Compared with the traditional self-attention mechanism, the proxy attention mechanism has a larger receptive field and has stronger performance in obtaining long-distance dependencies. The Softmax function in the proxy attention mechanism is improved to be the SoftmaxOne function. The SoftmaxOne function allows the attention head to ignore irrelevant information or features by modifying the Softmax function, thereby reducing feature noise and redundancy. The SoftmaxOne proxy attention mechanism is combined with LSTM to obtain context information in the conversation, wherein LSTM is used to capture neighboring context information and emotional inertia, and the SoftmaxOne proxy attention mechanism is used to capture long-distance context information and use the rich global information implicit in long-distance discourse to help classify discourse that lacks clear emotions, thereby further obtaining better emotion recognition from the text. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 A flowchart of an emotion recognition method based on the SoftmaxOne proxy attention mechanism of the present invention;
[0056] Figure 2 A schematic diagram of the process structure of an emotion recognition method based on the SoftmaxOne proxy attention mechanism of the present invention;
[0057] Figure 3 This is a schematic diagram of the SoftmaxOne proxy attention mechanism structure of the present invention;
[0058] Figure 4 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION
[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0060] The existing factors for emotion recognition in conversations are mainly based on the discourse itself and its contextual information, speaker information, and emotions expressed in previous discourses. Existing methods based on dictionaries and deep learning are used to model the context and understand user emotions in conversation patterns. Existing methods usually focus on how to effectively obtain the discourse itself and its contextual information, ignoring factors specific to the conversation, which leads to poor results of these emotion recognition methods.
[0061] In view of the above problems, the present invention provides an emotion recognition method based on SoftmaxOne proxy attention mechanism. Figure 1 A flow chart of an emotion recognition method based on a SoftmaxOne proxy attention mechanism provided in an embodiment of the present invention, the method comprising:
[0062] S1. Use the pre-trained language model RoBERTa to extract context-independent discourse-level feature vectors as initial text features.
[0063] Specifically, it includes: adding a special token [CLS] to the beginning of the utterance so that the output sequence is [CLS], x1, x2, …, xN; the [CLS] sequence passes through the model and then uses the activation of the last layer corresponding to the [CLS] token in a small feed-forward network to classify it as a sentiment class; after fine-tuning the model for sentiment label classification, the BPE tokenized utterance appended with the [CLS] sequence is passed to it and the activations from the last four layers corresponding to the [CLS] token are extracted; these four activation vectors are then averaged and passed through a linear layer to obtain a context-independent utterance-level feature vector with a dimension of 100 as the initial text feature u i .
[0064] S2. Use a dual-channel bidirectional LSTM model to contextually encode the initial text features and obtain the initial context information features:
[0065] like Figure 2 As shown, a dual-channel bidirectional long short-term memory network LSTM model is used to analyze the initial text feature u i Context encoding is performed. The bidirectional LSTM model can capture the temporal relationship in text features, which helps to better utilize the information in sequence data. For each utterance, it interacts with the hidden layer of the previous layer and the hidden layer of the next layer to obtain the context information vector l i , the formula is:
[0066]
[0067] Among them, i represents the i-th speech sample; u i The context-independent initial text features in the representation, represents the contextual relationship of the input features obtained from the forward sequence, represents the contextual relationship of the input features obtained from the reverse sequence, and [·] represents the concatenation operation.
[0068] S3, embedding the speaker information into the initial context information features, inputting them into the Transformer encoder for processing, and then performing layer normalization to obtain a coded normalized feature sequence;
[0069] The speaker information is embedded into the initial context information features. The formula is:
[0070] l si =w ps s i +l i
[0071] Where W ps represents the learnable weight parameter, S i is the speaker information, l si It is the feature matrix after speaker information is embedded.
[0072] The speaker information is embedded into l si , input into the Transformer encoder for processing. The Transformer encoder can obtain long-distance context information, and due to the introduction of position encoding, it can capture the position of each element in the sequence, thereby better capturing contextual dependencies.
[0073] The Transformer encoder can be extended to different input lengths. The self-attention mechanism in the Transformer allows short and long sequences to be effectively processed without significantly increasing the computation. Layer normalization can effectively prevent gradient vanishing and gradient explosion. The encoded normalized feature sequence T is obtained through the Transformer encoder and layer normalization. i , the formula is:
[0074] T i =LayerNorm(TE(l i ))
[0075] Among them, TE stands for Transformer encoder and LayerNorm stands for layer normalization.
[0076] S4, using the SoftmaxOne proxy attention mechanism, selectively add long-distance attention information to the feature sequence to obtain optimized context information features;
[0077] Since the loop-based method will gradually attenuate during the process of information transmission, the loop-based method cannot effectively obtain long-distance context dependencies, while the self-attention mechanism can provide the interaction ability of long-distance information. The distance of self-attention to obtain context dependencies usually depends on the size of the receptive field. The agent attention mechanism (AgentAttention) is introduced. As a variant of self-attention, it has low computational complexity and high model expression ability. Since the agent attention mechanism has linear complexity, it can have a larger or even global receptive field than the self-attention mechanism, so it has better performance in long-distance context modeling.
[0078] like Figure 3 As shown in the figure, in the proxy attention mechanism, the proxy vector A is used to avoid the pairwise similarity calculation of the query vector Q and the key vector K. At the same time, the proxy vector acts as a proxy for the query vector to aggregate the global information from the key vector K (Key) and the value vector V (Value), and then broadcasts it back to Q (Query). A simple pooling strategy is used to obtain the proxy token. The strategy is very effective and is expressed as follows:
[0079]
[0080] Among them, B1, B2 are biases, DWC(Vi) means that the deep convolution function is equivalent to a sparse matrix, Q i =W Q T i , K i =W K T i , V i =W V T i , A i =pool(Q i ) represents the proxy vector, W Q , W K , W V represents the learnable weight matrix, d k =d T / h,d T Indicates T i The number of dimensions.
[0081] The traditional self-attention mechanism usually uses softmax attention. Softmax attention enables the model to better distinguish the importance of input elements while maintaining the smoothness of the output. However, Softmax forces the attention head to pay attention to information, even if no useful information can be added to the output vector. This causes Softmax to pay attention to irrelevant information when no information is available, that is, this causes noise in the output vector. Since multi-head attention contains more attention heads, this problem is more serious in multi-head attention. The traditional Softmax function formula is:
[0082]
[0083] Among them, x represents a K-dimensional vector, x j Represents the jth element. When x is much smaller than 0 and does not need to be paid attention to, the result of Softmax cannot approach 0, as shown in the formula:
[0084]
[0085] According to the shortcomings of Softmax, it can be improved to SoftmaxOne, and the formula is expressed as:
[0086]
[0087] At this time, when x is much smaller than 0 and does not need to be paid attention to, the result of Softmax can approach 0, as shown in the formula:
[0088]
[0089] This allows the vector to approach 0 when needed, and in other cases, only a portion of the value will be reduced. This reduction will be compensated in the subsequent normalization process, which allows the attention head to selectively focus on information without adding irrelevant information to the output vector.
[0090] In the self-attention mechanism, the improved SoftmaxOne is used to better obtain the context information in the feature sequence, provide the interaction capability of long-distance information, and effectively obtain long-distance context information. The formula is:
[0091]
[0092] Among them, C i represents the optimized context information feature of the output, B1 and B2 are biases, DWC(Vi) represents the deep convolution function, and Q i =W Q T i , K i =WK T i , V i =W V T i , A i =pool(Q i ) represents the proxy vector, W Q , W K , W V represents the learnable weight matrix, d k =d T / h,d T Indicates T i The number of dimensions.
[0093] S5, jump-connect the processed initial text features with the optimized context information features to improve feature transfer efficiency and model training stability, prevent network overfitting, and obtain the final information features;
[0094] The fully connected layer connects all nodes in the previous layer and contains a large number of model parameters. Adding a normalization layer can accelerate the convergence of the model and convert the initial text feature u i The feature scale is changed through the fully connected layer and the feature b is obtained after layer normalization. i , the formula is:
[0095] bi=(LayerNorm(FC(ui))
[0096] Among them, FC represents the fully connected layer, u i It is the original text feature, LayerNorm represents layer normalization;
[0097] In order to improve the efficiency of feature transfer and the stability of model training, a skip connection is used to transfer feature b i and optimized context information feature C i Add and normalize again through the layer to generate the final information feature representation:
[0098] e i =(LayerNorm(b i +C i ))
[0099] Among them, C i is the output from the proxy attention mechanism, + represents matrix addition, and LayerNrom represents layer normalization.
[0100] S6. Send the final information features into a multi-layer perceptron for sentiment recognition and classification.
[0101] The final information feature e iThe data is sent to a multi-layer perceptron (MLP). MLP is commonly used for classification tasks and usually includes an input layer, several hidden layers, and an output layer. There are many neurons in each layer, and the neurons are connected by weights. Then there is an activation function that performs a nonlinear transformation on the output of the neurons for emotion recognition and classification. The formula is:
[0102] l i =sigmoid(W l e i +b l )
[0103] P i =Softmax(W smax l i +b smax )
[0104]
[0105] Among them, Sigmoid represents the activation function, W l The weight matrix, b l , b smax represents the offset parameter, P i represents the probability of each category predicted by the model, Softmax represents the normalization function, and W smax represents the weight matrix, Represents finding the i [k] The largest category k, k represents the sentiment classification, Represents the predicted sentiment label.
[0106] Furthermore, the classification cross entropy loss function used by the model is expressed as:
[0107]
[0108] Where N is the number of dialogues, c(i) is the number of utterances in dialogue i, and P i,j is the probability distribution of the predicted emotion label of utterance j in conversation i, y i,j is the expected class label of utterance j in dialogue i, and the training loss function is given by And L2 regularization, the formula is:
[0109]
[0110] Among them, λm is the weight of L2 regularization, and θm represents all trainable parameters in the training model.
[0111] This paper proposes a new emotion recognition framework and introduces a proxy attention mechanism. Compared with the traditional self-attention mechanism, the proxy attention mechanism has a larger receptive field and better performance in acquiring long-distance dependencies. The Softmax function in the proxy attention mechanism is improved, and the improved proxy attention mechanism is named SoftmaxOne proxy attention. By modifying the Softmax function, the attention head is allowed to ignore irrelevant information or features, thereby reducing feature noise and redundancy. The SoftmaxOne proxy attention mechanism is combined with LSTM to obtain contextual information in the conversation, in which LSTM is used to capture neighboring contextual information and emotional inertia, and the SoftmaxOne proxy attention mechanism is used to capture long-distance contextual information and use the rich global information implied in long-distance discourse to help classify discourse that lacks clear emotions.
[0112] The present invention also provides an electronic device, Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 4 As shown, the electronic device may include: a processor, a communications interface, a memory, and a communications bus, wherein the processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may call the logic instructions in the memory to execute the above method, for example, including:
[0113] S1. Use the pre-trained language model RoBERTa to extract context-independent discourse-level feature vectors as initial text features;
[0114] S2. Use a dual-channel bidirectional LSTM model to contextually encode the initial text features and obtain the initial context information features:
[0115] S3, embedding the speaker information into the initial context information features, inputting them into the Transformer encoder for processing, and then performing layer normalization to obtain a coded normalized feature sequence;
[0116] S4, using the SoftmaxOne proxy attention mechanism, selectively add long-distance attention information to the feature sequence to obtain optimized context information features;
[0117] S5, jump-connecting the processed initial text features with the optimized context information features to obtain the final information features;
[0118] S6. Send the final information features into a multi-layer perceptron for sentiment recognition and classification.
[0119] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0120] An embodiment of the present invention further provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method provided in each of the above embodiments is implemented, for example, including:
[0121] S1. Use the pre-trained language model RoBERTa to extract context-independent discourse-level feature vectors as initial text features;
[0122] S2. Use a dual-channel bidirectional LSTM model to contextually encode the initial text features and obtain the initial context information features:
[0123] S3, embedding the speaker information into the initial context information features, inputting them into the Transformer encoder for processing, and then performing layer normalization to obtain a coded normalized feature sequence;
[0124] S4, using the SoftmaxOne proxy attention mechanism, selectively add long-distance attention information to the feature sequence to obtain optimized context information features;
[0125] S5, jump-connecting the processed initial text features with the optimized context information features to obtain the final information features;
[0126] S6. Send the final information features into a multi-layer perceptron for sentiment recognition and classification.
[0127] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A sentiment recognition method based on SoftmaxOne proxy attention mechanism, It is characterized in that Includes steps: S1. Use the pre-trained language model RoBERTa to extract context-independent discourse-level feature vectors as initial text features; S2. Use a dual-channel bidirectional LSTM model to contextually encode the initial text features and obtain the initial context information features: S3, embedding the speaker information into the initial context information features, inputting them into the Transformer encoder for processing, and then performing layer normalization to obtain a coded normalized feature sequence; S4, using the SoftmaxOne proxy attention mechanism, selectively add long-distance attention information to the feature sequence to obtain optimized context information features; S5, jump-connecting the processed initial text features with the optimized context information features to obtain the final information features; S6. Send the final information features into a multi-layer perceptron for sentiment recognition and classification.
2. According to claim 1, a method for emotion recognition based on SoftmaxOne proxy attention mechanism, It is characterized in that S1 uses the pre-trained language model RoBERTa to extract context-independent discourse level feature vectors as initial text features, including the following steps: S11. Add a special token [CLS] at the beginning of the utterance, and the output sequence is [CLS], x1, x2, …, xN; S12, pass the [CLS] sequence through the model, and then use the activation of the last layer corresponding to the [CLS] token in a feed-forward network to classify it into the sentiment class; S13 passes it the BPE tokenized utterance appended with the [CLS] sequence and extracts the activations from the last four layers corresponding to the [CLS] tokens; S14. The four activation vectors are averaged and then passed through a linear layer to obtain a context-independent discourse-level feature vector with a dimension of 100 as the initial text feature u i .
3. According to claim 2, a method for emotion recognition based on SoftmaxOne proxy attention mechanism, It is characterized in that In S2, a dual-channel bidirectional LSTM model is used to contextually encode the initial text features to obtain initial context information features, including: Use a dual-channel bidirectional LSTM model to train the initial text feature u i Perform context encoding to obtain the context information vector l i , the formula is: Among them, i represents the i-th speech sample; u i The context-independent text features in the representation, represents the contextual relationship of the input features obtained from the forward sequence, represents the contextual relationship of the input features obtained from the reverse sequence, and [·] represents the concatenation operation.
4. According to claim 3, a method for emotion recognition based on SoftmaxOne proxy attention mechanism, It is characterized in that In S3, the speaker information is embedded into the initial context information feature, input into the Transformer encoder for processing, and then layer normalization is performed to obtain a coded normalized feature sequence, including the following steps: S31. Speaker information is embedded into the initial context information features. The formula is: l si =w ps s i +l i Where W ps represents the learnable weight parameter, S i is the speaker information, l si It is the feature matrix after the speaker information is embedded; S32, the feature matrix l si Input the Transformer encoder for processing, then perform layer normalization to obtain the encoded normalized feature sequence T i , the formula is: T i =LayerNorm(TE(l i )) Among them, TE stands for Transformer encoder and LayerNorm stands for layer normalization.
5. The emotion recognition method based on SoftmaxOne proxy attention mechanism according to claim 4, It is characterized in that In the S4, the SoftmaxOne proxy attention mechanism is used to selectively add long-distance attention information to the feature sequence to obtain optimized context information features; including: The improved SoftmaxOne is obtained and added to the proxy attention mechanism to obtain the context information features in the optimized feature sequence. The formula is expressed as: Among them, C i Represents the optimized context information feature of the output, B 1 , B 2 is the bias, DWC(Vi) represents the deep convolution function, Q i =W Q T i , K i =W K T i , V i =W V T i , A i =pool(Q i ) represents the proxy vector, W Q , W K , W V represents the learnable weight matrix, d k =d T / h,d T Indicates T i The number of dimensions.
6. The emotion recognition method based on SoftmaxOne proxy attention mechanism according to claim 5, It is characterized in that The improved SoftmaxOne is expressed as follows: Among them, x represents a K-dimensional vector, x j Represents the jth element.
7. The emotion recognition method based on SoftmaxOne proxy attention mechanism according to claim 5, It is characterized in that In S5, the initial text feature is processed and then jump-connected with the optimized context information feature to obtain the final information feature, including: The initial text feature u i The feature scale is changed through the fully connected layer and the feature b is obtained after layer normalization. i , the formula is: bi=(LayerNorm(FC(ui)) Among them, FC represents the fully connected layer, u i It is the original text feature, LayerNorm represents layer normalization; Using skip connection method, feature b i and optimized context information feature C i Add and normalize again through the layer to generate the final information feature, the formula is expressed as: e i =(LayerNorm(b i +C i )) Among them, C i is the output from the proxy attention mechanism, + represents matrix addition, and LayerNrom represents layer normalization.
8. The emotion recognition method based on SoftmaxOne proxy attention mechanism according to claim 7, It is characterized in that In S6, the final information features are sent to a multi-layer perceptron for emotion recognition and classification, including: The final information feature e i It is sent to the multi-layer perceptron MLP for emotion recognition and classification. The formula is: l i =sigmoid(W l e i +b l ) P i =Softmax(W smax l i +b smax ) Among them, Sigmoid represents the activation function, W l The weight matrix, b l , b smax represents the offset parameter, P i represents the probability of each category predicted by the model, Softmax represents the normalization function, and W smax represents the weight matrix, Represents finding the i [k] The largest category k, k represents the sentiment classification, Represents the predicted sentiment label.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of an emotion recognition method based on a SoftmaxOne proxy attention mechanism as described in any one of claims 1 to 8 are implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of an emotion recognition method based on a SoftmaxOne proxy attention mechanism as described in any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Root cause analysis method, system and equipment based on large model causal reasoning
CN120017426A
AI-based intelligent focus detection method in virtual hysteroscope operation training
CN120125921A
