Easily-confused emotion recognition method based on multi-path feature extraction and gating fusion
By employing a multi-path feature extraction and gating fusion method, and utilizing adaptive wavelet transform and deep learning networks, the accuracy problem of easily confused emotion recognition is solved, and efficient speech emotion classification is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-15
- Publication Date
- 2026-04-10
AI Technical Summary
Existing speech emotion recognition methods have difficulty recognizing easily confused emotion combinations such as "happiness-anger", "sadness-neutral", and "fear-surprise". Feature extraction networks lead to reduced inter-class separation, making it difficult to effectively distinguish between them.
A multi-path feature extraction network is constructed, multi-scale wavelet filter kernels are generated through adaptive wavelet transform, and feature fusion is performed by combining gated attention mechanism. The deep learning model is trained with dual supervision of weighted cross-entropy loss and contrastive loss.
It improved the accuracy of easily confused emotion recognition, especially the recognition accuracy of emotion combinations such as "happy/angry", "sad/neutral" and "fear/surprise", significantly reduced the confusion error rate, and enhanced the robustness of the model in complex scenarios.
Smart Images
Figure CN121838818A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of human-computer interaction and affective computing, specifically to a method for identifying easily confused emotions based on multi-path feature extraction and gating fusion. Background Technology
[0002] Speech emotion recognition determines a speaker's emotional state by analyzing the acoustic and prosodic features of a speech signal. Speech signals contain multi-dimensional information such as energy, fundamental frequency, formants, and temporal variations. Different emotional speech patterns differ in energy distribution, speech rate, and frequency band variations. In practical applications, due to individual speaker differences, contextual factors, and the diversity of emotional expression methods, the acoustic feature distributions of different emotion categories tend to overlap, leading to unclear category boundaries and increased recognition difficulty.
[0003] This is particularly evident in easily confused emotion combinations such as "happiness-anger," "sadness-neutral," and "fear-surprise." From an intra-class perspective, these emotions exhibit high similarity in overall energy levels, speech rate variations, and spectral distribution. Differences are primarily found in local time-frequency structure, transient change patterns, and prosodic features. For example, anger and happiness overlap in spectral energy distribution and share similar transient characteristics, both exhibiting significant energy distribution in the mid-to-high frequency range, leading to reduced distinguishability. Fear and surprise both present with sudden energy shifts and increased speech rate, sharing similar transient characteristics and making them difficult to differentiate effectively using conventional methods. Calmness and sadness both exhibit low-frequency stability, with differences limited to weak extension features; the speech signal characteristics of these non-stationary, short-term easily confused emotions are difficult to capture. From an inter-class perspective, there are significant differences in acoustic features between different easily confused emotion groups. The happiness-anger group exhibits enhanced mid-to-high frequency energy. The fear-surprise group exhibits short-term sudden changes and increased speech rate. The sadness-neutral group exhibits low-frequency dominance and stable energy. Inter-class differences mainly lie in the dominant frequency band and the intensity of dynamic changes.
[0004] Existing speech emotion recognition methods often use the same feature extraction network for speech signals across all emotion categories during the feature extraction stage. Common methods are based on Mel-frequency cepstral coefficients (MFCC), linear predictive cepstral coefficients (LPCC), or spectrogram features, combined with deep neural networks for feature learning. When different emotion categories share the same feature extraction network, the training process is more likely to reinforce high-energy correlated components shared across categories, easily confusing emotions. Insufficient representation of differences in local time-frequency structure and short-term dynamic changes leads to reduced inter-class separation in the feature space and unclear discrimination boundaries.
[0005] The key issue in easily confused emotion recognition lies in the significant feature similarities between emotion categories. The emotion groups "happiness-anger," "sadness-neutral," and "fear-surprise" share similar spectral structures, leading to high misclassification rates for the same feature extraction network. Within each class, feature distributions overlap, with intra-group differences concentrated in local discriminative cues. Between classes, the dominant acoustic patterns of different similar emotion groups differ, providing a basis for modeling based on similarity grouping. After grouping emotion features by similarity, each emotion group employs a separate feature extraction structure, and the deep learning feature network structure forms a multi-path feature extraction framework. Each path corresponds to a specific emotion group, extracting features based on local temporal-frequency differences within the group to reduce intra-group confusion.
[0006] To improve the recognition of easily confused emotion features, this invention proposes a speech emotion recognition method based on multi-path feature extraction and fusion. For three easily confused emotion groups—"happiness-anger," "sadness-neutral," and "fear-surprise"—multiple feature extraction paths are constructed in the model based on the similarity of emotion features, with each path corresponding to a group of easily confused emotions. Within each path, an adaptive wavelet transform module is invoked, setting different parameter combinations to generate different wavelet kernel functions, extracting multi-path features. The features output from each path are then weighted and fused using a gated attention mechanism. During the training phase, a total loss is constructed using weighted cross-entropy loss and contrastive loss, and a deep learning network completes parameter learning and classification decisions, improving the accuracy of easily confused emotion recognition. Summary of the Invention
[0007] This invention targets three easily confused emotion combinations: "happiness-anger," "sadness-neutrality," and "fear-surprise." It constructs multiple feature paths, each corresponding to a different set of wavelet parameters. Here, 'c' represents the path number. All paths share the same learnable mother wavelet function. Different wavelet kernel functions are generated from different parameter sets, constructing wavelet filter kernel groups that focus on different emotion categories. This generates a multi-path feature extraction structure. Convolution operations are performed on the same speech signal at different resolutions to obtain the time-frequency features of the 'c'-th path, ultimately resulting in a multi-path emotion feature set representing different time-frequency resolutions. Then, the outputs of each path are input into a gating attention mechanism, and after calculating the path weights, a weighted fusion is performed to obtain fused features for easily confused emotion discrimination.
[0008] (1) After preprocessing, the speech signal is fed into the adaptive wavelet multi-path feature extraction module. Different wavelet kernel functions are generated using different sets of wavelet parameters, resulting in multiple independent time-frequency feature paths on the same speech signal. These paths have the same structure but different parameter configurations. Convolution operations are performed on the same speech signal at different resolutions to construct c feature paths, each corresponding to a different set of wavelet parameters. 'c' represents the path number, and the formula is as follows:
[0009] This process yields a set of multi-path sentiment features { , , ... , This set of features represents the features extracted by wavelet functions with different time-frequency resolutions for different paths of the same speech signal.
[0010] (2) Input the multipath time-frequency features obtained in step (1) into the gated attention fusion module.
[0011] Perform Global Average Pooling (GAP) on each path feature to obtain path statistical features:
[0012] in, Let F represent the time-frequency feature map of the c-th feature path, where T is the number of time frames and F is the number of frequency bands. The input is a fully connected layer, which undergoes a linear transformation. A sigmoid activation function is then used to generate dynamic weights for each set of emotion features. The formula is as follows:
[0013] in, For the weights of the fully connected layer, For bias terms, This represents the Sigmoid function, which controls the weights within the range of [0,1]. This reflects the importance of the c-th path in the current sample's sentiment assessment. Based on the weights... The multi-path features are weighted and fused to obtain the fused features. The formula is as follows:
[0014] in, This indicates an element-wise multiplication operation. The discriminative information in the multi-path features is preserved, and the weighted and integrated time-frequency features are used as input for subsequent classification networks.
[0015] (3) Input the fused emotion features into the deep learning classification module, use the softmax function to calculate the probability distribution of each emotion category, and use the weighted cross-entropy loss function combined with the contrast loss to calculate the total loss.
[0016] Fusion features The input is used to classify the network for emotion determination, and the feature representation vector of the sample is obtained. The probability distribution of each emotion category is calculated using fully connected layers and a softmax function. The classification process employs weighted cross-entropy loss and contrastive loss, introducing a category weight term into the cross-entropy loss function. This dynamically adjusts the contribution of different categories to the loss function, enhancing the influence of minority sentiment samples and mitigating the model's bias towards the mainstream category. The weighted cross-entropy loss function takes the following form:
[0017] Where N is the total number of samples, and k is the total number of emotion categories. This represents the feature representation of the i-th sample. Let k be the classification vector of class k. Let i be the true category label for sample i.
[0018] Building upon the weighted cross-entropy loss, a contrastive loss is introduced to enhance the inter-class separation in the feature space. The contrastive loss takes the following form:
[0019] in, This represents the set of sample pairs constructed from three sets of easily confused sentiment pairs. The set distance threshold controls the minimum expected interval between feature vectors, which is used to constrain the feature spacing of different categories and ensure the formation of stable inter-class boundaries.
[0020] The total loss function is defined as the weighted sum of the weighted cross-entropy loss and the contrastive loss:
[0021] in, The weighting coefficients for the comparison loss are used to balance the influence of the two types of supervision signals.
[0022] (4) Total loss Based on this, the Adam optimization algorithm is used to update the model parameters. The set of trainable parameters is as follows: ,in These are the weights of the fully connected layer. It is a bias term. Classifier parameters. Calculate the gradient of the total loss with respect to each trainable parameter. Update parameters: in, For learning rate, This represents the set of all learnable parameters. After multiple iterations until the model converges, the final parameter configuration and classification model are output.
[0023] S1: Voice data input and preprocessing; S2: Multipath emotion feature extraction based on adaptive wavelet transform; S3: Weighted fusion of multipath emotional features by gating attention mechanism; S4: Construct a deep learning classification network and perform classification and loss calculation under the dual supervision of weighted cross-entropy loss and contrastive loss; S5: Use the Adam optimization algorithm to train the model parameters until the performance of the validation set meets the convergence condition, and obtain the final trained emotion recognition model; Testing phase: S6: Input the speech to be tested into the trained emotion recognition model and output the predicted category.
[0024] Step S1 (data input and preprocessing) specifically includes the following steps: S1: Prepare a publicly available standard dataset for speech emotion recognition as the training data source. The dataset contains speech samples from multiple speakers and multiple emotion categories, divided into sets of easily confused emotion pairs such as "happy-angry," "sad-neutral," and "fear-surprised." The input raw speech signal is pre-emphasized, framed, and windowed.
[0025] Step S2 (multi-path emotion feature extraction based on adaptive wavelet transform) specifically includes the following steps: S2: Input the preprocessed speech signal x(t) obtained in step S1 into the adaptive wavelet transform module for multi-scale time-frequency analysis. Set c feature paths, where c represents the path number, c∈[1,3]. Each path shares the same learnable mother wavelet function, determined by the parameter set. The wavelet kernel function for this path is generated, forming a multi-scale wavelet filter kernel group. Within the ccc-th path, convolution operations are performed on x(t) at multiple scales to obtain the time-frequency features of this path:
[0026] By summarizing the outputs from each path, a multi-path sentiment feature set is obtained: { , , ... , }
[0027] Step S3 (weighted fusion of multi-path emotion features by gating attention mechanism) specifically includes the following steps: S3.1: Obtain the time-frequency characteristics of each path from step S2. Then, the feature map of the c-th path... Perform global average pooling (GAP), iterating along the time dimension T, calculating the average value for each frequency sub-band, and obtaining the feature vector after average pooling. :
[0028] in, This is the feature vector obtained after average pooling of the feature map of this path. .
[0029] S3.2: Calculate the gating attention weights. Initialize the fully connected layer weights. and bias terms ,in For feature dimension, For the output dimension. The feature vector after average pooling. The input is fed into a fully connected layer, and the gated attention weights are calculated using a sigmoid activation function. :
[0030] in, This represents the Sigmoid activation function and its output. Weights indicate the importance of this set of features. This indicates the importance of the path feature in the current sample sentiment judgment and controls the contribution of the path in the fusion process.
[0031] S3.3: Calculate the weighted fusion features. This involves converting the original feature matrix of each path... With the calculated weights Perform element-wise multiplication, then sum the weighted results of all paths to obtain the final fused feature matrix. :
[0032] in, This represents element-wise multiplication. The final weighted fusion features are the fused emotional features. This information will be used for subsequent loss calculations and model training. Through this step, the originally independent multi-path features are selectively integrated along the path dimension, highlighting more discriminative path information and suppressing redundant paths.
[0033] Step S4 (Classification and Loss Calculation under the Dual Loss Supervision Mechanism) specifically includes the following steps: S4.1: Construct a deep learning classification network and use the fused feature matrix obtained in step S3. It is converted into a one-dimensional feature vector through a pooling layer. , where i represents the sample number. Then... Input the fully connected classification layer to obtain the raw scores for each emotion category. Apply the SoftMax function to the raw scores to convert them into probability distributions for each emotion category.
[0034] S4.2: Compare the predicted probabilities with the true labels, and adjust them according to the weights of each category. Calculate weighted loss The formula is as follows:
[0035] in, This represents the depth feature representation of the i-th sample. Let be the classification vector for the k-th class, where k is the total number of emotion categories. Let i be the true category label for sample i.
[0036] S4.3: Within the same training batch, the program constructs a set P of sample pairs based on the true labels. For each pair of samples in the set, it calculates their deep features. and The Euclidean distance is calculated, and the contrast loss is calculated based on the threshold δ. Furthermore, the inter-class separation in the feature space is enhanced by contrastive loss, as shown in the following formula:
[0037] in, The set of sample pairs constructed for easily confused sentiment pairs, For the set distance threshold, ∈ This controls the minimum expected interval between samples of different categories.
[0038] S4.4: Calculate the total loss, which is a weighted sum of the weighted cross-entropy loss and the contrastive loss:
[0039] in, To compare the weighting coefficients of the loss, This step allows the model to optimize overall classification accuracy while also taking into account inter-class margin constraints in the feature space, further improving the reliability and stability of easily confused emotion recognition.
[0040] Step S5 (updating model parameters using the Adam optimization algorithm) specifically includes the following steps: S5: The program uses the Adam optimization algorithm to update the model parameters. Input total loss. Let the set of trainable parameters be... ,in These are the weights of the fully connected layer. It is a bias term. Classifier parameters. Calculate the gradient of the total loss with respect to each trainable parameter. Update parameters: in, For learning rate, This represents the set of all learnable parameters. After multiple iterations until the model converges, the final parameter configuration and classification model are output.
[0041] Step S6 (testing section) specifically includes the following steps: S6: After completing the training in steps S1–S5, fix the model parameters, input the speech to be tested or the speech samples in the test set into the trained emotion recognition model, perform forward inference, output the predicted emotion category, do not perform backpropagation or parameter update, calculate the accuracy, precision, recall, F1-score and confusion matrix based on the prediction results, and use them to evaluate the performance of the model in easily confused emotion recognition tasks and in real-world use environments.
[0042] Compared with existing technologies, this method has the following advantages: By invoking the adaptive wavelet transform module and setting different parameter combinations, different wavelet kernel functions are generated, producing corresponding multi-scale wavelet filter kernels for each path. This allows for targeted extraction of feature differences between easily confused sentiment pairs of different categories. The output features of each path are then processed by a gated attention mechanism to calculate fusion weights, completing the weighted fusion of multi-path features. During the training phase, a joint loss function of weighted cross-entropy loss and contrastive loss is constructed to constrain classification learning and feature separability. Parameter updates are completed during the deep learning network training process. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0044] Figure 1 This is a schematic diagram of the training process of the easily confused emotion recognition method based on multi-path feature extraction and gating fusion of the present invention; Figure 2 This is a schematic diagram illustrating the model construction of the easily confused emotion recognition method based on multi-path feature extraction and gating fusion according to the present invention. Figure 3 This is a flowchart of the model of the easily confused emotion recognition method based on multi-path feature extraction and gating fusion of the present invention. Detailed Implementation
[0045] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0046] As shown in the model construction diagram Figure 2 As shown, the specific implementation of this invention is divided into two stages and the following steps: S1: Data Input and Preprocessing S2: Multipath emotion feature extraction based on adaptive wavelet transform; S3: Weighted fusion of multipath emotional features by gating attention mechanism; S4: Construct a deep learning classification network and perform classification and loss calculation under the dual supervision of weighted cross-entropy loss and contrastive loss; S5: Use the Adam optimization algorithm to train the model parameters until the performance of the validation set meets the convergence condition, and obtain the final trained emotion recognition model; Testing phase: S6: Input the speech to be tested into the trained emotion recognition model, output the predicted category, and evaluate the model's recognition performance using metrics such as confusion matrix, accuracy, and recall, with a focus on its classification ability on easily confused emotion pairs.
[0047] Step S1 (data input and preprocessing) specifically includes the following steps: S1: Prepare a publicly available standard dataset for speech emotion recognition as the training data source. The dataset contains speech samples from multiple speakers and multiple emotion categories, divided into easily confused emotion pairs such as "happy-angry," "sad-neutral," and "fear-surprised." The input raw speech signal is pre-emphasized, framed, and windowed.
[0048] Step S2 (multi-path emotion feature extraction based on adaptive wavelet transform) specifically includes the following steps: S2: After obtaining the preprocessed emotional speech signal x(t) in step S1, it is input into the adaptive wavelet transform module to perform multi-scale time-frequency analysis on the speech signal. The wavelet basis function is controlled by learnable parameters β, γ, and a. These parameters are used to set the shape, center frequency, and scale of the wavelet basis.
[0049] This invention sets up a feature path for each type of easily confused emotion. c feature paths are constructed, each corresponding to a different set of wavelet parameters. c represents the path number, c∈[1,3]. Each path shares the same learnable mother wavelet function. Different wavelet kernel functions are generated from different parameter sets, forming a multi-scale wavelet filter kernel group within each path. Convolution operations are performed on the same speech signal at different resolutions to obtain the time-frequency features of the c-th path.
[0050] After processing through c feature paths, a set of multi-path sentiment features is obtained { , , ... , } represents multiple independent time-frequency feature paths formed by the same speech signal under different parameter configurations. Each path has the same structure but different parameters.
[0051] Step S3 (weighted fusion of multi-path emotion features by gating attention mechanism) specifically includes the following steps: S3.1: Obtain the time-frequency characteristics of each path from step S2. Then, the feature map of the c-th path... Perform global average pooling (GAP), iterating along the time dimension T, calculating the average value for each frequency sub-band, and obtaining the feature vector after average pooling. :
[0052] in, This is the feature vector obtained after average pooling of the feature map of this path. .
[0053] S3.2: Calculate the gating attention weights. Initialize the fully connected layer weights. and bias terms ,in For feature dimension, For output dimensions, bias terms Initialize as zero vector The feature vector after average pooling The input is fed into a fully connected layer, and the gated attention weights are calculated using a sigmoid activation function. :
[0054] in, This represents the Sigmoid activation function and its output. Weights indicate the importance of this set of features. This indicates the importance of the path feature in the current sample sentiment judgment and controls the contribution of the path in the fusion process.
[0055] S3.3: Calculate the weighted fusion features. This involves converting the original feature matrix of each path... With the calculated weights Perform element-wise multiplication, then sum the weighted results of all paths to obtain the final fused feature matrix. :
[0056] in, This represents element-wise multiplication. For the final weighted fusion feature, =3, Emotional characteristics after fusion This will be used for subsequent loss calculations and model training.
[0057] Step S4 (Classification and Loss Calculation under the Dual Loss Supervision Mechanism) specifically includes the following steps: S4.1: Construct a deep learning classification network and use the fused feature matrix obtained in step S3. It is converted into a one-dimensional feature vector through a pooling layer. , where i represents the sample number. Then... The input to the fully connected classification layer yields the raw scores for each emotion category. A softmax function is then applied to these raw scores to transform them into probability distributions for each emotion category. The input dimension of the fully connected layer is... The output dimension is Number of SoftMax output layer categories .
[0058] S4.2: Compare the predicted probabilities with the true labels, and adjust them according to the weights of each category. Calculate weighted loss The formula is as follows:
[0059] in, This represents the depth feature representation of the i-th sample. Let be the classification vector for the k-th class, where k is the total number of emotion categories. Let i be the true category label for sample i.
[0060] S4.3: Within the same training batch, the program constructs a set P of sample pairs based on the true labels. For each pair of samples in the set, it calculates their deep features. and The Euclidean distance is calculated, and the contrast loss is calculated based on the threshold δ. Furthermore, the inter-class separation in the feature space is enhanced by contrastive loss, as shown in the following formula:
[0061] in, The set of sample pairs constructed for easily confused sentiment pairs, For the set distance threshold, ∈ It is initialized to 0.8 and controls the minimum expected interval between samples of different categories.
[0062] S4.4: Calculate the total loss, which is a weighted sum of the weighted cross-entropy loss and the contrastive loss:
[0063] in, To compare the weighting coefficients of the loss, , Initialized to 0.5.
[0064] Step S5 (training model parameters using the Adam optimization algorithm) specifically includes the following steps: S5: The program uses the Adam optimization algorithm to update the model parameters. Input total loss. Let the set of trainable parameters be... ,in These are the weights of the fully connected layer. It is a bias term. Classifier parameters. Calculate the gradient of the total loss with respect to each trainable parameter. Update parameters: in, For learning rate, This represents the set of all learnable parameters. After multiple iterations until the model converges, the final parameter configuration and classification model are output.
[0065] Step S6 specifically includes the following steps: S6: The trained model is input using untrained speech samples. Only forward computation is performed, outputting the emotion category. Backpropagation is not performed based on the prediction results, and model parameters are not updated. Test samples are only used to calculate performance metrics such as accuracy, precision, recall, F1-score, and confusion error rate to evaluate the model's performance on easily confused emotion recognition tasks. For easily confused emotion pairs such as "happy / angry," "sad / neutral," and "fear / surprised," the Confusion Error Rate (CER) is introduced for quantitative analysis. Experimental results show that the method of this invention achieves an overall recognition accuracy of 91.7%, significantly improving upon traditional methods such as MFCC+SVM, CNN+Attention, and BiLSTM+Softmax. Specifically, the CER for "happy / angry" decreases from 21.37% to 9.72%, and for "sad / neutral" from 9.46% to 6.94%. Even in a noisy environment (SNR=10dB), the accuracy remains at 84.7%, significantly better than baseline methods, verifying the robustness and practicality of this invention in complex scenarios.
[0066] Table 1 shows the model's performance on the dataset.
[0067] In summary, the easily confused emotion recognition method based on multi-path feature extraction and gating fusion can effectively identify and distinguish easily confused emotions.
[0068] It should be noted that the above implementation of the easily confused emotion recognition method based on multi-path feature extraction and gating fusion provides the model structure, input-output relationship, parameter setting method and training process.
[0069] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for easily confused emotion recognition based on multi-path feature extraction and gated fusion, comprising the following steps: S1: data input and preprocessing S2: multi-path emotion feature extraction based on adaptive wavelet transform; S3: weighted fusion of multi-path emotion features by gated attention mechanism; S4: constructing a deep learning classification network, and performing classification and loss calculation under the dual supervision of weighted cross-entropy loss and contrastive loss; S5: training model parameters using the Adam optimization algorithm until the performance of the validation set meets the convergence condition to obtain a finally trained emotion recognition model; S6: inputting the test speech into the trained emotion recognition model to output the predicted category.
2. The method of claim 1, wherein the method is based on multi-path feature extraction and gated fusion. The step S1 includes: preparing a publicly available speech emotion recognition standard dataset as a training data source, which contains speech samples of multiple speakers and multiple emotion categories, and is divided into "happy-angry" "sad-neutral" "fear-surprise" easily confused emotion pair sample set; inputting the original speech signal to perform pre-emphasis, framing and windowing processing.
3. The method of claim 1, wherein the method is based on multi-path feature extraction and gated fusion. The step S2 comprises: inputting the preprocessed voice signal x(t) obtained in the step S1 into an adaptive wavelet transform module, performing multi-scale time-frequency analysis, setting c characteristic paths, c representing path number, c∈[1,3], each path sharing the same learnable mother wavelet function, and the parameters of the mother wavelet function are updated by the gradient descent method Generating the wavelet kernel function of the path, constituting a multi-scale wavelet filter kernel group, performing convolution operation on x(t) under multi-scale in the cth path to obtain the time-frequency feature of the path: , aggregate the path outputs to obtain a multi-path emotion feature set: { , ,..., }.
4. The method of claim 1, wherein the method is based on multi-path feature extraction and gated fusion. The step S3 includes: obtaining the time-frequency feature of each path from step S2 After that, the feature map of the cth path Perform global average pooling (GAP), traverse in the time dimension T, calculate the average value of each frequency sub-band, and obtain the average-pooled feature vector : （2） wherein is the average-pooled feature vector of the path feature map, ; Compute the gating attention weights, initialize the fully connected layer weights and bias terms where is the feature dimension, is the output dimension, the averaged pooled feature vector is input to the fully connected layer and the gating attention weights are computed by the Sigmoid activation function : （3） wherein, denotes a Sigmoid activation function, output denotes the importance of the feature set, weight denotes the importance of the path feature in the current sample emotion discrimination, controls the contribution size of the path in the fusion process; The weighted fusion features are calculated, the original feature matrix of each path is element-wise multiplied with the calculated weight, and then the weighted results of all paths are added to obtain a final fusion feature matrix : （4） wherein, denotes an element-wise multiplication operation, is the final weighted fused feature, the fused emotion feature which will be used for subsequent loss calculation and model training. Through this step, the originally independent multi-path features are selectively integrated in the path dimension, highlighting the path information with higher discrimination and suppressing redundant paths.
5. The method of claim 1, wherein the method is based on multi-path feature extraction and gated fusion. The step S4 includes: constructing a deep learning classification network, converting the fusion feature matrix obtained in step S3 into a one-dimensional feature vector through a pooling layer , i represents a sample number, and then inputting the one-dimensional feature vector into a full connection classification layer to obtain original scores corresponding to each emotion category, and applying a softmax function to the original scores to convert them into a probability distribution of each emotion category. The predicted probabilities are compared to the true labels and a weighted loss is computed for each class according to the weights The weighted loss is computed as follows: （5） wherein, represents a depth feature representation of the i-th sample, is a classification vector of the k-th class, k is the total number of emotion classes, is the true class label of the sample i; Within the same training batch, the program constructs a set of sample pairs P according to the true labels, for each pair of samples in the set, calculates their deep features and The Euclidean distance, and according to the threshold value δ, calculates the contrast loss , and then enhances the class separation in the feature space through the contrast loss, the formula is as follows: （6） wherein, is a set of sample pairs constructed for the confusing emotions, is a set distance threshold, ∈ controls the minimum expected separation between samples of different classes. The total loss is calculated, which is the weighted sum of the weighted cross-entropy loss and the contrastive loss: （7） wherein, is a weight coefficient of the contrast loss, By this step, the model considers the class interval constraint of the feature space while optimizing the overall classification accuracy, further improving the reliability and stability of the easily confused emotion recognition.
6. The method of claim 1, wherein the method is based on multi-path feature extraction and gated fusion. The step S5 includes: the program updates the model parameters by using the Adam optimization algorithm, and inputs the total loss , let the set of trainable parameters be , where is the weight of the fully connected layer, is the bias term, is the classifier parameter, and the gradient of the total loss with respect to each trainable parameter is calculated ; update the parameters: （8） wherein, is the learning rate, denotes the set of all learnable parameters, after multiple iterations until the model converges, outputting the final parameter configuration and classification model.
7. The method of claim 1, wherein the method is based on multi-path feature extraction and gated fusion. The step S6 includes: after completing the training of steps S1-S5, fixing the model parameters, inputting the test speech or test set speech samples into the trained emotion recognition model, performing forward inference, outputting the predicted emotion category, not performing back propagation and parameter updating, calculating the accuracy, precision, recall, F1-score and confusion matrix, etc. according to the prediction result, which are used to evaluate the performance of the model in the easily confused emotion recognition task and the actual use environment.