A speech emotion recognition method and system based on multi-dimensional information perception strategy

Abstract

The present invention provides a method and system for speech emotion recognition based on a multidimensional information perception strategy, which relates to the field of speech emotion recognition technology, including: extracting WavLM features and MFCC features; constructing a multidimensional information perception and cross-dimensional interleaving module through a Transformer layer, an MDIP layer, a CDI layer, and a convolutional layer; processing the WavLM features and the MFCC features respectively through the multidimensional information perception and cross-dimensional interleaving module to obtain a first feature map and a second feature map; inputting the first feature map and the second feature map into SENet for feature fusion, and outputting fused features through a full connection operation and a batch normalization operation; classifying the fused features through a classifier to predict the emotion of the speech to be recognized. The multidimensional information perception and cross-dimensional interleaving module proposed in the present invention can effectively improve the accuracy, robustness, and versatility of the emotion recognition model in multiple data sets and multiple situations.

Description

Technical Field

[0001] The present invention relates to the technical field of speech emotion recognition, and in particular to a speech emotion recognition method and system based on a multi-dimensional information perception strategy. Background Art

[0002] Speech is one of the basic means of human communication and can effectively convey the speaker's emotions through sound wave signals. With the rapid development of human-computer interaction systems, in-depth analysis of speech signals has become one of the key technologies to improve the quality of human-computer interaction. Speech not only contains rich information, but also carries complex emotional data and can express emotional reactions to objects, scenes or events. The process of automatically identifying human emotions from speech signals is called Speech Emotion Recognition (SER), which has become one of the most important research and development fields in the past few decades. At present, speech emotion recognition technology has been widely used in many fields such as education, medical care and services, and continues to receive widespread attention. The key technology of SER lies in the extraction of emotional features and the construction of emotion recognition models. In the process of realizing the present invention, the applicant found that most of the current research still focuses on the fusion of traditional features, and there is a lack of in-depth exploration of the correlation and complementarity between traditional features and self-supervised pre-training features. Most methods still directly use MFCC as network input, fail to fully extract the spatiotemporal features in speech, and ignore the balance between time domain and frequency domain features, making it difficult to effectively construct a complete emotional representation. Furthermore, traditional feature extraction networks typically model at a fixed scale, while emotional information is often distributed across scales. Single-scale modeling fails to fully capture emotional characteristics. Existing speech emotion recognition methods still have significant shortcomings in feature extraction and multi-source information fusion, limiting their application and widespread adoption in complex environments. Emotional information in speech signals exhibits complex distributional characteristics, often existing simultaneously across different time scales and frequency ranges. Traditional feature extraction methods, limited by fixed receptive fields or shallow perceptual capabilities, are unable to fully capture the underlying emotional characteristics of speech, resulting in poor performance in recognizing strong, subtle, or rapidly changing emotional states. Most existing methods rely solely on handcrafted acoustic features such as MFCCs, ignoring their complementary relationship with the deeper semantic features extracted by self-supervised learning models. This limits the models' expressive and discriminative capabilities, making them incapable of adapting to diverse and complex emotional expression scenarios. Furthermore, most current speech emotion recognition models employ a single-scale, single-path feature extraction network architecture, lacking flexible modeling mechanisms for multi-scale features. This makes them incapable of performing stably across varying speech rates, speaking styles, and signal-to-noise ratios, resulting in poor generalization performance.

[0003] Therefore, how to improve the accuracy, robustness and versatility of speech emotion recognition models in multiple data sets and multiple scenarios has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related art, and discloses a speech emotion recognition method and system based on a multi-dimensional information perception strategy, which can extract more comprehensive and prominent emotional information from speech signals, improve the emotion recognition ability of the model, and effectively improve the accuracy, robustness and versatility of the emotion recognition model in multiple data sets and multiple situations.

[0005] The first aspect of the present invention discloses a speech emotion recognition method based on a multidimensional information perception strategy, comprising: extracting WavLM features: extracting features of the speech to be recognized through the WavLM-Large model to obtain WavLM features; extracting MFCC features: extracting Mel-frequency cepstral coefficients of the speech to be recognized through an audio feature extraction tool to obtain MFCC features; constructing a multidimensional information perception and cross-dimensional interleaving module: the multidimensional information perception and cross-dimensional interleaving module includes a Transformer layer, an MDIP layer, a CDI layer and a convolutional layer (conv) connected in sequence; wherein the Transformer layer processes the WavLM features or MFCC features to generate a frequency feature map and a time feature map; the MDIP layer processes the WavLM features or the MFCC features to generate a frequency feature map and a time feature map; the MDIP layer processes the WavLM features or the MFCC features to generate a frequency feature map and a time feature map through a frequency multidimensional The information perception operation obtains frequency features and obtains time features through the time multidimensional information perception operation; the CDI layer takes the time features and frequency features from the MDIP layer as input, and enhances the representation ability of time features and frequency features by feature interleaving; processing WavLM features: processing WavLM features in turn through multi-layer multidimensional information perception and cross-dimensional interleaving modules to obtain the first feature map; processing MFCC features: processing MFCC features in turn through multi-layer multidimensional information perception and cross-dimensional interleaving modules to obtain the second feature map; feature fusion: input the first feature map and the second feature map into SENet for feature fusion, and output the fused features through full connection operation and batch normalization operation; emotion classification: classify the fused features through the classifier to predict the emotion of the speech to be recognized.

[0006] In this technical solution, WavLM-Large is a Transformer-based self-supervised pre-training model proposed by Microsoft. It has powerful context modeling capabilities and the ability to extract deep representations of audio signals. The extracted speech embeddings are used for general speech understanding tasks. Mel-Frequency Cepstral Coefficients (MFCC) is a feature extraction method commonly used in audio signal processing. MFCC converts audio signals into a series of cepstral coefficients by simulating the auditory characteristics of the human ear. These coefficients can capture important features of the sound. The present invention uses T×39-dimensional MFCC (composed of its static features T×13 dimensions and its first and second order differences, where T is the number of frames). The MDIP layer can extract more comprehensive and prominent emotional information from speech signals by adaptively perceiving multi-granular emotional information features in different dimensions of time and frequency, thereby improving the model's emotion recognition ability. The CDI layer can interactively calibrate time domain and frequency domain features, model the complementary relationship between feature information of different dimensions, thereby more richly describing emotional information and enhancing the feature's emotional representation ability.

[0007] According to the speech emotion recognition method based on the multi-dimensional information perception strategy disclosed in the present invention, preferably, a multi-scale perception attention mechanism is introduced between the Transformer layer and the MDIP layer, and the multi-scale time-frequency features are modeled respectively to obtain multi-scale time feature maps and frequency feature maps.

[0008] According to the speech emotion recognition method based on the multidimensional information perception strategy disclosed in the present invention, preferably, the calculation process of the frequency multidimensional information perception operation specifically includes: applying a sliding window on the frequency dimension of each time frame to extract local frequency domain features, thereby selecting specific frequency points to capture the emotional correlation between different frequency bands in the frequency domain.

[0009] According to the speech emotion recognition method based on the multidimensional information perception strategy disclosed in the present invention, preferably, the calculation process of the time multidimensional information perception operation specifically includes: applying a sliding window along the time dimension at each frequency point to extract local time features to capture short-term and long-term dynamic changes in the time domain, wherein short-term dynamic changes are instantaneous changes in emotions, and long-term dynamic changes refer to the dependency trends between emotion changes.

[0010] According to the speech emotion recognition method based on the multi-dimensional information perception strategy disclosed in the present invention, preferably, the calculation process of the CDI layer specifically includes:

[0011] Information enhancement:

[0012] Receives the frequency feature X generated by the MDIP layer freq and time feature X timeAs input, generate enhanced features:

[0013]

[0014]

[0015] IEM(x)=conv(Contrast(Conv(x)))

[0016] Contrast(x)=s(conv(x-AP(x)))·x+x

[0017] Among them, IEM(x) represents the information enhancement operation, x represents the input feature, represents the enhanced features of temporal features, Represents the enhanced features of frequency features, AP() represents the average pooling operation, Contrast represents the contrast enhancement operation, Conv represents the convolution operation, and s represents the Sigmoid activation function;

[0018] Interdimensional Interweaving:

[0019] First, receive Two different attention maps are obtained through the spatial attention operation:

[0020]

[0021] Among them, W time represents the temporal feature attention map, W freq represents the frequency feature attention map, SA represents the spatial attention operation;

[0022] Then, in and Cross-dimensional feature complementation:

[0023]

[0024] in, represents the weighted time feature, Represents the weighted frequency feature;

[0025] Subsequently, the weighted features are concatenated and fused through two convolutional layers to generate the collective weight map W:

[0026]

[0027] Among them, Sigmoid() represents the Sigmoid activation function, conv represents the convolution operation, Indicates channel-based splicing operation;

[0028] Finally, use the set weight graph W to and Re-correct and fuse through the convolution layer to generate output features of multi-dimensional information:

[0029]

[0030] Among them, X MDI Represents a multi-dimensional information feature map.

[0031] According to the speech emotion recognition method based on multi-dimensional information perception strategy disclosed by the present invention, preferably, the Transformer layer is a Vanilla Transformer Encoder.

[0032] The second aspect of the present invention discloses a speech emotion recognition system based on a multi-dimensional information perception strategy, comprising: a memory for storing program instructions; a processor for calling the program instructions stored in the memory to implement a speech emotion recognition method based on a multi-dimensional information perception strategy as any of the above technical solutions.

[0033] Compared with the prior art, the beneficial effects of the present invention include at least the following: MDIP solves the problem of large differences in the distribution of emotional information in speech signals between the frequency domain and the time domain, and the model can adaptively obtain emotional feature information of different scales. The CDI proposed in the present invention is mainly used to fuse emotional information of different dimensions in the time domain and the frequency domain, reduce redundant information in speech emotion embedding, and model the complementary relationship between feature information of different dimensions, thereby enhancing the emotional representation ability of speech embedding. In addition, the present invention adopts a dual-channel network structure, and uses a combination of traditional manual features MFCC and self-supervised pre-training features WavLM to perform speech emotion recognition, which has brought certain development to speech emotion recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flow chart of a method for speech emotion recognition based on a multi-dimensional information perception strategy according to an embodiment of the present invention is shown.

[0035] Figure 2 A schematic block diagram of a speech emotion recognition system based on a multi-dimensional information perception strategy according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0037] like Figure 1 As shown, according to one embodiment of the invention, a method for speech emotion recognition based on a multi-dimensional information perception strategy is disclosed, comprising:

[0038] Step 1: Extract WavLM features: Extract features of the speech to be recognized using the WavLM-Large model to obtain WavLM features.

[0039] Step 2: Extract MFCC features: Use the audio feature extraction tool to extract the Mel-frequency cepstral coefficients of the speech to be recognized to obtain MFCC features;

[0040] Step 3: Construct a multidimensional information perception and cross-dimensional interleaving module: The multidimensional information perception and cross-dimensional interleaving module includes a Transformer layer, an MDIP layer, a CDI layer, and a convolutional layer connected in sequence; wherein the Transformer layer (Vanilla Transformer Encoder) processes the WavLM features or MFCC features to generate frequency feature maps and time feature maps; the MDIP layer obtains frequency features through frequency multidimensional information perception operations and obtains time features through time multidimensional information perception operations; the CDI layer uses the time features and frequency features from the MDIP layer as input and enhances the representation capabilities of the time features and frequency features by feature interleaving;

[0041] Step 4: Process WavLM features: Process WavLM features through multiple layers of multi-dimensional information perception and cross-dimensional interleaving modules to obtain the first feature map;

[0042] Step 5: Processing MFCC features: Processing MFCC features through multiple layers of multi-dimensional information perception and cross-dimensional interleaving modules to obtain a second feature map;

[0043] Step 6, feature fusion: Input the first feature map and the second feature map into SENet for feature fusion, and output the fused features through full connection operation and batch normalization operation;

[0044] Step 7, emotion classification: classify the fused features through a classifier to predict the emotion of the speech to be recognized.

[0045] Furthermore, a multi-scale perceptual attention mechanism is introduced between the Transformer layer and the MDIP layer to model the multi-scale time-frequency features respectively to obtain multi-scale time feature maps and frequency feature maps.

[0046] Furthermore, the computational process for frequency multi-dimensional information perception specifically involves applying a sliding window along the frequency dimension of each time frame to extract local frequency domain features, thereby selecting specific frequency points to capture the emotional correlation between different frequency bands in the frequency domain. The computational process for temporal multi-dimensional information perception specifically involves applying a sliding window along the time dimension at each frequency point to extract local temporal features to capture both short-term and long-term dynamic changes in the temporal domain.

[0047] Furthermore, the calculation process of the CDI layer specifically includes:

[0048] Information enhancement:

[0049] Receives the frequency feature X generated by the MDIP layer freq and time feature X time As input, generate enhanced features:

[0050]

[0051]

[0052] IEM(x)=conv(Contrast(Conv(x)))

[0053] Contrast(x)=s(conv(x-AP(x)))·x+x

[0054] Among them, IEM(x) represents the information enhancement operation, x represents the input feature, represents the enhanced features of temporal features, Represents the enhanced features of frequency features, AP() represents the average pooling operation, Contrast represents the contrast enhancement operation, Conv represents the convolution operation, and s represents

[0055] Interdimensional Interweaving:

[0056] First, receive Two different attention maps are obtained through the spatial attention operation:

[0057]

[0058] Among them, W time represents the temporal feature attention map, W freq represents the frequency feature attention map, SA represents the spatial attention operation;

[0059] Then, in and Cross-dimensional feature complementation:

[0060]

[0061] in, represents the weighted time feature, Represents the weighted frequency feature;

[0062] Subsequently, the weighted features are concatenated and fused through two convolutional layers to generate the collective weight map W:

[0063]

[0064] Among them, Sigmoid() represents the Sigmoid activation function, and conv represents the convolution operation; Indicates channel-by-channel splicing operation.

[0065] Finally, use the set weight graph W to and Re-correct and fuse through the convolution layer to generate output features of multi-dimensional information:

[0066]

[0067] Among them, X MDI Represents a multi-dimensional information feature map.

[0068] According to another embodiment of the present invention, the specific application process and principle of the above-mentioned speech emotion recognition method based on multidimensional information perception strategy are also disclosed: corresponding to the speech emotion recognition method based on multidimensional information perception strategy disclosed in the above embodiment, a network structure of a speech emotion recognition model based on multidimensional information perception and interweaving strategy is built (named MDIPI-Net in the present invention). In this model, WavLM features and MFCC features are processed respectively based on the MDIPI module (multidimensional information perception and cross-dimensional interweaving module) through two branches, and then the feature maps of the two branches are fused through SENet (Squeeze-and-Excitation Networks), and after full connection operation and batch normalization operation (FC+BN), they are input into the classifier for emotion prediction. SENet can enhance the perception ability of the convolutional neural network to different features by adaptively adjusting the feature map weights of each channel, thereby improving the classification performance of the model. In the present invention, SENet adaptively assigns weights to different channels through global information, so that the fused features have stronger emotion differentiation ability.

[0069] The specific steps of MDIPI-Net are:

[0070] S1, data preprocessing:

[0071] WavLM features are deep features extracted by the self-supervised pre-training model WavLM-Large. WavLM-Large is a Transformer-based self-supervised pre-training model proposed by Microsoft. It has powerful context modeling capabilities and the ability to extract deep representations of audio signals. The extracted speech embeddings are used for general speech understanding tasks.

[0072] MFCC features are traditional hand-crafted features extracted using audio feature extraction tools, namely Mel-Frequency Cepstral Coefficients (MFCCs). They use T×39-dimensional MFCCs (composed of their static features T×13 dimensions and their first- and second-order differences, where T is the number of frames).

[0073] S2, the present invention uses a dual-channel network structure to extract emotional features, aiming to extract rich time domain information and frequency domain emotional information:

[0074] S21: In the field of speech emotion recognition, emotional information is typically distributed across different time-frequency ranges. Traditional neural networks struggle to capture long-range dependencies due to issues such as limited receptive fields and vanishing gradients. To enhance the model's ability to perceive emotional information, we propose a MDIP module (MDIP layer) based on a dilated convolution strategy.

[0075] MDIP network structure: In order to extract the frequency domain information of emotional features, the frequency multi-dimensional information perception operation (FMDP) uses a sliding window method to extract local frequency domain features in the frequency dimension of each time frame. f , FMDP performs sparse sampling on the sliding window, thereby selecting specific frequency points to capture the emotional correlation between different frequency bands in the frequency domain. The formal description of FMDP is as follows:

[0076]

[0077] in, Indicates the frequency domain position (f i , query on t), K f and V f are the key and value selected from the frequency domain; r f ∈N + It is the frequency expansion rate used to adjust the sparsity of the selected frequency point. The position of the selected frequency point in the sliding window (f′ i , t) is defined as follows:

[0078]

[0079] Among them, w f Indicates the size of the frequency domain window.

[0080] Similarly, to extract sentiment features in the temporal domain, Temporal Multi-Dimensional Perception (TMDP) applies a sliding window along the temporal dimension at each frequency point to extract local temporal features. This atrous sliding window selects a time frame based on the expansion rate, aiming to capture both short-term and long-term dynamics in the temporal domain. The formal description of TMDP is as follows:

[0081]

[0082] in, Indicates the time domain position (f, t i ), K t and V t are the keys and values ​​selected from the time domain; r t ∈N + is the time dilation rate used to adjust the sparsity of the selected time frame. The selected time frame position (f,t′) in the sliding window i ) is defined as follows:

[0083]

[0084] where w t Indicates the size of the time window.

[0085] In order to capture multi-scale frequency and time features, the present invention introduces a multi-scale perceptual attention mechanism (MSPA) based on the extracted source features. MSPA models multi-scale time-frequency features separately. Specifically, for a given feature map X, the corresponding query (Q), key (K), and value (V) are obtained by linear projection. Subsequently, the channel is divided into n different heads, and a multi-scale information perception (MDP) operation is performed at different expansion rates. The calculation formula of MSPA is as follows:

[0086] h i =MDP(Q i , K i , i , r i , i)

[0087] X=Linear(Concat(h1,h2,...,h n ))

[0088] Among them, h irepresents the emotional information features obtained under the i-th head, r i represents the expansion coefficient of the i-th head, h1, h2, ... h n Belong to h i , Q i ,、K i ,、V i Represents the feature map of the i-th head, Concat represents the splicing operation, and Linear represents the full connection operation.

[0089] In order to capture the importance differences of multi-scale feature information, this paper introduces a weight calculation unit (WCU) to calculate the importance of features based on the attention weights obtained from the Transformer. The formal description of WCU is as follows:

[0090]

[0091] Among them, aw (i) represents the attention weight from the i-th head of MSPA, and (r, c) represents the weight aw (i) The element in row r and column c, where R and C are the row length and height of matrix aw. Concat represents the concatenation operation, and Softmax represents the Softmax activation function. The feature weight of the i-th head representing the frequency feature, belong Represents the feature weight of the i-th head of the time domain feature (time feature), belong imp freq Indicates the weight coefficient of frequency feature in emotional information, imp time Represents the weight coefficient of the time domain feature (time feature) in the emotional information.

[0092] S22, in the field of SER, the interaction between features of different dimensions plays a vital role. Time features contain more dynamic information about the changes of the signal over time, while frequency features reflect more characteristics related to frequency distribution. Therefore, in order to more effectively coordinate the properties of features of different dimensions, the present invention proposes a cross-dimensional interweaving (CDI) module. The CDI module (CDI layer) fuses time and frequency features and enhances the emotional discriminant feature representation in the output.

[0093] The CDI module takes the temporal and frequency features from MDIP as input and enhances the representation capability of these two types of features in an interleaved manner. CDI is divided into two stages:

[0094] Information Enhancement Module (IEM);

[0095] Cross-Dimensional Interweaving Unit (CDIU).

[0096] Specifically:

[0097] The IEM module receives the frequency signature X generated by the MDIP freq and time feature X time As input, the mutated edge enhancement features are generated:

[0098]

[0099] Specifically, the fusion of features from different dimensions typically involves upsampling or downsampling operations, implemented through element-by-element addition. However, these operations can dilute boundary information in important regions, leading to feature redundancy. In speech signals, emotionally charged speech is often accompanied by dramatic energy fluctuations. By extracting and enhancing these boundary features, core emotional characteristics can be more accurately captured.

[0100] Therefore, the present invention adopts the IEM module to effectively enhance the boundary area in the feature map while retaining the original feature information. Its formal description is as follows:

[0101] IEM(x)=conv(Contrast(Conv(x)))

[0102] Contrast(x)=s(conv(x-AP(x)))·x+x

[0103] Where AP() stands for average pooling operation.

[0104] In CDIU, the input features come from the enhanced feature information of the previous layer. First, two different attention maps are obtained through the spatial attention operation:

[0105]

[0106] Among them, SA represents the spatial attention mechanism, which is implemented through global average pooling and maximum pooling operations, followed by convolutional layers and Sigmoid activation functions.

[0107] Next, in order to obtain a more comprehensive feature expression, cross-dimensional feature complementation is performed between frequency features and time features to obtain weighted features:

[0108]

[0109] In order to further explore the interaction information between these two types of features, the present invention concatenates the weighted features and fuses them through two convolutional layers to generate a set weight map W, which is calculated as follows:

[0110]

[0111] Finally, use the set weight graph W to and The re-correction is performed and fused through the convolutional layer to generate the output features of multi-dimensional information, as shown below:

[0112]

[0113] In this way, features from different dimensions are used and combined with the attention mechanism to generate an adaptive attention map, while calibrating the information of features from different dimensions. In addition, in the process of generating the set weight map, the module also uses the convolution layer to re-optimize the input features, thereby generating a more refined and more adaptable output feature X MDI .

[0114] S3, combines X from different source features MDI The input is into SENet, and weights are adaptively assigned to different channels through global information, so that the fused features have stronger emotion differentiation ability. The final emotion feature is input into the classifier to predict the emotion expressed by the current speech.

[0115] like Figure 2 As shown, according to another embodiment of the present invention, a speech emotion recognition system 500 based on a multi-dimensional information perception strategy is also disclosed, including: a memory 501 for storing program instructions; a processor 502 for calling the program instructions stored in the memory to implement the speech emotion recognition method based on the multi-dimensional information perception strategy as in the above embodiment.

[0116] All or part of the steps in the various methods of the above embodiments can be completed by controlling related hardware through a program. The program can be stored in a readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other readable medium that can be used to carry or store data.

[0117] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A speech emotion recognition method based on multi-dimensional information perception strategy, characterized in that: include: Extract WavLM features: Extract features of the speech to be recognized using the WavLM-Large model to obtain WavLM features; Extracting MFCC features: extracting Mel-frequency cepstral coefficients of the speech to be recognized using an audio feature extraction tool to obtain MFCC features; Constructing a multidimensional information perception and cross-dimensional interleaving module: the multidimensional information perception and cross-dimensional interleaving module includes a Transformer layer, an MDIP layer, a CDI layer, and a convolutional layer connected in sequence; wherein the Transformer layer processes the WavLM features or the MFCC features to generate a frequency feature map and a time feature map; the MDIP layer obtains frequency features through a frequency multidimensional information perception operation and obtains time features through a time multidimensional information perception operation; the CDI layer uses the time features and frequency features from the MDIP layer as input and enhances the representation capabilities of the time features and frequency features by feature interleaving; The calculation process of the frequency multi-dimensional information perception operation specifically includes: Applying a sliding window to extract local frequency domain features in the frequency dimension of each time frame, thereby selecting specific frequency points to capture the emotional correlation between different frequency bands in the frequency domain, wherein the specific frequency points refer to the frequency points selected within the sliding window; The calculation process of the temporal multi-dimensional information perception operation specifically includes: Apply a sliding window along the time dimension at each frequency point to extract local time features to capture the instantaneous changes of emotions in the time domain and the dependency trends between emotion changes; Processing WavLM features: sequentially processing the WavLM features through multiple layers of the multi-dimensional information perception and cross-dimensional interleaving modules to obtain a first feature map; Processing MFCC features: sequentially processing the MFCC features through multiple layers of the multi-dimensional information perception and cross-dimensional interleaving modules to obtain a second feature map; Feature fusion: Input the first feature map and the second feature map into SENet for feature fusion, and output the fused features through full connection operation and batch normalization operation; Emotion classification: classifying the fused features through a classifier to predict the emotion of the speech to be recognized.

2. The speech emotion recognition method based on multidimensional information perception strategy according to claim 1 is characterized in that: A multi-scale perceptual attention mechanism is introduced between the Transformer layer and the MDIP layer to model multi-scale time-frequency features respectively to obtain multi-scale time feature maps and frequency feature maps.

3. The speech emotion recognition method based on multidimensional information perception strategy according to claim 1 is characterized in that: The calculation process of the CDI layer specifically includes: Information enhancement: Receives the frequency feature X generated by the MDIP layer freq and time feature X time As input, generate enhanced features: IEM(x)=conv(Contrast(Conv(x))) Contrast(x)=s(conv(x-AP(x)))·x+x Among them, IEM(x) represents the information enhancement operation, x represents the input feature, represents the enhanced features of temporal features, Represents the enhanced features of frequency features, AP() represents the average pooling operation, Contrast represents the contrast enhancement operation, Conv represents the convolution operation, and s represents the Sigmoid activation function; Interdimensional Interweaving: First, receive Two different attention maps are obtained through the spatial attention operation: Among them, W time represents the temporal feature attention map, W freq represents the frequency feature attention map, SA represents the spatial attention operation; Then, in and Cross-dimensional feature complementation: in, represents the weighted time feature, Represents the weighted frequency feature; Subsequently, the weighted features are concatenated and fused through two convolutional layers to generate the collective weight map W: Among them, Sigmoid() represents the Sigmoid activation function, conv represents the convolution operation, and ○ represents the channel-by-channel splicing operation; Finally, use the set weight graph W to and Re-correct and fuse through convolutional layers to generate output features of multi-dimensional information: Among them, X MDI Represents a multi-dimensional information feature map.

4. The speech emotion recognition method based on multidimensional information perception strategy according to claim 1 is characterized in that: The Transformer layer is a Vanilla Transformer Encoder.

5. A speech emotion recognition system based on multi-dimensional information perception strategy, characterized in that: include: a memory for storing program instructions; A processor, configured to call the program instructions stored in the memory to implement the speech emotion recognition method based on a multidimensional information perception strategy as described in any one of claims 1 to 4.

Images