Emotion Recognition Method Based on Multi-Domain Attention Fusion Network

Abstract

This invention discloses an emotion recognition method based on a multi-domain attention fusion network. The method first acquires EEG signals and preprocesses them, then extracts channel features using a spatial-frequency domain hybrid channel attention module to capture weighted attention features. Next, the weighted attention features are processed by a CNN module to extract spatial and frequency domain features from the EEG signals. Then, an improved multidimensional self-attention mechanism is combined with BiSRU to capture the combined features of spatial and frequency domains. Finally, the combined features are classified using a Softmax function to output the emotion recognition result. This invention enhances the ability to capture long-term dependencies in EEG signals and effectively improves the performance and efficiency of emotion recognition.

Description

Technical Field

[0001] This invention relates to methods for electroencephalogram (EEG) signal processing and emotion recognition, and particularly to an EEG signal emotion recognition method based on a multi-domain attention fusion network. Background Technology

[0002] Emotions play a vital role in daily life. The presence of negative emotions increases the risk of mental health problems such as depression and schizophrenia. Therefore, the recognition and identification of emotions are crucial for emotion regulation. Electroencephalography (EEG), as a non-invasive method for monitoring brain activity, has been widely used in fields such as emotion recognition, medical diagnosis, and human-computer interaction.

[0003] Electroencephalogram (EEG) signals possess multidimensional features, including temporal, spatial, and spectral characteristics, which must be fully considered in emotion analysis. However, current methods fail to fully utilize these diverse EEG features and the local patterns among emotional features. In EEG-based emotion recognition, the introduction of deep learning models has significantly improved classification performance and generalization ability.

[0004] Some researchers have also extracted spectra from different frequency bands and fed them into convolutional neural networks (CNNs) for emotion analysis. However, CNNs neglect global information, while recurrent neural networks (RNNs) fail to capture spatial information and have low parallel efficiency. To address these issues, researchers have begun combining CNNs with RNNs for emotion recognition. Meanwhile, attention mechanisms have also attracted researchers' attention. While the self-attention mechanism in transformers can uncover temporal relationships in EEG, it does not consider the crucial spatial information in EEG.

[0005] Furthermore, it's important to note that most existing models utilize only a single feature or a combination of two features, failing to fully leverage the complementarity between spatial, frequency, and temporal features. Therefore, these models are limited to some extent. In addition, capturing local patterns of features across different dimensions also presents a challenge for EEG emotion recognition. Commonly used Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) have relatively more complex structures, leading to higher computational costs for training and inference. They typically have more parameters, increasing model complexity. Currently, the pooling methods used for EEG emotion recognition tasks mainly include max pooling and average pooling. Max pooling tends to capture the most salient local features, while average pooling focuses more on global information. However, emotion recognition tasks require capturing both local features and considering global patterns, which may prevent pooling operations from fully meeting task requirements. Furthermore, the methods for processing EEG signals can be further optimized to improve the accuracy and efficiency of emotion recognition. Summary of the Invention

[0006] This invention proposes a multi-domain attention fusion network-based emotion recognition method, aiming to address the problems of insufficient feature extraction, limited classification accuracy, and information loss during pooling in current EEG emotion recognition methods. Unlike traditional methods, this invention comprehensively considers features in the frequency, spatial, and temporal domains. It extracts the differential entropy features of each channel of the EEG signal and combines them with channel information. A spatial-frequency domain hybrid channel attention mechanism dynamically assigns weights to the importance of each channel, thereby extracting spatial information of the EEG signal more accurately. Compared with traditional pooling methods, this invention employs bilinear downsampling pooling (BD pooling), which can retain more detailed information while highlighting important features, thus improving model performance. Furthermore, this invention introduces a bidirectional simple recurrent unit (BiSRU) structure instead of the traditional RNN structure, utilizing its bidirectional characteristics to capture long-term dependencies in the EEG signal. An improved multidimensional self-attention mechanism considers the intrinsic correlation between samples, further enhancing the model's performance and making it more accurate and efficient in EEG emotion recognition tasks.

[0007] According to the invention, the specific steps include:

[0008] Step 1: Acquire EEG signals and preprocess the EEG signals.

[0009] Step 2: The preprocessed EEG signal is passed through a spatial-frequency domain hybrid channel attention module to extract channel features and capture weighted attention features. Specifically:

[0010] Channel features are extracted from the preprocessed EEG signal, and differential entropy features are extracted for each channel. The channel features and differential entropy features are used to calculate the attention weights for each channel. These two weights are then summed, normalized, and applied to the original EEG signal, achieving a dynamic allocation of the importance of each channel in emotion recognition. A spatial-frequency domain hybrid channel attention module can simultaneously capture the spatial and frequency domain features of the EEG signal. Existing methods for EEG signal channels typically focus only on single-dimensional features, failing to fully consider the comprehensive feature information of the spatial and frequency domains. This invention, by fusing multi-dimensional features, can more comprehensively reflect the complexity of the EEG signal, thereby improving the accuracy of emotion recognition. This fusion approach can better preserve and utilize important information in the signal, improving model performance.

[0011] Step 3: Weighted attention features are extracted from the spatial and frequency domain features of the EEG signal using the CNN module.

[0012] The CNN module further extracts spatial and frequency domain information from the EEG signal. The BD pooling method proposed in this invention calculates corresponding values ​​by considering the data throughout the pooling window. This allows for the preservation of some detailed features while highlighting the most important features, thus improving classification performance and efficiency. Existing commonly used pooling methods, such as max pooling, may lose some detailed information, while BD pooling selectively preserves important details, thereby improving feature representativeness and classification accuracy.

[0013] Step 4: Combine the improved multidimensional self-attention with BiSRU to capture the combined features of spatial and frequency domain characteristics.

[0014] To fully consider the temporal information of EEG signals, an improved multidimensional self-attention mechanism is combined with BiSRU to capture the long-term dependencies of features in the output of step 3. While general RNN models such as LSTM and GRU can handle temporal dependencies, they have limitations in capturing complex multidimensional features. Introducing a self-attention mechanism allows the model to focus on the importance of each position in the input sequence, thus better capturing temporal information. Furthermore, BiSRU further enhances the model's ability to capture temporal dependencies through bidirectional information flow. Theoretically, the combination of self-attention and BiSRU can significantly improve the model's understanding of temporal features without significantly increasing computational costs.

[0015] Step 5: Classify the comprehensive features using the Softmax function and output the emotion recognition results.

[0016] The beneficial effects of this invention are as follows:

[0017] By introducing a multi-domain attention mechanism and a CNN-BSA architecture, this invention effectively integrates the frequency, spatial, and temporal features of EEG signals. Through a novel spatial-frequency hybrid channel attention mechanism, this invention dynamically assigns weights to each channel to more accurately extract key information from EEG signals. The BiSRU module, combined with improved multidimensional self-attention, not only enhances the ability to capture long-term dependencies in EEG signals but also effectively improves the performance and efficiency of emotion recognition. Furthermore, the BD pooling method proposed in this invention effectively reduces the computational cost of model parameters, better highlights important features, and retains key details, thereby providing a more reliable feature representation for emotion recognition and further improving classification accuracy. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the present invention;

[0019] Figure 2 A schematic diagram of the principle of spatial frequency domain hybrid channel attention;

[0020] Figure 3 This is a diagram illustrating the structure of a convolutional layer and the principle of BD pooling.

[0021] Figure 4 This is the schematic diagram of the SRU unit;

[0022] Figure 5 A schematic diagram of the improved multidimensional self-attention mechanism;

[0023] Figure 6 This is a graph showing the classification results for each subject in the dataset;

[0024] Figure 7 The result of channel weights for the spatial frequency domain hybrid channel attention on the dataset;

[0025] Figure 8 This is a comparison chart of the BD pooling method with max pooling and average pooling. Detailed Implementation

[0026] The present invention will be further described below with reference to specific embodiments. The following description is for illustrative and explanatory purposes only and does not constitute any limitation on the present invention.

[0027] like Figure 1 As shown, the implementation steps of the embodiments of the present invention are as follows:

[0028] Step 1: Acquire EEG signals and preprocess them, including downsampling to 128Hz and removing EOG artifacts. A 1-second baseline signal was also subtracted from the experimental brain spot signals. Furthermore, the EEG signals were divided into 3-second intervals for easier subsequent analysis.

[0029] Step 2: Apply a spatial-frequency domain hybrid channel attention module to the preprocessed EEG signal for feature weighting. For example, perform spatial-frequency domain hybrid channel attention weighting on the preprocessed EEG signal. Figure 2 As shown, the importance of each channel in emotion recognition can be dynamically adjusted, and spatial and frequency domain information are considered simultaneously, thereby more accurately determining the weights of key channels. The specific steps are as follows:

[0030] Step 2-1: Perform a Fourier transform on the EEG signal and use the differential entropy formula to extract the frequency domain features H(x) of a single channel:

[0031] H(x)=-∫p(x)log[p(x)]dx (1)

[0032] Where x represents the EEG signal, and p(x) represents the probability density function of x; the frequency domain features of each sample are concatenated from the c channels to obtain the frequency domain features of each sample.

[0033] Step 2-2: Perform mean pooling on each channel of the preprocessed EEG sample to obtain the following channel statistics:

[0034]

[0035] in It is the average value of the j-th channel. Spatial domain features are obtained through dimensionality transformation.

[0036] Steps 2-3: The extracted frequency domain features and spatial domain features are weighted using a weight matrix. The mechanism of the spatial-frequency domain hybrid channel attention is represented as follows:

[0037] p = sigmoid(W s2 ·(tanh(W s1 ·F s +b s1 )+b s2 )+W f2 ·(tanh(W f1 ·F f +b f1 )+b f2 )+b)(3)

[0038] Among them W s1 and W f1 W represents the dimension-reduced (dimension reduction ratio R) weighted matrices of the spatial and frequency domain features, respectively. s2 and W f2 b represents the increased-dimensional weighted matrix representing the spatial and frequency domain features, respectively. s1 and b f1 b is the bias term for the dimensionality reduction layer. s2 and b f2 is the bias term for the dimensionality-enhancing layer, and b is the bias term for the mixed feature weights. The tanh function is used for the nonlinear activation of the dimensionality-reduction layer. The Sigmoid function transforms the importance of channels into different probability distributions p = [p1, p2, ..., p...]. c ], which represents the importance of different channels.

[0039] Steps 2-4: Finally, EEG sample S i =[s1,s2,...,s c The probability is used as the weight for re-encoding. Therefore, the attention channel feature of the j-th (j = 1, 2, ..., c) channel extracted through spatial-frequency domain hybrid channel attention can be expressed as:

[0040] c j =p j ·s j (4)

[0041] Therefore, C = {C1, C2, ..., C} n} represents the extracted channel attention features, where the i-th extracted feature C i =[c1,c2,...,c c [Can be generated by S] i =[s1,s2,...,s c The corresponding channel in ] is p = [p1, p2, ..., p c The corresponding elements in the [] are multiplied one by one to obtain the result.

[0042] Step 3: Use a convolutional neural network to extract features from the attention-weighted signal, such as... Figure 3 As shown, bilinear downsampling pooling (BD pooling) is used to further extract features. The specific steps are as follows:

[0043] Step 3-1: The number of convolutional kernels is K, with a height equal to the number of channels, primarily used to capture spatial features, while the kernel width is designed to explore temporal information. This invention selects the Exponential Linear Unit (ELU) as the activation function for the convolution operation. Therefore, the i-th convolutional feature C′... i (i = 1, 2, ..., n) is formed by the i-th weighted attention feature C i It is obtained through convolution and activation operations.

[0044] Step 3-2: BD Pooling is used to pool the convolutional features. BD Pooling first performs a bilinear interpolation-like calculation on the input within each stride and window size, obtaining n corresponding interpolations. Then, the average of these interpolations is used to represent the features within the pooling window. Each feature encodes Q... i The result, expressed as the average of the pooled window interpolation, is given by the following formula:

[0045]

[0046] Among them, P i,j This is the value obtained after bilinear interpolation within the pooling window i. Specifically, the pooling window size is [h, w], and the stride is s. Bilinear interpolation is performed within each pooling window to obtain the aggregated feature representation.

[0047] Step 3-3: Finally, the feature dimensions are converted to conform to the input shape of the subsequent bidirectional simple recurrent unit, and dropout is applied to prevent overfitting and improve the generalization ability of the model, resulting in spatial and frequency domain features.

[0048] Step 4: Use a bidirectional simple recurrent unit (BiSRU) combined with an improved multidimensional self-attention mechanism to extract temporal features from the extracted features. The specific steps are as follows:

[0049] Step 4-1: Simple loop unit, such as Figure 4 As shown, it receives two inputs: the spatial and frequency domain features x at the current time step. t The input c of the previous time step t-1 The simple loop unit outputs two results: the output c of the current time step. t and hidden state h t , where state c t This can be viewed as a feature extracted from the current time step. The SRU unit performs state updates and output computation in the following manner:

[0050]

[0051] h t =r t ⊙tanh(c t )+(1-r t )⊙x t (7)

[0052] Where f t It's a forget gate, used to control the degree to which the current state is mixed with the previous state. It is a candidate state for the current time step. t It is a reset gate, used to control the impact of the previous state on the current state update.

[0053] The bidirectional simple recurrent unit (BRU) is an extension of the simple recurrent unit (CRU). By simultaneously processing the forward and reverse information of the sequence, the i-th output of the BRU network is ultimately {h′}. i |h′ i =BiSRU(Q i ),i=1,2,...,s};

[0054] Step 4-2: Through improved multidimensional self-attention mechanisms, such as Figure 5 As shown, the combined features of spatial and frequency domains are extracted from the output features of the bidirectional simple cyclic unit. Specifically:

[0055] Step 4-2-1: The improved multidimensional self-attention mechanism calculates the similarity within each sample from different points. Feature score vector z i It can be regarded as the i-th feature h′ i A representation of . The improved multidimensional self-attention mechanism adds two bias terms inside and outside the activation function, where z i Represented as:

[0056] z i =f(h′) i ) = W T σ(W1h i ′+b1)+b (8)

[0057] Among them, z i This represents the intrinsic similarity of the i-th coded EEG sample. The activation function σ uses ELU, where W and b are the weight and bias terms of the function, respectively. W1 is the weight parameter, and b1 is the bias term.

[0058] Step 4-3: Then, calculate the probability matrix P for each sample. ki =[p1,p2,...p i ,...,p s ]:

[0059] The probability of the i-th EEG sample can be expressed as:

[0060]

[0061] Step 4-4: Finally, the comprehensive feature extracted by the improved multidimensional self-attention mechanism is F. att ={F1,F2,...,F n The i-th attentional feature extracted by the improved multidimensional self-attention mechanism can be represented as:

[0062] F i =p i ·h′ i (10)

[0063] Step 5: Use the SoftMax classifier to process the extracted features F att ={F1,F2,...,F n Categorize:

[0064] P emo =SoftMax(W·F att +b) (11)

[0065] Where P i (i = 1, 2, ..., n) represents the predicted probability of the i-th EEG sample, P emo ={P1,P2,...,P n} represents the set of these samples, and W and b represent the weights and biases of SoftMax, respectively.

[0066] Finally, calculate the cross-entropy loss for all samples:

[0067]

[0068] Among them, Y iFor the true label of the i-th EEG sample, a lower cross-entropy loss is achieved. This corresponds to a higher recognition accuracy.

[0069] This invention utilizes the Demographic Emotion Analysis Database (DEAP), a multimodal dataset designed to study human emotional states. It records the electroencephalograms (EEGs) and peripheral physiological signals of 32 participants while they watch 40 one-minute video clips. These signals are recorded using 32 electrodes according to the international 10-20 system. Each participant rated their emotional experience on arousal, valence, liking / disliking, dominance, and familiarity, ranging from 1 to 9. Each video trial includes 60 seconds of experimental signal and 3 seconds of baseline signal. The dataset is preprocessed, including downsampling to 128 Hz and removal of electrooculography (EOG) artifacts. In this embodiment, only EEG signals are used, with a threshold of 5, and the experimental data are categorized into high and low classes based on arousal and valence levels.

[0070] For the dataset, a baseline signal of 1 second was subtracted from the experimental signal per second for each subject. In the recognition task, the EEG signal was segmented into 3-second segments. For the DEAP dataset, the 60-second experimental signal for each trial was divided into 20 segments. Since the DEAP dataset has 40 trials, 800 EEG samples were generated for each subject. The sampling rate of the dataset is 128Hz, so each sample in the DEAP dataset is 32×384 in size. In the experiments, 10-fold cross-validation was used, and the model was evaluated in a subject-dependent manner. In each cross-validation, the DEAP dataset included 720 training samples and 80 test samples. The average accuracy through 10 cross-validations was used as the result for a single subject, and the average result for all subjects was used as the final accuracy.

[0071] Table 1. Mean accuracy (%) and standard deviation of different methods for the Arousal and Valence classification tasks in DEAP.

[0072]

[0073] Table 1 shows the mean accuracy and standard deviation of this invention compared to other methods on the DEAP dataset. Compared to the three traditional methods (SVM, DT, and MLP), this invention demonstrates the best performance in emotion recognition from EEG signals. Specifically, this invention achieves mean accuracy of 99.78% and 99.81% in Arousal and Valence, respectively, surpassing seven state-of-the-art methods (DGCNN, Conti-CNN, CRAM, ACRNN, gcForest, FP-CapsNet, and TNAS). TNAS, which follows closely with the second-highest mean accuracy, still lags behind this invention by 1.1%. Figure 6 The report details the classification accuracy of the proposed method in terms of arousal and valence for the top 20 subjects in the DEAP dataset. Furthermore, the standard deviation of the proposed method is consistently below 1%, indicating minimal variation in results across different subjects and demonstrating higher consistency and reliability.

[0074] Table 2 Ablation experiments on DEAP

[0075]

[0076] To further demonstrate the effectiveness of the method of this invention, ablation experiments were conducted on the DEAP dataset to illustrate the impact of each module within the model. The results of the ablation experiments are shown in Table 2. First, the impact of BD Pooling was evaluated. Considering the necessity of pooling layers, the commonly used max pooling was used instead. The classification accuracy of max pooling was 99.04% and 99.03% on Arousal and Valence, respectively, which was lower than the 99.78% and 99.81% of this method. The results show that BD Pooling is more effective in extracting features. Second, the classification accuracy of extracting temporal features using traditional LSTM reached 87.28% and 83.73%, respectively, which was 11.76% and 15.3% lower than that using the BiSRU self-attention method (BSA), respectively. This indicates that BiSRU combined with the improved multidimensional self-attention mechanism can enable the model to more effectively capture the temporal feature information of EEG signals and fully consider the intrinsic connections between EEG signals. Furthermore, when the spatial and frequency domain information of EEG is not comprehensively considered through the Spatial-Frequency Domain Hybrid Channel Attention (SSDFCA) mechanism, the accuracy is reduced by 34.69% and 41.28% respectively compared to the proposed method. The final experimental results also show that each module has a significant impact on improving the performance of emotion recognition and classification.

[0077] To further explore the impact of spatial-frequency domain hybrid channel attention on EEG signals, an experiment was conducted to calculate the channel weights of EEG signals. Figure 7This shows the average channel weights for all subjects in the DEAP dataset. The DEAP dataset contains 32 channels. Analysis... Figure 7 It can be observed that in the DEAP dataset, the FC5, C4, P8, and O2 channels have significantly higher weights in both dimensions than other channels. A higher channel weight indicates greater importance for capturing emotion-related signals. This suggests that the spatial-frequency domain hybrid channel attention mechanism can effectively identify EEG signal channels closely related to emotions, which is of great significance for further improving the accuracy of EEG emotion recognition.

[0078] The advantages of the proposed BD pooling method over max pooling and average pooling, which are commonly used for emotion recognition in EEG signals, are further demonstrated. Figure 8 As shown in the figure, the average test accuracy of 32 subjects on the DEAP dataset in the Arousal dimension at each epoch of the 10x cross-validation experiment is as follows. The figure shows that the proposed BD pooling method achieves high accuracy at the 20th epoch, indicating that the method can highlight important features while retaining more detailed information, thereby improving classification performance and efficiency.

Claims

1. An emotion recognition method based on a multi-domain attention fusion network, characterized in that, Includes the following steps: Step 1: Acquire EEG signals and preprocess the EEG signals; Step 2: The preprocessed EEG signal is passed through the spatial-frequency domain hybrid channel attention module to extract channel features and capture weighted attention features; Step 3: Weighted attention features. The spatial and frequency domain features of the EEG signal are extracted through the CNN module. The specific implementation process is as follows: Step 3-1: The number of convolutional kernels is K, and their height is the same as the number of channels. The Exponential Linear Unit (ELU) is chosen as the activation function for the convolution operation. The i-th convolutional feature C... i ′,i=1,2,...,n is the i-th weighted attention feature C i Obtained through convolution and activation operations; Step 3-2: BD Pooling is used to pool the convolutional features. BD Pooling first performs bilinear interpolation on the input within each stride and window size, calculating n corresponding interpolations. Then, the average of these interpolations is used to represent the features within the pooling window; each feature is encoded as Q. i The result, expressed as the average of the pooled window interpolation, is given by the following formula: Among them, P i,j It is the value obtained after bilinear interpolation within pooling window i. Bilinear interpolation is performed within each pooling window to obtain the summarized feature representation. Step 3-3: Apply dropout to the summarized feature representation to obtain spatial and frequency domain features; Step 4: Combine the improved multidimensional self-attention with BiSRU to capture the comprehensive features of spatial and frequency domain features. The specific implementation process is as follows: Step 4-1: Bidirectional simple cyclic unit (BiSRU) receives two inputs: the spatial and frequency domain characteristics of the current time step and the input of the previous time step. The bidirectional simple recurrent unit (BRU) is an extension of the simple recurrent unit (CRU). By processing both the forward and reverse information of the sequence simultaneously, the i-th output of the BRU network is {h}. i ′|h i =BiSRU(Q) i ),i=1,2,...,s}; Step 4-2: Through an improved multidimensional self-attention mechanism, combined spatial and frequency domain features are extracted from the output features of the bidirectional simple recurrent unit. The specific implementation is as follows: Step 4-2-1: Calculate the similarity within each sample from different points; feature score vector z i Consider it as the i-th feature h i A representation of '; the improved multidimensional self-attention mechanism adds two bias terms inside and outside the activation function, where z i Represented as: With i =f(h i ′)=W T σ(W1h i ′+b1)+b Among them, z i The intrinsic similarity of the i-th encoded EEG sample is represented by σ; the activation function σ adopts ELU, W and b are the weight and bias terms of the function, respectively, W1 is the weight parameter and b1 is the bias term; Step 4-2-2: Calculate the probability matrix P for each sample. ki =[p1,p2,...p i ,...,p s ]: The probability of the i-th EEG sample is expressed as: Step 4-2-3: Extracting the comprehensive feature F using the improved multidimensional self-attention mechanism att ={F1,F2,...,F n The i-th attentional feature extracted by the improved multidimensional self-attention mechanism is represented as: F i =p i ·h i ′; Step 5: Classify the comprehensive features using the Softmax function and output the emotion recognition results.

2. The emotion recognition method based on a multi-domain attention fusion network according to claim 1, characterized in that, The specific implementation process of step 2 is as follows: Step 2-1: Perform Fourier transform on the EEG signal and extract the frequency domain features H(x) of a single channel using the differential entropy formula: Where x represents the EEG signal, and p(x) represents the probability density function of x; the frequency domain features of each sample are concatenated from the c channels to obtain the frequency domain features of each sample. Step 2-2: Perform mean pooling on each channel of the preprocessed EEG to obtain channel statistics: in It is the average value of the j-th channel. Spatial domain features are obtained through dimensionality transformation. Steps 2-3: The extracted frequency domain features and spatial domain features are weighted using a weight matrix; the mechanism of the spatial-frequency domain hybrid channel attention is represented as follows: p=sigmoid(W s2 ·(tanh(W s1 ·F s +b s1 )+b s2 )+W f2 ·(tanh(W f1 ·F f +b f1 )+b f2 )+b) Among them W s1 and W f1 W represents the dimension-reduced weighted matrix of the spatial domain and frequency domain features, respectively. s2 and W f2 b represents the increased-dimensional weighted matrix representing the spatial and frequency domain features, respectively. s1 and b f1 b is the bias term for the dimensionality reduction layer. s2 and b f2 b is the bias term for the dimension-enhancing layer, and b is the bias term for the weights of the mixed features; The tanh function is used for nonlinear activation of the dimensionality reduction layer; the sigmoid function transforms the importance of channels into different probability distributions p = [p1, p2, ..., p c This represents the importance of different channels; Steps 2-4: Preprocessed EEG signal S i =[s1,s2,...,s c Using probability as weights for re-encoding, the attention channel feature of the j-th channel extracted through spatial-frequency domain hybrid channel attention is represented as: c j =p j ·s j C = {C1,C2,...,C} n } represents the extracted channel attention features, where the i-th extracted feature C i =[c1,c2,...,c c By S i =[s1,s2,...,s c The corresponding channel in ] is p = [p1, p2, ..., p c The corresponding elements in the [] are multiplied one by one to obtain the result.

Images