An electroencephalogram and electrooculogram combined method for detecting auditory spatial attention and a system thereof

Abstract

The application discloses an ear-brain electrical signal auditory spatial attention detection method and system fusing channel attention and residual learning, and belongs to the technical field of brain-computer interfaces and biological signal processing. First, ear-brain electrical signals are collected and preprocessed through filtering, artifact removal, bad channel interpolation, average re-reference and down-sampling. Then, short decision window samples are constructed through a sliding window. Channel attention mechanism is used to dynamically allocate weights to different brain electrical channels, and to strengthen the expression of key channel features. Subsequently, deep spatio-temporal feature extraction is completed based on a light residual network, which relieves the gradient vanishing problem of deep networks and improves the feature learning ability under a short window. Finally, the auditory spatial attention direction is output through a fully connected layer and a Softmax classification. The application can realize low-delay and high-precision detection only by relying on ear-brain electrical signals, and excellent performance is maintained in a 0.1s-2s decision window.

Description

Technical Field

[0001] This invention belongs to the fields of brain-computer interface, biosignal processing and pattern recognition technology. Specifically, it relates to an ear-brain electroencephalogram (EEG) method and system for detecting auditory spatial attention that integrates channel attention and residual learning. It can be applied to brain-computer interface scenarios such as smart headphones, neuromodulation hearing aids, and assessment and rehabilitation of auditory attention disorders. Background Technology

[0002] In recent years, auditory spatial attention detection (ASAD) technology based on electroencephalogram (EEG) signals has developed rapidly. It can determine the spatial direction of a subject's auditory attention by decoding EEG signals and has significant application value in fields such as intelligent hearing devices and brain-computer interfaces.

[0003] Significant progress has been made in deep learning-based ASAD technology. The application of models such as convolutional neural networks (CNN), attention mechanisms, and graph neural networks has enabled detection accuracy of approximately 95% within a 1-2 second decision window using full-scalp EEG. However, existing technologies still suffer from two major drawbacks: First, current high-performance ASAD methods heavily rely on full-scalp EEG signals, requiring subjects to wear electrode caps, which are cumbersome, impractical, and lack portability and concealment, failing to meet the needs of everyday wear and real-time applications. Second, while emerging ear-EEG technology has solved the portability issue, existing related ASAD methods rely on long decision windows exceeding 1 second, resulting in insufficient real-time system response capabilities. Furthermore, detection accuracy drops drastically under short decision windows, severely limiting practical applications.

[0004] Therefore, developing an ASAD method that combines portability, low latency, and high precision has become a pressing technical problem to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned deficiencies in existing technologies by providing a method and system for detecting auditory spatial attention using electroencephalography (EEG) that integrates channel attention and residual learning. This solution addresses the problems of poor portability of full-scalp EEG devices, high latency in EEG ASAD detection, insufficient feature extraction under short decision windows, and inadequate classification accuracy in existing technologies. It achieves low-latency, high-precision auditory spatial attention detection using EEG signals, providing core technical support for wearable intelligent hearing devices.

[0006] To achieve the objectives of this invention, the specific technical solution provided by this invention is as follows: First aspect This invention discloses a method for detecting auditory spatial attention based on electroencephalography (EEG) that integrates channel attention and residual learning, comprising the following steps: Step S1: Acquire the electroencephalogram (EEG) signals of the subject; Step S2: Preprocess the acquired EEG signals to obtain preprocessed EEG signals; Step S3: Use a sliding window strategy to slice the preprocessed EEG signal into windows to obtain EEG samples that conform to the model input format; Step S4: Employ channel attention mechanism to assign dynamic weights to EEG signals from different channels, thereby enhancing the expression of key channel information; Step S5: Based on the residual network, perform spatiotemporal feature extraction to automatically extract deep auditory attention features from the EEG signal; Step S6: Classify the EEG signals from which deep auditory attention features have been extracted and output the probability of auditory attention direction.

[0007] Second aspect Corresponding to the above method, the present invention also provides an EEG-based auditory spatial attention detection system that integrates channel attention and residual learning, comprising the following modules: signal acquisition module, preprocessing module, windowing module, channel attention module, spatiotemporal feature extraction module, and classification module; The signal acquisition module is used to acquire the electroencephalogram (EEG) signals of the subject to be tested; The preprocessing module is used to preprocess the acquired EEG signals to obtain preprocessed EEG signals. The windowing module is used to slice the preprocessed EEG signal into windows using a sliding window strategy to obtain EEG samples that conform to the model input format. The channel attention module is used to assign dynamic weights to EEG signals from different channels using a channel attention mechanism, thereby enhancing the expression of key channel information. The spatiotemporal feature extraction module is used to extract spatiotemporal features based on residual networks and automatically extract deep auditory attention features from EEG signals. The classification module is used to classify EEG signals from which deep auditory attention features have been extracted and output the probability of auditory attention direction.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The fusion channel attention and residual learning-based EEG auditory spatial attention detection method establishes an end-to-end EEG auditory spatial attention classification model by fusing dynamic channel weight allocation and deep residual network, which effectively solves the problems of high latency and insufficient feature extraction under short decision window in the existing EEG ASAD detection.

[0009] (2) By using the channel attention mechanism to assign dynamic weights to different EEG channels, the brain regions related to auditory attention can be adaptively enhanced and the noise interference of irrelevant brain regions can be suppressed, effectively solving the problem of key information loss caused by the traditional CNN model treating all channels equally.

[0010] (3) By introducing residual connections, the residual network structure effectively alleviates the gradient vanishing problem of deep networks, enabling the model to better learn the deep spatiotemporal features in EEG signals and improve the model's feature extraction ability and classification accuracy under short decision windows.

[0011] (4) Auditory spatial attention detection is performed using EEG signals, which effectively solves the problems of poor portability, cumbersome wearing, and inability to meet the application needs of daily wearable scenarios of full scalp EEG devices, and provides technical support for wearable smart hearing devices. Attached Figure Description

[0012] Figure 1 A schematic flowchart of the electroencephalogram-electroencephalogram-auditory spatial attention detection method based on fusion channel attention and residual learning provided in the embodiments of this application.

[0013] Figure 2 This is a schematic diagram illustrating the principle of the fusion channel attention and residual learning mechanism in an embodiment of the present invention; Figure 3 This is a schematic diagram of the ENT spatial attention detection system module that integrates channel attention and residual learning in an embodiment of the present invention. Detailed Implementation

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0015] It should be noted that the acquisition of data and collection of information in this application are legal, compliant, or [illegible]. Obtain the consent of the data subject.

[0016] Example 1 like Figure 1 As shown, this embodiment provides a method for detecting auditory spatial attention using electroencephalography (EEG) that integrates channel attention and residual learning, including the following steps: Step S1: Acquire the electroencephalogram (EEG) signals of the subject; Step S2: Preprocess the acquired EEG signals to obtain preprocessed EEG signals; The acquired EEG signals are preprocessed to obtain preprocessed EEG signals, specifically as follows: The EEG signal preprocessing was performed using the EEGLAB toolkit to standardize the raw EEG signal. The specific steps are as follows: S201: Bandpass Filtering: A Butterworth bandpass filter is used to extract EEG signals in the 1-32Hz frequency band; the transfer function of the Butterworth filter is shown in the following formula: ; in, Let the amplitude squared response be the filter. The angular frequency of the input signal. Here, represents the passband ripple coefficient, and n is the filter order. The cutoff frequency, These are the passband edge frequencies; S202: Artifact Removal: ASR detection and removal of high-amplitude non-EEG activity artifacts such as eye movement and electromyography are performed using the artifact subspace reconstruction method; S203: Damaged Channel Interpolation: The spherical spline interpolation method is used to interpolate and repair damaged or excessively noisy channels; S204: Average Rereference: Average rereference processing is performed on the full-channel EEG signals, as follows: The filtered EEG signal is Xi(t) ,in, i For the first i One electrode, t For the current electrode at t The average rereferenced EEG signal at each sampling point at time t is shown in the following formula: ; in, N For the number of electrodes, For rereferenced EEG signals, The original electroencephalogram (EEG) signal; S205: Downsampling: The preprocessed EEG signal is downsampled to 128Hz to reduce the amount of subsequent computation.

[0017] Step S3: Use a sliding window strategy to slice the preprocessed EEG signal into windows to obtain EEG samples that conform to the model input format; Specifically, a sliding window strategy was used to window and slice the preprocessed EEG signals to construct EEG samples, as follows: A sliding window is used to segment continuous EEG signals into multiple independent decision windows. The length of the decision window can be selected from different time lengths such as 0.1s, 0.2s, 0.5s, 1s, and 2s. Each window is denoted as ,in For the first ii Time series of EEG channels Nt This represents the total number of brainwave channels. T For each channel's time sampling point, T = sampling rate × decision window length.

[0018] Step S4: Employ channel attention mechanism to assign dynamic weights to EEG signals from different channels, thereby enhancing the expression of key channel information; like Figure 2 As shown, a channel attention mechanism is used to assign dynamic weights to EEG signals from different channels, enhancing the expression of key channel information. Specifically, this includes: Global average pooling and global max pooling operations were performed on each EEG channel to extract the average and maximum features in the time dimension. The calculation formula is as follows: ; ; in and These represent global average pooling and global max pooling operations, respectively, and the outputs are... and All Nt The feature vectors are 3D, representing the time-averaged feature and the time-maximum feature of each channel, respectively; The pooled features are transformed through two fully connected layers to generate channel attention weights, calculated as follows: ; in, f1 and f2 These represent the first fully connected layer and the second fully connected layer, respectively. Represents the ReLU activation function; the first fully connected layer will... Nt 3D feature compression to Nt / r dimension, r To reduce the coefficients, the second fully connected layer restores the dimension to [the original value]. Nt Dimension, Output for Nt A dimensional vector representing the initial channel attention weights; The attention weights are normalized using the Sigmoid activation function, constraining the values ​​to the range [0, 1]. The raw EEG signal is then weighted channel by channel, and the calculation formula is as follows: ; ; Where σ(·) represents the Sigmoid activation function, and the formula for calculating the Sigmoid activation function is as follows: ; This represents element-wise multiplication. The weighted EEG signal has the same dimension as the original EEG signal C. Through the channel attention mechanism, the model can adaptively learn the importance of different EEG channels for auditory spatial attention classification tasks, assigning greater weights to key channels while suppressing the influence of noise and irrelevant channels.

[0019] Step S5: Based on the residual network, perform spatiotemporal feature extraction to automatically extract deep auditory attention features from the EEG signal; Among them, spatiotemporal feature extraction based on residual networks automatically extracts deep auditory attention features from EEG signals, specifically including: The weighted EEG signal was initially processed using a 7×7 convolutional layer to capture local spatiotemporal features. Batch normalization, ReLU activation, and max pooling were then performed. The calculation formula is as follows: ; Where Conv(·), BN(·), ReLU(·), and MaxPool(·) represent convolution, batch normalization, ReLU activation, and max pooling operations, respectively, and the output F1 is a 64-channel feature map with spatial dimension compression. Nt / 4×T / 4.

[0020] Then, deep feature extraction is performed by stacking two residual modules. Each residual module contains two 3×3 convolutional layers, followed by batch normalization and summation layers. ReLU Activation and residual connection structures effectively alleviate the gradient vanishing problem. The calculation formula is as follows: ; in, The residual module is represented by the following formula for calculating a single residual block: ; in, Conv2 and Conv3 These represent two 3×3 convolutional layers. F in and Fout These represent the input and output feature maps of the residual block, respectively. When the input and output dimensions are inconsistent, a 1×1 convolution is used for dimension matching. The output F2 is a 512-channel feature map, with spatial dimension compressed to [value missing]. Nt / 8× T / 8; Subsequently, an average pooling layer is used to compress the feature map to a fixed size, and after flattening, the final feature vector is output. The calculation formula is as follows: ; in, The expression represents global average pooling, which compresses the feature map to a fixed size. Flatten(·) converts the pooled feature map into a one-dimensional vector. The output F3 is a 512-dimensional vector, which serves as the final feature representation for subsequent classification tasks.

[0021] Step S6: Classify the EEG signals from which deep auditory attention features have been extracted and output the probability of auditory attention direction.

[0022] Among them, the EEG signals from which deep auditory attention features are extracted are classified as follows: The extracted deep auditory attention features are classified through a fully connected layer, and the probability distribution is obtained using the softmax function. The calculation formula is as follows: ; Where W and b are the parameters of the fully connected layer, and P represents the probability of the predicted direction.

[0023] Example 2 like Figure 3 As shown, this embodiment provides an EEG-based auditory spatial attention detection system that integrates channel attention and residual learning, including the following modules: signal acquisition module, preprocessing module, windowing module, channel attention module, spatiotemporal feature extraction module, and classification module; The signal acquisition module is used to acquire the electroencephalogram (EEG) signals of the subject to be tested; The preprocessing module is used to preprocess the acquired EEG signals to obtain preprocessed EEG signals. The acquired EEG signals are preprocessed to obtain preprocessed EEG signals, specifically as follows: The EEG signal preprocessing was performed using the EEGLAB toolkit to standardize the raw EEG signal. The specific steps are as follows: S201: Bandpass Filtering: A Butterworth bandpass filter is used to extract EEG signals in the 1-32Hz frequency band. The transfer function of the Butterworth filter is shown in the following equation: ; in, Let the amplitude squared response be the filter. The angular frequency of the input signal. Here, represents the passband ripple coefficient, and n is the filter order. The cutoff frequency, These are the passband edge frequencies; S202: Artifact Removal: ASR detection and removal of high-amplitude non-EEG activity artifacts from eye movement and electromyography are performed using the artifact subspace reconstruction method; S203: Damaged Channel Interpolation: The spherical spline interpolation method is used to interpolate and repair damaged or excessively noisy channels; S204: Average Rereference: Average rereference processing is performed on the full-channel EEG signals, as follows: The filtered EEG signal is Xi(t) ,in, i For the first i One electrode, t For the current electrode at t The average rereferenced EEG signal at each sampling point at time t is shown in the following formula: ; in, N For the number of electrodes, For rereferenced EEG signals, The original electroencephalogram (EEG) signal; S205: Downsampling: The preprocessed EEG signals are downsampled to 128Hz to reduce the amount of subsequent computation. The windowing module is used to slice the preprocessed EEG signal into windows using a sliding window strategy to obtain EEG samples that conform to the model input format. Specifically, a sliding window strategy was used to window and slice the preprocessed EEG signals to construct EEG samples, as follows: The continuous EEG signal was segmented into multiple independent decision windows using a sliding window, with the decision window length selected as 0.1s, 0.2s, 0.5s, 1s, or 2s. Each window is denoted as ,in For the first ii Time series of EEG channels Nt This represents the total number of brainwave channels. T For each channel's time sampling point, T = sampling rate × decision window length; The channel attention module is used to assign dynamic weights to EEG signals from different channels using a channel attention mechanism, thereby enhancing the expression of key channel information. Among these measures, a channel attention mechanism is employed to assign dynamic weights to EEG signals from different channels, enhancing the expression of key channel information. Specifically, this includes: Global average pooling and global max pooling operations were performed on each EEG channel to extract the average and maximum features in the time dimension. The calculation formula is as follows: ; ; in and These represent global average pooling and global max pooling operations, respectively, and the outputs are... and All Nt The feature vectors are 3D, representing the time-averaged feature and the time-maximum feature of each channel, respectively; The pooled features are transformed through two fully connected layers to generate channel attention weights, calculated as follows: ; in, f1 and f2 These represent the first fully connected layer and the second fully connected layer, respectively. Represents the ReLU activation function; the first fully connected layer will... Nt 3D feature compression to Nt / r dimension, r To reduce the coefficients, the second fully connected layer restores the dimension to [the original value]. Nt Dimension, Output for Nt A dimensional vector representing the initial channel attention weights; The attention weights are normalized using the Sigmoid activation function, constraining the values ​​to the range [0, 1]. The raw EEG signal is then weighted channel by channel, and the calculation formula is as follows: ; ; Where σ(·) represents the Sigmoid activation function, and the formula for calculating the Sigmoid activation function is as follows: ; This represents element-wise multiplication. The weighted EEG signal has the same dimension as the original EEG signal C. Through the channel attention mechanism, the model can adaptively learn the importance of different EEG channels for auditory spatial attention classification tasks, assigning greater weights to key channels while suppressing the influence of noise and irrelevant channels.

[0024] The spatiotemporal feature extraction module is used to extract spatiotemporal features based on residual networks and automatically extract deep auditory attention features from EEG signals. The classification module is used to classify EEG signals from which deep auditory attention features have been extracted and output the probability of auditory attention direction.

[0025] Specifically, the method of extracting spatiotemporal features based on residual networks to automatically extract deep auditory attention features from EEG signals includes: The weighted EEG signal was initially processed using a 7×7 convolutional layer to capture local spatiotemporal features. Batch normalization, ReLU activation, and max pooling were then performed. The calculation formula is as follows: ; Where Conv(·), BN(·), ReLU(·), and MaxPool(·) represent convolution, batch normalization, ReLU activation, and max pooling operations, respectively, and the output F1 is a 64-channel feature map with spatial dimension compression. Nt / 4×T / 4.

[0026] Then, deep feature extraction is performed by stacking two residual modules. Each residual module contains two 3×3 convolutional layers, followed by batch normalization and summation layers. ReLU Activation and residual connection structures effectively alleviate the gradient vanishing problem. The calculation formula is as follows: ; in, The residual module is represented by the following formula for calculating a single residual block: ; in, Conv2 and Conv3 These represent two 3×3 convolutional layers. F in and Fout These represent the input and output feature maps of the residual block, respectively. When the input and output dimensions are inconsistent, a 1×1 convolution is used for dimension matching. The output F2 is a 512-channel feature map, with spatial dimension compressed to [value missing]. Nt / 8× T / 8; Subsequently, an average pooling layer is used to compress the feature map to a fixed size, and after flattening, the final feature vector is output. The calculation formula is as follows: ; in, The expression represents a global average pooling operation, which compresses the feature map to a fixed size. Flatten(·) converts the pooled feature map into a one-dimensional vector. The output F3 is a 512-dimensional vector, which serves as the final feature representation for subsequent classification tasks. The classification of EEG signals for extracting deep auditory attention features is specifically as follows: The extracted deep auditory attention features are classified through a fully connected layer, and the probability distribution is obtained using the softmax function. The calculation formula is as follows: ; Where W and b are the parameters of the fully connected layer, and P represents the probability of the predicted direction.

[0027] The following is an introduction to this dataset: The first dataset used was the cEEGrid dataset. This dataset was collected from 36 participants. All participants were native German speakers with normal hearing and no reported mental or neurological disorders. Participants were required to selectively focus on one of two simultaneously played audiobooks and answer questions related to the selected audiobook to maintain attention. The two audiobooks were presented at ±30° or ±45° angles to the left and right of the participants, respectively. The data used in this study consisted of three 10-minute recordings from each participant, totaling 30 minutes of data per participant. The cEEGrid data was recorded using a 24-channel portable amplifier with electrodes placed around the participants' ears. The electrode impedance was maintained below 20 kΩ, and the data was transmitted to the recording computer via Bluetooth. The cEEGrid system recorded EEG data at a sampling rate of 500 Hz.

[0028] The second dataset is the KUL dataset. This dataset contains EEG data collected from 16 hearing-normal participants. Participants were required to selectively attend to one speaker in a two-speaker scenario, with the two speakers positioned 90° to the left and right of the participant, respectively. The speech stimuli were played through in-ear headphones, filtered at 4 kHz, and the sound pressure level was set to 60 dB. Each participant completed 8 sets of experiments, each lasting 6 minutes. The EEG data were recorded in a soundproof, electromagnetically shielded room using a 64-channel BioSemiActiveTwo system at a sampling rate of 8192 Hz.

[0029] Neural network models need to be trained on samples before they can be used for auditory attention classification.

[0030] Specifically, the training process includes: The experiment used a non-overlapping sliding window to divide the EEG data into five decision windows of 0.1s, 0.2s, 0.5s, 1s, and 2s. Each participant's data was divided into training, validation, and test sets in an 8:1:1 ratio. The stochastic gradient descent (SGD) optimizer was used to minimize the binary cross-entropy loss. The batch size was set to 128, and the initial learning rate was 0.1. The learning rate was halved at the 10th, 20th, and 30th epochs, with a maximum of 50 training epochs. The optimal model was selected based on the validation set accuracy and evaluated on the test set. All experiments were performed on a Tesla V100-SXM2-32GB GPU, and the model training was completed in an end-to-end manner.

[0031] To verify the effectiveness of the proposed model, this paper conducts comparative experiments with mainstream models such as CNN, DenseNet, EEG-Graph, and STA-Net, and performs performance verification on the whole-brain KUL dataset and the cEEGrid dataset of periauricular EEG.

[0032] Table 1 shows the comparison results of different models on the KUL dataset. Table 2 shows the comparison results of different models on the cEEGrid periauricular EEG dataset.

[0033] Table 1. Comparison of different models on the KUL dataset (accuracy %)

[0034] As shown in Table 1, compared with CNN, DenseNet, EEG-Graph, and STA-Net methods, the method proposed in this application achieves the highest accuracy across different time windows. In particular, the performance advantage of the proposed method is more significant in short time windows (0.1s and 0.2s), indicating that the channel attention mechanism and residual network structure can effectively improve the model's feature extraction capability under short decision windows.

[0035] Table 2 Comparison results of different models on the cEEGrid dataset (accuracy %)

[0036] As can be seen from Table 2, on the cEEGrid periauricular EEG dataset, the method of this application achieved the best performance under all decision windows. In particular, under the 0.1s decision window, the method of this application improved by 25.55% compared with the CNN method and by 3.28% compared with the DenseNet method, which fully demonstrates the superiority of the method in ear and brain electroencephalogram signal processing.

[0037] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

Claims

1. A method for detecting auditory spatial attention using electroencephalography (EEG) that integrates channel attention and residual learning, characterized in that, Includes the following steps: Step S1: Acquire the electroencephalogram (EEG) signals of the subject; Step S2: Preprocess the acquired EEG signals to obtain preprocessed EEG signals; Step S3: Use a sliding window strategy to slice the preprocessed EEG signal into windows to obtain EEG samples that conform to the model input format; Step S4: Employ channel attention mechanism to assign dynamic weights to EEG signals from different channels, thereby enhancing the expression of key channel information; Step S5: Based on the residual network, perform spatiotemporal feature extraction to automatically extract deep auditory attention features from the EEG signal; Step S6: Classify the EEG signals from which deep auditory attention features have been extracted and output the probability of auditory attention direction.

2. The method for detecting auditory spatial attention using electroencephalography (EEG) based on fusion of channel attention and residual learning as described in claim 1, characterized in that, In step S2, the acquired EEG signals are preprocessed to obtain preprocessed EEG signals, specifically as follows: The EEG signal preprocessing was performed using the EEGLAB toolkit to standardize the raw EEG signal. The specific steps are as follows: S201: Bandpass Filtering: A Butterworth bandpass filter is used to extract EEG signals in the 1-32Hz frequency band; the transfer function of the Butterworth filter is shown in the following formula: ； in, Let the amplitude squared response be the filter. The angular frequency of the input signal. Here, represents the passband ripple coefficient, and n is the filter order. The cutoff frequency, These are the passband edge frequencies; S202: Artifact Removal: ASR detection and removal of high-amplitude non-EEG activity artifacts from eye movement and electromyography are performed using the artifact subspace reconstruction method; S203: Damaged Channel Interpolation: The spherical spline interpolation method is used to interpolate and repair damaged or excessively noisy channels; S204: Average Rereference: Average rereference processing is performed on the full-channel EEG signals, as follows: The filtered EEG signal is Xi(t) ,in, i For the first i One electrode, t For the current electrode at t The average rereferenced EEG signal at each sampling point at time t is shown in the following formula: ； in, N For the number of electrodes, For rereferenced EEG signals, The original electroencephalogram (EEG) signal; S205: Downsampling: The preprocessed EEG signal is downsampled to 128Hz to reduce the amount of subsequent computation.

3. The method for detecting auditory spatial attention using electroencephalography (EEG) based on a fusion of channel attention and residual learning, as described in claim 2, is characterized in that... In step S3: A sliding window strategy is used to window and slice the preprocessed EEG signal to construct an EEG sample, specifically as follows: The continuous EEG signal was segmented into multiple independent decision windows using a sliding window, with the decision window length selected as 0.1s, 0.2s, 0.5s, 1s, or 2s. Each window is denoted as ,in For the first ii Time series of EEG channels Nt This represents the total number of brainwave channels. T For each channel's time sampling point, T = sampling rate × decision window length.

4. The method for detecting auditory spatial attention using electroencephalography (EEG) based on a fusion of channel attention and residual learning, as described in claim 3, is characterized in that... In step S4, a channel attention mechanism is used to assign dynamic weights to EEG signals from different channels, enhancing the expression of key channel information. Specifically, this includes: Global average pooling and global max pooling operations were performed on each EEG channel to extract the average and maximum features in the time dimension. The calculation formula is as follows: ； ； in and These represent global average pooling and global max pooling operations, respectively, and the outputs are... and All Nt The feature vectors are 3D, representing the time-averaged feature and the time-maximum feature of each channel, respectively; The pooled features are transformed through two fully connected layers to generate channel attention weights, calculated as follows: ； in, f1 and f2 These represent the first fully connected layer and the second fully connected layer, respectively. Represents the ReLU activation function; the first fully connected layer will... Nt 3D feature compression to Nt / r dimension, r To reduce the coefficients, the second fully connected layer restores the dimension to [the original value]. Nt Dimension, Output for Nt A dimensional vector representing the initial channel attention weights; The attention weights are normalized using the Sigmoid activation function, constraining the values ​​to the range [0, 1]. The raw EEG signal is then weighted channel by channel, and the calculation formula is as follows: ； ； Where σ(·) represents the Sigmoid activation function, and the formula for calculating the Sigmoid activation function is as follows: ； This represents element-wise multiplication. The weighted EEG signal has the same dimension as the original EEG signal C. Through the channel attention mechanism, the model can adaptively learn the importance of different EEG channels for auditory spatial attention classification tasks, assigning greater weights to key channels while suppressing the influence of noise and irrelevant channels.

5. The method for detecting auditory spatial attention using electroencephalography (EEG) based on a fusion of channel attention and residual learning as described in claim 4, characterized in that... In step S5, spatiotemporal feature extraction is performed based on the residual network, automatically extracting deep auditory attention features from the EEG signal, specifically including: The weighted EEG signal was initially processed using a 7×7 convolutional layer to capture local spatiotemporal features. Batch normalization, ReLU activation, and max pooling were then performed. The calculation formula is as follows: ； Where Conv(·), BN(·), ReLU(·), and MaxPool(·) represent convolution, batch normalization, ReLU activation, and max pooling operations, respectively, and the output F1 is a 64-channel feature map with spatial dimension compression. Nt / 4×T / 4; Then, deep feature extraction is performed by stacking two residual modules. Each residual module contains two 3×3 convolutional layers, followed by batch normalization and summation layers. ReLU Activation and residual connection structures effectively alleviate the gradient vanishing problem. The calculation formula is as follows: ； in, The residual module is represented by the following formula for calculating a single residual block: ； in, Conv2 and Conv3 These represent two 3×3 convolutional layers. F in and Fout These represent the input and output feature maps of the residual block, respectively. When the input and output dimensions are inconsistent, a 1×1 convolution is used for dimension matching. The output F2 is a 512-channel feature map, with spatial dimension compressed to [value missing]. Nt / 8× T / 8; Subsequently, an average pooling layer is used to compress the feature map to a fixed size, and after flattening, the final feature vector is output. The calculation formula is as follows: ； in, The expression represents global average pooling, which compresses the feature map to a fixed size. Flatten(·) converts the pooled feature map into a one-dimensional vector. The output F3 is a 512-dimensional vector, which serves as the final feature representation for subsequent classification tasks.

6. The method for detecting auditory spatial attention using electroencephalography (EEG) based on a fusion of channel attention and residual learning as described in claim 5, characterized in that... In step S6: The EEG signals from which deep auditory attention features have been extracted are classified, specifically as follows: The extracted deep auditory attention features are classified through a fully connected layer, and the probability distribution is obtained using the softmax function. The calculation formula is as follows: ； Where W and b are the parameters of the fully connected layer, and P represents the probability of the predicted direction.

7. A transcranial EEG auditory spatial attention detection system integrating channel attention and residual learning, characterized in that, It includes the following modules: signal acquisition module, preprocessing module, windowing module, channel attention module, spatiotemporal feature extraction module, and classification module; The signal acquisition module is used to acquire the electroencephalogram (EEG) signals of the subject to be tested; The preprocessing module is used to preprocess the acquired EEG signals to obtain preprocessed EEG signals. The windowing module is used to slice the preprocessed EEG signal into windows using a sliding window strategy to obtain EEG samples that conform to the model input format. The channel attention module is used to assign dynamic weights to EEG signals from different channels using a channel attention mechanism, thereby enhancing the expression of key channel information. The spatiotemporal feature extraction module is used to extract spatiotemporal features based on residual networks and automatically extract deep auditory attention features from EEG signals. The classification module is used to classify EEG signals from which deep auditory attention features have been extracted and output the probability of auditory attention direction.

8. The electroencephalogram (EEG) auditory spatial attention detection system according to claim 7, characterized in that, The acquired EEG signals were preprocessed to obtain preprocessed EEG signals, specifically: The EEG signal preprocessing was performed using the EEGLAB toolkit to standardize the raw EEG signal. The specific steps are as follows: S201: Bandpass Filtering: A Butterworth bandpass filter is used to extract EEG signals in the 1-32Hz frequency band; the transfer function of the Butterworth filter is shown in the following formula: ； in, Let the amplitude squared response be the filter. The angular frequency of the input signal. Here, represents the passband ripple coefficient, and n is the filter order. The cutoff frequency, These are the passband edge frequencies; S202: Artifact Removal: ASR detection and removal of high-amplitude non-EEG activity artifacts from eye movement and electromyography are performed using the artifact subspace reconstruction method; S203: Damaged Channel Interpolation: The spherical spline interpolation method is used to interpolate and repair damaged or excessively noisy channels; S204: Average Rereference: Average rereference processing is performed on the full-channel EEG signals, as follows: The filtered EEG signal is Xi(t) ,in, i For the first i One electrode, t For the current electrode at t The average rereferenced EEG signal at each sampling point at time t is shown in the following formula: ； in, N For the number of electrodes, For rereferenced EEG signals, The original electroencephalogram (EEG) signal; S205: Downsampling: The preprocessed EEG signal is downsampled to 128Hz to reduce the amount of subsequent computation.

9. The electroencephalogram (EEG) auditory spatial attention detection system according to claim 8, characterized in that, A sliding window strategy was used to window and slice the preprocessed EEG signals to construct EEG samples, specifically: The continuous EEG signal was segmented into multiple independent decision windows using a sliding window, with the decision window length selected as 0.1s, 0.2s, 0.5s, 1s, or 2s. Each window is denoted as ,in For the first ii Time series of EEG channels Nt This represents the total number of brainwave channels. T For each channel's time sampling point, T = sampling rate × decision window length; The channel attention mechanism is used to assign dynamic weights to EEG signals from different channels, enhancing the expression of key channel information. Specifically, this includes: Global average pooling and global max pooling operations were performed on each EEG channel to extract the average and maximum features in the time dimension. The calculation formula is as follows: ； ； in and These represent global average pooling and global max pooling operations, respectively, and the outputs are... and All Nt The feature vectors are 3D, representing the time-averaged feature and the time-maximum feature of each channel, respectively; The pooled features are transformed through two fully connected layers to generate channel attention weights, calculated as follows: ； in, f1 and f2 These represent the first fully connected layer and the second fully connected layer, respectively. Represents the ReLU activation function; the first fully connected layer will... Nt 3D feature compression to Nt / r dimension, r To reduce the coefficients, the second fully connected layer restores the dimension to [the original value]. Nt Dimension, Output for Nt A dimensional vector representing the initial channel attention weights; The attention weights are normalized using the Sigmoid activation function, constraining the values ​​to the range [0, 1]. The raw EEG signal is then weighted channel by channel, and the calculation formula is as follows: ； ； Where σ(·) represents the Sigmoid activation function, and the formula for calculating the Sigmoid activation function is as follows: ； This represents element-wise multiplication. The weighted EEG signal has the same dimension as the original EEG signal C. Through the channel attention mechanism, the model can adaptively learn the importance of different EEG channels for auditory spatial attention classification tasks, assigning greater weights to key channels while suppressing the influence of noise and irrelevant channels.

10. The electroencephalogram (EEG) auditory spatial attention detection system according to claim 9, characterized in that, The method of extracting spatiotemporal features based on residual networks, and automatically extracting deep auditory attention features from EEG signals, specifically includes: The weighted EEG signal was initially processed using a 7×7 convolutional layer to capture local spatiotemporal features. Batch normalization, ReLU activation, and max pooling were then performed. The calculation formula is as follows: ； Where Conv(·), BN(·), ReLU(·), and MaxPool(·) represent convolution, batch normalization, ReLU activation, and max pooling operations, respectively, and the output F1 is a 64-channel feature map with spatial dimension compression. Nt / 4×T / 4; Then, deep feature extraction is performed by stacking two residual modules. Each residual module contains two 3×3 convolutional layers, followed by batch normalization and summation layers. ReLU Activation and residual connection structures effectively alleviate the gradient vanishing problem. The calculation formula is as follows: ； in, The residual module is represented by the following formula for calculating a single residual block: ； in, Conv2 and Conv3 These represent two 3×3 convolutional layers. F in and Fout These represent the input and output feature maps of the residual block, respectively. When the input and output dimensions are inconsistent, a 1×1 convolution is used for dimension matching. The output F2 is a 512-channel feature map, with spatial dimension compressed to [value missing]. Nt / 8× T / 8; Subsequently, an average pooling layer is used to compress the feature map to a fixed size, and after flattening, the final feature vector is output. The calculation formula is as follows: ； in, The expression represents a global average pooling operation, which compresses the feature map to a fixed size. Flatten(·) converts the pooled feature map into a one-dimensional vector. The output F3 is a 512-dimensional vector, which serves as the final feature representation for subsequent classification tasks. The classification of EEG signals for extracting deep auditory attention features is specifically as follows: The extracted deep auditory attention features are classified through a fully connected layer, and the probability distribution is obtained using the softmax function. The calculation formula is as follows: ； Where W and b are the parameters of the fully connected layer, and P represents the probability of the predicted direction.

Images