Noise robustness hearing decoding method and system based on adversarial domain adaptation network

Abstract

The application discloses a noise robustness hearing decoding method and system based on an adversarial domain adaptation network, relates to the technical field of hearing decoding, and comprises the following steps: acquiring a multi-channel electroencephalogram signal, and performing parallel analysis and feature fusion; inputting a multi-dimensional signal integrity feature vector into a noise evaluation model to output a noise complexity score; calling a feature decoupling module to output a task-related feature vector and a noise-related feature vector; inputting the task-related feature vector into an attention decoder to obtain a hearing attention allocation probability distribution; and taking the hearing attention allocation probability distribution and a noise tracing and influence report as a noise robustness hearing decoding result of a target user. The technical problem of low decoding accuracy and insufficient robustness of existing hearing decoding methods in a noisy scene is solved.

Description

Technical Field

[0001] This application relates to the field of auditory decoding technology, specifically to a noise-robust auditory decoding method and system based on adversarial domain adaptation networks. Background Technology

[0002] With the rapid development of brain-computer interface technology and neural engineering, auditory decoding, as a key technology for extracting users' auditory cognitive intentions from electroencephalogram (EEG) signals, has shown great application potential in fields such as hearing aids, neurorehabilitation, and intelligent interaction.

[0003] However, EEG signals are inevitably subject to various noise interferences. Most existing auditory decoding methods rely on EEG data collected in an ideal, clean environment for model training. When applied to noisy scenarios, noise severely contaminates the effective features relevant to the task, leading to a significant decrease in the decoding accuracy of the model for core tasks such as auditory attention allocation and sound source identification. Furthermore, traditional methods typically employ fixed denoising strategies or feature extraction methods, making it difficult to adaptively adjust the decoding process according to dynamic changes in noise. They also lack the ability to accurately trace and quantify the source of noise, resulting in insufficient reliability and robustness of the decoding results, thus limiting the application of auditory decoding technology in complex real-world environments. Summary of the Invention

[0004] This application provides a noise-robust auditory decoding method and system based on adversarial domain adaptation networks, which solves the technical problems of low decoding accuracy and insufficient robustness of existing auditory decoding methods in noisy scenarios.

[0005] The technical solution to the above-mentioned technical problems in this application is as follows:

[0006] In a first aspect, this application provides a noise-robust auditory decoding method based on an adversarial domain adaptation network, the method comprising:

[0007] The system acquires multi-channel EEG signals of the target user in an auditory scenario, and performs parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector.

[0008] The multi-dimensional signal integrity feature vector is input into a pre-trained noise evaluation model, and the noise complexity score is output.

[0009] Based on the noise complexity score, the feature decoupling module is dynamically invoked from the progressive feature decoupling network constructed based on the adversarial domain adaptation network, and the multi-channel EEG signal is input to output the task-related feature vector and the noise-related feature vector.

[0010] The task-related feature vector is input into a pre-trained attention decoder, and the output is the auditory attention allocation probability distribution. The noise-related feature vector is fused with the multi-dimensional signal integrity feature vector and then input into a pre-trained noise quantizer, and the output is the noise source tracing and impact report.

[0011] The auditory attention allocation probability distribution and the noise source and impact report are output as the noise robustness auditory decoding results for the target user.

[0012] Secondly, this application provides a noise-robust auditory decoding system based on an adversarial domain adaptation network, comprising:

[0013] The information acquisition module is used to acquire multi-channel EEG signals of the target user in an auditory scenario, and to perform parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector.

[0014] The noise model training module is used to input the multi-dimensional signal integrity feature vector into the pre-trained noise evaluation model and output a noise complexity score.

[0015] The feature vector output module is used to dynamically call the feature decoupling module from the progressive feature decoupling network constructed based on the adversarial domain adaptation network based on the noise complexity score, and input the multi-channel EEG signal to output task-related feature vectors and noise-related feature vectors.

[0016] The probability distribution output module is used to input the task-related feature vector into a pre-trained attention decoder and output an auditory attention distribution probability distribution. The noise-related feature vector is fused with the multi-dimensional signal integrity feature vector and then input into a pre-trained noise quantizer to output a noise source tracing and impact report.

[0017] The decoding result acquisition module is used to output the auditory attention allocation probability distribution and the noise source tracing and impact report as the noise robustness auditory decoding result of the target user.

[0018] This application provides one or more technical solutions, which have at least the following technical effects or advantages:

[0019] This application provides a noise-robust auditory decoding method and system based on an adversarial domain adaptation network. First, it acquires multi-channel EEG signals from a target user in an auditory scenario, performs parallel analysis and feature fusion to obtain a multi-dimensional signal integrity feature vector. Second, this feature vector is input into a pre-trained noise assessment model, which outputs a noise complexity score to quantify the complexity of noise in the current EEG signal. Subsequently, based on this noise complexity score, a feature decoupling module of the appropriate level is dynamically invoked from a progressive feature decoupling network constructed based on an adversarial domain adaptation network to process the multi-channel EEG signal, achieving effective separation of task-related feature vectors and noise-related feature vectors. This allows the model to adaptively adjust the depth and intensity of feature decoupling according to the noise situation. The task-related feature vector is then input into a pre-trained attention decoder to obtain an auditory attention allocation probability distribution, presenting the user's attention to different sound sources. Simultaneously, the noise-related feature vector is fused with the multi-dimensional signal integrity feature vector and input into a pre-trained noise quantizer, outputting a noise source tracing and impact report, clarifying information such as the type and intensity of the noise. Finally, the output of the auditory attention allocation probability distribution and noise source and impact report are used as decoding results.

[0020] Through the above technical solution, this application, based on the dynamic invocation feature decoupling module, can better cope with interference of different noise types and intensities, improve the accuracy and robustness of auditory decoding in noisy scenarios, and at the same time realize accurate assessment and source tracing of noise. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the noise-robust auditory decoding method based on an adversarial domain adaptation network provided in an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of the structure of a noise-robust auditory decoding system based on an adversarial domain adaptation network provided in an embodiment of this application.

[0024] The components represented by each number in the attached diagram are explained below:

[0025] Information acquisition module 11, noise model training module 12, feature vector output module 13, probability allocation output module 14, decoding result acquisition module 15. Detailed Implementation

[0026] This application provides a noise-robust auditory decoding method and system based on adversarial domain adaptation networks, which addresses the technical problems of low decoding accuracy and insufficient robustness of existing auditory decoding methods in noisy scenarios.

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0029] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessarily obscuring the description of this application. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0030] Example 1, as Figure 1 As shown, embodiments of this application provide a noise-robust auditory decoding method based on an adversarial domain adaptation network, including:

[0031] S10: Acquire multi-channel EEG signals of the target user in an auditory scenario, and perform parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector;

[0032] In this embodiment, firstly, multi-channel EEG signals of the target user in a specific auditory scenario are acquired, with the multi-channel EEG signals covering different regions of the cerebral cortex. The acquired raw EEG signals are preprocessed, including but not limited to baseline drift removal, power line interference filtering, and artifact removal, to ensure signal quality.

[0033] Subsequently, the preprocessed multi-channel EEG signals were analyzed in parallel to extract their time-domain, frequency-domain, and time-frequency-domain features. Simultaneously, spatial features such as EEG topographic features and coherence features between different channels were extracted. After the independent extraction of features from each channel, feature fusion techniques were used to integrate features from different channels and of different types to obtain a multi-dimensional signal integrity feature vector.

[0034] Specifically, step S10 in the method includes:

[0035] Acquire multi-channel EEG signals of target users in auditory scenarios;

[0036] For the multi-channel EEG signal, the amplitude squared coherence coefficient between all channel pairs within a preset time window is calculated;

[0037] The amplitude squared coherence coefficients of all channel pairs are arranged into a symmetric channel coherence matrix. The upper triangular elements of the channel coherence matrix are extracted and arranged into a one-dimensional vector, which serves as the spatial coherence feature vector.

[0038] For each channel of the multichannel EEG signal, the average power spectral density in the δ band, θ band, α band and β band is calculated respectively;

[0039] Calculate the arithmetic mean of the average power spectral density of each channel in the δ band, θ band, α band and β band respectively, and combine them to form the cross-channel average frequency band power vector;

[0040] The spatial coherence feature vector and the cross-channel average frequency band power vector are concatenated to form a multi-dimensional signal integrity feature vector.

[0041] In this embodiment of the application, firstly, the multi-channel EEG signal of the target user in an auditory scene is acquired, for example, by using a 32-channel or 64-channel EEG acquisition device to simultaneously record the user's EEG activity while the user listens to mixed audio containing multiple sound sources.

[0042] Secondly, for multi-channel EEG signals, a preset time window is selected, such as 1 second in duration and 50% overlap rate, and the amplitude squared coherence coefficient between all channel pairs within the window is calculated to measure the synchronicity of neuronal activity in different brain regions.

[0043] For example, for a 32-channel EEG signal, 32 × 32 = 1024 channel pairs are generated. The amplitude squared coherence coefficient C(i,j) of each channel pair (i,j) is obtained by the formula C(i,j) = |E[Xi(f)Xj*(f)]| 2 / (E[|Xi(f)| 2]E[|Xj(f)| 2 The calculation is performed, where Xi(f) and Xj(f) are the Fourier transforms of the signals in channels i and j, respectively, E[·] represents the expectation operation, and * represents the conjugate. After arranging the coefficients into a symmetric channel coherence matrix, the upper triangular elements are extracted, resulting in a total of 32×31 / 2=496 elements. The diagonal and lower triangular duplicate elements are excluded and arranged into a one-dimensional vector, thus obtaining the spatial coherence eigenvector.

[0044] Simultaneously, for each channel of the multi-channel EEG signal, a Fast Fourier Transform (FFT) was used to convert the signal to the frequency domain, and the power spectral density was calculated in the delta (δ), theta (θ), alpha (α), and beta (β) bands. Different bands represent different frequency ranges: the delta band is 0.5–4 Hz, the theta band is 4–8 Hz, the alpha band is 8–13 Hz, and the beta band is 13–30 Hz. The power spectral density in each band was then integrated and averaged to obtain the average power spectral density value for each channel across the four bands.

[0045] Furthermore, the arithmetic mean of the average power spectral density of all channels in the δ band, the arithmetic mean of the average power spectral density of the θ band, the arithmetic mean of the average power spectral density of the α band, and the arithmetic mean of the average power spectral density of the β band are calculated respectively. The four values ​​are combined to form the cross-channel average frequency band power vector.

[0046] Finally, the 496-dimensional spatial coherence feature vector obtained above is concatenated with the 4-dimensional cross-channel average frequency band power vector to form a 500-dimensional multi-dimensional signal integrity feature vector, which comprehensively reflects the spatial synchronization characteristics and cross-frequency band energy distribution characteristics of EEG signals.

[0047] Specifically, for the multi-channel EEG signal, the amplitude squared coherence coefficient between all channel pairs within a preset time window is calculated, including:

[0048] For the multi-channel EEG signal segments within the preset time window, the Welch average periodogram method is used to calculate the power spectral density of each channel signal and the cross-power spectral density between any two channel signals.

[0049] For any two channels, the average coherence coefficient within the preset frequency band is calculated according to the corresponding power spectral density and cross power spectral density using the amplitude squared coherence formula, and is used as the amplitude squared coherence coefficient of the channel pair.

[0050] Fill the amplitude squared coherence coefficients of all channel pairs into a C-row C-column symmetric matrix, where C is the total number of EEG channels;

[0051] Extract all elements in the symmetric matrix whose row index is less than their column index, and arrange them in row priority order to form a spatial coherence feature vector.

[0052] In this embodiment, firstly, the multi-channel EEG signal segments within a preset time window are processed by frame segmentation. For example, a 1-second signal is divided into 200ms sub-windows with 50% overlap, and a Hanning window is applied to each sub-window to reduce spectral leakage.

[0053] Then, using the Welch average periodogram method, a Fast Fourier Transform is performed on each sub-window to calculate the power spectral density (PSD) of each channel signal and the cross-power spectral density (CPSD) between any two channel signals. For any two channels i and j, their amplitude squared coherence coefficient γ is calculated. 2 (i,j,f) is defined as γ at frequency f. 2 (i,j,f)=|CPSDij(f)| 2 / [PSD_i(f)×PSDj(f)]. To obtain the average coherence coefficient within a preset frequency band, the frequency range is limited to, for example, an effective EEG frequency band of 0.5Hz to 30Hz, and the γ value at all frequency points within this band is calculated. 2 The arithmetic mean of (i,j,f) is used to obtain the average amplitude squared coherence coefficient of the channel pair within the preset frequency band. Among them, PSDi_(f) is the power spectral density of channel i, PSDj(f) is the power spectral density of channel j, and CPSDij(f) is the cross-power spectral density between channel i and channel j.

[0054] Among them, the Welch average periodogram method estimates the power spectral density by segmenting the long signal, windowing it, calculating the periodogram of each segment, and then averaging the results, thereby reducing the variance of the spectrum estimation.

[0055] The average coherence coefficients of all C channel pairs are filled into a C x C symmetric matrix. The diagonal element (i, i) of the matrix represents the coherence of the channel itself, which is theoretically 1, but may have slight deviations due to noise in actual calculations. To avoid information redundancy, all upper triangular elements in the symmetric matrix whose row indices are less than their column indices are extracted. For example, for 32 channels, elements i=1 to 31 and j=i+1 to 32 are extracted, and arranged in row-major order, i.e., first by row index, then by column index within the same row, to form a one-dimensional vector. This vector is the spatial coherence feature vector, with a dimension of C*(C-1) / 2.

[0056] S20: Input the multi-dimensional signal integrity feature vector into the pre-trained noise evaluation model and output the noise complexity score;

[0057] In this embodiment, a multi-dimensional signal integrity feature vector is input into a pre-trained noise assessment model. This model employs a multilayer perceptron architecture, with its input layer dimension matching the dimension of the multi-dimensional signal integrity feature vector. The training process of the noise assessment model is based on a database of EEG signal samples containing different types and intensities of noise, with each sample labeled with a corresponding noise complexity tag.

[0058] The model learns the mapping relationship between multi-dimensional signal integrity feature vectors and noise complexity labels to achieve a quantitative assessment of the noise complexity in the EEG signal represented by the input feature vectors and outputs a noise complexity score.

[0059] The construction process of the noise assessment model includes:

[0060] Multiple multi-channel EEG signal samples containing different noise types and intensities were collected, and multi-dimensional signal integrity feature vectors were extracted from each multi-channel EEG signal sample to form a feature vector training set.

[0061] Each feature vector is labeled with a noise complexity score label according to a preset standard, thus forming a labeled training set.

[0062] An initial noise assessment model is constructed based on a multilayer perceptron.

[0063] The noise assessment initial model is trained in a supervised manner using the feature vector training set and the label training set until the verification convergence, thus obtaining the pre-trained noise assessment model.

[0064] In this embodiment, firstly, a multi-channel EEG signal sample library is collected. Noise types include eye movement artifacts, electromyography interference, power frequency and its harmonic interference, electrode contact noise, and environmental electromagnetic radiation noise. Each noise type must contain at least five gradient intensity levels, ranging from weak to strong interference. For each noisy EEG signal sample, its multi-dimensional signal integrity feature vector is extracted simultaneously. This vector specifically includes time-domain statistical features, frequency-domain features, time-frequency-domain features, and spatial features. These features are standardized and then concatenated to form a feature vector of dimension D, constituting a feature vector training set.

[0065] Secondly, each feature vector is labeled according to a pre-defined noise complexity scoring system. This scoring system uses a quantification standard of 1-10 points, considering the degree of distortion of key signal features by noise (weight 40%), the degree of noise type mixing (weight 30%), and the dynamic fluctuation of noise intensity (weight 30%).

[0066] Subsequently, an initial noise assessment model was constructed based on a multilayer perceptron. The model structure includes an input layer, hidden layers, and an output layer. The output layer contains one neuron with a linear activation function, outputting a continuous noise complexity score. The Adam optimizer was used with an initial learning rate of 0.001, decaying by 10% every 50 rounds. The loss function was mean squared error. Five-fold cross-validation was employed during training. Convergence was defined as the model's parameters being saved as a pre-trained noise assessment model when the change in MSE on the validation set was less than 1e-5 over 100 consecutive rounds. This model quantifies the overall interference level of noise in the current EEG signal based on the multi-dimensional signal integrity feature vector of the input.

[0067] S30: Based on the noise complexity score, the feature decoupling module is dynamically invoked from the progressive feature decoupling network constructed based on the adversarial domain adaptation network, and the multi-channel EEG signal is input to output the task-related feature vector and the noise-related feature vector.

[0068] In this embodiment, feature decoupling modules of the corresponding level are dynamically invoked from the progressive feature decoupling network based on noise complexity scores. The progressive feature decoupling network includes three levels: basic decoupling module, intermediate decoupling module, and advanced decoupling module, corresponding to low, medium, and high noise complexity scores, respectively. When the noise complexity score is 1-3, the basic decoupling module is invoked. This module adopts a shallow convolutional neural network architecture, containing two convolutional layers and one pooling layer, mainly for preliminary feature separation of a single type of weak noise. When the score is 4-7, the intermediate decoupling module is invoked. It adds a bidirectional long short-term memory network layer and an attention mechanism to the basic module architecture to handle scenarios with multiple medium-intensity noise mixtures. It improves decoupling accuracy through temporal dependency modeling and key feature weight allocation. When the score reaches 8-10, the advanced decoupling module is invoked. It introduces a multi-scale feature fusion module and a residual connection structure on the basis of the intermediate module. The multi-scale feature fusion module extracts and fuses features in parallel through convolutional kernels of different sizes. The residual connection structure alleviates the gradient vanishing problem during deep network training to cope with complex interference with high intensity, multiple types of noise mixtures and dynamic fluctuations.

[0069] After calling the corresponding feature decoupling module, the preprocessed multi-channel EEG signal is input into the module, which outputs a temporal feature vector. The weight of each time step is calculated through the attention mechanism, and the weighted sum is used to obtain the preliminary decoupling features.

[0070] Furthermore, the initial decoupled features are passed to subsequent convolutional and fully connected layers through residual connection structures, and finally output task-related feature vectors and noise-related feature vectors through two parallel output branches.

[0071] Specifically, step S30 in the method includes:

[0072] Obtain the total number of levels K of the feature decoupling modules contained in the pre-trained progressive feature decoupling network;

[0073] The noise complexity score is multiplied by the total number of levels K, and the product is rounded down to obtain the activation number N, where the activation number N is an integer between 1 and K.

[0074] The multi-channel EEG signal is input into the first N-level feature decoupling module of the progressive feature decoupling network for decoupling processing. The task-related feature vector output by the N-level feature decoupling module is used as the final task-related feature vector, and the noise-related feature vector output by the N-level feature decoupling module is used as the final noise-related feature vector.

[0075] In this embodiment, firstly, the total number of levels K of the feature decoupling modules included in the progressive feature decoupling network is set to 3, corresponding to the basic, intermediate, and advanced levels. The noise complexity score S output by the noise evaluation model, ranging from 1 to 10, is multiplied by the total number of levels K to obtain the product S×K. To ensure that the activation level N is an integer between 1 and K, the product result is normalized and rounded. Specifically, the calculation method is N=round(S-1) / (10-1)(K-1))+1.

[0076] For example, when the noise complexity score S=2, (2-1) / (10-1)(3-1)=1 / 9×2≈0.222, which is rounded to 0, N=0+1=1, that is, the first level basic decoupling module is activated;

[0077] When S=5, (5-1) / 9×2≈0.888, which is rounded to 1, N=1+1=2, and the second-level intermediate decoupling module is activated;

[0078] When S=9, (9-1) / 9×2≈1.777, which is rounded to 2. N=2+1=3, thus activating the third-level advanced decoupling module.

[0079] Subsequently, the preprocessed multi-channel EEG signals are input into the first N levels of the progressive feature decoupling network for step-by-step decoupling. Each level of the decoupling module uses the output of the previous level as input to perform deeper feature decoupling and purification.

[0080] Finally, the task-related feature vector output by the Nth-level feature decoupling module is used as the final task-related feature vector; at the same time, the noise-related feature vector output by the Nth-level feature decoupling module is used as the final noise-related feature vector.

[0081] The multi-channel EEG signal is input into the first N-level feature decoupling module of the progressive feature decoupling network for decoupling processing, including:

[0082] The multi-channel EEG signal is input into the first-level feature decoupling module of the progressive feature decoupling network, and the first-level task-related feature vector and the first-level noise-related feature vector are output.

[0083] The first-level task-related feature vector is input into the second-level feature decoupling module of the progressive feature decoupling network, and the output is the second-level task-related feature vector and the second-level noise-related feature vector.

[0084] Repeat the input and output process until the task-related feature vector output by the (N-1)th level feature decoupling module is input into the Nth level feature decoupling module, and the Nth level task-related feature vector and the Nth level noise-related feature vector are output.

[0085] In this embodiment, the preprocessed multi-channel EEG signal is first input to the first-level feature decoupling module of the progressive feature decoupling network, namely the basic decoupling module. This module first performs preliminary feature extraction on the input EEG signal through the first convolutional layer, capturing the local temporal and spatial patterns of the signal, and then performs dimensionality reduction and nonlinear transformation through the ReLU activation function and a 2x2 max pooling layer.

[0086] Secondly, the second convolutional layer further extracts more abstract features, also undergoing ReLU activation and max pooling. After two layers of convolution and pooling operations, the output feature map is flattened into a one-dimensional vector, serving as the preliminary form of the first-level task-related feature vector. Simultaneously, the corresponding first-level noise-related feature vector is output, which contains the feature components identified as noise-related during the initial feature separation process.

[0087] Furthermore, if the activation level N is greater than 1, for example, N=2, then the first-level task-related feature vector output by the first-level feature decoupling module is input into the second-level feature decoupling module, i.e., the intermediate-level decoupling module. The intermediate-level decoupling module adds a bidirectional long short-term memory (Bi-LSTM) layer and an attention mechanism to the architecture of the basic decoupling module.

[0088] First, the flattened first-level task-related feature vector is fed into a Bi-LSTM layer, for example, with 128 hidden units. This layer effectively models the temporal dependencies of EEG signals, capturing the dynamic correlations between features at different time points. The output of the Bi-LSTM is then fed into an attention layer, which learns the importance weights of features at each time step and performs a weighted summation of the Bi-LSTM output, thereby highlighting the temporal features more critical to subsequent decoding tasks. After processing by the Bi-LSTM and attention mechanisms, the feature vector is mapped and integrated through a fully connected layer to obtain the second-level task-related feature vector. Correspondingly, the intermediate decoupling module also outputs a second-level noise-related feature vector. Compared to the first-level noise-related feature vector, this vector separates the features of various moderate-intensity mixed noises, because the Bi-LSTM and attention mechanisms help distinguish the dynamic patterns of noise and the temporal characteristics of task-related signals.

[0089] Similarly, when the activation level N=3, the second-level task-related feature vector is input to the third-level feature decoupling module, i.e., the advanced decoupling module. The advanced decoupling module introduces a multi-scale feature fusion module and a residual connection structure based on the intermediate decoupling module. The multi-scale feature fusion module typically contains multiple parallel convolutional operation branches, each using convolutional kernels of different sizes to capture feature information at different temporal or spatial scales. Then, the feature maps from different branches are concatenated and fused along the channel dimension. Through the synergistic effect of multi-scale feature fusion and residual connections, the advanced decoupling module can perform deeper decoupling and purification of the second-level task-related feature vector, ultimately outputting a third-level task-related feature vector and a third-level noise-related feature vector. This addresses complex interference scenarios with high-intensity, multi-type noise mixing and dynamic fluctuations, maximizing the retention of feature information closely related to the auditory decoding task and separating noise-related features.

[0090] Furthermore, the construction process of the progressive feature decoupling network includes:

[0091] Construct an initial decoupling network containing K-level initial feature decoupling modules, and simultaneously construct K independent noise domain discriminators;

[0092] Clean EEG data with accurate attention labels are prepared as source domain training data, and noisy EEG data without accurate attention labels are prepared as target domain training data.

[0093] The source domain training data and the target domain training data are respectively input into the initial decoupling network for processing, and the corresponding task-related feature vectors and noise-related feature vectors are output.

[0094] Based on the task-related feature vectors and accurate attention labels corresponding to the source domain training data, the task decoding loss is calculated using the cross-entropy loss function.

[0095] The noise-related feature vectors output by each feature decoupling module for the source domain training data and the target domain training data are respectively input into the corresponding noise domain discriminator. The domain classification loss is calculated by the binary cross-entropy loss function, and the total domain classification loss is obtained by summing all the domain classification losses.

[0096] With the joint optimization objective of minimizing the task decoding loss and maximizing the total domain classification loss, the parameters of the initial decoupled network are updated through the gradient backpropagation algorithm.

[0097] With minimizing the total domain classification loss as the optimization objective, the parameters of each noise domain discriminator are updated using the gradient backpropagation algorithm;

[0098] Once the iterative training process reaches the preset convergence condition, the parameters of the initial decoupling network are saved to obtain the pre-trained progressive feature decoupling network.

[0099] In this embodiment, firstly, an initial decoupling network containing three levels of initial feature decoupling modules is constructed. Each level of module corresponds to the basic, intermediate, and advanced initial architectures, respectively. Simultaneously, an independent noise domain discriminator is constructed for each level of module. Each discriminator adopts a three-layer fully connected network structure. The input is the noise-related feature vector of the corresponding level, and the output is the probability that the feature vector comes from the source domain or the target domain.

[0100] Secondly, when preparing the source domain training data, multi-channel auditory evoked EEG signals were collected from 20 healthy subjects in a quiet environment for 10 hours, with accurate attentional labels for specific auditory stimuli, such as the time of appearance of the target stimulus, recorded simultaneously. For the target domain training data, EEG signals from the same subjects were collected in different noise environments, including all the aforementioned noise types and intensity levels, for 50 hours, without attentional labels. The source domain data and target domain data were divided into training and validation sets in a 4:1 ratio.

[0101] Secondly, during the training phase, the source domain training data is input into the initial decoupled network. After processing by each module, it outputs task-related feature vectors and noise-related feature vectors. The target domain training data is also input into the network, outputting only noise-related feature vectors. When calculating the task decoding loss, the source domain task-related feature vectors are input into the classifier, and cross-entropy loss is calculated with the accurate attention labels, with a weight of 1.0. For the domain classification loss, each level of noise-related feature vector is input into its corresponding noise domain discriminator. For example, the first level of noise-related feature vector is input into the first discriminator. The discriminator learns to distinguish the distribution differences of noise features between the source and target domains, outputs the classification probability, and calculates the binary cross-entropy loss. The losses of the three discriminators are summed with a weight of 1:1:1 to obtain the total domain classification loss.

[0102] Furthermore, the network parameter update employs an alternating training strategy to fix the discriminator parameters in order to minimize the task decoding loss. The parameters of the initial decoupled network are updated with the goal of gradient descent and maximizing the total domain classification loss. This process ensures that the task-related features generated by the decoupled network retain the information required for decoding while making the distribution of noise-related features as similar as possible in the source and target domains, i.e., to confuse the discriminator. Subsequently, the decoupled network parameters are fixed, and the discriminator parameters are updated with the goal of minimizing the total domain classification loss to improve its discriminative ability.

[0103] Specifically, during iterative training, each training round contains 1000 batches, with each batch containing 64 samples. When the task decoding accuracy on the validation set improves by less than 0.1% for 20 consecutive rounds and the total domain classification loss stabilizes, convergence is determined, and the network parameters at this point are saved, completing the pre-training of the progressive feature decoupling network. This pre-training process enhances the network's adaptability to different noise distributions through an adversarial domain adaptation mechanism.

[0104] S40: Input the task-related feature vector into the pre-trained attention decoder and output the auditory attention allocation probability distribution. Then, fuse the noise-related feature vector with the multi-dimensional signal integrity feature vector and input it into the pre-trained noise quantizer to output the noise source tracing and impact report.

[0105] In this embodiment, the pre-trained attention decoder adopts a Transformer-based encoder-decoder architecture, wherein the encoder part consists of 6 stacked Transformer encoders, each layer containing a multi-head self-attention mechanism and a feedforward neural network, which is used to perform deep semantic modeling on the input task-related feature vectors.

[0106] The decoder is a 3-layer Transformer decoder that focuses on key features of the encoder output through a cross-attention mechanism. Finally, it outputs an auditory attention allocation probability distribution after passing through a linear layer and a softmax activation function. This distribution covers attention weights for different auditory stimuli, ranging from 0 to 1, with the sum of all weights equal to 1. For example, when the input EEG signal corresponds to the subject focusing on the target speech played through the left earphone, the weight of the left-side speech source in the output probability distribution is higher than that of other interfering sound sources.

[0107] Furthermore, the noise quantizer consists of a feature fusion layer, a noise type identification subnetwork, and an impact assessment subnetwork. By fusing the noise-related feature vector with the multi-dimensional signal integrity feature vector and inputting it into the pre-trained noise quantizer, the output is a noise source tracing and impact report.

[0108] Specifically, the task-related feature vector is input into a pre-trained attention decoder, and the output is the auditory attention allocation probability distribution, including:

[0109] Collect multiple historical users' EEG signal samples under known attentional target sound source states, extract corresponding task-related feature vectors, and construct a decoder feature training set;

[0110] Obtain the known attention target sound source label corresponding to each sample to form the decoder label training set;

[0111] Construct a neural network with fully connected layers and softmax activation function as output layers as the initial attention decoder;

[0112] The initial attention decoder is trained in a supervised manner using the decoder feature training set and the decoder label training set until convergence is verified, thus obtaining the pre-trained attention decoder.

[0113] The task-related feature vector is input into a pre-trained attention decoder, which outputs an auditory attention allocation probability distribution, wherein each dimension of the auditory attention allocation probability distribution represents the probability of the target user allocating attention to a specific sound source in the auditory scene.

[0114] In this embodiment, firstly, 500 EEG signal samples containing a clear state of attention to a target sound source are selected from a publicly available EEG signal database. These samples cover data from healthy subjects of different ages and genders in various auditory scenarios, such as situations where clear attention is paid to sound sources on the left, right, in front, or at a specific frequency in scenarios involving dual-source competition or multi-source mixing. For each sample, the corresponding task-related feature vector is extracted according to the aforementioned preprocessing and progressive feature decoupling network processing flow. The feature vector dimension is uniformly set to 256 dimensions to construct the decoder feature training set.

[0115] At the same time, each sample is labeled with the corresponding known target sound source. For example, in a scenario containing three sound sources A, B, and C, if the subject pays attention to sound source A, the label is set to [1,0,0], and if the subject pays attention to sound source B, the label is set to [0,1,0], and so on, to form the decoder label training set.

[0116] Secondly, an initial attention decoder is constructed, with the input layer dimension set to 256 to match the dimension of the task-related feature vectors. The hidden layer consists of two fully connected layers: the first layer has 128 neurons using the ReLU activation function, and the second layer has 64 neurons, also using the ReLU activation function. The number of neurons in the output layer is consistent with the number of possible sound source categories; for example, 3 neurons for 3 sound sources. A softmax activation function is used to output the probability distribution.

[0117] Secondly, in the supervised training phase, the decoder feature training set and label training set are divided into training and validation sets in a 7:3 ratio. The Adam optimizer is used, with an initial learning rate of 0.001, a batch size of 32, and the cross-entropy loss function. During training, the model performance is evaluated on the validation set after each iteration. Training is stopped when the classification accuracy on the validation set no longer improves for 15 consecutive iterations or the loss value decreases to less than 0.0001 for 15 consecutive iterations. The model parameters at this point are saved, resulting in the pre-trained attention decoder.

[0118] Finally, the final task-related feature vector output by the progressive feature decoupling network is input into the pre-trained attention decoder. The decoder then outputs an auditory attention allocation probability distribution based on the mapping relationship between the learned features and attention labels. Each dimension of this distribution represents the probability of the target user allocating attention to a specific sound source in the current auditory scene.

[0119] For example, in a complex auditory environment containing television sound, traffic noise outside the window, and conversations of others, if the probability distribution of the output is [0.85, 0.05, 0.10], it indicates that the target user currently allocates approximately 85% of their attention to the television sound, 5% to the traffic noise, and 10% to conversations.

[0120] Furthermore, the noise-related feature vector and the multi-dimensional signal integrity feature vector are fused and input into a pre-trained noise quantizer, outputting a noise source tracing and impact report, including:

[0121] Multiple noisy EEG signal samples containing known noise types and intensity labels were collected. The corresponding multi-dimensional signal integrity feature vectors and noise-related feature vectors were extracted and concatenated to form a quantizer feature training set.

[0122] Obtain the known noise type and intensity label corresponding to each sample to form the quantizer label training set;

[0123] Construct a neural network with multiple fully connected layers as the initial noise quantizer;

[0124] The initial noise quantizer is trained in a supervised manner using the quantizer feature training set and the quantizer label training set until convergence is verified, thus obtaining a pre-trained noise quantizer.

[0125] The noise-related feature vector is concatenated with the multi-dimensional signal integrity feature vector to generate a fused noise feature vector;

[0126] The fused noise feature vector is input into a multi-layer neural network of a pre-trained noise quantizer for calculation, and the noise source and impact report calculated by the last layer of the noise quantizer is output. The noise source and impact report includes the main noise type identifier and the intensity quantization value of each noise type.

[0127] In this embodiment, firstly, 1000 noisy EEG signal samples with clearly defined noise types and intensity labels are selected, with each noise type containing 5 intensity levels, ranging from weak to extremely strong, and including complex cases of mixed noise types. The two feature vectors are concatenated along the channel dimension to form the quantizer feature training set.

[0128] Simultaneously, each sample is labeled with its corresponding known noise type and intensity label. The noise type label adopts a one-hot encoding form, for example, white noise corresponds to [1,0,0,...], power frequency interference corresponds to [0,1,0,...], etc.; the intensity label assigns a quantization value between 0 and 1 to each identified noise type, where 0 represents no noise and 1 represents the strongest intensity of the noise type, forming the quantizer label training set.

[0129] Secondly, an initial noise quantizer is constructed, consisting of four fully connected layers. The input layer dimension is set to 136, matching the dimension of the fused noise feature vector. The first fully connected layer has 256 neurons and uses the Leaky ReLU activation function; the second layer has 128 neurons, also using the Leaky ReLU activation function; the third layer has 64 neurons and uses the ELU activation function. The number of neurons in the output layer is set according to the preset maximum number of noise types plus the corresponding number of intensity quantization values. For example, for 10 noise types, the output layer has 20 neurons, representing 10 type probabilities plus 10 intensity values. The first 10 neurons output the probability distribution of the main noise types using the softmax activation function, and the last 10 neurons output the intensity quantization values ​​of the corresponding noise types using the sigmoid activation function.

[0130] Secondly, during the supervised training phase, the quantizer feature training set and the label training set are divided into training, validation, and test sets in a 6:2:2 ratio. The AdamW optimizer is used with an initial learning rate of 0.0005, a weight decay coefficient of 1e-5, and a batch size of 128. A combined loss function is employed: for noise type identification, the cross-entropy loss between the output type probability distribution and the one-hot encoded label is calculated; for noise intensity quantization, the mean square error loss between the output intensity value and the true intensity label is calculated, and these two losses are added together with a 1:1 weighting to obtain the total loss. During training, the model performance is evaluated on the validation set after each iteration. Convergence is determined when the total loss on the validation set decreases to less than 0.0001 for 30 consecutive iterations and the noise type identification accuracy and intensity prediction root mean square error on the test set tend to stabilize. The model parameters at this point are then saved, resulting in the pre-trained noise quantizer.

[0131] Finally, the final noise-related feature vector output by the progressive feature decoupling network is concatenated with the multi-dimensional signal integrity feature vector obtained in step S30 to generate a fused noise feature vector. This fused noise feature vector is then input into a multi-layer fully connected network of a pre-trained noise quantizer for computation. The quantizer first performs deep nonlinear transformation and feature abstraction on the fused features through the first three fully connected layers, and finally outputs a noise source and impact report from the output layer.

[0132] S50: Output the auditory attention allocation probability distribution and the noise source and impact report as the noise robustness auditory decoding result for the target user.

[0133] In this embodiment, the output auditory attention allocation probability distribution is integrated with the noise source tracing and impact report to form the final noise robust auditory decoding result, which is then presented to the user or downstream application system in a visual interface or structured data format.

[0134] For example, the probability distribution of auditory attention allocation can be visually displayed using bar charts or pie charts to show the attention weight of each sound source, with values ​​accurate to three decimal places. The noise source tracing and impact report lists the main noise types, such as "white noise 0.65, power frequency interference 0.23", with the intensity quantification value in parentheses. It can also include an explanation of the estimated impact of this noise combination on the current auditory decoding accuracy, such as "the overall noise intensity is moderate, which may lead to a decrease in decoding accuracy of approximately 5.2%". Downstream applications can use this result; for example, hearing aids can enhance the target sound source and suppress interfering sound sources based on the attention distribution, or researchers can optimize experimental environment settings based on the noise report.

[0135] In summary, compared with existing technologies, this application effectively separates task-related features from noise-related features by introducing a progressive feature decoupling network and an adversarial domain adaptation mechanism. This ensures that even when there are differences in noise distribution between the source and target domains, task-related features can still retain sufficient decoding information, thereby significantly improving the robustness of attention decoding.

[0136] In summary, the embodiments of this application have at least the following technical effects:

[0137] This application provides a noise-robust auditory decoding method based on an adversarial domain adaptation network. First, it acquires multi-channel EEG signals from a target user in an auditory scenario, performs parallel analysis and feature fusion to obtain a multi-dimensional signal integrity feature vector. Second, this feature vector is input into a pre-trained noise assessment model, which outputs a noise complexity score to quantify the complexity of noise in the current EEG signal. Subsequently, based on this noise complexity score, a feature decoupling module of the appropriate level is dynamically invoked from a progressive feature decoupling network constructed based on an adversarial domain adaptation network to process the multi-channel EEG signal, achieving effective separation of task-related feature vectors and noise-related feature vectors. This allows the model to adaptively adjust the depth and intensity of feature decoupling according to the noise situation. The task-related feature vector is then input into a pre-trained attention decoder to obtain an auditory attention allocation probability distribution, presenting the user's attention to different sound sources. Simultaneously, the noise-related feature vector is fused with the multi-dimensional signal integrity feature vector and input into a pre-trained noise quantizer, outputting a noise source tracing and impact report, clarifying information such as the type and intensity of the noise. Finally, the output of the auditory attention allocation probability distribution and noise source and impact report are used as decoding results.

[0138] Through the above technical solution, this application, based on the dynamic invocation feature decoupling module, can better cope with interference of different noise types and intensities, improve the accuracy and robustness of auditory decoding in noisy scenarios, and at the same time realize accurate assessment and source tracing of noise.

[0139] Example 2, as Figure 2 As shown, based on the same inventive concept as the noise robust auditory decoding method based on adversarial domain adaptation networks provided in Embodiment 1, this application also provides a noise robust auditory decoding system based on adversarial domain adaptation networks, including:

[0140] The information acquisition module 11 is used to acquire the multi-channel EEG signals of the target user in an auditory scene, and to perform parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector.

[0141] The noise model training module 12 is used to input the multi-dimensional signal integrity feature vector into the pre-trained noise evaluation model and output a noise complexity score.

[0142] The feature vector output module 13 is used to dynamically call the feature decoupling module from the progressive feature decoupling network constructed based on the adversarial domain adaptation network based on the noise complexity score, and input the multi-channel EEG signal to output task-related feature vectors and noise-related feature vectors.

[0143] The probability distribution output module 14 is used to input the task-related feature vector into a pre-trained attention decoder and output an auditory attention distribution probability distribution. It also inputs the noise-related feature vector and the multi-dimensional signal integrity feature vector into a pre-trained noise quantizer and outputs a noise source tracing and impact report.

[0144] The decoding result acquisition module 15 is used to output the auditory attention allocation probability distribution and the noise source tracing and impact report as the noise robustness auditory decoding result of the target user.

[0145] In one embodiment, the information acquisition module 11 is specifically used for:

[0146] Acquire multi-channel EEG signals of target users in auditory scenarios;

[0147] For the multi-channel EEG signal, the amplitude squared coherence coefficient between all channel pairs within a preset time window is calculated;

[0148] The amplitude squared coherence coefficients of all channel pairs are arranged into a symmetric channel coherence matrix. The upper triangular elements of the channel coherence matrix are extracted and arranged into a one-dimensional vector, which serves as the spatial coherence feature vector.

[0149] For each channel of the multichannel EEG signal, the average power spectral density in the δ band, θ band, α band and β band is calculated respectively;

[0150] Calculate the arithmetic mean of the average power spectral density of each channel in the δ band, θ band, α band and β band respectively, and combine them to form the cross-channel average frequency band power vector;

[0151] The spatial coherence feature vector and the cross-channel average frequency band power vector are concatenated to form a multi-dimensional signal integrity feature vector.

[0152] Further, in one embodiment of the application, for the multi-channel EEG signal, calculating the amplitude squared coherence coefficient between all channel pairs within a preset time window includes:

[0153] For the multi-channel EEG signal segments within the preset time window, the Welch average periodogram method is used to calculate the power spectral density of each channel signal and the cross-power spectral density between any two channel signals.

[0154] For any two channels, the average coherence coefficient within the preset frequency band is calculated according to the corresponding power spectral density and cross power spectral density using the amplitude squared coherence formula, and is used as the amplitude squared coherence coefficient of the channel pair.

[0155] Fill the amplitude squared coherence coefficients of all channel pairs into a C-row C-column symmetric matrix, where C is the total number of EEG channels;

[0156] Extract all elements in the symmetric matrix whose row index is less than their column index, and arrange them in row priority order to form a spatial coherence feature vector.

[0157] In one embodiment, the feature vector output module 13 is specifically used for:

[0158] Obtain the total number of levels K of the feature decoupling modules contained in the pre-trained progressive feature decoupling network;

[0159] The noise complexity score is multiplied by the total number of levels K, and the product is rounded down to obtain the activation number N, where the activation number N is an integer between 1 and K.

[0160] The multi-channel EEG signal is input into the first N-level feature decoupling module of the progressive feature decoupling network for decoupling processing. The task-related feature vector output by the N-level feature decoupling module is used as the final task-related feature vector, and the noise-related feature vector output by the N-level feature decoupling module is used as the final noise-related feature vector.

[0161] Furthermore, in one embodiment, the multi-channel EEG signal is input into the first N levels of the progressive feature decoupling network for decoupling processing, including:

[0162] The multi-channel EEG signal is input into the first-level feature decoupling module of the progressive feature decoupling network, and the first-level task-related feature vector and the first-level noise-related feature vector are output.

[0163] The first-level task-related feature vector is input into the second-level feature decoupling module of the progressive feature decoupling network, and the output is the second-level task-related feature vector and the second-level noise-related feature vector.

[0164] Repeat the input and output process until the task-related feature vector output by the (N-1)th level feature decoupling module is input into the Nth level feature decoupling module, and the Nth level task-related feature vector and the Nth level noise-related feature vector are output.

[0165] Furthermore, the construction process of the progressive feature decoupling network includes:

[0166] Construct an initial decoupling network containing K-level initial feature decoupling modules, and simultaneously construct K independent noise domain discriminators;

[0167] Clean EEG data with accurate attention labels are prepared as source domain training data, and noisy EEG data without accurate attention labels are prepared as target domain training data.

[0168] The source domain training data and the target domain training data are respectively input into the initial decoupling network for processing, and the corresponding task-related feature vectors and noise-related feature vectors are output.

[0169] Based on the task-related feature vectors and accurate attention labels corresponding to the source domain training data, the task decoding loss is calculated using the cross-entropy loss function.

[0170] The noise-related feature vectors output by each feature decoupling module for the source domain training data and the target domain training data are respectively input into the corresponding noise domain discriminator. The domain classification loss is calculated by the binary cross-entropy loss function, and the total domain classification loss is obtained by summing all the domain classification losses.

[0171] With the joint optimization objective of minimizing the task decoding loss and maximizing the total domain classification loss, the parameters of the initial decoupled network are updated through the gradient backpropagation algorithm.

[0172] With minimizing the total domain classification loss as the optimization objective, the parameters of each noise domain discriminator are updated using the gradient backpropagation algorithm;

[0173] Once the iterative training process reaches the preset convergence condition, the parameters of the initial decoupling network are saved to obtain the pre-trained progressive feature decoupling network.

[0174] In one embodiment, the probability allocation output module 14 is specifically used for:

[0175] Collect multiple historical users' EEG signal samples under known attentional target sound source states, extract corresponding task-related feature vectors, and construct a decoder feature training set;

[0176] Obtain the known attention target sound source label corresponding to each sample to form the decoder label training set;

[0177] Construct a neural network with fully connected layers and softmax activation function as output layers as the initial attention decoder;

[0178] The initial attention decoder is trained in a supervised manner using the decoder feature training set and the decoder label training set until convergence is verified, thus obtaining the pre-trained attention decoder.

[0179] The task-related feature vector is input into a pre-trained attention decoder, which outputs an auditory attention allocation probability distribution, wherein each dimension of the auditory attention allocation probability distribution represents the probability of the target user allocating attention to a specific sound source in the auditory scene.

[0180] In one embodiment, the probability distribution output module 14 is further configured to:

[0181] Multiple noisy EEG signal samples containing known noise types and intensity labels were collected. The corresponding multi-dimensional signal integrity feature vectors and noise-related feature vectors were extracted and concatenated to form a quantizer feature training set.

[0182] Obtain the known noise type and intensity label corresponding to each sample to form the quantizer label training set;

[0183] Construct a neural network with multiple fully connected layers as the initial noise quantizer;

[0184] The initial noise quantizer is trained in a supervised manner using the quantizer feature training set and the quantizer label training set until convergence is verified, thus obtaining a pre-trained noise quantizer.

[0185] The noise-related feature vector is concatenated with the multi-dimensional signal integrity feature vector to generate a fused noise feature vector;

[0186] The fused noise feature vector is input into a multi-layer neural network of a pre-trained noise quantizer for calculation, and the noise source and impact report calculated by the last layer of the noise quantizer is output. The noise source and impact report includes the main noise type identifier and the intensity quantization value of each noise type.

[0187] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0188] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0189] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A noise-robust auditory decoding method based on adversarial domain adaptation networks, characterized in that, The method includes: The system acquires multi-channel EEG signals of the target user in an auditory scenario, and performs parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector. The multi-dimensional signal integrity feature vector is input into a pre-trained noise evaluation model, and the noise complexity score is output. Based on the noise complexity score, the feature decoupling module is dynamically invoked from the progressive feature decoupling network constructed based on the adversarial domain adaptation network, and the multi-channel EEG signal is input to output the task-related feature vector and the noise-related feature vector. The task-related feature vector is input into a pre-trained attention decoder, and the output is the auditory attention allocation probability distribution. The noise-related feature vector is fused with the multi-dimensional signal integrity feature vector and then input into a pre-trained noise quantizer, and the output is the noise source tracing and impact report. The auditory attention allocation probability distribution and the noise source and impact report are output as the noise robust auditory decoding results for the target user; Specifically, this involves acquiring multi-channel EEG signals of the target user in an auditory context, and performing parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector, including: Acquire multi-channel EEG signals of target users in auditory scenarios; For the multi-channel EEG signal, the amplitude squared coherence coefficient between all channel pairs within a preset time window is calculated; The amplitude squared coherence coefficients of all channel pairs are arranged into a symmetric channel coherence matrix. The upper triangular elements of the channel coherence matrix are extracted and arranged into a one-dimensional vector, which serves as the spatial coherence feature vector. For each channel of the multichannel EEG signal, the average power spectral density in the δ band, θ band, α band and β band is calculated respectively; Calculate the arithmetic mean of the average power spectral density of each channel in the δ band, θ band, α band and β band respectively, and combine them to form the cross-channel average frequency band power vector; The spatial coherence feature vector and the cross-channel average frequency band power vector are concatenated to form a multi-dimensional signal integrity feature vector; Specifically, based on the noise complexity score, the feature decoupling module is dynamically invoked from the progressive feature decoupling network constructed based on the adversarial domain adaptation network, and the multi-channel EEG signal is input to output task-related feature vectors and noise-related feature vectors, including: Obtain the total number of levels K of the feature decoupling modules contained in the pre-trained progressive feature decoupling network; The noise complexity score is multiplied by the total number of levels K, and the product is rounded down to obtain the activation number N, where the activation number N is an integer between 1 and K. The multi-channel EEG signal is input into the first N-level feature decoupling module of the progressive feature decoupling network for decoupling processing. The task-related feature vector output by the N-level feature decoupling module is used as the final task-related feature vector, and the noise-related feature vector output by the N-level feature decoupling module is used as the final noise-related feature vector. Specifically, the task-related feature vector is input into a pre-trained attention decoder, and the output is the auditory attention allocation probability distribution, including: Collect multiple historical users' EEG signal samples under known attentional target sound source states, extract corresponding task-related feature vectors, and construct a decoder feature training set; Obtain the known attention target sound source label corresponding to each sample to form the decoder label training set; Construct a neural network with fully connected layers and softmax activation function as output layers as the initial attention decoder; The initial attention decoder is trained in a supervised manner using the decoder feature training set and the decoder label training set until convergence is verified, thus obtaining the pre-trained attention decoder. The task-related feature vector is input into a pre-trained attention decoder, which outputs an auditory attention allocation probability distribution, wherein each dimension of the auditory attention allocation probability distribution represents the probability of the target user allocating attention to a specific sound source in the auditory scene.

2. The noise-robust auditory decoding method based on adversarial domain adaptation networks according to claim 1, characterized in that, For the multi-channel EEG signal, the amplitude squared coherence coefficient between all channel pairs within a preset time window is calculated, including: For the multi-channel EEG signal segments within the preset time window, the Welch average periodogram method is used to calculate the power spectral density of each channel signal and the cross-power spectral density between any two channel signals. For any two channels, the average coherence coefficient within the preset frequency band is calculated according to the corresponding power spectral density and cross power spectral density using the amplitude squared coherence formula, and is used as the amplitude squared coherence coefficient of the channel pair. Fill the amplitude squared coherence coefficients of all channel pairs into a C-row C-column symmetric matrix, where C is the total number of EEG channels; Extract all elements in the symmetric matrix whose row index is less than their column index, and arrange them in row priority order to form a spatial coherence feature vector.

3. The noise-robust auditory decoding method based on adversarial domain adaptation networks according to claim 1, characterized in that, The process of constructing a noise assessment model includes: Multiple multi-channel EEG signal samples containing different noise types and intensities were collected, and multi-dimensional signal integrity feature vectors were extracted from each multi-channel EEG signal sample to form a feature vector training set. Each feature vector is labeled with a noise complexity score label according to a preset standard, thus forming a labeled training set. An initial noise assessment model is constructed based on a multilayer perceptron. The noise assessment initial model is trained in a supervised manner using the feature vector training set and the label training set until the verification convergence, thus obtaining the pre-trained noise assessment model.

4. The noise-robust auditory decoding method based on adversarial domain adaptation networks according to claim 1, characterized in that, The multi-channel EEG signals are input into the first N levels of the progressive feature decoupling network for decoupling processing, including: The multi-channel EEG signal is input into the first-level feature decoupling module of the progressive feature decoupling network, and the first-level task-related feature vector and the first-level noise-related feature vector are output. The first-level task-related feature vector is input into the second-level feature decoupling module of the progressive feature decoupling network, and the output is the second-level task-related feature vector and the second-level noise-related feature vector. Repeat the input and output process until the task-related feature vector output by the (N-1)th level feature decoupling module is input into the Nth level feature decoupling module, and the Nth level task-related feature vector and the Nth level noise-related feature vector are output.

5. The noise-robust auditory decoding method based on adversarial domain adaptation networks according to claim 1, characterized in that, The construction process of a progressive feature decoupling network includes: Construct an initial decoupling network containing K-level initial feature decoupling modules, and simultaneously construct K independent noise domain discriminators; Clean EEG data with accurate attention labels are prepared as source domain training data, and noisy EEG data without accurate attention labels are prepared as target domain training data. The source domain training data and the target domain training data are respectively input into the initial decoupling network for processing, and the corresponding task-related feature vectors and noise-related feature vectors are output. Based on the task-related feature vectors and accurate attention labels corresponding to the source domain training data, the task decoding loss is calculated using the cross-entropy loss function. The noise-related feature vectors output by each feature decoupling module for the source domain training data and the target domain training data are respectively input into the corresponding noise domain discriminator. The domain classification loss is calculated by the binary cross-entropy loss function, and the total domain classification loss is obtained by summing all the domain classification losses. With the joint optimization objective of minimizing the task decoding loss and maximizing the total domain classification loss, the parameters of the initial decoupled network are updated through the gradient backpropagation algorithm. With minimizing the total domain classification loss as the optimization objective, the parameters of each noise domain discriminator are updated using the gradient backpropagation algorithm; Once the iterative training process reaches the preset convergence condition, the parameters of the initial decoupling network are saved to obtain the pre-trained progressive feature decoupling network.

6. The noise-robust auditory decoding method based on adversarial domain adaptation networks according to claim 1, characterized in that, The noise-related feature vector and the multi-dimensional signal integrity feature vector are fused and then input into a pre-trained noise quantizer. The output is a noise source tracing and impact report, including: Multiple noisy EEG signal samples containing known noise types and intensity labels were collected. The corresponding multi-dimensional signal integrity feature vectors and noise-related feature vectors were extracted and concatenated to form a quantizer feature training set. Obtain the known noise type and intensity label corresponding to each sample to form the quantizer label training set; Construct a neural network with multiple fully connected layers as the initial noise quantizer; The initial noise quantizer is trained in a supervised manner using the quantizer feature training set and the quantizer label training set until convergence is verified, thus obtaining a pre-trained noise quantizer. The noise-related feature vector is concatenated with the multi-dimensional signal integrity feature vector to generate a fused noise feature vector; The fused noise feature vector is input into a multi-layer neural network of a pre-trained noise quantizer for calculation, and the noise source and impact report calculated by the last layer of the noise quantizer is output. The noise source and impact report includes the main noise type identifier and the intensity quantization value of each noise type.

7. A noise-robust auditory decoding system based on an adversarial domain adaptation network, characterized in that, The method for performing the noise-robust auditory decoding method based on adversarial domain adaptation networks as described in any one of claims 1-6 includes: The information acquisition module is used to acquire multi-channel EEG signals of the target user in an auditory scenario, and to perform parallel analysis and feature fusion on the multi-channel EEG signals to obtain a multi-dimensional signal integrity feature vector. The noise model training module is used to input the multi-dimensional signal integrity feature vector into the pre-trained noise evaluation model and output a noise complexity score. The feature vector output module is used to dynamically call the feature decoupling module from the progressive feature decoupling network constructed based on the adversarial domain adaptation network based on the noise complexity score, and input the multi-channel EEG signal to output task-related feature vectors and noise-related feature vectors. The probability distribution output module is used to input the task-related feature vector into a pre-trained attention decoder and output an auditory attention distribution probability distribution. The noise-related feature vector is fused with the multi-dimensional signal integrity feature vector and then input into a pre-trained noise quantizer to output a noise source tracing and impact report. The decoding result acquisition module is used to output the auditory attention allocation probability distribution and the noise source tracing and impact report as the noise robustness auditory decoding result of the target user.

Images