Auditory attention detection method and system based on electroencephalogram signal

Abstract

The present invention relates to the field of signal processing technology, and more specifically, to an auditory attention detection method and system based on electroencephalogram (EEG) signals. The present invention first performs sliding window processing on the EEG signals to obtain data of different decision time window lengths, then divides the processed EEG signals into frequency bands, extracts differential entropy features in each frequency band, and finally extracts spatial features in the EEG signals, thereby improving the accuracy of auditory spatial attention detection by making full use of the global and local spatial information contained in the leads. In addition, experimental verification shows that the accuracy of the present invention is better than the mainstream level of the prior art.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and more specifically, to an auditory attention detection method and system based on electroencephalogram signals. Background Art

[0002] The cocktail party effect refers to an attention mechanism in the cerebral cortex when processing sound signals, which can distinguish the target sound from the noisy environment and ignore the interfering sound. Auditory attention detection is the listener's brain using the cocktail party effect to detect the speaker of attention in a multi-speaker scene. However, due to hearing loss, hearing-impaired people cannot effectively use the cocktail party effect, which makes it difficult to distinguish the target sound from the interfering sound in a noisy environment, which leads to communication difficulties and emotional problems.

[0003] With the continuous development of technology, more and more advanced auditory attention detection algorithms are integrated into existing hearing aids. These algorithms can help the hearing-impaired to capture target sounds more accurately, ignore interfering sounds, and improve their hearing performance in multi-speaker scenarios.

[0004] There are two main paradigms in the current research on auditory attention detection, namely detecting speaker identity and detecting the spatial trajectory of attention. The former requires EEG signals and clean auditory stimuli as input, while the latter only requires EEG signals. However, in daily life, such clean auditory stimuli are difficult to obtain.

[0005] At present, auditory attention detection methods are mainly divided into traditional linear methods and nonlinear methods. In recent years, neuroscience research has shown that the brain processes auditory stimuli through nonlinear mapping. Traditional linear methods are difficult to study this nonlinear mapping, and this type of method has an important disadvantage. That is, since it takes about 1s to 2s for a person with normal hearing to shift attention from one speaker to another, the decoding accuracy of the linear method will drop sharply as the time window shortens (the accuracy is less than 60% when the time window is 1s). Therefore, the research trend is gradually moving towards nonlinear methods based on EEG signals, which can achieve high performance in a short time window.

[0006] When performing auditory spatial attention detection based on nonlinear methods, it is also very important to make full use of the spatial information in EEG signals. Because when using leads to collect EEG signals, the EEG signals collected by different lead placements have obvious spatial position information, and the association between different leads can supplement the spatial information. The most commonly used nonlinear method at present is the convolutional neural network. However, for discrete signals with different spatial scales, the existing convolutional neural network uses a fixed-size convolution kernel to extract its features, which limits the convolutional neural network's perception of spatial structures of different scales and levels. Therefore, the convolutional neural network may not be able to fully capture the complex relationship between local and global spatial information. EEG signals are discrete in the spatial domain, so the existing nonlinear methods are not very suitable. Summary of the invention

[0007] Based on this, it is necessary to provide an auditory attention detection method and system based on EEG signals to address the problem that the existing nonlinear method based on convolutional neural networks cannot effectively extract features when processing EEG signals.

[0008] The present invention is implemented by the following technical solutions:

[0009] In a first aspect, the present invention discloses an auditory attention detection method based on electroencephalogram signals, which is used to detect the speaker that the listener is interested in in a multi-speaker scene.

[0010] The auditory attention detection method comprises the following steps:

[0011] Step 1: Perform sliding window processing on the real-time EEG signals of the multi-speaker scene to obtain EEG signal data of N time periods;

[0012] Step 2: Divide the EEG signal data of each time period into five frequency bands, and extract multi-band differential entropy features to obtain EEG signal data with multi-band differential entropy features;

[0013] Step 3: Use the trained graph self-distillation model to process the EEG signal data with multi-band differential entropy features to obtain the detection results;

[0014] The graph self-distillation model includes a four-layer graph convolutional network, and each layer of the graph convolutional network is connected to a fully connected classifier; the four-layer graph convolutional network in the trained graph self-distillation model is also loaded with an optimal adjacency matrix;

[0015] In step 3, the spatial features of the EEG signal data with multi-band differential entropy features are extracted through a four-layer graph convolutional network loaded with the optimal adjacency matrix; the probability of the listener paying attention to the left / right space is predicted through the fully connected classifier of the fourth layer;

[0016] If the probability that the listener pays attention to the left side of the space is greater than the probability that the listener pays attention to the right side of the space, then the speaker that the listener is interested in is on the left side of the listener; if the probability that the listener pays attention to the left side of the space is less than the probability that the listener pays attention to the right side of the space, then the speaker that the listener is interested in is on the right side of the listener.

[0017] The auditory attention detection method based on EEG signals implements the method or process according to the embodiment of the present disclosure.

[0018] In a second aspect, the present invention discloses an auditory attention detection system based on EEG signals, which uses the auditory attention detection method based on EEG signals of the first aspect.

[0019] The auditory attention detection system based on EEG signals includes: an EEG signal data processing module, a multi-band differential entropy feature extraction module, and a graph self-distillation module.

[0020] The EEG signal data processing module is used to perform sliding window processing on the real-time EEG signals of the multi-talker scene to obtain EEG signal data of N time periods. The multi-band differential entropy feature extraction module is used to divide the EEG signal data of each time period into five frequency bands and extract multi-band differential entropy features to obtain EEG signal data with multi-band differential entropy features. The image self-distillation module is used to use the trained image self-distillation model to process the EEG signal data with multi-band differential entropy features to obtain the detection results.

[0021] The auditory attention detection system based on EEG signals implements the method or process according to the embodiment of the present disclosure.

[0022] In a third aspect, the present invention discloses a readable storage medium having computer program instructions stored therein, and when the computer program instructions are read and executed by a processor, the auditory attention detection method based on EEG signals disclosed in the first aspect is executed.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] (1) When processing EEG signal data, the present invention uses a sliding window to cut the listener's EEG signal into a series of time periods according to the frequency of the pre-processed EEG signal, so as to better measure the time scale of the listener's attention switching.

[0025] (2) The present invention divides the EEG signal data of each time period into five frequency bands to describe the spatial characteristics and functional status of the EEG signal, avoid the defects and deficiencies of single frequency band information, and improve the stability and robustness of the features; and extracts differential entropy features from each frequency band respectively, which can better reflect the complexity and randomness between brain regions and better express the dynamic changes of the signal, thereby improving the sensitivity to changes in auditory spatial attention.

[0026] (3) The present invention uses a graph to construct the spatial position relationship of EEG signal data, and utilizes multi-layer graph convolution to extract the spatial features of the EEG signal, and then predicts the probability of the listener paying attention to the left / right space through a fully connected classifier to obtain the detection result; wherein, the adopted graph self-distillation model uses graph self-distillation during training to balance the difference between the features extracted by the deepest graph convolution and the first three layers of graph convolution, so that the prediction result after balancing the features will be more accurate than that before prediction, thereby improving the listener's auditory spatial attention detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the auditory attention detection method based on EEG signals in Example 1 of the present invention;

[0028] Figure 2 for Figure 1 Working diagram of the EEG signal data processing module;

[0029] Figure 3 for Figure 1 Working diagram of the multi-band differential entropy feature extraction module;

[0030] Figure 4 for Figure 1 Schematic diagram of the working of the graph self-distillation module in the trained graph self-distillation model;

[0031] Figure 5 for Figure 1 Schematic diagram of the working of the graph self-distillation module during training of the graph self-distillation model.

[0032] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0033] 1. EEG signal data processing module, 2. Multi-band differential entropy feature extraction module, 3. Image self-distillation module. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component at the same time. When a component is considered to be "fixed to" another component, it may be directly fixed on the other component or there may be a central component at the same time.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" used herein includes any and all combinations of one or more of the related listed items.

[0037] Example 1

[0038] See also Figure 1 , Figure 1 It is a flow chart of the auditory attention detection method based on EEG signals in the present invention, and the auditory attention detection method is used to detect the speaker that the listener is interested in in a multi-speaker scene. Figure 1 An auditory attention detection system based on EEG signals is also disclosed simultaneously.

[0039] Among them, the auditory attention detection system based on EEG signals includes: an EEG signal data processing module 1, a multi-band differential entropy feature extraction module 2, and a graph self-distillation module 3.

[0040] The EEG signal data processing module 1 is used to perform sliding window processing on the real-time EEG signals of the multi-speaker scene to obtain EEG signal data of N time periods. That is, the EEG signal data processing module 1 performs sliding window processing on the real-time EEG signals in order to better measure the time scale of the listener's attention switching.

[0041] The multi-band differential entropy feature extraction module 2 is used to divide the EEG signal data of each time period into five frequency bands, and extract the multi-band differential entropy features to obtain EEG signal data with multi-band differential entropy features. That is, the multi-band differential entropy feature extraction module 2 extracts the differential entropy features of the five frequency bands in the EEG signal data so as to capture the changes in auditory spatial attention within different frequency ranges.

[0042] The image self-distillation module 3 is used to process the EEG signal data with multi-band differential entropy features using the trained image self-distillation model to obtain the detection result. That is, the image self-distillation module 3 makes full use of the spatial feature information of the EEG signal data, and finally achieves the purpose of improving the auditory spatial attention detection performance.

[0043] The following is a detailed description of each module in conjunction with the flowchart of the auditory attention detection method based on EEG signals:

[0044] The auditory attention detection method based on EEG signals includes the following steps:

[0045] Step 1: Perform sliding window processing on the real-time EEG signals of the multi-speaker scene to obtain EEG signal data of N time periods.

[0046] See also Figure 2 , for the EEG signal data processing module 1:

[0047] The real-time EEG signals in multi-talker scenarios must be preprocessed to improve signal quality. Preprocessing includes: first, filtering out line noise and harmonics at 50Hz in the EEG signals; second, performing a 0.1Hz to 50Hz bandpass filter on the EEG signals to reduce noise; then, using a fast Fourier transform (FFT)-based resampling method to downsample the EEG data to a set frequency (e.g., 64Hz, 128Hz, etc.); finally, normalizing the EEG data channels to ensure that the mean and unit variance of each trial are zero.

[0048] Then, according to the frequency of the preprocessed real-time EEG signal, the preprocessed real-time EEG signal is cut into N time periods using a sliding window with an overlap of 50%, to obtain EEG signal data of N time periods.

[0049] Among them, as the frequency of the preprocessed real-time EEG signal is different, the length of the corresponding time period is also different. For example, if the frequency is 128Hz, the length of the time period of 1s is 1*128, and the length of the time period of 2s is 2*128; if the frequency is 64Hz, the length of the time period of 1s is 1*64, and the length of the time period of 2s is 2*64.

[0050] Of course, the preprocessed real-time EEG signal can also be used as the input of the EEG signal data processing module 1. In this case, a preprocessing module needs to be connected in front of the EEG signal data processing module 1 to preprocess the real-time EEG signal of the multi-speaker scene.

[0051] Step 2: Divide the EEG signal data of each time period into five frequency bands, and extract multi-band differential entropy features to obtain EEG signal data with multi-band differential entropy features.

[0052] See also Figure 3 , for the multi-band differential entropy feature extraction module 2:

[0053] The EEG data in each time window is divided into frequency bands, and the differential entropy features in each frequency band are extracted.

[0054] Specifically, the EEG signal data of the nth time period is taken as the nth time window; n∈[1,N];

[0055] For the nth time window, the power spectral density function PSD in the jth frequency band is extracted from the EEG signal of the ith lead. i,j (n,f), f represents frequency;

[0056] PSD-based i,j (n,f) calculates differential entropy DE i,j (n);

[0057] Then DE i,j (n) Take the average along the time window direction to obtain the multi-band differential entropy feature F of the i-th lead in the j-th frequency band i,j .

[0058] in,

[0059]

[0060]

[0061] In the formula, K j is the number of intervals used to calculate differential entropy; p i,j,k (n) is the probability density function of the kth interval of the ith lead in the jth frequency band; f j,k-1 and f j,k are the lower and upper limits of the kth interval respectively; f l is the frequency when the time window is at l in the kth interval;

[0062] L j It is used to calculate p i,j,k (n) is the window length; x is a two-dimensional matrix, the first dimension of which is the lead and the second dimension is the differential entropy features of the five frequency bands of each lead.

[0063] In this way, the multi-band differential entropy feature F is obtained i,j , and is represented by a two-dimensional matrix x.

[0064] Step three, use the trained graph self-distillation model to process the EEG signal data with multi-band differential entropy features to obtain the detection results.

[0065] See also Figure 4 , for the graph distillation module 3:

[0066] The graph self-distillation model includes a four-layer graph convolutional network, and each layer of the graph convolutional network is connected to a fully connected classifier. The four-layer graph convolutional network of the graph self-distillation model is a graph convolution improved by K-order Chebyshev polynomials, and its convolution kernel is

[0067] In the formula, θ k are the coefficients of the Chebyshev polynomial, is the normalized Laplacian matrix, It is used To calculate the K-order Chebyshev polynomial.

[0068] It should be noted that the four-layer graph convolutional network in the trained graph self-distillation model is also loaded with the optimal adjacency matrix.

[0069] The spatial features of EEG signal data with multi-band differential entropy features are extracted through a four-layer graph convolutional network loaded with the optimal adjacency matrix; the probability of the listener paying attention to the left / right space is predicted through the fully connected classifier of the fourth layer.

[0070] Specifically, the EEG signal data with multi-band differential entropy features is first constructed into a graph G. G = {V, W}, V = {v1, v2, ..., v N′} is a set of vertices, where each vertex v i Corresponding to one lead, N′ is the number of leads in the EEG recording device. N′×N′ is the adjacency matrix of G, and the element w in W ij Non-negative, its magnitude represents v i and v j The strength of the functional connection between them.

[0071] Each vertex is associated with d differential entropy features, d = 5; that is, the feature matrix (i.e., two-dimensional matrix) x∈R that can constitute all vertices N′×d , each column of x is a signal defined on a vertex.

[0072] That is, the feature matrix x of all vertices is constructed into a graph G, where x contains the differential entropy features of d frequency bands.

[0073] Next, the spatial features of x in the graph G are extracted through a four-layer graph convolutional network loaded with the optimal adjacency matrix, and the balanced signal x4 is obtained.

[0074] Among them, the m-th layer of graph convolution is connected to a fully connected classifier, that is, the m-th layer of fully connected classifier.

[0075] x m-1 As the input of the m-th layer of graph convolution, first get x m , then x mRefine to get F m , and then pass through the m-th layer of fully connected classifier to get p m . Among them, m = 1 ~ 4, x0 = x, x1 ~ x4 are balanced signals.

[0076] F m is the output feature of the mth layer of graph convolution; p m is the output of the m-th layer fully connected classifier.

[0077] Among them, x m =x m-1 +GCN m (x,a m ), F m =avgpool(conv(x m ));

[0078] In the formula, GCN m represents the mth layer of graph convolution, a m represents the mth row of A, A is The matrix generated by the K-order Chebyshev polynomial; avgpool represents average pooling, and conv represents one-dimensional convolution.

[0079] Then, x4 is processed by the fully connected classifier in the fourth layer to obtain the output p4, which predicts the probability of the listener paying attention to the left / right side of the space.

[0080] Here, the purpose is to detect the listener's left or right spatial attention, thereby predicting the probability that the listener will pay attention to the left / right side of the space.

[0081] If the probability that the listener pays attention to the left side of the space is greater than the probability that the listener pays attention to the right side of the space, then the speaker that the listener is interested in is on the left side of the listener; if the probability that the listener pays attention to the left side of the space is less than the probability that the listener pays attention to the right side of the space, then the speaker that the listener is interested in is on the right side of the listener.

[0082] In addition, the graph self-distillation model needs to be trained first to obtain a trained graph self-distillation model.

[0083] See also Figure 5 ,Specifically, the training method of the graph self-distillation model includes:

[0084] Obtain sample EEG signals of a multi-speaker scene and process them to obtain a sample data set. The sample data set includes: sample EEG signal data with multi-band differential entropy features and sample true labels. The sample EEG signals are processed using steps 1 and 2 to obtain sample EEG signal data with multi-band differential entropy features. The sample true label is the real position of the speaker that the listener is interested in, which is the information built into the sample EEG signal.

[0085] A four-layer graph convolutional network is used to extract spatial features from sample EEG signal data with multi-band differential entropy features. As mentioned above, sample EEG signal data with multi-band differential entropy features should also be constructed into a graph G′ to provide the two-dimensional matrix x′ required for model training. Among them, the deepest graph convolution extracts global spatial features, and the first three layers of graph convolution extract local spatial features.

[0086] Specifically, referring to the above record, the m-th layer of graph convolution is connected to a fully connected classifier, that is, the m-th layer of fully connected classifier.

[0087] During the model training phase, x′ m-1 As the input of the mth layer of graph convolution, first get x′ m , and then for x′ m Refine to get F m ′, and then pass through the m-th layer of fully connected classifier to get p′ m . Wherein, m=1~4, x0′=x′, x1′~x3′ are signals containing local spatial feature information, and x4′ is a signal containing global spatial feature information.

[0088] F m ′ is the output feature of the mth layer of graph convolution in the model training stage; p′ m It is the output of the mth layer fully connected classifier in the model training phase.

[0089] Among them, x′ m = x′ m-1 +GCN m (x′,a m ), F m ′=avgpool(conv(x′ m ));

[0090] In the formula, GCN m represents the mth layer of graph convolution, a m represents the mth row of A, A is The matrix generated by the K-order Chebyshev polynomial; avgpool represents average pooling, and conv represents one-dimensional convolution.

[0091] The difference between global and local spatial features is balanced through graph self-distillation, that is, x1′~x4′ is balanced. This is the focus of this model training - it can improve the accuracy of listeners' auditory spatial attention detection.

[0092] Among them, methods to balance differences include:

[0093] 1. Calculate the cross entropy loss loss1 between the output of the deepest graph convolution after the fully connected classifier and the true label of the sample through cross entropy;

[0094] loss1=CrossEntropy(p′4,y);

[0095] Where p′4 is the output of the deepest graph convolution after the fully connected classifier (i.e., the output of the 4th layer fully connected classifier), and y is the true label of the sample.

[0096] 2. Calculate the feature loss loss2 of the output features of the first three layers of graph convolution and the deepest layer of graph convolution through the L2 loss function;

[0097]

[0098] In the formula, F′ g is the output feature of the graph convolution of the g-th layer, F′4 is the output feature of the deepest graph convolution, and L2 represents the L2 loss function.

[0099] That is, the L2 loss function is used to calculate the feature mapping between the deepest graph convolution and each shallow graph convolution, and the global spatial feature information in the deepest feature is introduced into the shallow feature to balance the global / local spatial feature information, so that the shallow graph convolution can better match the results of the deep graph convolution when predicting the spatial classification results.

[0100] 3. Calculate the output loss loss3 of the first three layers of fully connected classifiers and the deepest fully connected classifier through KL divergence;

[0101]

[0102] Where p′ g is the output of the g-th layer graph convolution after the fully connected classifier (i.e., the output of the g-th layer fully connected classifier), p′4 is the output of the deepest layer graph convolution after the fully connected classifier (i.e., the output of the 4th layer fully connected classifier), and KL represents the KL divergence function.

[0103] That is to say, the deepest graph convolution can be used as the teacher model, and the remaining shallow graph convolutions can be used as student models to form a teacher-student model; and then the KL divergence is used to calculate the loss loss3 in the teacher-student model. This loss can obtain the difference between the two output distributions, which can better guide the shallow network to learn spatial features.

[0104] 4. Linearly combine loss1, loss2, and loss3 according to the weights to obtain the final loss, and backpropagate to update the adjacency matrix W.

[0105] loss=αloss1+(1-α)loss2+βloss3;

[0106] Among them, α and β are hyperparameters for balancing loss1, loss2, and loss3.

[0107] Since back propagation (BP) is used to iteratively update network parameters during model training to achieve the optimal or suboptimal solution, it is necessary to calculate the partial derivative of loss with respect to W, that is,

[0108] Among them, W is the dynamically learned adjacency matrix, and the goal is to obtain the optimal adjacency matrix.

[0109] The i-th row and j-th column element of W is denoted as w ij .

[0110] According to the chain rule, the composition Each element of The calculation is expressed as:

[0111]

[0112] The partial derivative is obtained Then, update W using the following rules: Where ρ is the hyperparameter that needs to be learned.

[0113] Finally, the spatial features in the EEG signals are continuously learned through the graph self-distillation module 3, and the adjacency matrix is ​​continuously optimized to obtain a higher accuracy of auditory spatial attention detection.

[0114] The probability of the listener paying attention to the left / right space is predicted by the fully connected classifier in the fourth layer to obtain the detection result (the detection result is obtained based on the probability, the specific method is the same as above, and will not be repeated here), and compared with the real label of the sample to verify the accuracy of the model;

[0115] After multiple iterations, the graph self-distillation model with the highest accuracy is obtained, and its optimal adjacency matrix is ​​saved as the trained graph self-distillation model.

[0116] Example 2

[0117] This embodiment also discloses a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the auditory attention detection method based on electroencephalogram signals of embodiment 1 is executed.

[0118] When the method of Example 1 is applied, it can be applied in the form of software, such as a program designed as a computer-readable storage medium that can run independently. The computer-readable storage medium can be a USB flash drive designed as a USB shield, and the USB flash drive is designed to start the program of the entire method through an external trigger.

[0119] Example 3

[0120] This embodiment 3 is experimentally verified for the method of embodiment 1:

[0121] Two public datasets, KUL and DTU, were selected, both of which were dual-speaker scenarios. Experiments were conducted on KUL and DTU using the method in Example 1 and some mainstream models in the prior art to obtain detection results. The detection accuracy was calculated by comparing the results with the actual attention direction.

[0122] Table 1 and Table 2 show the results on the KUL and DTU datasets respectively.

[0123] Table 1 Auditory spatial attention detection accuracy achieved on the KUL dataset

[0124]

[0125] Table 2 Auditory spatial attention detection accuracy achieved on the DTU dataset

[0126]

[0127]

[0128] Models marked with * in the table are linear models.

[0129] It can be seen from Table 1 and Table 2 that no matter in which data set, the detection accuracy of the method in Example 1 can reach the best at the time scale of 1s and 2s. This result illustrates the effectiveness of the present invention in auditory attention detection.

[0130] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An auditory attention detection method based on EEG signals, used to detect the speaker of interest to the listener in a multi-speaker scene, characterized in that: The auditory attention detection method comprises the following steps: Step 1: Perform sliding window processing on the real-time EEG signals of the multi-speaker scene to obtain EEG signal data of N time periods; Step 2: Divide the EEG signal data of each time period into five frequency bands, and extract multi-band differential entropy features to obtain EEG signal data with multi-band differential entropy features; Step 3: Use the trained graph self-distillation model to process the EEG signal data with multi-band differential entropy features to obtain the detection results; The graph self-distillation model includes a four-layer graph convolutional network, and each layer of the graph convolutional network is also connected to a fully connected classifier; wherein the four-layer graph convolutional network in the trained graph self-distillation model is also loaded with an optimal adjacency matrix; In step 3, the spatial features of the EEG signal data with multi-band differential entropy features are extracted through a four-layer graph convolutional network loaded with the optimal adjacency matrix; the probability of the listener paying attention to the left / right space is predicted through the fully connected classifier of the fourth layer; If the probability that the listener pays attention to the left side of the space is greater than the probability that the listener pays attention to the right side of the space, then the speaker that the listener is interested in is on the left side of the listener; if the probability that the listener pays attention to the left side of the space is less than the probability that the listener pays attention to the right side of the space, then the speaker that the listener is interested in is on the right side of the listener.

2. The auditory attention detection method based on EEG signals according to claim 1 is characterized in that: In step one, Preprocess the real-time EEG signals in multi-speaker scenarios to improve signal quality; According to the frequency of the preprocessed real-time EEG signal, the preprocessed real-time EEG signal is cut into N time periods using a sliding window with an overlap of 50%, to obtain EEG signal data of N time periods.

3. The auditory attention detection method based on EEG signals according to claim 1, characterized in that: In step 2, The EEG signal data of the nth time period is taken as the nth time window; n∈[1,N]; For the nth time window, the power spectral density function PSD in the jth frequency band is extracted from the EEG signal of the ith lead. i,j (n,f), f represents frequency; PSD-based i,j (n,f) calculates differential entropy DE i,j (n); Then DE i,j (n) Take the average along the time window direction to obtain the multi-band differential entropy feature F of the i-th lead in the j-th frequency band i,j .

4. The auditory attention detection method based on EEG signals according to claim 3 is characterized in that: In the formula, K j is the number of intervals used to calculate differential entropy; p i,j,k (n) is the probability density function of the kth interval of the ith lead in the jth frequency band; f j,k-1 and f j,k are the lower and upper limits of the kth interval respectively; f l is the frequency when the time window is at l in the kth interval; L j It is used to calculate p i,j,k (n) is the window length; x is a two-dimensional matrix, the first dimension of which is the lead and the second dimension is the differential entropy features of the five frequency bands of each lead.

5. The auditory attention detection method based on EEG signals according to claim 4 is characterized in that: The four-layer graph convolutional network of the graph distillation model is a graph convolution improved by K-order Chebyshev polynomials.

6. The auditory attention detection method based on EEG signals according to claim 5, characterized in that: In step three: The EEG signal data with multi-band differential entropy features is constructed into a graph G; The spatial features of x in the graph G are extracted through a four-layer graph convolutional network loaded with the optimal adjacency matrix, and the balanced signal x4 is obtained; The output p4 is obtained by processing x4 through the fully connected classifier of the fourth layer, which predicts the probability that the listener pays attention to the left / right side of the space.

7. The auditory attention detection method based on EEG signals according to claim 1, characterized in that: The training method of the graph self-distillation model includes: Obtaining sample EEG signals of a multi-speaker scene and processing them to obtain a sample data set; wherein the sample data set includes: sample EEG signal data with multi-band differential entropy features and sample true labels; A four-layer graph convolutional network is used to extract spatial features from sample EEG signal data with multi-band differential entropy features. The deepest layer of graph convolution extracts global spatial features, and the first three layers of graph convolution extract local spatial features. Balance the difference between global and local spatial features through graph self-distillation; The probability of the listener paying attention to the left / right space is predicted through the fourth-layer fully connected classifier to obtain the detection result, and compared with the real label of the sample to verify the accuracy of the model; After multiple iterations, the graph self-distillation model with the highest accuracy is obtained, its optimal adjacency matrix is ​​saved, and it is used as the trained graph self-distillation model.

8. The auditory attention detection method based on EEG signals according to claim 7, characterized in that: Ways to balance the differences include: The cross entropy loss loss1 between the output of the deepest graph convolution after the fully connected classifier and the true label of the sample is calculated through cross entropy; The feature loss loss2 of the output features of the first three layers of graph convolution and the deepest layer of graph convolution is calculated through the L2 loss function; The output loss loss3 of the first three layers of fully connected classifiers and the deepest fully connected classifier is calculated through KL divergence; Loss1, loss2, and loss3 are linearly combined according to the weights to obtain the final loss, and back-propagated to update the adjacency matrix.

9. Auditory attention detection system based on EEG signal, characterized in that: It uses the auditory attention detection method based on EEG signals as described in any one of claims 1 to 8; The auditory attention detection system based on EEG signals comprises: An EEG signal data processing module is used to perform sliding window processing on real-time EEG signals in a multi-speaker scenario to obtain EEG signal data for N time periods; A multi-band differential entropy feature extraction module, which is used to divide the EEG signal data of each time period into five frequency bands, and extract the multi-band differential entropy features to obtain EEG signal data with multi-band differential entropy features; The image self-distillation module is used to use the trained image self-distillation model to process the EEG signal data with multi-band differential entropy features to obtain the detection results.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the auditory attention detection method based on EEG signals as described in any one of claims 1 to 8 is executed.

Images