An attention evaluation method based on electroencephalogram channel and frequency weight redistribution

Abstract

The application discloses a kind of attention evaluation method based on electroencephalogram channel and frequency weight redistribution.The application is by electrode channel of multi-channel electroencephalogram signal, and the feature weight of each channel signal on electroencephalogram frequency is redistributed;Then by constructing end-to-end deep neural network, the optimal feature weight distribution is automatically obtained, to construct fast and effective intelligent attention state evaluation.The application comprises the following steps: step 1: data preprocessing;Step 2: time-frequency diagram construction;Step 3: constructing weight redistribution network branch;Step 4: constructing end-to-end deep convolutional neural network;Step 5: deep neural network training and reasoning.The method proposed in the application can quickly, simply and accurately evaluate the attention state, and the channel weight redistribution method is used to redistribute the weight of the electrode channel and the frequency, to give greater weight to the meaningful channel / frequency, so as to amplify the effective channel / frequency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of biomedical signal processing, intelligent medical treatment and clinical auxiliary detection, and relates to a method for evaluating attention based on EEG channel and frequency weight redistribution. BACKGROUND

[0002] Attention plays an important role in the interaction between humans and the surrounding environment. Attention state is a behavior corresponding to the brain state, in which the participant allocates a certain amount of resources to perform a task. Accurate evaluation of attention is of great significance for many applications in reality. Attention Deficit Hyperactivity Disorder (ADHD) is a common mental health disorder characterized by inattention, which is mainly manifested in inattention, hyperactivity and impulsivity. Two-thirds of children with attention deficit hyperactivity disorder will continue to adulthood, which seriously affects their lives. Effective and rapid attention evaluation is an important prerequisite and guarantee for effective intervention and treatment of ADHD and follow-up evaluation. For example, in online education based on virtual reality, rapid attention evaluation can provide real-time feedback on the learning of the educated, and help the educator adjust his teaching methods. The attention state can be evaluated by different methods, including questionnaire survey, neurophysiological task and nervous system test. However, these methods have the problems of being greatly affected by the subjective thinking of the tested person, large deviation of the results, and too complicated test process. At present, there is a lack of fast and effective intelligent attention evaluation method. There are some related researches that use specific questionnaires and tasks to evaluate attention, use Go / noGo Test, Continuous Performance Test and The Wisconsin Card Sorting Test to evaluate attention state, and use electroencephalogram signals from biomedical recorders to evaluate attention. However, there are problems such as low precision or inconvenience of needing to train the user, which leads to unsatisfactory actual effect. As for the electroencephalogram signal, its biggest feature is multi-channel, and the correlation between channels is rarely studied. Therefore, the present method improves the accuracy and accuracy of existing attention evaluation by studying the channel relationship between electroencephalogram signals. SUMMARY

[0003] The purpose of the present application is to overcome the shortcomings of the prior art in attention state evaluation, and provide an attention evaluation method based on electroencephalogram channel and frequency weight redistribution. By redistributing the electrode channels of the multi-channel electroencephalogram signal and the characteristic weight of each channel signal on the electroencephalogram frequency, the meaningful channels and frequencies are highlighted, and the channels and frequencies with little significance are suppressed. By constructing an end-to-end deep neural network, the optimal feature weight distribution is automatically obtained, thereby constructing a fast and effective intelligent attention state evaluation method.

[0004] The technical scheme of the present application mainly includes the following steps:

[0005] Step 1: data preprocessing:

[0006] 1-1. Artifact removal. Use high-pass and low-pass filters to remove artifacts caused by frequencies less than 0.5HZ and greater than 50HZ, that is, only keep the electroencephalogram signals between 0.5HZ and 50HZ.

[0007] 1-2. Mean filtering. The electroencephalogram signal is subjected to mean filtering, that is, the signal of each electrode is subtracted from the mean value of all signals of the electrode.

[0008] 1-3. The electroencephalogram data record of each participant is retained according to the position of the stimulus, and the electroencephalogram data one second before the stimulus appears and three seconds after the stimulus appears is retained.

[0009] Step 2: time-frequency diagram construction.

[0010] 2-1. Frame and window the single electroencephalogram channel, set the frame length as L and the frame shift as L / 2. The window function uses a Hamming window.

[0011] 2-2. Assuming that the sampling frequency is Fs, the sampled signal is x n , N is the sample number, and the fast discrete Fourier transform (FFT) is performed on each frame signal after framing.

[0012]

[0013] Then remove the symmetric part and the zero frequency point amplitude.

[0014] 2-3. Take consecutive M frames of signals for splicing, the horizontal coordinate is the frame number, the vertical coordinate is the frequency, and the pixel is the corresponding amplitude, thereby constructing a (L / 2) x M gray time-frequency diagram.

[0015] 2-4. Steps 2-1, 2-2 and 2-3 are performed on the E electroencephalogram channels respectively, thereby constructing an E x (L / 2) x M multi-channel time-frequency diagram.

[0016] Step 3: constructing a weight redistribution network branch.

[0017] There are two main implementation methods: weight redistribution based on EEG channels and weight redistribution based on frequency. The specific process is as follows:

[0018] The channel-based weight redistribution steps are as follows:

[0019] First, adaptive average pooling (L / 2)×M is applied to each of the E channels to compress the features, resulting in an output tensor of shape E×1×1, as shown in the following formula:

[0020] U(i,j) is a grayscale image with shape (L / 2)×M.

[0021] Then, a multilayer perceptron is constructed, assuming it has K... M There are n hidden layers, let H be the number of neurons in the i-th hidden layer. i i = 1, ..., K M Then the output of the i-th layer is y (i) =w i x (i) , where the input x (i) The shape is 1×H i w i These are the weights of the i-th hidden layer, with shape H. i ×H i+1 The output y is obtained. (i) The shape is 1×H i+1 .

[0022] After passing through a multilayer perceptron, the final weight parameters 1×K after weight redistribution are obtained. M Adjusted to K M ×1×1, this weight parameter is multiplied by the feature of the original multi-channel time-frequency plot to obtain the weight redistribution feature I.

[0023] The frequency-weighted redistribution steps are as follows:

[0024] For an E×(L / 2)×M multi-channel time-frequency plot, global average pooling and global max pooling are performed simultaneously on the (L / 2) frequency dimension. Global average pooling calculates an average value for each E×M dimension tensor data, resulting in an output tensor of shape 1×(L / 2)×1. Global max pooling takes the maximum value for each E×M dimension tensor data, also resulting in an output tensor of shape 1×(L / 2)×1. The two tensors obtained from global average pooling and global max pooling are concatenated to obtain a tensor of shape 2×(L / 2)×1. Then, k 7*7 convolutional layers are used to adaptively adjust the weights of the (L / 2) frequency dimension, resulting in a weight parameter of shape 1×(L / 2)×1. This weight parameter is multiplied by the original E×(L / 2)×M multi-channel time-frequency plot features to obtain the weight redistribution feature II.

[0025] Step 4: Construct an end-to-end deep convolutional neural network.

[0026] The weight redistribution features obtained in step 3 are used as input to the deep convolutional neural network.

[0027] The input is either weight redistribution feature I, weight redistribution feature II, or both weight redistribution feature I and weight redistribution feature II.

[0028] When the input to a deep convolutional neural network is either weight redistribution feature I or weight redistribution feature II, the internal structure of the deep convolutional neural network consists of a weight redistribution network module and multiple blocks. Each block consists of the following modules: convolution, batch normalization (BN), ReLU, convolution, ReLU, and max pooling. After passing through several blocks, the final classification result is obtained through fully connected (FC).

[0029] When the input to a deep convolutional neural network is weight redistribution feature I and weight redistribution feature II, the final weight parameter K is obtained through weight redistribution feature I. M The final weight parameters 1×(L / 2)×1 are obtained from the weight redistribution model II and ×1×1. Multiplying both weight parameters by the original multi-channel time-frequency plot features of E×(L / 2)×M simultaneously yields the weight redistribution features that utilize both methods. The feature shape obtained in this way is the same as that obtained from weight redistribution feature I and weight redistribution feature II, which is E×(L / 2)×M, and is used as the input to the deep convolutional neural network.

[0030] Furthermore, the structure of each module in a deep convolutional neural network is as follows:

[0031] 1) Convolutional module. Input size is D×H i ×W i If the convolution kernel is F×P, the stride is s1×s2, and the padding is padding, then the formula for the output size D×H×W is:

[0032]

[0033]

[0034] 2) Max Pooling Module. In addition to convolutional modules, convolutional networks also frequently use pooling layers to reduce model size, improve computation speed, and enhance the robustness of extracted features.

[0035] 3) ReLU module. This is a commonly used activation function in artificial neural networks, with the following formula:

[0036]

[0037] Where x is the input of the function.

[0038] 4) Batch Normalization Module (BN). For an input x with d dimensions, we will normalize each dimension as follows:

[0039]

[0040] Where k is 1, 2, ..., d.

[0041] 5) Fully Connected Module (FC). This module acts as a "classifier." The input is H×D, and the output is H×P obtained through the fully connected layer function y=wx, where w is D×P. The softmax function maps the result to the real space of the probability distribution represented by [0,1].

[0042] Step 5: Deep Neural Network Training and Inference.

[0043] The model uses the cross-entropy function as the loss function, and the formula is:

[0044]

[0045] in The actual output value, y (i) The label value is used. Stochastic gradient descent is employed for optimization, adjusting various parameters in the network to continuously reduce the loss value.

[0046] For each new data sample, by going through steps 1, 2, 3, and 4, we obtain the probability of it being distributed in [0,1], and use this to determine its attention state.

[0047] The benefits of this invention are as follows:

[0048] The method proposed in this invention can quickly, simply, and accurately assess attentional states. It employs a channel weight redistribution method, reallocating weights to electrode channels and frequencies, assigning greater weight to meaningful channels / frequencies to amplify effective channels / frequencies. By constructing an end-to-end deep neural network structure, the deep network acquires features with stronger representational capabilities during training, thus enabling rapid and accurate attentional state assessment. This method has broad application prospects, such as in virtual reality-based online education, where rapid attention assessment can provide real-time feedback on learners' learning progress and help instructors adjust their teaching methods. It can also be used in the auxiliary diagnosis of ADHD; effective and rapid attention assessment is a crucial prerequisite and guarantee for effective intervention and follow-up evaluation of ADHD, providing doctors with objective and scientific diagnostic evidence. Attached Figure Description

[0049] Figure 1 Flowchart of the present invention;

[0050] Figure 2 A practical application diagram of the present invention. Detailed Implementation

[0051] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0052] The first major step of this invention, data preprocessing, is implemented as follows:

[0053] 1-1. A dataset contains EEG data from multiple volunteers. High-pass and low-pass filters are used to remove artifacts caused by frequencies less than 0.5 Hz and greater than 50 Hz, respectively.

[0054] 1-2. Perform mean filtering, that is, subtract the mean of all signals of that electrode from the signal of each electrode.

[0055] 1-3. Record the EEG data of each participant, retaining the EEG data one second before and three seconds after the stimulus occurs, based on the location of the stimulus.

[0056] The second main step of this invention is the construction of a time-frequency graph, the specific implementation steps of which are as follows:

[0057] 2-1. Frame-by-frame windowing is applied to a single EEG channel, with a frame length of 100 and a frame shift of 50. A Hamming window is used as the window function.

[0058] 2-2. Assume the sampling frequency is Fs and the sampled signal is x. n Where N is the number of samples, a Discrete Fourier Transform (FFT) is performed on each frame of the signal after framing. The formula for the Discrete Fourier Transform is:

[0059]

[0060] Then remove the symmetrical part and the amplitude at the zero frequency point.

[0061] 2-3. Take 10 consecutive frames of signal and stitch them together. The horizontal axis represents the frame number, the vertical axis represents the frequency, and the pixel represents the corresponding amplitude, thus constructing a 50×10 grayscale time-frequency diagram.

[0062] 2-4. Perform steps 2-1 to 2-3 on each of the 31 EEG channels to construct a 31×50×10 multi-channel time-frequency map.

[0063] The third main step of this invention is to construct a weight redistribution network branch, the specific implementation steps of which are as follows:

[0064] 1) The channel-based weight redistribution steps are as follows: For channel E, weight redistribution is performed. First, adaptive average pooling of 50×10 is used to compress the features of each of the 31 channels, resulting in an output tensor of shape 31×1×1, as shown in the following formula:

[0065]

[0066] Where U(i,j) is a 50×10 grayscale image. Then there is a two-layer perceptron, the first layer being y... (1) =w1x (1) , where the input x (1) The shape of w1 is 1×31, and the shape of w1 is 31×2, resulting in the output y. (1) The shape is 1×2. The second layer is y. (2) =w2x (2) Where w2 has a shape of 2×31, the output y is obtained. (2) The shape is 1×31, which is adjusted to 31×1×1. This weight parameter is multiplied by the original feature to obtain the feature after weight redistribution.

[0067] 2) The frequency-based weight redistribution steps are as follows: For a 31×50×10 multi-channel time-frequency map, global average pooling and global max pooling are performed on the 50-frequency dimension. Global average pooling calculates an average value for each 31×10-dimensional tensor data, obtaining an output tensor of shape 1×50×1. Global max pooling takes the maximum value from the 31×10-dimensional tensor data, obtaining an output tensor of shape 1×50×1. The two tensors obtained from global average pooling and global max pooling are concatenated to obtain a tensor of shape 2×50×1. Then, a 7*7 convolutional layer adaptively adjusts the weights for the 50-frequency dimension, obtaining a 1×50×10 weight parameter. This weight is multiplied by the original 31×50×10 time-frequency map features to obtain the weight redistribution feature.

[0068] The fourth main step of this invention is to construct an end-to-end deep convolutional neural network:

[0069] The weight redistribution features obtained in step 3 are used as input to a deep convolutional neural network. The internal structure of this deep convolutional neural network consists of a weight redistribution network module and three blocks. Each block comprises the following modules: convolution, batch normalization (BN), ReLU, convolution, ReLU, and max pooling. After passing through these three blocks, the final classification result is obtained via fully connected (FC).

[0070] The fifth main step of this invention is deep neural network training and inference.

[0071] The method used to evaluate model accuracy is leave-one-out cross-validation. In this method, we consider all 32 participants except for one participant to train the model (i.e., a length of 31), and use the remaining participant (1 participant) for validation. This step is repeated for each participant, and the mean and standard deviation are calculated. Adam is used as the optimizer, and the learning rate is set to 1e-3. The cross-entropy function is used as the loss function, with the following formula:

[0072]

[0073] in The actual output value, y (i) This is the tag value.

[0074] Stochastic gradient descent is used for optimization, yielding one output in each training iteration. The softmax function maps the result to the real-valued space of a probability distribution represented by [0,1]. Each training iteration captures 64 data samples, resulting in one output. The loss value is calculated using a loss function, and gradient correction is applied to adjust the network parameters to continuously reduce the loss. Typically, 100 epochs are sufficient for a good fit. For each new data sample, steps 1, 2, 3, and 4 are followed to obtain its probability within the [0,1] distribution, thus determining its attention state.

[0075] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the concept of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An attention assessment method based on EEG channel and frequency weight redistribution, characterized in that, Includes the following steps: Step 1: Data Preprocessing Step 2: Time-frequency graph construction; Step 3: Construct the weight redistribution network branch; Step 4: Construct an end-to-end deep convolutional neural network; Step 5: Deep Neural Network Training and Inference; The time-frequency diagram construction described in step 2 is implemented as follows: 2-1. Frame-by-frame windowing is applied to a single EEG channel, with a frame length of L and a frame shift of L / 2; a Hamming window is used as the window function. 2-2. Assume the sampling frequency is Fs and the sampled signal is x. n Where N is the number of samples, a Fast Discrete Fourier Transform (FFT) is performed on each frame of the signal after framing. The FFT formula is as follows: Then remove the symmetrical portion and the amplitude at the zero frequency point; 2-3. Take M consecutive frames of signal and stitch them together. The horizontal axis represents the frame number, the vertical axis represents the frequency, and the pixel represents the corresponding amplitude, so as to construct a (L / 2)×M grayscale time-frequency diagram. 2-4. Perform steps 2-1, 2-2, and 2-3 on each of the E EEG channels to construct a multi-channel time-frequency diagram of E×(L / 2)×M; The construction of the weight redistribution network branch described in step 3 includes two implementation methods: weight redistribution based on EEG channels and weight redistribution based on frequency. The specific implementation of the weight redistribution based on EEG channels is as follows: First, adaptive average pooling (L / 2)×M is applied to each of the E channels to compress the features, resulting in an output tensor of shape E×1×1, as shown in the following formula: U(i,j) is a grayscale image with shape (L / 2)×M; Then, a multilayer perceptron is constructed, assuming it has K... M There are n hidden layers, let H be the number of neurons in the i-th hidden layer. i i = 1, ..., K M Then the output of the i-th layer is y (i) =w i x (i) , where the input x (i) The shape is 1×H i w i These are the weights of the i-th hidden layer, with shape H. i ×H i+1 The output y is obtained. (i) The shape is 1×H i+1 ; After passing through a multilayer perceptron, the final weight parameters 1×K after weight redistribution are obtained. M Adjusted to K M ×1×1, this weight parameter is multiplied by the feature of the original multi-channel time-frequency plot to obtain the weight redistribution feature I; The specific implementation of the frequency-weighted reallocation is as follows: For a multi-channel time-frequency plot of E×(L / 2)×M, global average pooling and global max pooling are performed simultaneously on the (L / 2) frequency dimension. Global average pooling calculates an average value for each E×M dimension tensor data to obtain an output tensor of shape 1×(L / 2)×1. Global max pooling takes the maximum value for each E×M dimension tensor data to obtain an output tensor of shape 1×(L / 2)×1. The two tensors obtained from global average pooling and global max pooling are concatenated to obtain a tensor of shape 2×(L / 2)×1. Then, the frequency dimension (L / 2) is adaptively adjusted through k 7*7 convolutional layers to obtain a weight parameter of 1×(L / 2)×1. This weight parameter is multiplied with the original multi-channel time-frequency plot feature of E×(L / 2)×M to obtain the weight redistribution feature II. In step 4, the weight redistribution features obtained in step 3 are used as the input to the deep convolutional neural network. This input uses both weight redistribution feature I and weight redistribution feature II. When the input to a deep convolutional neural network is weight redistribution feature I and weight redistribution feature II, the final weight parameter K is obtained through weight redistribution feature I. M The final weight parameter 1×(L / 2)×1 is obtained by combining ×1×1 and weight redistribution model II. Multiplying the two weight parameters by the original multi-channel time-frequency plot feature E×(L / 2)×M simultaneously yields the weight redistribution feature using both methods. The feature shape obtained by this method is the same as the feature shape obtained by weight redistribution feature I and weight redistribution feature II, which is E×(L / 2)×M, and is used as the input of the deep convolutional neural network.

Images