An electroencephalogram analysis method based on modeling of left and right brain differences of autistic children

Abstract

The application discloses an electroencephalogram analysis method based on autism children left and right brain difference modeling, and belongs to the field of biomedical information, which comprises the following steps: collecting resting-state electroencephalogram data of normal children and autism children by using an electroencephalogram data acquisition module, and preprocessing the data into a multivariate time series matrix required by a model input; capturing features of different scales in the time dimension of the electroencephalogram signal by using a time sequence multi-scale feature learning module, and fusing the multi-scale time features by using a convolution layer; performing convolution operation on the spatial dimension of the left and right hemispheres by using a spatial feature learning and fusion module, extracting more abundant inter-channel features, and maximizing the feature difference between the left and right hemispheres by using an attention mechanism, and finally performing data fusion through weighted average; and completing a training process by optimizing the model through a classification loss in a classification module, and obtaining the autism probability of an analyzed person. The application can extract autism electroencephalogram space-time features, and improves the performance and precision of auxiliary analysis and judgment.

Description

Technical Field

[0001] This invention relates to the field of biomedical information technology, and in particular to an electroencephalogram (EEG) analysis method based on modeling the differences between the left and right hemispheres in children with autism. Background Technology

[0002] Autism is a pervasive developmental disorder characterized by impaired social interaction, language development, and restricted interests / repetitive behaviors. The cause remains unknown. Autism typically begins before the age of three. Clinical studies indicate that earlier intervention, education, and clinical treatment can significantly improve cognitive and communication functions, with some mild cases achieving treatment outcomes comparable to those of typically developing autism through early intervention. However, due to the young age of children with autism, the widely used techniques in research, such as functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and eye-tracking, have certain limitations.

[0003] Electroencephalography (EEG), as a non-invasive acquisition method, has high temporal resolution and is a powerful tool for studying complex neurological disorders. Resting-state EEG, which measures brain activity without external stimuli or tasks, has certain advantages for younger autistic children with low cognitive levels and task cooperation, and is therefore easier to apply in the field of clinical auxiliary analysis and judgment.

[0004] Numerous studies on the classification of EEG signals have incorporated current popular computer technology—convolutional neural networks. Most existing techniques first provide EEG quantitative features such as power entropy and sample entropy, and then use a simple classifier for classification. However, EEG signals are highly complex, and the above techniques rely on the premise that EEG signals have random stationarity, which is not actually the case.

[0005] Therefore, it is necessary to develop an EEG analysis method based on modeling the differences between the left and right hemispheres of children with autism to solve this problem. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide an EEG analysis method based on modeling the differences between the left and right hemispheres of children with autism. The method uses a convolutional neural network to perform end-to-end feature extraction on the raw EEG signals, which can learn feature patterns that are difficult to explain with current quantitative techniques, thereby improving classification accuracy and increasing work efficiency in practical applications.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] An electroencephalogram (EEG) analysis method based on modeling the differences between the left and right hemispheres in children with autism includes:

[0009] The EEG data acquisition module was used to collect resting-state EEG data from normal children and children with autism, and the data was preprocessed into a multivariate time series matrix required for model input.

[0010] A temporal multi-scale feature learning module is used to capture features of EEG signals at different scales in the time dimension, and a convolutional layer is used to fuse the multi-scale temporal features;

[0011] The spatial feature learning and fusion module performs convolution operations on the left and right hemispheres in the spatial dimension to extract richer inter-channel features. An attention mechanism is used to maximize the feature differences between the left and right hemispheres. Finally, the data is fused by weighted averaging.

[0012] The classification module is used to optimize the model through classification loss to complete the training process and obtain the probability of the analyzed person having autism.

[0013] A further improvement of the technical solution of the present invention is that: the method of acquiring resting-state EEG data of normal children and children with autism by using an EEG data acquisition module is to use a 128-channel EEG system for acquisition, select 46 electrodes covering four functional brain regions, and control the impedance of all electrodes to below 50kΩ with Cz as the reference.

[0014] A further improvement to the technical solution of this invention lies in the following: the preprocessing specifically includes: slicing the collected raw resting-state EEG data into segments at a sampling frequency fs, treating each segment as a sample, and then dividing each sample data into left and right hemisphere data, with the number of channels for the left and right hemispheres being n1 and n2, respectively. The model's inputs are left hemisphere EEG data. and right hemisphere EEG data

[0015] A further improvement of the technical solution of the present invention is that: the temporal multi-scale feature learning module includes a multi-scale temporal feature extraction sub-module and a multi-scale temporal feature fusion sub-module;

[0016] The process involves capturing features at different scales along the time dimension of EEG signals and fusing these multi-scale temporal features using convolutional layers; specifically, it includes the following steps:

[0017] Step 21: Input the preprocessed raw EEG data matrix of the left and right hemispheres into the multi-scale temporal feature extraction submodule;

[0018] Step 22, the multi-scale temporal feature extraction submodule includes convolutional layers and pooling layers, and obtains temporal feature matrices at three different scales through the ReLU activation function, where the specific scales are 1 / 2, 1 / 4 and 1 / 8 of the time dimension of two-dimensional convolutions;

[0019] Step 23: Each multi-scale temporal feature extraction submodule is followed by a pooling layer after a convolutional layer. The input data is compressed using mean pooling in the temporal dimension to half its original size, thus obtaining the temporal features of the EEG data at different scales in the left and right hemispheres. and

[0020]

[0021]

[0022] Where AP(·) is the average pooling with a pooling window of (1,2), and Conv2D(·,s) is the average pooling with a pooling window of (1,2). i ) represents the convolution kernel with a scale of s i The two-dimensional convolution, i = {1, 2, 3}, represents three convolution kernels of different scales, respectively. and ReLU(·) is the ReLU activation function;

[0023] Step 24: Use the multi-scale temporal feature fusion submodule to combine the spliced ​​left and right hemisphere multi-scale temporal features M L and M R Convolutional operations are used to fuse temporal features across three scales, and mean pooling layers are used to compress them to half their original size. These compressed features are then used as the outputs of the temporal multi-scale feature learning module, TemL and TemR.

[0024]

[0025]

[0026]

[0027]

[0028] Where BN{·} represents the batch normalization operation, and [·,] represents the concatenation of the channel dimension. For a Dropout layer with a randomness rate of 0.5, S t The kernel scale used for the convolutional layers in the multi-scale temporal feature fusion module.

[0029] A further improvement of the technical solution of the present invention is that the spatial feature learning and fusion module includes a spatial feature extraction submodule and an attention spatial feature fusion submodule;

[0030] The process involves performing convolution operations on the left and right hemispheres in the spatial dimension to extract richer inter-channel features, maximizing the feature differences between the left and right hemispheres using an attention mechanism, and finally fusing the data through a weighted average. Specifically, this includes the following steps:

[0031] Step 31: The spatial feature extraction submodule extracts the temporal feature matrices of the left and right hemispheres' EEG in the spatial dimension using a method including a two-dimensional convolutional layer, a ReLU activation function, and an average pooling layer, to obtain the spatiotemporal feature matrices SpaL and SpaR of the left and right hemispheres.

[0032]

[0033]

[0034] The convolutional kernel scale used in the convolutional layers of the spatial feature extraction submodule is as follows:

[0035] Step 32: Concatenate the features in the spatial dimension and use batch normalization to standardize the spatiotemporal feature matrix in the batch dimension to obtain the input features of the attention spatial feature fusion submodule:

[0036] Features=BN{[SpaL,SpaR]}=[L,R] (9)

[0037] Step 33: The attention spatial feature fusion submodule uses the attention mechanism Att(·) to obtain the feature weights β1 and β2 of the left and right brains, multiplies them with the spatiotemporal feature matrices of the left and right brains, and reweights the spatiotemporal feature matrices in space using attention to obtain the spatiotemporal feature matrix enhanced with the difference between the left and right brains.

[0038] Step 34: Use a global average pooling layer to fuse the spatiotemporal feature matrices of the left and right hemispheres, and use this as the output of the attention space feature fusion submodule, Fusion.

[0039]

[0040] Where GAP(·) is global average pooling, k″ is the number of convolutional kernels in the previous convolutional layer, and Q is the compression ratio in the attention mechanism.

[0041] A further improvement to the technical solution of this invention lies in the following: The step of completing the training process through classification loss optimization to obtain the probability of autism in the analyzed individual specifically involves: inputting the spatiotemporal fusion feature matrix into a classification layer containing two fully connected layers, a ReLU activation function, and a Softmax activation function, and then using the cross-entropy loss function to optimize the training to obtain the final analysis result. The specific formula for the cross-entropy loss function is as follows:

[0042]

[0043] Among them, y i The true label for subject i is 1 for positive and 0 for negative. i This represents the probability that subject i predicts the positive class.

[0044] The technological advancements achieved by this invention due to the adoption of the above technical solutions are as follows:

[0045] This invention leverages the abnormal distribution of frequency bands and the asymmetry between the left and right hemispheres in children with autism to design a network capable of efficiently extracting spatiotemporal learning features from raw EEG data of children with autism. By proposing a left-right hemisphere spatiotemporal feature fusion strategy, it uses multi-scale extraction of multi-frequency band features and employs an attention mechanism to maximize the differences between the left and right hemispheres—that is, reweighting the left and right hemispheres using an attention mechanism. This effectively extracts and fuses spatiotemporal features while addressing the issue of large individual differences to some extent, increasing the model's robustness. It can extract more efficient features, improving the efficiency of autism analysis and judgment, enhancing the performance and accuracy of autism analysis and judgment, and providing a new approach for autism-assisted analysis and judgment. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of the method in an embodiment of the present invention;

[0048] Figure 2 This is a method framework diagram in an embodiment of the present invention;

[0049] Figure 3 This is a diagram of the multi-scale temporal feature extraction framework in an embodiment of the present invention;

[0050] Figure 4 This is a framework diagram of the attention mechanism in an embodiment of the present invention;

[0051] Figure 5 This is the confusion matrix result of the method in the embodiments of the present invention and other control methods. Detailed Implementation

[0052] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0053] This invention is inspired by the spatial differences in EEG signals between the left and right hemispheres in children with autism spectrum disorder. It comprises an EEG data acquisition module, a temporal multi-scale feature learning module (including a multi-scale temporal feature extraction submodule and a multi-scale temporal feature fusion submodule), a spatial feature learning and fusion module (including a spatial feature extraction submodule and an attention-based spatial feature fusion submodule), and a classification module. The multi-scale temporal feature extraction submodule extracts temporal features from different frequency bands of each EEG electrode channel. The spatial feature extraction submodule learns spatial features from two local regions of the left and right hemispheres, and the attention-based fusion module achieves the fusion of spatiotemporal features between the left and right hemispheres. The raw EEG data from both hemispheres are processed into matrices as input, and feature matrices of different resolutions are obtained through temporal feature learning at three different scales. The temporal feature matrices are then used by the temporal feature extraction module to learn the spatial feature matrices of the left and right hemispheres respectively. The spatial features of the left and right hemispheres are weighted and fused using an attention mechanism. Finally, the training process is completed using a classification loss optimization model. The results are then used to test and analyze suspected autism spectrum disorder children to obtain the probability of the analyzed individuals having autism.

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments:

[0055] like Figure 1-2 As shown, an EEG analysis method based on modeling the differences between the left and right hemispheres in children with autism, in a channel-first data format, includes the following steps:

[0056] Step 1: Use the EEG data acquisition module to collect resting-state EEG data from normal children and children with autism, and preprocess it into a multivariate time series matrix required for model input;

[0057] Specifically, the following steps are included:

[0058] Step 11, Resting-state EEG data acquisition:

[0059] The EEG data acquisition module used a 128-channel EEG system, selecting 46 electrodes that could cover four functional brain regions extensively. All electrodes were referenced at Cz, with impedance controlled below 50kΩ. Data acquisition time was ensured to be between 5 and 10 minutes. During the acquisition, the children sat quietly in comfortable chairs with their eyes open, typically accompanied by caregivers in a quiet room. Resting-state EEG data were acquired from 45 children aged 3-6 years with autism and 45 children aged 3-6 years with generalized development.

[0060] Step 12, Resting-state EEG data preprocessing:

[0061] The collected raw data was sliced ​​at a sampling frequency fs. Each subject selected 30 seconds of data, resulting in 30 slices. Each slice was considered a sample, and each sample was further divided into left and right hemisphere data, with n1 channels respectively. The model's inputs are left hemisphere EEG data. and right hemisphere EEG data In this experiment, 23 electrode data points were selected from both the left and right hemispheres, i.e., n1 = n2 = 23, to facilitate subsequent multi-scale feature learning in the temporal and channel dimensions, serving as the model input X = [X]. L ,X R ], and

[0062] Step 2: The temporal multi-scale feature learning module is used to capture features at different scales in the time dimension of the EEG signal, and convolutional layers are used to fuse the multi-scale temporal features.

[0063] The temporal multi-scale feature learning module includes a multi-scale temporal feature extraction submodule and a multi-scale temporal feature fusion submodule; specifically, it includes the following steps:

[0064] Step 21: Input the preprocessed raw EEG data matrix of the left and right hemispheres into the multi-scale temporal feature extraction submodule;

[0065] Step 22, as follows Figure 3 As shown, the multi-scale temporal feature extraction submodule includes convolutional layers and pooling layers, and obtains temporal feature matrices at different scales through the ReLU activation function, where the specific scales are 1 / 2, 1 / 4 and 1 / 8 of the time dimension of two-dimensional convolutions.

[0066] Step 23: Each multi-scale temporal feature extraction submodule is followed by a pooling layer after a convolutional layer. The input data is compressed using mean pooling in the temporal dimension to half its original size, thus obtaining the temporal features of the EEG data at different scales in the left and right hemispheres. and

[0067]

[0068]

[0069] Where AP(·) is the average pooling with a pooling window of (1,2), and Conv2D(·,s) is the average pooling with a pooling window of (1,2). i ) represents the convolution kernel with a scale of s i The two-dimensional convolution, i = {1, 2, 3}, represents three convolution kernels of different scales, respectively. and ReLU(·) is the ReLU activation function;

[0070] Step 24: Use the multi-scale temporal feature fusion submodule to combine the spliced ​​left and right hemisphere multi-scale temporal features M L and M R Convolutional operations are used to fuse temporal features across three scales, and mean pooling layers are used to compress them to half their original size. These compressed features are then used as the outputs of the temporal multi-scale feature learning module, TemL and TemR.

[0071]

[0072]

[0073]

[0074]

[0075] Where BN{·} represents the batch normalization operation, and [·,] represents the concatenation of the channel dimension. For a Dropout layer with a randomness rate of 0.5, S t The kernel scale used for the convolutional layers in the multi-scale temporal feature fusion module.

[0076] Step 3: The spatial feature learning and fusion module is used to perform convolution operations on the left and right hemispheres in the spatial dimension to extract richer inter-channel features. The attention mechanism is used to maximize the feature differences between the left and right hemispheres. Finally, the data is fused by weighted averaging.

[0077] The spatial feature learning and fusion module includes a spatial feature extraction submodule and an attention-based spatial feature fusion submodule; specifically, it includes the following steps:

[0078] Step 31: Design a spatial feature extraction submodule to extract the temporal feature matrices of the left and right hemispheres' EEG in the spatial dimension using a method including a two-dimensional convolutional layer, a ReLU activation function, and an average pooling layer, to obtain the spatiotemporal feature matrices SpaL and SpaR of the left and right hemispheres:

[0079]

[0080]

[0081] Step 32: Concatenate the features in the spatial dimension and use batch normalization to standardize the spatiotemporal feature matrix in the batch dimension to obtain the input features of the attention spatial feature fusion submodule:

[0082] Features=BN{[SpaL,SpaR]}=[L,R] (9)

[0083] Among them, the convolutional kernel scale S used in the convolutional layers of the spatial feature extraction submodule is... s = (23,1);

[0084] Step 33, as follows Figure 4 As shown, the attention spatial feature fusion submodule uses the attention mechanism to obtain the feature weights of the left and right hemispheres, multiplies them with the spatiotemporal feature matrices of the left and right hemispheres, and reweights the spatiotemporal feature matrices in space using attention to obtain the spatiotemporal feature matrix enhanced with the differences between the left and right hemispheres.

[0085] Step 34: Use a global average pooling layer to fuse the spatiotemporal feature matrices of the left and right hemispheres, and use this as the output of the attention space feature fusion submodule, Fusion.

[0086]

[0087] Where GAP(·) is global average pooling, k″ is the number of convolutional kernels in the previous convolutional layer, and Q is the compression ratio in the attention mechanism.

[0088] Step 4: The classification module is used to optimize the model through classification loss to complete the training process and obtain the probability of the analyzed individual having autism; the specific method is as follows:

[0089] The spatiotemporal fusion feature matrix is ​​input into a classification layer containing two fully connected layers, a ReLU activation function, and a Softmax activation function. The training is then optimized using the cross-entropy loss function to obtain the final analysis result. The specific formula for the cross-entropy loss function is as follows:

[0090]

[0091] Among them, y iThe true label for subject i is 1 for positive and 0 for negative. i This represents the probability that subject i predicts the positive class.

[0092] This embodiment uses the following comparison method:

[0093] TSception: A deep learning framework for emotion recognition based on electroencephalography (EEG). The spatial feature learning part utilizes the asymmetry of emotional responses in different brain regions to learn and recognize features of the left and right hemispheres of the brain.

[0094] EEGNet: A lightweight convolutional neural network for EEG analysis that uses depthwise convolutions and separable convolutions to build EEG feature extraction models. It is also a multi-paradigm framework and is currently popular in the field of electroencephalography (EEG).

[0095] ShallowConvNet: A fairly general-purpose architecture specifically designed for decoding band power characteristics based on DeepConvNet.

[0096] During the experiment, the sample set from 90 participants was divided into five equal parts independently by each participant. Four parts were used as the training set for model training, and the remaining part was used as the test set to evaluate model performance. The five data sets were used sequentially as the test set for five experiments. The accuracy rates of the proposed method and the control method are shown in Table 1, and the confusion matrix results are as follows: Figure 5 As shown, all results are the average of the results from five experiments.

[0097] Table 1 Comparison of test results of the method of the present invention with those of the control method.

[0098]

[0099]

[0100] As shown in Table 1 above, the performance of the method of the present invention is significantly better than that of other baseline methods. This indicates that the present invention can effectively utilize the left and right brain differences in EEG data of autistic children and can better analyze and predict the results of the tests.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An electroencephalogram (EEG) analysis method based on modeling the differences between the left and right hemispheres in children with autism, characterized in that: include: Step 1: Collect resting-state EEG data from normal children and children with autism using the EEG data acquisition module, and preprocess the data into a multivariate time series matrix required for model input; The preprocessing specifically includes: processing the collected raw resting-state EEG data at a sampling frequency... The data is processed into slices, with each slice representing a sample. Each sample is then divided into left and right hemisphere data, with the number of channels in each hemisphere being [number missing]. The model's inputs are left hemisphere EEG data. and right hemisphere EEG data ; Step 2: The temporal multi-scale feature learning module is used to capture features at different scales in the time dimension of the EEG signal, and convolutional layers are used to fuse the multi-scale temporal features; the temporal multi-scale feature learning module includes a multi-scale temporal feature extraction sub-module and a multi-scale temporal feature fusion sub-module; Step 2 specifically includes the following steps: Step 21: Input the preprocessed raw EEG data matrix of the left and right hemispheres into the multi-scale temporal feature extraction submodule; Step 22, the multi-scale temporal feature extraction submodule includes convolutional layers and pooling layers, and is implemented through... The activation function yields temporal feature matrices at three different scales, specifically two-dimensional convolutions at scales of 1 / 2, 1 / 4, and 1 / 8 of the time dimension. Step 23: Each multi-scale temporal feature extraction submodule is followed by a pooling layer after a convolutional layer. The input data is compressed using mean pooling in the temporal dimension to half its original size, thus obtaining the temporal features of the EEG data at different scales in the left and right hemispheres. and : （1） （2） in, It is average pooling with a pooling window of (1,2). The kernel size is Two-dimensional convolution, , representing three convolution kernels of different scales, respectively , and , for Activation function; Step 24: Use the multi-scale temporal feature fusion submodule to combine the spliced ​​left and right hemisphere multi-scale temporal features. and Convolutional operations are used to fuse temporal features at three scales along the time dimension, and average pooling layers are used to compress them to half their original size, which is then used as the output of the temporal multi-scale feature learning module. and （3） （4） （5） （6） in, For batch normalization operations, [ [This refers to the concatenation of channel dimensions.] For a Dropout layer with a randomness rate of 0.5, The convolution kernel scale used by the convolutional layers in the multi-scale temporal feature fusion module; Step 3: The spatial feature learning and fusion module performs convolution operations on the left and right hemispheres in the spatial dimension to extract richer inter-channel features, and uses an attention mechanism to maximize the feature differences between the left and right hemispheres. Finally, the data is fused by weighted averaging. The spatial feature learning and fusion module includes a spatial feature extraction submodule and an attention spatial feature fusion submodule. Step 3 specifically includes the following steps: Step 31: The spatial feature extraction submodule extracts the temporal feature matrices of the left and right brain EEG in the spatial dimension, using a method including a two-dimensional convolutional layer and a... Feature extraction is performed using an activation function and an average pooling layer to obtain the spatiotemporal feature matrices of the left and right hemispheres. and （7） （8） The convolutional kernel scale used in the convolutional layers of the spatial feature extraction submodule is as follows: ; Step 32 involves cascading the features in the spatial dimension and using batch normalization to standardize the spatiotemporal feature matrix in the batch dimension, thus obtaining the input to the attention spatial feature fusion submodule. : （9） Step 33: The attention space feature fusion submodule uses an attention mechanism. Obtain the feature weights of the left and right hemispheres and Multiplying the spatiotemporal feature matrices of the left and right hemispheres by applying attentional reweighting to the spatiotemporal feature matrices in space yields a spatiotemporal feature matrix enhanced with left-right hemisphere differences. Step 34: Use a global average pooling layer to fuse the spatiotemporal feature matrices of the left and right hemispheres, and use this as the output of the attention space feature fusion submodule. : （10） in, It is global average pooling. This represents the number of convolutional kernels in the previous convolutional layer. This refers to the compression ratio in the attention mechanism; Step 4: Use the classification module to optimize the model through classification loss to complete the training process and obtain the probability of the analyzed person having autism.

2. The EEG analysis method based on modeling the differences between the left and right hemispheres of children with autism according to claim 1, characterized in that: The method of acquiring resting-state EEG data from normal children and children with autism using an EEG data acquisition module employs a 128-channel EEG system, selecting 46 electrodes covering four functional brain regions. All electrodes are referenced to Cz, and the impedance is controlled at 50kΩ. the following.

3. The electroencephalogram (EEG) analysis method based on modeling the differences between the left and right hemispheres of children with autism, as described in claim 1, is characterized in that... The process of training the model by optimizing the classification loss to obtain the probability of autism in the analyzed individual specifically refers to: inputting the spatiotemporal fusion feature matrix into a layer containing two fully connected layers, one... Activation function and a The classification layer uses an activation function, and the training is optimized using the cross-entropy loss function to obtain the final analysis results. The specific formula for the cross-entropy loss function is as follows: （11） in, Let be the true label of the subject j, with 1 for positive and 0 for negative. This represents the probability that the predicted value of subject j is positive; N is the total number of subject samples participating in the training.

Images