Brain network blind source separation method based on multi-scale convolutional autoencoder

The multi-scale convolutional autoencoder-based method addresses the limitations of GLM and ICA by extracting and fusing multi-scale fMRI features, achieving improved brain network separation and discovery.

CN115191946BActive Publication Date: 2024-12-06SHANXI BETHUNE HOSPITAL (SHANXI ACAD OF MEDICAL SCI SHANXI HOSPITAL OF TONGJI HOSPITAL AFFILIATED TO TONGJI MEDICAL COLLEGE OF HUAZHONG UNIV OF SCI & TECH SHANXI MEDICAL UNIV THIRD HOSPITAL SHANXI MEDICAL UNIV THIRD CLINICAL COLLEGE OF MEDICINE)
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210843038.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-18
Publication Date
2024-12-06
Estimated Expiration
2042-07-18

AI Technical Summary

Technical Problem

Existing methods for brain function research using general linear model (GLM) and independent component analysis (ICA) fail to effectively capture the complex multi-scale time domain characteristics of functional magnetic resonance imaging (fMRI) data, leading to inaccurate analysis of brain activities.

Method used

A brain network blind source separation method utilizing a multi-scale convolutional autoencoder, incorporating multi-scale convolution and long short-term memory networks to extract and fuse time series information at different scales, followed by unsupervised learning and blind source separation.

Benefits of technology

The method provides a more accurate and reliable extraction of brain networks by capturing multi-scale features in fMRI data, enhancing the discovery and exploration of brain functions compared to previous methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115191946B_ABST
    Figure CN115191946B_ABST
Patent Text Reader

Abstract

The present invention discloses a brain network blind source separation method based on a multi-scale convolutional autoencoder, and relates to the technical field of brain network separation. The brain network blind source separation method based on a multi-scale convolutional autoencoder includes the following steps: step one: extraction and preprocessing of brain signals; step two: brain signal feature extraction process based on multi-scale convolution; step three: multi-scale feature fusion process based on long short-term memory network. The present invention uses an innovative and effective unsupervised autoencoder learning paradigm to perform blind source separation of brain networks. Different from previous blind source separation technologies, the model directly uses the time series information of different scales that can be extracted from brain signals, and uses the long short-term memory network to fuse the features of each scale, and at the same time performs sparse restrictions on the fused response signal, thereby realizing the extraction of brain networks from brain signals. The new model architecture has made outstanding progress in brain network discovery and exploration compared with previous methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of brain network separation, and in particular to a brain network blind source separation method based on a multi-scale convolutional autoencoder. Background Art

[0002] The brain is the most advanced and complex information processing system known to humans so far. All human consciousness and behavior are efficiently controlled by the brain. As the most important component of the central nervous system, it is highly complex and highly correlated in structure and function. The exploration of human brain functions can not only help us better interpret various advanced cognitive functions such as emotions, psychology and memory, but also provide new ideas for the exploration and treatment of brain diseases and neurological diseases.

[0003] Functional magnetic resonance imaging (FMRI), as one of many magnetic resonance imaging technologies, has become the mainstream imaging method and technical means in the study of brain functional activities because of its good spatiotemporal resolution. But at the same time, as a highly complex giant system, the human brain contains very rich spatiotemporal correlations and multi-scale time domain components in fMRI data. However, how these correlations or time domain components at different scales are generated, what is their significance, and how they can help the study of brain-like science has always been the direction that researchers have been working hard to explore. At present, the research methods commonly used in the field of brain function research are still the general linear model method (GLM) and the independent component analysis method (ICA). Although these methods have promoted the research progress in the field of fMRI to a certain extent, as a shallow model, its ability in feature mining or relationship extraction still has many deficiencies.

[0004] The autoencoder model is the most typical unsupervised learning framework in deep neural networks. It has also been initially applied to the analysis of fMRI data and has achieved certain results in blind source separation. These existing research works generally analyze the time series signals of voxels at a single time domain scale. However, related research works have shown that the functional activities of the cerebral cortex have very complex time domain multi-scale receptive fields. The time domain receptive fields of some cognitive activities (such as instantaneous visual and auditory information perception) are short, while the time domain receptive fields of some cognitive activities (such as situational perception and emotional perception) are long. Ignoring the time domain multi-scale characteristics of fMRI brain functional activities will lead to analysis results that are far from the actual situation. Summary of the invention

[0005] The present invention provides a brain network blind source separation method based on a multi-scale convolutional autoencoder to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A brain network blind source separation method based on a multi-scale convolutional autoencoder, the brain network blind source separation method based on a multi-scale convolutional autoencoder, comprises the following steps:

[0008] Step 1: Extraction and preprocessing of brain signals;

[0009] Step 2: Brain signal feature extraction process based on multi-scale convolution;

[0010] Step 3: Multi-scale feature fusion process based on long short-term memory network;

[0011] Step 4: The process of restoring the timing signals in brain signals.

[0012] A further improvement of the technical solution of the present invention is that: the step one includes: using the standard fMRI data preprocessing tool DPARSF to perform time series processing on the brain signal, and the specific processing method includes: time layer correction, head movement correction, functional image and structural image alignment, spatial smoothing and bandpass filtering, so as to obtain the processed fMRI signal and use it as the input of the subsequent multi-scale convolution part.

[0013] The further improvement of the technical solution of the present invention is that: the step 2 includes: using 3 convolution kernels of different scales, the model extracts information of different scales in fMRI time series data by using multi-scale convolution kernels, wherein the number of convolution kernels of each scale is 16, so a total of 3×16=48 feature maps can be generated in the first layer of the multi-scale convolution part. In the second layer, the model also uses convolution kernels of different scales with lengths of 4, 8, and 16 for convolution operations. In order to further enhance the model's ability to extract information of different time series scales in fMRI data, the number of each convolution kernel in this layer becomes 32, and the number of feature maps is correspondingly increased from 48 to 3×32=96. At this point, the multi-scale convolution part in the MCA model ends.

[0014] A further improvement of the technical solution of the present invention is that it should be particularly pointed out in step 2 that the entire model does not use any data dimensionality reduction method of pooling or adjusting the stride. Therefore, in the 96 feature maps finally obtained, the length of each feature map is still the time series length t of the input sample.

[0015] A further improvement of the technical solution of the present invention is that the step three further includes the following steps:

[0016] A1: The output of the last hidden layer node of LSTM can be used to construct a multi-scale fusion feature matrix, and then the fused high-level features can be mapped back to the brain to obtain the brain network spatial distribution map;

[0017] A2: For the fully connected decoding part of the model, in the context of fMRI blind source separation, the weights of the fully connected layer reflect the source components in the input data. The model restores the output to the original input space by multiplying high-level features with the source components, thereby ultimately achieving the purpose of unsupervised learning and blind source separation of fMRI data.

[0018] A further improvement of the technical solution of the present invention is that the step 4 further includes the following steps:

[0019] S1: O∈R h×(n×m) is the multi-scale high-level feature obtained after the encoder's multi-scale feature learning and LSTM's feature fusion for each input fMRI sample, where h is the number of neurons in the LSTM hidden state, and n is the total number of samples in the input sample, that is, each row of O reflects the response of all input voxel samples on a certain fMRI source component feature, and the value of the vector reflects the degree to which the voxel belongs to a certain brain network source component, and thus the spatial distribution under the component can be obtained;

[0020] S2: Divide O in space to obtain the spatial distribution of brain networks of each subject;

[0021] S3: Perform a single-sample T-TEST statistical hypothesis test on the SMs of multiple subjects to test whether the response of each voxel position under the source component is significantly greater than 0, so as to obtain the statistically significant spatial distribution of the brain network in the group of subjects.

[0022] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art:

[0023] 1. The present invention provides a brain network blind source separation method based on a multi-scale convolutional autoencoder, which uses an innovative and effective unsupervised autoencoder learning paradigm to perform blind source separation of brain networks. Different from previous blind source separation technologies, this model directly uses the time series information of different scales that can be extracted from brain signals, and uses a long short-term memory network to fuse features of each scale, while sparsely restricting the fused response signal, thereby realizing brain network extraction from brain signals. The new model architecture has made outstanding progress in brain network discovery and exploration compared with previous methods.

[0024] 2. The present invention provides a brain network blind source separation method based on a multi-scale convolutional autoencoder. The time domain multi-scale fusion analysis of fMRI data is realized through the analysis model of the time domain multi-scale convolutional autoencoder. The method uses the time series signal of the voxel as input and uses a multi-scale time domain convolutional neural network to realize the time domain multi-scale feature expression of fMRI data. After that, a feature fusion module based on a long short-term memory network is used to achieve the purpose of fMRI time domain multi-scale feature fusion and brain network separation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the process of the present invention;

[0026] Figure 2 It is a schematic diagram of the network structure of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described in detail below in conjunction with embodiments:

[0028] Example 1

[0029] like Figure 1-2 As shown, the present invention provides a brain network blind source separation method based on a multi-scale convolutional autoencoder, and the brain network blind source separation method based on a multi-scale convolutional autoencoder comprises the following steps:

[0030] Step 1: Extraction and preprocessing of brain signals;

[0031] Step 2: Brain signal feature extraction process based on multi-scale convolution;

[0032] Step 3: Multi-scale feature fusion process based on long short-term memory network;

[0033] Step 4: The process of restoring the timing signals in brain signals.

[0034] Step 1 includes: using the standard fMRI data preprocessing tool DPARSF to perform time series processing on brain signals. The specific processing methods include: time layer correction, head motion correction, functional image and structural image registration, spatial smoothing and bandpass filtering, so as to obtain the processed fMRI signal as the input of the subsequent multi-scale convolution part.

[0035] In this embodiment, a Siemens 3T Trio scanner is used for data scanning during the experiment. In each scan, the participants close their eyes and enter a resting state for 8 minutes to obtain resting-state fMRI data. The parameters of the fMRI data are: TR 2200ms, FOV 192mm×192mm, TE 30ms, Flip Angle 79°, and voxel resolution 3mm×3mm×3mm. In addition, the scanning parameters of the T1 structural image are TR 4000ms, TE 2.89ms, Flip Angle 9°, FOV 240mm×256mm, and voxel resolution 1mm×1mm×1mm. The entire preprocessing step uses the most widely used statistical parameter mapping toolbox in the fMRI field to preprocess the functional images. The preprocessing process includes time layer correction, head motion correction using 6 parameters, functional image and structural image registration, and standardization to MN 2mm. In I space, spatial smoothing and band-pass filtering (0.0085-0.15Hz) of 6mmFWHM were performed. After band-pass filtering, resting-state data processing software was used to regress confounding covariates including white matter, cerebrospinal fluid and head movement parameters to reduce the potential influence of physiological confounding variables. To ensure that the time series data of all subjects were within the same standard range, the whole-brain voxel signals of each subject after preprocessing were standardized to ensure that the time series mean in each voxel was 0 and the standard deviation was 1.

[0036] Example 2

[0037] like Figure 1-2 As shown, on the basis of Example 1, the present invention provides a technical solution: preferably, step 2 includes: using 3 convolution kernels of different scales, the model extracts information of different scales in fMRI time series data by using multi-scale convolution kernels, wherein the number of convolution kernels of each scale is 16, so a total of 3×16=48 feature maps can be generated in the first layer of the multi-scale convolution part. In the second layer, the model also uses convolution kernels of different scales with lengths of 4, 8, and 16 for convolution operations. In order to further enhance the ability of the model to extract information of different time series scales in fMRI data, the number of each convolution kernel in this layer becomes 32, and the number of feature maps is correspondingly increased from 48 to 3×32=96. At this point, the multi-scale convolution part in the MCA model ends. It should be particularly pointed out in step 2 that the entire model does not use any data dimensionality reduction method of pooling or adjusting the stride, so in the 96 feature maps finally obtained, the length of each feature map is still the time series length t of the input sample.

[0038] In this embodiment, the number of whole-brain voxels of each subject is 228453, and the length of the time series signal is t=215. The resting-state fMRI data of different subjects are concatenated in time series to obtain the entire input sample, that is, the dimension of the input sample is S∈R 215×228453 , the present invention sets three one-dimensional convolution kernels of different lengths, which are 4, 8, and 16 respectively. The depth of the convolution layer is 2, where the number of convolution kernels in the first layer is set to 16, and the number of convolution kernels in the second layer is set to 96. Therefore, the present invention obtains C∈R 215×96×228453 Convolutional feature maps.

[0039] Example 3

[0040] like Figure 1-2 As shown, based on Example 1, the present invention provides a technical solution: preferably, the step three further includes the following steps:

[0041] A1: The output of the last hidden layer node of LSTM can be used to construct a multi-scale fusion feature matrix, and then the fused high-level features can be mapped back to the brain to obtain the brain network spatial distribution map;

[0042] A2: For the fully connected decoding part of the model, in the context of fMRI blind source separation, the weights of the fully connected layer reflect the source components in the input data. The model restores the output to the original input space by multiplying high-level features with the source components, thereby ultimately achieving the purpose of unsupervised learning and blind source separation of fMRI data.

[0043] In this embodiment, the multi-scale features obtained in the previous stage are concatenated to obtain the input vector of LSTM, where the initial state of LSTM is set to a full 0 matrix with a length of 100, and the number of LSTM layers is set to 1. C is sent to LSTM to obtain O∈R 100×228453 , and set the source component matrix W∈R 100×215 , multiply the matrices O and C to restore the input matrix and perform regression calculations.

[0044] Example 4

[0045] like Figure 1-2 As shown, based on Example 1, the present invention provides a technical solution: preferably, the step 4 further includes the following steps:

[0046] S1: O∈R h×(n×m)is the multi-scale high-level feature obtained after the encoder's multi-scale feature learning and LSTM's feature fusion for each input fMRI sample, where h is the number of neurons in the LSTM hidden state, and n is the total number of samples in the input sample, that is, each row of O reflects the response of all input voxel samples on a certain fMRI source component feature, and the value of the vector reflects the degree to which the voxel belongs to a certain brain network source component, and thus the spatial distribution under the component can be obtained;

[0047] S2: Divide O in space to obtain the spatial distribution of brain networks of each subject;

[0048] S3: Perform a single-sample T-TEST statistical hypothesis test on the SMs of multiple subjects to test whether the response of each voxel position under the source component is significantly greater than 0, so as to obtain the statistically significant spatial distribution of the brain network in the group of subjects.

[0049] In this embodiment, the obtained O matrix is ​​subjected to a T-TEST test, that is, a single-sample T-TEST statistical hypothesis test is performed on the spatial distribution maps of multiple subjects to test whether the response of each voxel position under the source component is significantly greater than 0, so as to obtain a statistically significant spatial distribution of the brain network in the group of subjects.

[0050] In addition, the present invention also illustrates one aspect of model training: the number of neurons in the LSTM hidden state is set to 100, and when the coefficient of L2 regularization is 0.1, L1 regularization is also introduced for the output of the LSTM hidden state to impose sparsity constraints on it, and its weight is 0.0005. In addition, the present invention uses dropout=0.5 to prevent overfitting in the feature fusion part of LSTM. The basic learning rate of the MCA model of the present invention is 0.001, and the size of the training batch is 256. The present invention also uses a gradually decreasing learning rate attenuation algorithm. After every 5000 iterations, the learning rate of the model is reduced to the original 0.99.

[0051] The present invention also illustrates another aspect of model application: in order to prove that the resting-state brain network obtained by the MCA model used in the present invention has similarity with the standard resting-state network template, this section of the experiment calculated the spatial overlap rate of the brain network obtained by the ICA method and the MCA method and the RSN template to verify the reliability of the MCA method, wherein the RSN template is a standard resting-state functional network (RSN). Table 1 shows the spatial overlap rate of 10 typical resting-state networks, and 1 to 10 in the table are the serial numbers of the ten RSN templates. Combined with Table 1, it can be seen that both the ICA method and the MCA method have a high overlap rate with the RSN true value, but at the same time, it can also be found that the MCA method can indeed have a relatively higher overlap rate than the traditional ICA method in some difficult-to-distinguish brain networks such as the ventral visual network (No. 2) and the cerebellum network (No. 8), which also proves that the MCA method does have a higher reliability for blind source separation of resting-state fMRI data.

[0052] Table 1 Overlap rate between the spatial distribution of brain networks extracted by MCA and ICA methods and the resting state network template

[0053]

[0054] The above generally describes the present invention in detail, but it is obvious to a person skilled in the art that some modifications or improvements can be made to the present invention. Therefore, modifications or improvements that do not depart from the spirit of the present invention are within the scope of protection of the present invention.

Claims

1. A brain network blind source separation method based on a multi-scale convolutional autoencoder, characterized by: The brain network blind source separation method based on multi-scale convolutional autoencoder includes the following steps: Step 1: Extraction and preprocessing of brain signals; Step 2: Brain signal feature extraction process based on multi-scale convolution; Step 3: Multi-scale feature fusion process based on long short-term memory network; Step 4: The process of restoring the time series signal in the brain signal; The step 1 includes: using a standard fMRI data preprocessing tool DPARSF to perform time series processing on brain signals; The step 2 includes: using 3 convolution kernels of different scales, the model extracts information of different scales in fMRI time series data by using multi-scale convolution kernels, wherein the number of convolution kernels of each scale is 16, so a total of 48 feature maps can be generated in the first layer of the multi-scale convolution part, and in the second layer, the model also uses convolution kernels of different scales with lengths of 4, 8, and 16 for convolution operations. In order to further enhance the model's ability to extract information of different time series scales in fMRI data, the number of convolution kernels of each scale in this layer is changed to 32, and the number of feature maps is correspondingly increased from 48 to 96. At this point, the multi-scale convolution part in the MCA model ends; The step 4 also includes the following steps: S1: O∈Rh×(n×m) is the multi-scale high-level feature obtained after the encoder’s multi-scale feature learning and LSTM’s feature fusion for each input fMRI sample, where h is the number of neurons in the LSTM hidden state, and n is the total number of samples in the input sample. That is, each row of O reflects the response of all input voxel samples to a certain fMRI source component feature, and the value of the vector in each row of O reflects the degree to which the voxel belongs to a certain brain network source component, and thus the spatial distribution under the source component can be obtained; S2: Divide O in space to obtain the spatial distribution of the brain network for each subject; S3: Perform a single-sample T-TEST statistical hypothesis test on the SMs of multiple subjects to test whether the response of each voxel position under the source component is significantly greater than 0, so as to obtain a statistically significant spatial distribution of the brain network in the group of subjects.

2. The method for blind source separation of brain networks based on multi-scale convolutional autoencoders according to claim 1, characterized in that: The specific processing method of step 1 includes: time layer correction, head movement correction, functional image and structural image registration, spatial smoothing and bandpass filtering, so as to obtain the processed fMRI signal and serve as the input of the subsequent multi-scale convolution part.

3. The method for blind source separation of brain networks based on multi-scale convolutional autoencoders according to claim 1, characterized in that: It should be noted in particular in step 2 that the entire model does not use any data dimensionality reduction method of pooling or adjusting the stride. Therefore, in the 96 feature maps finally obtained, the length of each feature map is still the time series length t of the input sample.

4. The method for blind source separation of brain networks based on multi-scale convolutional autoencoders according to claim 1, characterized in that: The step three also includes the following steps: A1: The output of the last hidden layer node of LSTM can be used to construct a multi-scale fusion feature matrix, and then the fused high-level features can be mapped back to the brain to obtain the brain network spatial distribution map; A2: For the fully connected decoding part of the model, in the context of fMRI blind source separation, the weights of the fully connected layer reflect the source components in the input data. The model restores the output to the original input space by multiplying high-level features with the source components, thereby ultimately achieving the purpose of unsupervised learning and blind source separation of fMRI data.

Citation Information

Patent Citations

  • Method for classifying electroencephalogram (EEG) signals based on multi-scale brain function network

    CN110522412A

  • Adaptive variable-scale convolution kernel method based on brain MRI three-dimensional image confidence

    CN112330674A