Classification method of brain fMRI data
By combining the brain fMRI data classification method of multi-scale residual blocks and timing extraction modules, the problem of insufficient utilization of time dynamic information in the prior art is solved, and a higher accuracy diagnosis of brain diseases is achieved.
Patent Information
- Application Number
- CN202510484802.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-17
- Publication Date
- 2025-08-08
AI Technical Summary
When processing fMRI data, existing deep learning models are difficult to fully capture and utilize time dynamic information, which limits the classification ability of brain disease diagnosis.
A method of classification of brain fMRI data is adopted, and multi-scale residual block EMSR and timing extraction module STT are combined with the FGB imaging module and the 3D GAN network to extract multi-scale spatial features and timing features, and the final feature vector is generated through the feature fusion module for classification.
It improves the classification accuracy and robustness of fMRI data, can capture abnormalities on a finer grain, and improves the accuracy of brain disease diagnosis.
Smart Images
Figure CN120451629A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data classification, and provides a classification method for brain fMRI data. Background Art
[0002] Functional magnetic resonance imaging (fMRI) is a non-invasive brain imaging technique that can monitor changes in brain activity in real time while the brain is performing a specific task or at rest. fMRI reflects the activity state of local brain regions by measuring changes in the brain's blood oxygen level-dependent (BOLD) signal.
[0003] fMRI is widely used in the study of brain diseases due to its advantages, such as non-invasiveness, high spatial resolution, and ability to precisely localize abnormal brain activity. In fMRI images of patients with certain brain disorders, activity patterns in specific brain regions (such as the prefrontal cortex and striatum) can differ significantly from those in healthy individuals. These differences can be used to detect abnormal activity in various brain regions by analyzing large amounts of fMRI data.
[0004] With the rapid development of artificial intelligence (AI), the application of deep learning in assisted diagnosis has demonstrated powerful neuroimaging processing capabilities. Convolutional neural networks (CNNs), a typical example of deep learning models, excel at processing image and video data. However, since fMRI data is essentially a three-dimensional image sequence (i.e., four-dimensional data) that changes over time, using only 2D or 3D CNNs cannot fully capture and utilize this temporal dynamics, limiting the classification capabilities of deep neural networks. Summary of the Invention
[0005] In view of this, the present application provides a classification method for brain fMRI data, aiming to improve at least one of the above problems.
[0006] Specifically, the following technical solutions are included:
[0007] On the one hand, the present invention provides a method for classifying brain fMRI data, which is as follows:
[0008] S1, detecting the fMRI data stream of the brain of the person to be tested;
[0009] S2. The collected fMRI data is input into the classification model, and the classification model outputs the classification result corresponding to the fMRI data, and the classification result is normal or abnormal.
[0010] Classification models include:
[0011] A first multi-scale residual block EMSR, and a timing extraction module STT connected to the first multi-scale residual block EMSR; an FGB imaging module, and a second multi-scale residual block EMSR connected to the FGB imaging module; a feature fusion module connected to the timing extraction module STT and the second multi-scale residual block EMSR, and a classifier connected to the feature fusion module.
[0012] In some embodiments of the present invention, the collected fMRI data stream is respectively input into the first multi-scale residual block EMSR and the FGB imaging module. The first multi-scale residual block EMSR extracts the multi-scale spatial features of each time point in the fMRI data, and the multi-scale spatial features of each time point are passed to the time series extraction module STT. The time series extraction module STT captures the time series features in the multi-scale spatial features, enhances the perception of dynamic features, and inputs the feature fusion module; the FGB imaging module randomly selects fMRI data frames from the input fMRI data stream to input the adversarial network, the adversarial network generates high-quality fMRI data, and inputs the second multi-scale residual block EMSR. The second multi-scale residual block EMSR extracts brain structure features in the fMRI data and inputs the feature fusion module; the feature fusion module fuses the time series features in the multi-scale spatial features with the brain structure features, inputs the formed fusion features into the classifier, and the classifier outputs the classification results corresponding to the fusion features.
[0013] In some embodiments of the present invention, the FGB imaging module is composed of a random sampling module and a 3D GAN network. The random sampling module randomly extracts MRI data frames from the input fMRI data stream and inputs them into the 3D GAN network. The 3D GAN network performs quality enhancement on the input MRI data frames.
[0014] In some embodiments of the present invention, the multi-scale residual block EMSR includes:
[0015] The first 3×3×3 convolutional layer and the second 5×5×5 convolutional layer connected to the input end; the first ReLU nonlinear activation function connected to the first 3×3×3 convolutional layer, the second ReLU nonlinear activation function connected to the second 5×5×5 convolutional layer; the first ReLU nonlinear activation function and the second ReLU nonlinear activation function are connected to the first Concat layer and the second Concat layer; the first Concat layer is connected to the second 3×3×3 convolutional layer, and the second Concat layer is connected to the second 5×5×5 convolutional layer; the second 3×3×3 convolutional layer is connected to the third ReLU nonlinear activation function, and the second 5×5×5 convolutional layer is connected to the fourth ReLU nonlinear activation function; the third ReLU nonlinear activation function and the fourth ReLU nonlinear activation function are connected to the third Concat layer, the third Concat layer is connected to the 1×1×1 convolutional layer, and the output of the 1×1×1 convolutional layer is connected to the input residual of the input end.
[0016] In some embodiments of the present invention, the timing extraction module STT is specifically as follows:
[0017] Based on the time sequence, the multi-scale spatial feature maps F at different time points are transformed into n A multi-scale spatial feature sequence is formed. The input multi-scale spatial feature sequence is patch-segmented using a patch of a specified size. The multi-scale spatial feature sequence is divided into a group of multi-scale spatial feature subsequences that exist for a certain period of time. By changing the size of the patch, three groups of multi-scale spatial feature subsequences of different lengths are obtained. These subsequences are input into three parallel connected Transformer layers respectively. The features output by the three Transformer layers are fused. The fusion process includes flattening, splicing, and linear transformation in sequence.
[0018] In some embodiments of the present invention, each Transformer layer includes: a multi-head self-attention layer, a first layer normalization connected to the multi-head self-attention layer, the input of the multi-head self-attention layer is connected to the output residual of the first layer normalization and then connected to the feedforward layer, the feedforward layer is connected to the second layer normalization, and the input of the feedforward network is connected to the output residual of the second layer normalization.
[0019] In some embodiments of the present invention, the patch segmentation method is as follows:
[0020]
[0021] Among them, N is the number of multi-scale spatial feature subsequences after segmentation, c is the length of the multi-scale spatial feature sequence, m is the size of the patch, and r is the segmentation stride.
[0022] In some embodiments of the present invention, the classifier is a fully connected network.
[0023] By fusing spatial features and temporal features, the model can capture anomalies at a finer granularity, thereby improving the accuracy and robustness of classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0025] Figure 1 A flowchart of a method for classifying brain fMRI data provided by an embodiment of the present invention;
[0026] Figure 2A schematic diagram of the structure of a classification model provided by an embodiment of the present invention;
[0027] Figure 3 A schematic diagram of the structure of a multi-scale residual block EMSR provided in an embodiment of the present invention;
[0028] Figure 4 A schematic diagram of the structure of a timing extraction module STT provided in an embodiment of the present invention;
[0029] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0031] Unless otherwise defined, all technical terms used in the embodiments of the present application have the same meanings as commonly understood by those skilled in the art.
[0032] For ease of understanding, the following is a detailed description of the processing flow of the DMS-Former model:
[0033] The fMRI data is first processed into two parallel branches. The first branch extracts spatial features through the first multi-scale residual block (EMSR). In this module, the data is convolved with 3×3×3 and 5×5×5 kernels along two paths, respectively. The outputs are activated by ReLU and then concatenated and fed into the next convolution layer, maintaining the kernel size. The concatenated result is then connected to the module input through a 1×1×1 convolution layer to form a residual connection, resulting in the output of the first multi-scale residual block (EMSR). The output is then flattened into a (samples, embedding) tensor and fed into the temporal extraction module (STT). In the STT, the data is segmented according to three strides. The resulting multi-scale spatial feature subsequences are fed into different Transformer layers, where a multi-head attention mechanism is used to extract temporal features. The multi-head attention results from each Transformer layer are concatenated to more fully capture the temporal characteristics of the data. The fMRI data from the second branch first passes through the FGB module, where a randomly selected frame of data is used to generate a 3D MRI image using an adversarial network. The resulting image is passed to the second multi-scale residual block, EMSR, to extract brain structural features. This process is similar to the first branch's feature extraction process, resulting in brain structural information. The features output by the two branches are fused and fed into a linear classifier to determine if the subject's fMRI data is abnormal.
[0034] Figure 1 This is a flow chart of a method for classifying brain fMRI data provided by an embodiment of the present invention. The method is specifically as follows:
[0035] S1. Detect the fMRI data stream of the brain of the person being tested. The fMRI data stream reflects the changes in the BOLD signal at various locations in the brain over time and is 4D fMRI data.
[0036] S2. The collected fMRI data is input into the classification model, and the classification model outputs the classification result corresponding to the fMRI data, and the classification result is normal or abnormal.
[0037] Figure 2 A schematic diagram of the structure of a classification model provided in an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown. The classification model includes:
[0038] A first multi-scale residual block EMSR and a time series extraction module STT connected to the first multi-scale residual block EMSR; an FGB imaging module and a second multi-scale residual block EMSR connected to the FGB imaging module; a feature fusion module connected to the time series extraction module STT and the second multi-scale residual block EMSR, and a classifier connected to the feature fusion module;
[0039] The collected fMRI data stream is respectively input into the first multi-scale residual block EMSR and FGB imaging module. The first multi-scale residual block EMSR extracts the multi-scale spatial features of each time point in the fMRI data. The multi-scale spatial features of each time point are passed to the time series extraction module STT. The time series extraction module STT captures the temporal features in the multi-scale spatial features, enhances the perception of dynamic features, and inputs them into the feature fusion module.
[0040] The FGB imaging module randomly selects fMRI data frames from the input fMRI data stream to be input into the adversarial network. The adversarial network generates high-quality fMRI data and inputs it into the second multi-scale residual block EMSR. The second multi-scale residual block EMSR extracts brain structural features from the fMRI data and inputs them into the feature fusion module.
[0041] The feature fusion module fuses the temporal features of multi-scale spatial features with brain structural features, and inputs the resulting fused features into a classifier, which then outputs the classification result corresponding to the fused features. This classifier is a fully connected network that analyzes neural signals over time, identifies different states of brain activity, and then classifies the signals as normal or abnormal.
[0042] In an embodiment of the present invention, the FGB imaging module consists of a random sampling module and a 3D GAN network. The random sampling module randomly extracts MRI data frames from the input fMRI data stream and inputs them into the 3D GAN network. The 3D GAN network performs quality enhancement on the input MRI data frames.
[0043] The generator of the 3D GAN network uses three-dimensional convolutions, which can capture more complex spatial structural features, allowing the generator to better restore the spatial integrity of brain images when modeling them. To improve the ability to discriminate generated images, the 3D GAN network uses a three-dimensional residual network as a discriminator. Residual networks can effectively alleviate the gradient vanishing problem through skip connections, ensuring effective training of deep networks. In the three-dimensional residual discriminator, the input and generated three-dimensional images are subjected to feature extraction through multiple residual blocks, thereby more accurately judging the authenticity of the generated images. This structure not only captures image details but also focuses on a wider range of three-dimensional contextual information, thereby improving the ability to discriminate the authenticity of generated images.
[0044] To better measure the difference between generated images and real images, the FGB module introduces Learned Perceptual Image Patch Similarity (LPIPS) as an optimization objective. LPIPS is a perceptual-based image similarity metric that, by measuring the similarity between different images, can better reflect image quality under human perception. Specifically, the FGB module optimizes the generation process through the following steps: Computing perceptual similarity: During the GAN generation process, the LPIPS distance between the input image and the generated image k(s,o) is calculated, where s is the latent space representation and o is a 3D noise volume. Optimizing the latent space and noise volume: By minimizing the LPIPS distance, the latent space s and noise volume o are optimized, guiding the generator to produce images that are more consistent with the characteristics of real images. Because LPIPS is based on feature extraction from deep learning models, it can more accurately reflect the perceptual quality of images than traditional pixel differences (such as Mean Separation). Iteratively updating the generator: Using feedback from LPIPS, the generator parameters are continuously adjusted to gradually bring the generated images closer to the real images. This perceptual-based optimization approach effectively improves the quality of generated images, making them more visually realistic and consistent.
[0045] Figure 3 A schematic diagram of the structure of a multi-scale residual block EMSR provided in an embodiment of the present invention is provided. For ease of explanation, only the portion related to the embodiment of the present invention is shown. The multi-scale residual block EMS includes:
[0046] The first 3×3×3 convolutional layer and the second 5×5×5 convolutional layer connected to the input;
[0047] The first ReLU nonlinear activation function connected to the first 3×3×3 convolutional layer, and the second ReLU nonlinear activation function connected to the second 5×5×5 convolutional layer;
[0048] The first ReLU nonlinear activation function and the second ReLU nonlinear activation function connect the first Concat layer and the second Concat layer;
[0049] The first Concat layer is connected to the second 3×3×3 convolution layer, and the second Concat layer is connected to the second 5×5×5 convolution layer;
[0050] The second 3×3×3 convolutional layer is connected to the third ReLU nonlinear activation function, and the second 5×5×5 convolutional layer is connected to the fourth ReLU nonlinear activation function;
[0051] The third ReLU nonlinear activation function and the fourth ReLU nonlinear activation function are connected to the third Concat layer, the third Concat layer is connected to the 1×1×1 convolution layer, and the output of the 1×1×1 convolution layer is connected to the input residual of the input end.
[0052] The multi-scale residual block EMSR module consists of two parts: multi-scale feature fusion and local residual learning. Multi-scale feature fusion is responsible for extracting rich image features from different scales, while local residual learning ensures smooth information transmission and optimizes network performance.
[0053] In the multi-scale feature fusion part, the multi-scale residual block EMSR uses convolution kernels of different sizes to extract image features under different receptive fields. The multi-scale residual block EMSR contains two convolution paths. The input feature map is processed in parallel by the first 3×3×3 and 5×5×5 convolution layers. The 3×3×3 convolution focuses on capturing local details, while the 5×5×5 convolution is more suitable for capturing larger-scale contextual information. The output of each convolution layer is processed by a nonlinear activation function (ReLU) and then spliced through a Concat layer to form multi-scale features. The multi-scale features are convolved in parallel by the 3×3×3 and 5×5×5 convolution layers, and then processed by a nonlinear activation function (ReLU) and spliced again through a Concat layer, allowing features of different scales to interact with each other and enriching the feature expression capability. The feature map formed by the secondary splicing is input into the 1×1×1 convolution layer as a bottleneck layer to reduce the number of channels and fuse multi-scale features, thereby outputting the compressed feature map S′. The above operation can be defined as:
[0054] S1=σ(w 11 *F n-1 +b 11 );
[0055] S2=σ(w 21 *F n-1 +b 21 );
[0056] S3=σ(w 12 *[S1||S2]+b 12 );
[0057] S4=σ(w 22 *[S2||S1]+b 22 );
[0058] S′=w 3 *[S3||S4]+b 3 ;
[0059] Among them, w 11 、w 12They represent the weights of the first 3×3×3 convolutional layer (the convolutional layer with a convolution kernel size of 3×3×3) and the second 3×3×3 convolutional layer, respectively. 21 、w 22 They represent the weights of the first 5×5×5 convolutional layer (the convolutional layer with a convolution kernel size of 5×5×5), and the second 5×5×5 convolutional layer, w 3 represents the weight of the 1×1×1 convolutional layer, || is the concatenation symbol, and b 11 、b 12 They represent the deviations of the first 3×3×3 convolutional layer and the second 3×3×3 convolutional layer, respectively, and b 21 、b 22 They represent the deviations of the first 5×5×5 convolutional layer and the second 5×5×5 convolutional layer, respectively, and b 3 represents the deviation of the 1×1×1 convolution layer (the convolution kernel size is 1×1×1), σ(·) represents the ReLU nonlinear activation function, F n-1 is the feature map of the input multi-scale residual block EMSR, and S′ is the feature map output after the convolution operation of the 1×1×1 convolution layer.
[0060] In order to improve the efficiency of gradient propagation, the multi-scale residual block EMSR adopts the design of local residual connection. n-1 Add together to form the output feature map F of the multi-scale residual block EMSR n , through the above residual connection, the multi-scale residual block EMSR can learn the residual between the input and output, rather than the complete feature conversion, making the training more efficient and helping to alleviate the gradient vanishing problem. This design can ensure that information is effectively propagated in the deep network and significantly improve the training stability. This operation can be defined as: F n =S′+F n-1 .
[0061] Based on the multi-scale residual block EMSR, multi-scale spatial features are extracted from the fMRI data frame corresponding to each time point to form a multi-scale spatial feature stream corresponding to the fMRI data stream, that is, the changes of the extracted multi-scale spatial features over time. The multi-scale residual block EMSR is called the first multi-scale residual block EMSR. When extracting brain structure features in fMRI data based on the multi-scale residual block EMSR, the multi-scale residual block EMSR is called the second multi-scale residual block EMSR.
[0062] Figure 4 This is a schematic diagram of the structure of a timing extraction module STT provided in an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown. The timing extraction module STT includes:
[0063] Based on the time sequence, the multi-scale spatial feature maps F at different time points are transformed inton A multi-scale spatial feature sequence is formed, and the input multi-scale spatial feature sequence is patch-segmented using a patch of a specified size. The multi-scale spatial feature sequence is divided into a group of multi-scale spatial feature subsequences that exist for a certain period of time. Patch segmentation is performed in a sliding window manner. By changing the size of the patch, three groups of multi-scale spatial feature subsequences of different lengths are obtained. These subsequences are input into three parallel connected Transformer layers respectively, and the features output by the three Transformer layers are fused. The fusion process includes flattening, splicing, and linear transformation in sequence.
[0064] In an embodiment of the present invention, the patch segmentation method is as follows:
[0065]
[0066] Among them, N is the number of multi-scale spatial feature subsequences after segmentation, c is the length of the multi-scale spatial feature sequence, m is the size of the patch, and r is the segmentation stride.
[0067] Each Transformer layer consists of a multi-head self-attention layer, a normalized first layer connected to the multi-head self-attention layer, the input of the multi-head self-attention layer is connected to the normalized output residual of the first layer and then connected to the feedforward network, the feedforward layer is connected to the normalized second layer, and the input of the feedforward layer is connected to the normalized output residual of the second layer.
[0068] The time series extraction module (STT) applies a self-attention mechanism to the generated multi-scale spatial feature subsequences, calculating the similarity between each pair of multi-scale spatial feature subsequences, i.e., the attention weight, to capture the dependencies within the time series. Multi-head attention is used, i.e., multiple sets of attention matrices are calculated in each attention layer to enhance the model's ability to model complex features. The specific calculation of multi-head attention is as follows:
[0069]
[0070] Among them, W Q 、W K 、W V is the learnable weight matrix of query, key and value, p ij is the relative position encoding between the i-th and j-th multi-scale spatial feature subsequences, w pos is the weight of the position encoding, the attention score α ij It represents the degree of attention of the i-th multi-scale spatial feature subsequence to the j-th multi-scale spatial feature subsequence, and D is the dimension of the key vector.
[0071] Multi-head attention uses the attention score of the i-th multi-scale spatial feature sub-sequence to all other multi-scale spatial feature sub-sequences to the weight matrix W V Perform weighted summation to obtain the attention output of the i-th multi-scale spatial feature subsequence
[0072]
[0073] At each attention layer, the Time Series Extraction Module (STT) employs a multi-head attention mechanism. This involves computing multiple attention heads and concatenating their outputs to improve the model's ability to capture complex features. To better capture periodic features in time series, the Time Series Extraction Module (STT) employs relative position encoding within its self-attention mechanism, rather than traditional absolute position encoding. Relative position encoding effectively represents the relative positional relationships between subsequences of multi-scale spatial features and is particularly well-suited for modeling periodic time series.
[0074] In an embodiment of the present invention, the classification model needs to be trained with a large number of samples before use. The sample data in the present invention is derived from the internationally published ADHD-200 database (http: / / fcon_1000.projects.nitrc.org / indi / adhd200 / ). The ADHD-200 dataset is a public neuroimaging dataset specifically used to study the neural mechanisms of Attention Deficit Hyperactivity Disorder (ADHD). It contains 8 datasets, namely the databases of the Child Study Center of New York University, Brown University, University of Pittsburgh, University of Washington, NeuroImage, Kennedy Krieger Institute, Oregon Health and Science University, and Peking University Child Study Center. The ADHD-200 dataset contains multiple fMRI studies with different numbers of subjects. The training set and validation set will be used in a mixed manner to reduce the error between different datasets.
[0075] fMRI is an imaging technique used to observe brain activity. It primarily does this by detecting changes in blood flow in the brain, which are related to blood oxygen levels and are therefore also known as the BOLD signal. The brain can be spatially divided into many small cubic units, each called a voxel. Each voxel records a series of data points that change over time during an fMRI scan. This means that fMRI data is not just a three-dimensional image but also includes a time dimension, and can therefore be considered four-dimensional data. In short, fMRI captures spatial and temporal information about brain activity by analyzing changes in blood flow and oxygenation levels in the brain.
[0076] In order to improve the quality of fMRI data, the collected fMRI data were preprocessed, mainly for denoising. The preprocessing was completed by the DPARSF (Data Processing Assistant for Resting-State fMRI, http: / / www.restfmri.net) toolbox and the spm12 software package (http: / / www.fil.ion.ucl.ac.uk / spm / software).
[0077] The present invention uses one branch to extract spatiotemporal features from raw data, while another branch extracts spatial features from generated high-definition MRI images. This fusion generates a final feature vector, enabling the model to identify key brain regions associated with different brain dysfunctions from the imaging data, thereby discovering biomarkers associated with patterns of brain dysfunction. Furthermore, the present invention combines spatial and temporal feature extraction to effectively capture both short-range and long-range dependencies in the data, enabling the model to more accurately identify potential abnormal patterns when processing complex time series data, thereby improving overall recognition performance.
[0078] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the present invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only.
[0079] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A classification method for brain fMRI data, characterized in that: The method is specifically as follows: S1, collect fMRI data stream of the brain of the person to be tested; S2. The collected fMRI data is input into the classification model, and the classification model outputs the classification result corresponding to the fMRI data, and the classification result is normal or abnormal.
2. The method for classifying brain fMRI data according to claim 1, wherein: Classification models include: A first multi-scale residual block EMSR, and a timing extraction module STT connected to the first multi-scale residual block EMSR; an FGB imaging module, and a second multi-scale residual block EMSR connected to the FGB imaging module; a feature fusion module connected to the timing extraction module STT and the second multi-scale residual block EMSR, and a classifier connected to the feature fusion module.
3. The brain fMRI data classification method according to claim 2, wherein: The collected fMRI data stream is input into the first multi-scale residual block EMSR and FGB imaging module respectively. The first multi-scale residual block EMSR extracts the multi-scale spatial features of each time point in the fMRI data. The multi-scale spatial features of each time point are passed to the time series extraction module STT. The time series extraction module STT captures the temporal features in the multi-scale spatial features, enhances the perception of dynamic features, and inputs them into the feature fusion module. The FGB imaging module randomly selects fMRI data frames from the input fMRI data stream to be input into the adversarial network. The adversarial network generates high-quality fMRI data and inputs it into the second multi-scale residual block EMSR. The second multi-scale residual block EMSR extracts brain structural features from the fMRI data and inputs them into the feature fusion module. The feature fusion module fuses the temporal features in the multi-scale spatial features with the brain structure features, inputs the formed fusion features into the classifier, and the classifier outputs the classification results corresponding to the fusion features.
4. The method for classifying brain fMRI data according to claim 2, wherein: The FGB imaging module consists of a random sampling module and a 3D GAN network. The random sampling module randomly extracts MRI data frames from the input fMRI data stream and inputs them into the 3D GAN network. The 3D GAN network enhances the quality of the input MRI data frames.
5. The method for classifying brain fMRI data according to claim 2, wherein: The multi-scale residual block EMSR includes: The first 3×3×3 convolutional layer and the second 5×5×5 convolutional layer connected to the input end; the first ReLU nonlinear activation function connected to the first 3×3×3 convolutional layer, the second ReLU nonlinear activation function connected to the second 5×5×5 convolutional layer; the first ReLU nonlinear activation function and the second ReLU nonlinear activation function are connected to the first Concat layer and the second Concat layer; the first Concat layer is connected to the second 3×3×3 convolutional layer, and the second Concat layer is connected to the second 5×5×5 convolutional layer; the second 3×3×3 convolutional layer is connected to the third ReLU nonlinear activation function, and the second 5×5×5 convolutional layer is connected to the fourth ReLU nonlinear activation function; the third ReLU nonlinear activation function and the fourth ReLU nonlinear activation function are connected to the third Concat layer, the third Concat layer is connected to the 1×1×1 convolutional layer, and the output of the 1×1×1 convolutional layer is connected to the input residual of the input end.
6. The method for classifying brain fMRI data according to claim 2, wherein: The timing extraction module STT is as follows: Based on the time sequence, the multi-scale spatial feature maps F at different time points are transformed into n A multi-scale spatial feature sequence is formed. The input multi-scale spatial feature sequence is patch-segmented using a patch of a specified size. The multi-scale spatial feature sequence is divided into a group of multi-scale spatial feature subsequences that exist for a certain period of time. By changing the size of the patch, three groups of multi-scale spatial feature subsequences of different lengths are obtained. These subsequences are input into three parallel connected Transformer layers respectively. The features output by the three Transformer layers are fused. The fusion process includes flattening, splicing, and linear transformation in sequence.
7. The method for classifying brain fMRI data according to claim 6, wherein: Each Transformer layer consists of a multi-head self-attention layer, a normalized first layer connected to the multi-head self-attention layer, the input of the multi-head self-attention layer is connected to the normalized output residual of the first layer and then connected to the feedforward network, the feedforward layer is connected to the normalized second layer, and the input of the feedforward layer is connected to the normalized output residual of the second layer.
8. The method for classifying brain fMRI data according to claim 6, wherein: The patch segmentation method is as follows: Among them, N is the number of multi-scale spatial feature subsequences after segmentation, c is the length of the multi-scale spatial feature sequence, m is the size of the patch, and r is the segmentation stride.
9. The method for classifying brain fMRI data according to claim 1, wherein: The classifier is a fully connected network.