A classification method for Alzheimer's disease based on multi-view hybrid attention network

Abstract

The present invention provides an Alzheimer's disease classification method based on a multi-view hybrid attention network, comprising: obtaining and preprocessing an sMRI dataset; establishing an Alzheimer's disease classification model, comprising sequentially connected: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier; inputting the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training; obtaining sMRI image data of a subject to be classified and inputting the data into the trained Alzheimer's disease classification model for classification; the present invention uses a full-image-level method, by combining two deep learning network frameworks with complementary advantages, while using strategies such as multi-view, attention mechanism, and multi-scale feature fusion, and introducing MMSE clinical examination scores as auxiliary supervision information, which can effectively improve the reliability of computer-aided diagnosis and classification of Alzheimer's disease.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and structural magnetic resonance imaging (sMRI)-assisted diagnosis of Alzheimer's disease, and more specifically, to an Alzheimer's disease classification method based on a multi-view hybrid attention network. Background Art

[0002] Alzheimer's disease (AD) is an irreversible neurodegenerative disorder that is particularly common in people over 65. Clinical manifestations of AD include progressive deterioration of functions such as language, memory, and emotion, and neuropathological changes manifest as progressive brain atrophy, which is one of the main causes of dementia. Studies have shown that there is a progressive prodromal condition, mild cognitive impairment (MCI), before the onset of AD. This condition manifests as mild decline in memory, cognition, and sensory functions, but does not significantly affect daily life and is therefore not easily perceived by patients. MCI patients can be divided into progressive (pMCI) and stable (sMCI) types, depending on whether cognitive function remains relatively stable over time. Studies have shown that patients with pMCI can develop AD several years after diagnosis. However, there is currently no effective treatment for Alzheimer's disease (AD). Therefore, early preventive diagnosis and timely intervention are crucial to slowing the progression of AD. Structural magnetic resonance imaging technology shows high sensitivity in capturing image structural changes caused by brain atrophy. This technology can perform non-invasive imaging examinations under non-invasive conditions and present brain pathological changes in an intuitive manner. It has become an important auxiliary tool in many medical fields.

[0003] Machine learning (ML) and deep learning (DL) have been widely used in sMRI image analysis, particularly in computer-assisted diagnosis of early-stage Alzheimer's disease (AD). Traditional AD-assisted diagnosis methods first require experts to manually design and extract useful features, followed by feature learning using machine learning methods. For example, Kloppel et al. extracted features from whole-brain gray matter tissue, mapped them into a high-dimensional space, and then used a support vector machine (SVM) to classify subjects. Another approach, Silveira et al., divided the entire brain into 116 anatomical regions of interest (ROIs) and then used a boosting classifier for classification. However, traditional methods have limitations such as high expert knowledge requirements, subjective feature design, and a time-consuming and labor-intensive process, which have limited improvements in AD diagnostic performance. In contrast, deep learning (DL) uses an end-to-end approach to automatically learn low-level to high-level latent features from sMRI images, avoiding the complex process of manually designing features. Furthermore, deep learning methods based on sMRI images offer greater flexibility. Feature extraction methods are typically categorized into four types, depending on the model's data input: slice-level (2D slice), 3D patch-level, ROI (Region of Interest)-based, and subject-level. In recent years, convolutional neural networks (CNNs) have achieved significant success in sMRI-based AD diagnosis. For example, Lim et al. used a dataset of axial 2D slices of 3D sMRI images to construct a feature extractor using ResNet-50 and VGG-16, then combined them with fully connected layers for AD classification. However, this approach only captures limited lesion information in 2D slices and ignores structural information between slices. On the other hand, Zhu et al. first performed saliency analysis on 3D patches of AD and NC groups, identifying highly discriminative brain regions associated with AD diagnosis as the basis for classification in a multi-instance deep learning model. While this approach can reduce computational complexity and inference time, it also loses spatial information between patches, and selecting patch size remains a challenge. The CAM-CNN proposed by Zhang et al. takes the entire sMRI image as input, extends the convolution operation to 3D to better capture the spatial and imaging details of the sMRI, and applies a connected attention mechanism to integrate more compact high-level features. However, this method only uses 3D convolution to capture local information, making it difficult to effectively model the global dependencies between pathological regions in the 3D sMRI image space.The Vision Transformer (VIT) divides the original image into blocks, maps these blocks into vectors, and then uses the self-attention mechanism in the encoder for global modeling, thereby capturing long-range dependencies and enabling the model to better focus on important areas. For example, Hoang et al. studied the classification performance of different versions of the VIT model on a 2D slice dataset based on sMRI. However, the pure VIT model is not as good as CNN in capturing local features and has higher requirements for the size of the dataset, which limits its convergence and accuracy.

[0004] An existing patent document discloses a method for classifying Alzheimer's disease based on CNN and attention mechanism, including: obtaining a brain magnetic resonance imaging (MRI) image to be classified and slice images corresponding to three sections; obtaining local fine-grained features of the brain MRI image to be classified; increasing the channel features of the input feature map through a channel attention mechanism; adding the position information of each slice; capturing the long-distance dependency between slices through a multi-head attention mechanism; using one-dimensional convolution to enhance the local continuity of features between two adjacent slices, and globally averaging the fused feature vector and sending it to the classifier to obtain the classification result of the brain MRI image to be classified; although the solution of the prior art can combine the advantages of convolutional neural networks and attention mechanisms, and utilize the interaction between local lesion features and global information to improve classification accuracy, it is still difficult to comprehensively and accurately consider the relevant brain atrophy structural information contained in sMRI images and the correlation between them, thereby losing spatial and semantic information and limiting the diagnostic performance of the model. Summary of the Invention

[0005] In order to overcome the defects of the above-mentioned existing technologies in insufficient generalization ability and limited performance of atrophic lesion areas related to AD in sMRI images, the present invention provides an Alzheimer's disease classification method based on a multi-view hybrid attention network. By using a full-image-level research method, by combining two deep learning network frameworks with complementary advantages, and using strategies such as multi-view, attention mechanism and multi-scale feature fusion, the MMSE clinical examination score (Mini Mental Status Examination) is introduced as auxiliary supervision information, which can effectively improve the reliability of computer-assisted diagnosis and classification of Alzheimer's disease.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A method for Alzheimer's disease classification based on a multi-view hybrid attention network includes the following steps:

[0008] S1: Acquire sMRI dataset and perform preprocessing;

[0009] The sMRI data set includes three-dimensional sMRI image data of several subjects;

[0010] S2: Establishing a classification model for Alzheimer's disease;

[0011] The Alzheimer's disease classification model includes: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier connected in sequence;

[0012] The feature extraction backbone network also includes: a feature coarse extraction subnetwork, a plurality of multi-view feature fusion modules connected in sequence, and a cascade multi-scale feature fusion subnetwork; the output of each multi-view feature fusion module is connected to the input of the cascade multi-scale feature fusion subnetwork;

[0013] S3: Input the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training to obtain a trained Alzheimer's disease classification model;

[0014] S4: Obtain sMRI image data of the subject to be classified, input the sMRI image data of the subject to be classified into the trained Alzheimer's disease classification model for classification, and complete the classification of Alzheimer's disease.

[0015] Preferably, the preprocessing in step S1 includes:

[0016] The three-dimensional sMRI image data of each subject are subjected to skull stripping, spatial registration, smoothing and denoising, blank background removal, resampling, format conversion, data enhancement and data partitioning operations to complete preprocessing.

[0017] Preferably, the feature coarse extraction subnetwork in step S2 includes: a 3D convolution block, a Convnext feature extraction block 1 and a Convnext feature extraction block 2 connected in sequence;

[0018] The structure of the 3D convolutional block includes: a 4×4×4 convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer connected in sequence;

[0019] The structure of each Convnext feature extraction block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext feature extraction block also form a residual sum connection through a bypass.

[0020] Preferably, the multi-view feature fusion module in step S2 includes branches 1, 2 and 3 arranged in parallel, wherein branches 1, 2 and 3 correspond to the sagittal plane view, the coronal plane view and the axial plane view, respectively;

[0021] The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence;

[0022] The structure of each convolution block is the same, and includes: a 4×4×4 convolution layer, a batch normalization BN layer, a ReLU activation layer, and a 1×1×1 convolution layer connected in sequence;

[0023] The structure of each hybrid attention block is the same, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the spatial domain attention branch includes: a 1×1×1 convolution layer, a 3×3×3 convolution layer, a 3×3×3 convolution layer, a 1×1×1 convolution layer and a batch normalization BN layer connected in sequence; the channel domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence;

[0024] The outputs of the spatial domain attention branch and the channel domain attention branch are added and normalized by the Sigmoid activation function to obtain the attention weighted score of the hybrid domain, the attention weighted score of the hybrid domain is weighted multiplied by the input of the hybrid attention block, and the weighted multiplication result is used as the output of the hybrid attention block;

[0025] The outputs of branches 1, 2 and 3 are spliced ​​and fused as the output of the multi-view feature fusion module.

[0026] Preferably, in step S2, a transition layer is provided before each of the multi-view feature fusion modules, and the transition layer includes: a 1×1×1 convolution layer, a batch normalization (BN) layer, a ReLU activation layer, a maximum pooling downsampling layer, and a Convnext block connected in sequence;

[0027] The Convnext block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext block also form a residual sum connection through a bypass.

[0028] Preferably, in step S2, the output of each multi-view feature fusion module is recorded as: F1, F2, F3, ..., F n , n is a positive integer, indicating the number of multi-view feature fusion modules;

[0029] In the cascaded multi-scale feature fusion sub-network, F n After 1×1×1 convolution layer and upsampling layer, with F n-1 Perform feature splicing and fusion to obtain the intermediate fusion feature map M1;

[0030] After the intermediate fusion feature map M1 passes through the 1×1×1 convolution layer and the upsampling layer, it is combined with F n-2 Perform feature splicing and fusion to obtain the intermediate fusion feature map M2;

[0031] Repeat the above fusion steps recursively to obtain the intermediate fusion feature maps M1, M2, ..., M n-1 ;

[0032] The intermediate fusion feature map M n-1 After convolution operation and average pooling downsampling operation, it is fused with the intermediate feature map M n-2 Perform secondary fusion to obtain the intermediate fusion feature map P1;

[0033] The intermediate fusion feature map P1 is subjected to convolution operation and average pooling downsampling operation and is combined with the intermediate fusion feature map M n-3 Perform secondary fusion to obtain the intermediate fusion feature map P2;

[0034] Repeat the above secondary fusion steps recursively to obtain the intermediate fusion feature map P of the last secondary fusion n-2 , the intermediate fusion feature map P n-2 With F n An addition operation is performed, and the addition result is used as the output of the cascaded multi-scale feature fusion sub-network.

[0035] Preferably, the Transformer encoder in step S2 comprises: a flat linear layer and a plurality of Transformer blocks connected in sequence;

[0036] Each of the Transformer blocks has the same structure, and includes the following connected in sequence: layer normalization LN layer 1, multi-head self-attention layer, layer normalization LN layer 2 and multi-layer perceptron layer; the input of layer normalization LN layer 1 also forms a residual sum connection with the output of the multi-head self-attention layer; the input of layer normalization LN layer 2 also forms a residual sum connection with the output of the multi-layer perceptron layer.

[0037] Preferably, the MMSE score auxiliary classifier in step S2 includes: an MMSE score prediction branch and an Alzheimer's disease classification branch arranged in parallel;

[0038] The MMSE score prediction branch includes a linear layer FC1 and a linear layer FC2 connected in sequence; the Alzheimer's disease classification branch includes a linear layer FC3;

[0039] The output of the Transformer encoder is subjected to matrix slicing processing to obtain the classification projection vector c after global information mapping and the prediction projection vector m that additionally introduces the MMSE score;

[0040] The predicted projection vector m is input into the MMSE score prediction branch for nonlinear mapping to obtain the predicted value of the MMSE score; the classified projection vector c is input into the Alzheimer's disease classification branch to obtain the predicted value of the class probability of Alzheimer's disease.

[0041] Preferably, the Alzheimer's disease classification model is further provided with a mixed loss function L Hybird , the hybrid loss function L Hybird Including MMSE score loss function L MMSE And the classification loss function L Class , specifically:

[0042] L Hybird =L Class +αL MMSE

[0043]

[0044]

[0045]

[0046] Where N is the total number of batch samples; s i ,p i are the MMSE prediction score and category prediction probability of the i-th sample, respectively, x i ,y i are the MMSE true score and true category label of the i-th sample, represents the true category label of the i-th sample after smoothing; ε represents the label smoothing parameter; α is a hyperparameter.

[0047] The present invention also provides an Alzheimer's disease classification system based on a multi-view hybrid attention network, which applies the above-mentioned Alzheimer's disease classification method based on a multi-view hybrid attention network, including:

[0048] Data acquisition and preprocessing unit: used to acquire sMRI datasets and perform preprocessing;

[0049] The sMRI data set includes three-dimensional sMRI image data of several subjects;

[0050] Model building unit: used to build an Alzheimer's disease classification model;

[0051] The Alzheimer's disease classification model includes: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier connected in sequence;

[0052] The feature extraction backbone network also includes: a feature coarse extraction subnetwork, a plurality of multi-view feature fusion modules connected in sequence, and a cascade multi-scale feature fusion subnetwork; the output of each multi-view feature fusion module is connected to the input of the cascade multi-scale feature fusion subnetwork;

[0053] Model training unit: used to input the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training to obtain a trained Alzheimer's disease classification model;

[0054] Alzheimer's disease classification unit: used to obtain sMRI image data of the subject to be classified, input the sMRI image data of the subject to be classified into the trained Alzheimer's disease classification model for classification, and complete the classification of Alzheimer's disease.

[0055] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0056] The present invention provides an Alzheimer's disease classification method based on a multi-view hybrid attention network. A backbone network based on three-dimensional convolution is constructed to extract local features of sMRI and combined with a Transformer encoder to summarize global brain information. At the same time, an efficient spatial and channel-dimensional weighted hybrid attention module is designed to fuse the spatial structural features of sMRI images from different viewpoints (coronal, sagittal, and axial). A cascaded multi-scale feature fusion strategy is adopted to merge information at different granularity levels to further improve the representation ability and robustness of the overall model. In addition, by introducing the MMSE clinical diagnosis score as additional supervision information and designing a hybrid loss function, the model is guided to learn more efficiently and accurately. The present invention uses the full sMRI image information and MMSE clinical score of the subject, and utilizes the complementarity between the information from different viewpoints in the subject's image and the correlation between pathological areas to model the early diagnosis of AD and the prediction of MCI conversion, thereby realizing the fusion of two modal information of clinical diagnostic numerical features and image features, thereby enhancing the model's learning ability for AD-related identification features.

[0057] The Alzheimer's disease classification model in the present invention also has the following advantages:

[0058] 1) The multi-view feature fusion module in this paper is based on a hybrid attention mechanism, including channel attention and spatial attention mechanisms, to effectively capture the features of atrophic structures under different views and fuse them into a multi-view feature map, thereby producing a richer feature representation;

[0059] 2) The present invention constructs a cascaded multi-scale feature fusion module on the feature extraction backbone network, which is more computationally efficient when processing 3D sMRI images and can also provide a more hierarchical multi-scale feature representation. Specifically, the module fuses feature maps of adjacent scales layer by layer in descending order according to the fine-grained level of the feature maps to achieve multi-scale information fusion of sMRI images, thereby further improving the feature representation capability of the model.

[0060] 3) The present invention designs an auxiliary classifier at the top level of the model, introduces the MMSE score as auxiliary supervisory information, and uses it together with the sMRI category label information for supervised learning of the model; based on the characteristics of these two types of supervisory information, a hybrid loss function is also designed to control the weight ratio of auxiliary information in training by introducing hyperparameters, optimize the gradient distribution of model learning, and help improve the accuracy and reliability of AD diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of an Alzheimer's disease classification method based on a multi-view hybrid attention network provided in Example 1.

[0062] Figure 2 This is a partial pretreatment operation flow chart provided in Example 2.

[0063] Figure 3 This is a structural diagram of the Alzheimer's disease classification model provided in Example 2.

[0064] Figure 4 This is the structural diagram of the feature rough extraction subnetwork provided in Example 2.

[0065] Figure 5 This is the Convnext feature extraction block structure diagram provided in Example 2.

[0066] Figure 6 This is a structural diagram of the multi-view feature fusion module provided in Example 2.

[0067] Figure 7 This is the structure diagram of the hybrid attention block provided in Example 2.

[0068] Figure 8 This is a structural diagram of the transition layer provided in Example 2.

[0069] Figure 9 This is a structural diagram of the cascaded multi-scale feature fusion sub-network provided in Example 2.

[0070] Figure 10 This is a structural diagram of the Transformer encoder provided in Example 2.

[0071] Figure 11 This is a structural diagram of the MMSE score auxiliary classifier provided in Example 2. DETAILED DESCRIPTION

[0072] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0073] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0074] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0075] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0076] Example 1

[0077] like Figure 1 As shown, this embodiment provides an Alzheimer's disease classification method based on a multi-view hybrid attention network, comprising the following steps:

[0078] S1: Acquire sMRI dataset and perform preprocessing;

[0079] The sMRI data set includes three-dimensional sMRI image data of several subjects;

[0080] S2: Establishing a classification model for Alzheimer's disease;

[0081] The Alzheimer's disease classification model includes: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier connected in sequence;

[0082] The feature extraction backbone network also includes: a feature coarse extraction subnetwork, a plurality of multi-view feature fusion modules connected in sequence, and a cascade multi-scale feature fusion subnetwork; the output of each multi-view feature fusion module is connected to the input of the cascade multi-scale feature fusion subnetwork;

[0083] S3: Input the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training to obtain a trained Alzheimer's disease classification model;

[0084] S4: Obtain sMRI image data of the subject to be classified, input the sMRI image data of the subject to be classified into the trained Alzheimer's disease classification model for classification, and complete the classification of Alzheimer's disease.

[0085] In the specific implementation process, the original sMRI image dataset is first obtained and preprocessed, including skull stripping, spatial registration, smoothing and denoising, blank background clipping, and resampling. Then, data enhancement and data augmentation steps are performed. For example, before the data is loaded into the model, it undergoes random rotation, flipping, occlusion, 3D center cropping, and contrast enhancement in sequence with given probabilities to improve the dataset diversity and enhance the model's generalization ability.

[0086] An Alzheimer's disease classification model was established. The model structure mainly consists of three parts: the first part is the 3D convolutional feature extraction backbone network (Backbone), which is responsible for aggregating feature information from shallow to deep layers in the semantic and spatial dimensions of each local structure in the sMRI; the second part is the stacked Transformer encoder network. The high-dimensional local features extracted by the backbone network are serialized and then enter the stacked encoder for context modeling to capture global features; the third part is the MMSE score auxiliary classifier network (MMSE auxiliary classifier), which is composed of a fully connected layer responsible for classification result prediction and a multi-layer perceptron responsible for MMSE score prediction. By combining the MMSE prediction loss function and the classification loss function to construct a hybrid loss function, the MMSE score of each patient is introduced into the model as auxiliary supervision information to improve classification performance. The following is a detailed description of each part of the model;

[0087] First is the feature extraction backbone network of the first part:

[0088] According to relevant research, the brain tissue of patients with early Alzheimer's disease (AD) will experience progressive structural atrophy, involving damage to the cortex and gray matter tissue in different local brain regions such as the occipital lobe, parietal lobe, temporal lobe, and frontal lobe. Since these structures are distributed in different areas of the brain, specific imaging modalities and models need to be considered to provide support in computer-aided diagnosis research. Among many medical scanning imaging technologies, structural magnetic resonance imaging (sMRI) can accurately reflect subtle changes in the brain structure of AD patients due to its non-invasive and high-resolution characteristics. Therefore, the model of this method chooses to use sMRI as the main training data. In addition, in order to enable the model to better learn the spatial information of abnormal brain areas, this method adopts a full-image-level data input method, directly using preprocessed images as the input of the model, and uses three-dimensional convolution to extract features from the images.

[0089] This part is the feature extraction backbone network based on 3DCNN in the overall model. Its main task is to capture various local structural changes related to AD distributed in sMRI and summarize this information into high-dimensional image feature maps for use in the next part of the model. The main structure of this part includes: the network backbone (Stem) responsible for coarse feature extraction and parameter simplification, the transition layer responsible for feature resolution downsampling and dimensionality increase, the stacked multi-view feature fusion module (Multi-View Feature Fusion, MVFF) responsible for aggregating multi-view (sagittal, coronal, and axial) features, the cascading multi-scale fusion module (CMSF) responsible for aggregating multi-scale features of the model, and the convolution block (2x2x2 Conv) responsible for rescaling and dimensionality mapping of the high-dimensional feature map. Overall, this method introduces a multi-view feature fusion module and a cascaded multi-scale feature fusion strategy based on 3DCNN to improve the efficiency and richness of feature extraction of the backbone network, thereby further enhancing the model's ability to identify local atrophic areas in AD patients.

[0090] In the three-dimensional convolutional feature extraction backbone network of this method, when the scale of the sMRI dataset is small, simply using the Transformer encoder for diagnostic modeling will be subject to significant constraints on convergence and performance. Therefore, an inductive bias of the convolutional backbone network is introduced in the pre-model stage to compensate for the modeling difficulties of the Transformer and improve generalization ability. To improve the local feature induction efficiency of the convolutional backbone network, this embodiment constructs it into a four-layer feature pyramid hierarchy and uses the Convnext block as the feature aggregation unit to enhance the local feature representation capability of the backbone network.

[0091] Design of a multi-view feature fusion module in the 3D convolutional feature extraction backbone network: 3D sMRI feature maps from different viewpoints typically have complementary and correlated information. Therefore, this method designs a hybrid attention module with channel and spatial weighting in the backbone network to extract and fuse feature maps from different viewpoints. This enables the model to capture more comprehensive and richer high-dimensional feature representations, improving model performance and robustness.

[0092] Design of a cascaded multi-scale feature fusion subnetwork within a 3D convolutional feature extraction backbone network: Shallow feature maps in deep learning models can describe more detailed brain spatial information, but they lack the ability to generalize semantic information about brain structure. Deep feature maps, on the other hand, have the opposite characteristics. Specifically, to improve information reuse efficiency, this method pairs-wise fuses features of different scales in a cascaded manner, following the order of feature hierarchy from deep to shallow, maximizing the use of useful information in sMRI and improving diagnostic performance.

[0093] This is followed by the Transformer encoder in the second part:

[0094] Studies investigating the pathological progression of Alzheimer's disease (AD) have analyzed changes in neurodegenerative atrophy on longitudinal sMRI and found that as AD progresses from early to late stages, the patterns of neurodegenerative atrophy affecting different brain structures vary, with a temporal sequence and spatial diffusion. For example, atrophy and degeneration typically occur earlier in the left hemisphere than in the right hemisphere, and atrophy and degeneration of the nucleus accumbens can further extend to areas such as the amygdala. These studies suggest that there is a correlation between different local brain tissue structures experiencing atrophy.

[0095] In order to capture this local correlation and improve the diagnostic performance of AD, inspired by the success of the VIT model in computer vision (CV) classification applications, this section uses stacked Transformer encoders to construct the network, and confirms the optimal number of stacked Transformer encoders and the dimension of the local embedding vector through ablation studies; the Transformer encoder is good at analyzing sequence relationships, so before the data stream enters this section, the high-dimensional feature map obtained in the first section is converted into a local representation vector matrix through a dimensional transformation and flattening operation, and each local representation vector corresponds to an element of the target sequence. Through the self-attention mechanism of the Transformer encoder, the model can weight the local representation vectors of the input sMRI one by one and learn the dependencies between each other. After repeated processes of multiple encoders, the correlation representations between sequences are mapped to the classification vector and MMSE score prediction vector for the final category prediction, realizing contextual modeling of all atrophy structural features in the whole brain;

[0096] Construction of the Transformer encoder: In the local information feature map output by the feature extraction backbone network, each voxel corresponds to a large local receptive field in the original sMRI image. Research has shown that different parts of the brain have certain correlation information. Therefore, this is input into the encoder composed of a stack of Transformer blocks. The encoder learns the long-range dependencies between different voxels in the feature map and captures the internal correlations between features to produce a global feature representation of the brain sMRI. The global feature representation vector serves as the input to the model classifier.

[0097] Finally, the MMSE score auxiliary classifier in the third part:

[0098] The Mini-Mental State Examination (MMSE) score is an indicator used to measure cognitive function in patients with Alzheimer's disease (AD). Similar to other clinical assessment scores, it has a certain correlation with changes in the patient's brain structure. In deep learning-based early AD diagnosis research, the MMSE score can be used as a numerical modality feature to support disease classification, and can also be incorporated into the model's decision-making process as additional supervisory information through supervised learning.

[0099] This method constructs a classifier in which the MMSE score is introduced as additional auxiliary information. This classifier uses a parallel multi-layer perceptron (MLP) structure, taking the class label and the true MMSE score as supervision information. This method also uses two loss functions and combines them into a hybrid loss function to implement gradient backpropagation. In this way, the two modalities of clinical diagnostic numerical features and imaging features are fused, thereby enhancing the model's ability to learn discriminative features related to AD.

[0100] Design of an auxiliary classifier for MMSE scores and a hybrid loss function: To improve model performance, a dual-branch auxiliary classifier network was designed. One branch performs normal category prediction, while the other predicts the MMSE score. The true MMSE score serves as additional auxiliary information to guide model learning. A hybrid loss function is designed to integrate these two types of supervisory information. By minimizing the hybrid loss function, the learnable parameters of the entire model are jointly optimized.

[0101] After the model training is completed, sMRI image data of the subject to be classified is obtained, and the sMRI image data of the subject to be classified is input into the trained Alzheimer's disease classification model for classification, thereby completing the classification of Alzheimer's disease;

[0102] The model established by this method can fuse multi-perspective information and perform multi-scale information fusion on the output multi-perspective feature map, and then perform long-range dependency modeling to obtain higher-dimensional global features, enabling the model to characterize the correlation characteristics between different pathological areas of AD patients; in addition, the MMSE clinical examination score (Mini Mental Status Examination) is introduced as auxiliary supervision information, which can effectively improve the reliability of computer-assisted diagnosis and classification of Alzheimer's disease.

[0103] Example 2

[0104] This embodiment provides an Alzheimer's disease classification method based on a multi-view hybrid attention network, comprising the following steps:

[0105] S1: Acquire sMRI dataset and perform preprocessing;

[0106] The sMRI data set includes three-dimensional sMRI image data of several subjects;

[0107] S2: Establishing a classification model for Alzheimer's disease;

[0108] The Alzheimer's disease classification model includes: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier connected in sequence;

[0109] The feature extraction backbone network also includes: a feature coarse extraction subnetwork, a plurality of multi-view feature fusion modules connected in sequence, and a cascade multi-scale feature fusion subnetwork; the output of each multi-view feature fusion module is connected to the input of the cascade multi-scale feature fusion subnetwork;

[0110] S3: Input the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training to obtain a trained Alzheimer's disease classification model;

[0111] S4: Obtain sMRI image data of the subject to be classified, input the sMRI image data of the subject to be classified into the trained Alzheimer's disease classification model for classification, and complete the classification of Alzheimer's disease.

[0112] In the specific implementation process, first obtain the data set and preprocess it:

[0113] The datasets used in this method are all from the Alzheimer's Disease Neuroimaging Initiative (ADNI). The downloaded data include 4796 T1-weighted sMRI images of 1570 subjects from three different ADNI study phases (ADNI-1 / GO, ADNI-2, and ADNI3). Based on AD clinical diagnostic criteria, such as the Mini-Mental State Examination (MMSE) score and the Clinical Dementia Rating (CDR), ADNI divided these subjects into three groups: AD (patients with Alzheimer's disease), NC (healthy control), and MCI (patients with mild cognitive impairment). In the initial screening, subjects diagnosed with MCI were divided into pMCI (developmental mild cognitive impairment) and sMCI (stable mild cognitive impairment) groups based on whether they converted to AD during the subsequent 36-month follow-up. Detailed demographic information is shown in Table 1:

[0114] Table 1 Demographic information

[0115]

[0116] The original structural MRI images downloaded from ADNI are usually not directly usable for research and analysis. First, these images are acquired by scanners from different manufacturers (such as Siemens, Philips, etc.), and their image quality and specifications vary. Second, each subject's head size, shape, and position vary greatly, resulting in each subject's image not being in the same spatial coordinate system. Finally, the non-brain tissue and background contained in the original sMRI will cause unnecessary waste of computing resources; these interference factors outside the brain structure will affect the feature learning and efficiency of the model. Therefore, preprocessing of the original sMRI is necessary; the main preprocessing steps are as follows: Figure 2 As shown in the figure, the first three preprocessing steps are completed in the batch processing process of cat12 and spm toolbox on the matlab platform. The details of the steps are as follows:

[0117] 1) Skull stripping: Use the "APRG (adaptive probability region-growing)" algorithm to separate brain tissue and skull structures;

[0118] 2) Affine registration: We used the MNI152 as the standard brain space template to achieve spatial normalization. The resulting image had a voxel spacing of 1.5 mm × 1.5 mm × 1.5 mm and a size of 121 px × 145 px × 121 px.

[0119] 3) Image smoothing: A 4×4×4 Gaussian kernel function was used to perform spatial filtering on the sMRI to remove high-frequency noise in the image and improve the signal-to-noise ratio;

[0120] 4) Background space removal: By calculating the minimum distance of the current sMRI from the image edge in the X, Y, and Z directions, a mask of size 101px × 125px × 103px is generated to remove irrelevant background information;

[0121] 5) Resampling: To facilitate model analysis, each sMRI image was resampled to 112px × 112px × 112px by trilinear interpolation;

[0122] 6) Format conversion: Convert sMRI data in NIfTI format to NumPy format to facilitate model loading and processing;

[0123] After that, data augmentation is performed. The Alzheimer's disease diagnosis method used in this method is based on the full-image-level approach of sMRI. A single image corresponds to a single sample. Therefore, in order to fully converge the model, reduce overfitting, and improve robustness, data augmentation steps are used during model training to increase sample diversity. The following are the specific data augmentation steps used:

[0124] 1) The image was randomly rotated by ±15 degrees along the coronal, sagittal, and axial planes with a probability of 0.5;

[0125] 2) The image is randomly flipped vertically with a probability of 0.5;

[0126] 3) Randomly select a specified number and size of local areas on the image for masking, with a size of 10px×10px×10px, a number of 3, and a probability of 0.5;

[0127] 4) The image is randomly cropped from the 3D center to a size of 80px×80px×80px, and then resampled to the original size using bilinear interpolation with a probability of 0.5;

[0128] 5) Random contrast enhancement of images, using a uniformly distributed random number as the contrast scaling factor, with a value in the range of (0.8, 1.3), and limiting the voxel intensity to the range of (-1, 1);

[0129] After the dataset was preprocessed, it was randomly divided into three subsets: training set, validation set, and test set in a ratio of 4:1:1. Since each subject contains scans at one or more time points, in order to prevent data leakage, the division process ensured that all scans at each time point for each subject were only in the same subset, and a five-fold cross-validation evaluation method was used.

[0130] Establishing Alzheimer's disease classification model: The model overview of this method is as follows Figure 3 As shown in the figure, the following will introduce the specific implementation of the feature extraction backbone network (Backbone), Transformer encoder and MMSE score auxiliary classifier (MMSE auxiliary classifier) ​​in turn; the feature extraction backbone network contains three submodules: feature coarse extraction subnetwork (Stem), multi-view feature fusion module (MVFF), transition layer (Transition layer) and cascade multi-scale feature fusion subnetwork (CMSF subnetwork); in addition, because the Convnext block combines effective model optimization ideas such as the inverse bottleneck structure and depth-wise separable convolution in CNN, it shows strong feature aggregation ability. Therefore, in this method, it is applied to the feature extraction backbone network of the model and serves as a feature extraction subunit; the implementation of each part is introduced in turn below;

[0131] Part 1, feature extraction backbone network:

[0132] 1) Feature coarse extraction sub-network (stem):

[0133] like Figure 4As shown in Figure 2, after the sMRI data with a size of 1×112×112×112 (C×H×W×D) is input into the model, shallow feature aggregation is completed in the stem through two steps;

[0134] In step 1, a shallow feature extraction is performed through a 3D convolution layer with a convolution kernel size of 4×4×4, a stride of 2, and an output channel of 32, a BatchNorm (BN) layer, and a ReLU activation function to obtain the initial level feature map of the model.

[0135] Step 2: The output feature map of the previous step is passed through two stacked Convnext blocks for shallow feature aggregation; the Convnext block is as follows: Figure 5 As shown in the figure, the main path of the Convnext block consists of: a 3D convolutional layer with a convolution kernel size of 7×7×7 and a constant output channel, a LayerNorm (LN) layer; a 1×1×1 convolutional layer with 4C output channels (where C is the input channel of the block structure) and a GeRU activation function; a 1×1×1 convolutional layer with C output channels; in addition, the input and output are connected by a bypass residual connection with element-by-element addition; the Convnext block feature aggregation process is expressed as follows:

[0136] Conv next (x) = Conv s (Conv e (LN(Conv g (x))))

[0137] Among them, x is the input feature map, Conv g 、Conv e and Conv s The corresponding convolutional layers respectively;

[0138] 2) Multi-view feature fusion module (MVFF):

[0139] Different perspectives of 3D sMRI feature maps usually have complementary and relevant information. Fusion of feature maps from different perspectives can enable the model to capture more comprehensive and richer high-dimensional feature representations and enhance model robustness. To this end, this method designs an MVFF module to fuse the multi-perspective features of the input feature map to jointly locate the lesion area with multiple perspectives. The MVFF structure is as follows: Figure 6 As shown;

[0140] Step 3: Assume that the input sagittal plane view feature map of this level is After the dimension transformation, the remaining two view feature maps are obtained, that is, Get the coronal view feature map f (C) , for the same reason Get the axial view feature map f(A) , as follows:

[0141] f (C) =T C (f (S) ),f (A) =T A (f (S) )

[0142] Where, T A and T C They are respectively expressed as the dimensional transformation process of the coronal and axial plane perspectives;

[0143] Step 4: Design three parallel branches to aggregate the feature maps of each view, where each branch consists of two parts:

[0144] Part 1 is the convolutional stacking block responsible for extracting fine-grained features for each view (corresponding to AV Conv, SV Conv, and CVConv). They are all composed of 3×3×3 and 1×1×1 convolutional layers connected in sequence, where the 3×3×3 convolution is followed by a BN layer and ReLU nonlinear activation;

[0145] Part 2 is a simple and effective attention module that generates feature representations with spatial and channel attention perception to focus feature maps from different perspectives on the key lesion area. The attention module consists of a spatial domain attention branch and a channel domain attention branch. The specific structure is as follows: Figure 7 As shown in the figure, the design of the spatial domain attention branch is inspired by the inverse bottleneck structure of MobileNet V2 and adopts a similar structure. The attention map output by the spatial domain attention branch of any view feature map is defined as

[0146] SA (*) =BN(Conv g (Conv e2 (Conv e1 (Conv a (f (*) )))))

[0147] Where (*)∈[S,A,C] is used to identify different perspectives, Conv a Indicates that the 1×1×1 convolution layer is responsible for feature integration and dimensionality reduction mapping. Here, a channel scaling ratio r is introduced to make Conv a The output channel is reduced to C′ (C′=C / r), and in this embodiment, r is set to 4; Conv e1 and Conv e2 Represents the 3×3×3 convolutional layer responsible for extracting spatial context information, Conv gRepresents a 1×1×1 convolutional layer, which is responsible for spatial encoding to obtain attention scores, and adds a BN layer for batch normalization; a lightweight channel attention module ECA is used on the channel domain attention branch to generate an attention map, which only considers each channel and a certain number of adjacent channels for attention encoding, and has efficient cross-channel interaction capabilities. Any view feature map is output by this branch as a channel attention map

[0148] CA (*) =Conv 1D (Avgpool(f (*) ))

[0149] Where Avgpool represents the average pooling along the channel axis, which perceives the importance of the feature map channel; Conv 1D It represents an adaptive one-dimensional convolution that realizes channel interaction. Its convolution kernel k changes adaptively with the input channel, thereby focusing on more useful channel information. It is described as the formula:

[0150]

[0151] Where c represents the input channel; β and γ represent adjustable parameters, which are set to 2 and 1 respectively in this embodiment; |t| odd Indicates that the result takes the odd number closest to t as the convolution kernel size;

[0152] After obtaining the attention scores of the spatial domain and channel domain, they are integrated by adding them element by element, and then normalized by the Sigmoid activation function to obtain the attention weighted score of the mixed domain. The weighted score is directly multiplied with the input view feature map to obtain a feature map with spatial and channel perception.

[0153] f HA(*) =f (*) *(CA (*) +SA (*) )

[0154] Step 5: After completing the feature extraction of the three perspectives, the complementary perspective information is spliced ​​in the channel domain to obtain a more powerful feature representation; it is worth noting that in order to align the feature space of the complementary perspective feature map, the coronal plane perspective feature map f is centered before splicing. (C) and axial view feature diagram f (A) Perform dimensional inverse transformation; finally, the multi-view fusion feature output of the MVFF module It can be expressed as:

[0155]

[0156] Where Conv is a 1×1×1 convolutional layer responsible for channel downsampling, with 3C input channels and C output channels; Concat represents the channel concatenation operation; and They are the dimensionality recovery inverse transformation processes corresponding to the two view feature maps;

[0157] 3) Transition layer:

[0158] In the feature extraction backbone network of this embodiment, three MVFF modules are stacked to fuse the different perspective information of the feature map of this level. However, the increase in network depth will lead to a sharp increase in the number of network parameters, which may cause overfitting problems when training with a limited amount of training data. To address this, before the feature map enters the MVFF module, adding a transition layer for downsampling can achieve parameter simplification; the transition layer is constructed as follows: Figure 8 As shown, assuming that the feature map of the previous level is First, it is raised to 2C after passing through the convolution layer, then downsampled twice by maximum pooling, and finally a Convnext block is used to aggregate the features to obtain the feature map of this level. It is defined as the following formula:

[0159] f i =Conv next (Maxpool(Conv e (f i-1 )))

[0160] In the formula, Maxpool represents the maximum pooling downsampling (MAP), Conv e represents a 1×1×1 convolutional layer;

[0161] 4) Cascaded multi-scale feature fusion subnetwork (CMSF subnetwork):

[0162] In many classification tasks based on convolutional neural networks (CNNs), the strategy of fusing feature maps of information at different scales has been proven to effectively improve model performance. After extracting multiple perspective features from sMRI images through MVFF, the output feature maps of each MVFF module are saved in sequence. Due to the downsampling of the transition layer, a set of feature maps of different scales can be obtained. i},in:

[0163]

[0164] Where s0 and c0 are the feature map size and number of channels of the first MVFF module output feature map F1, and it is known that s0 = 28, c0 = 64; {F i} contains feature maps with scales from large to small and feature levels from shallow to deep. It is worth noting that shallow feature maps can describe more detailed brain spatial information, but the generalization of semantic information is insufficient, while deep feature maps are the opposite. Therefore, the fusion of {F i All scale features in} are beneficial to improving the model representation ability;

[0165] The cascaded multi-scale fusion sub-network (CMSF) structure used to fuse these different scale feature maps is as follows: Figure 9 As shown in Figure 2, in order to balance the representation ability of multi-scale fusion features and the amount of parameters introduced, feature maps of different scales are sequentially upsampled and combined, and then fused with the feature map of the next scale in a cascade information flow manner. The final multi-scale fusion feature is represented as

[0166] F MS =F3+P1

[0167] P1=Concat(M1,M2)

[0168] M1=y(Concat(F2,x(F3)))

[0169] M2=y(Concat(F1,x(Concat(F2,x(F3)))))

[0170] Where x represents the process of trilinear interpolation upsampling and 1×1×1 convolution channel alignment in sequence; y represents the process of convolution operation and average pooling downsampling in sequence. It is worth noting that due to the different granularity of brain structure features in F1 and F2, convolution layers with kernel sizes of 2×2×2 and 1×1×1 are used for local feature induction before downsampling in y. The detailed steps are explained as follows:

[0171] Step 6: First, the number of channels of F3 is reduced to the same as F2 through a 1×1×1 convolution layer, and then an interpolation operation is performed to align the spatial size of the features, and finally feature splicing is performed. This operation process is composed of Implementation; Finally, the intermediate output feature map M1 is obtained after the feature maps F3 and F2 are fused;

[0172] Step 7: Similar to step 6, the intermediate feature maps M1 and F1 are fused to obtain the intermediate output feature map M2;

[0173] Step 8: In order to make full use of the feature information at different scales, the intermediate feature maps M1 and M2 are aligned in feature space and channel dimension by δ, and then channel splicing is performed to obtain P1. Finally, P1 is added element by element to the deep feature map F3 to obtain a fusion feature representation F containing information at multiple scales. MS ;

[0174] Part 2, Transformer encoder:

[0175] After the feature extraction backbone network completes the local feature extraction of sMRI, it will obtain the output feature with a resolution of 256×7×7×7 Figure X , where each voxel corresponds to a large local receptive field in the original sMRI image; X is input into an encoder composed of a stack of Transformer blocks to learn the long-range dependencies between different voxels in the feature map and capture the internal correlation between features to produce a global feature representation of brain sMRI; Figure 10 As shown in the figure, the Transformer encoder consists of a cascade of consecutive Transformer blocks. Each Transformer block includes a multi-head self-attention layer (MSA) and a multi-layer perceptron (MLP), and the modules all apply layer normalization (LN) and skip connections. Here, the number of Transformer blocks and heads is 5 and 12 respectively.

[0176] Step 9: Before entering the Transformer encoder, the channel dimension is increased to the Transformer embedding dimension 576 by a 2×2×2 convolution layer. After feature map flattening and dimension transformation, in addition to the classification projection vector c, the MMSE score prediction projection vector m is additionally introduced, where Finally, after position encoding, we get the sequence matrix consisting of token embedding vectors (Token) Where N = 7 × 7 × 7 + 2, each token embedding vector can reflect the local features of the input sMRI image;

[0177] Step 10: Input Z to the Transformer encoder, and the token vectors learn the long-range dependencies between each other through layer-by-layer self-attention weighting; the output Z of the lth Transformer block is l Expressed as:

[0178] Z l =MLP(LN(Z′ l ))+Z′ l

[0179] Z′ l =MSA(LN(Z l-1 ))+Z l-1

[0180] Among them, each head in MSA performs self-attention weighted calculation separately to learn richer representation capabilities in different representation spaces. The calculation of SA is expressed as:

[0181]

[0182] Where Q, K, and V are the query matrix, key matrix, and value matrix respectively; d is the vector dimension of the key matrix; here Q, K, and V are all input token sequence matrix Z by the lth Transformer block l-1 After linear transformation, it is expressed as:

[0183] Q l =Z l-1 W Q ,K l =Z l-1 W K ,V l =Z l-1 W V

[0184] Where the linear transformation is achieved through the linear layer, W Q , W K , W V Corresponding to its trainable parameter matrix;

[0185] For MSA, by slicing Q, K, V into h groups (called h heads), and performing self-attention weighting in each of the h heads, we can obtain different subspace representation capabilities. The output of the i-th head of the l-th Transformer block can be expressed as:

[0186] head l i =Attention(Q l i ,K l i ,V l i )

[0187] Here i∈[1,...,12], after getting the output of each head, they are merged and passed through the linear layer W o Perform projection mapping to obtain the output Multihead of MSA l :

[0188] Multihead l =Concat(head l 1 ,...,head l 12 )W o

[0189] Step 11: The Token sequence matrix after global encoding is obtained through the Transformer encoder output

[0190] Part III, MMSE auxiliary classifier:

[0191] In order to obtain supervisory information sources other than sMRI images, the clinical diagnosis MMSE score is introduced as additional auxiliary information to guide model learning and improve model performance. Specifically, this embodiment designs an auxiliary classifier network with a dual-branch structure, in which one branch performs model classification and the other branch is used to predict the MMSE score value, thereby obtaining additional supervisory information. The auxiliary classifier is as follows: Figure 11 As shown;

[0192] Step 12: First, the output sequence vector matrix of the Transformer encoder (where N = 7 × 7 × 7 + 2) to perform matrix slicing to obtain the classification projection vector c after global information mapping and the additional MMSE score prediction projection vector

[0193] Step 13: The vector m is input into the MLP branch consisting of the linear layers FC1 and FC2 for nonlinear mapping to obtain the predicted value of MMSE, while the vector c is directly inferred by the linear layer FC3 to obtain the class score;

[0194] In step 14, a hybrid loss function is designed to integrate the two types of supervision information. By minimizing the hybrid loss function, the learnable parameters of the overall model are jointly optimized. Specifically, the mean square error (MSE) is used as the loss function to supervise the prediction of the MMSE score, and the label smoothing cross-entropy (LSCE) is used as the classification loss function to enhance the robustness of the model. Then, the two loss functions are added together to obtain the hybrid loss function L Hybird :

[0195] L Hybird =L Class +αL MMSE

[0196]

[0197]

[0198]

[0199] Where N is the total number of batch samples; s i ,p i are the MMSE prediction score and category prediction probability of the i-th sample, respectively, xi ,y i Corresponding to the MMSE true score and true category label of the i-th sample, represents the true category label of the i-th sample after smoothing; ε represents the label smoothing parameter; the hyperparameter α is the multiplication coefficient of the mean square loss function, which is used to adjust the weight of the auxiliary supervision information; in this embodiment, ε and α are set to 0.1 and 0.5 respectively;

[0200] Afterwards, we conducted experiments to verify that the model was implemented based on the Pytorch 1.8.1 framework. All experiments were conducted on a GPU configured with NVIDIA RTX. TM This was performed using Python 3.8.0 on a Linux server with an A5000 24G GPU. The ADNI dataset was randomly divided into training, validation, and test sets in a 4:1:1 ratio. The model was also tested on the OASIS dataset to further evaluate the generalization performance of the proposed model on heterogeneous data. In general, the validation experiment in this example studied two different Alzheimer's disease diagnosis tasks: AD classification (AD vs. NC) and MCI conversion prediction (sMCI vs. pMCI).

[0201] The training strategy includes setting the number of iterations to 100, the batch size to 8, selecting the Adam optimizer with a weight decay of 1e-4, setting the initial learning rate lr to 1e-4 at the beginning of training, using a learning rate decay strategy with a fixed step size of 15 and a decay coefficient of 0.9, and limiting the minimum learning rate to 1e-5; for model parameters, on the feature extraction backbone network of the model, the channel dimension is set to [32, 64, 128, 256], the number of stacked Transformer blocks on the Transformer global encoder is set to 5, and the dimension of the embedding vector is set to 576;

[0202] In the model classification performance evaluation, the evaluation indicators used include accuracy (ACC), sensitivity (SEN), specificity (SPE), and area under the receiver operating characteristic curve (AUC). These indicators are defined as:

[0203]

[0204]

[0205]

[0206] Where TP, TN, FP and FN represent true positive value, true negative value, false positive value and false negative value respectively;

[0207] In order to demonstrate the significant performance of the model, this embodiment compares the performance of the proposed method with eleven recently proposed methods. The compared methods are divided into 3DCNN, Transformer and hybrid methods combining CNN and visual transformer according to the basic model used, and divided into ROI, 2D slice, 3D image block and 3D full image according to the form of data features. The selected comparison methods are briefly described below:

[21] and

[22] The whole brain image is pre-segmented into multiple anatomical regions of interest based on the brain atlas, and then ROI features such as local image blocks of the hippocampus, cortical volume and thickness are extracted, and then 3DCNN or SVM is used as a classifier to obtain the classification results;

[23] LSTM is used to locate the local area of ​​significant atrophy in the sMRI two-dimensional slice, and the local features of each local area are learned through a convolutional subnetwork, and then the global slice features are integrated for AD classification;

[24] The slice sequence of each subject is input into multiple convolutional subnetworks to capture relevant information between slices;

[25]

[26] used statistical methods such as the Student t-test to obtain the discriminative locations in the AD and NC brains, and then obtained fixed-size 3D image blocks based on these locations, and applied the multi-instance learning (MIL) method to conduct AD diagnosis research;

[27] learned the channel attention scores of multi-view two-dimensional slices by aggregating each 2DCNN branch, and then performed channel weighting on the output feature map of the 3DCNN branch to improve accuracy;

[28] designed a global subnetwork based on 3DCNN and a multi-instance learning subnetwork in parallel, and then improved the generalization ability through a multi-level feature adversarial learning module;

[29] Based on the DAT model, adaptive positioning of the atrophic brain area was completed more accurately; The work of

[30] and

[31] is similar to this method, and the local feature extraction and global feature modeling from sMRI are completed by combining 3DCNN and Transformer encoder to achieve more powerful feature representation;

[0208] To ensure fairness in the comparison, only methods that used the ADNI dataset for model training and performance testing were compared; the compared classification tasks included AD diagnosis and MCI conversion prediction, where indicators such as ACC, SEN, SPE, and AUC were all average values ​​of five-fold cross-validation; as shown in Table 2, this method achieved the highest AUC, ACC, and SPE in the AD diagnosis task, and the highest AUC in the MCI conversion prediction task; first, compared with ROI-based and 2D slice-based methods, this method takes the entire image as input and uses 3DCNN to capture richer spatial and semantic information, thereby enabling more feature information to participate in model decision-making; secondly, compared with models based on 3D image blocks Since multi-instance learning for each local image block ignores the correlation between local regions, this method combines 3DCNN with a Transformer encoder to model the dependency relationship between lesion areas in brain space. Compared with the full-image-based method, this method still achieves competitive results. This is due to the introduction of multi-view feature fusion modules and multi-scale feature fusion modules on the 3DCNN backbone, which enables the model to capture complementary information from multiple feature map perspectives and multi-fine-scale feature representations, and can more effectively identify lesion areas. In addition, the introduction of MMSE clinical scores enables the model to obtain supervisory information outside of sMRI images, which can provide effective guidance for model learning.

[0209] Table 2 Performance comparison of the proposed model with other models on the ADNI data test set

[0210]

[0211] To verify the effectiveness of the model, this embodiment also conducted a series of ablation experiments on the entire model. As shown in Table 3, for a more concise representation, the multi-view feature fusion module, cascaded multi-scale feature fusion module, Transformer encoder, and MMSE score auxiliary classifier components in the model are respectively denoted as "MVFF", "CMSF", "VITE", and "MAC", while the feature extraction backbone network after removing "MVFF" and "CMSF" is denoted as "CN". This embodiment conducted ablation experiments in two aspects:

[0212] 1) Taking "CN+VITE" as the baseline model for the ablation process, the core components "MVFF", "CMSF" and "MAC" are added one by one to evaluate their respective contributions to model diagnosis. As shown in Table 3, compared with the baseline "CN+VITE", the combined models "MVFF+CN+VITE" and "CMSF+CN+VITE" have significant performance improvements. It is worth noting that the addition of the multi-view feature fusion module has achieved an improvement of 6.72% and 8.5% in the AUC and ACC indicators of the AD classification task, respectively, and an improvement of 10.43% and 7.62% in the MCI conversion prediction task. This fully demonstrates that the multi-view information of the fused feature map can greatly improve the performance of the model. Significantly improved model performance; by adding the CMSF module to MVFF, the "MVFF+CMSF+CN+VITE" model improved AUC and ACC by 0.92%, 1.65%, 1.15%, and 2.34% in the two tasks, respectively. This demonstrates the effectiveness of the CMSF module in fusing different levels of feature information in sMRI images. In addition, the MMSE clinical diagnostic score, as additional supervisory information, provided guidance for model learning, as demonstrated in the results of the "MVFF+CMSF+CN+VITE+MAC" (Proposal) model. By introducing the MMSE score prediction loss, the model learning approached the optimal point more easily, resulting in the optimal performance of the final model.

[0213] 2) To demonstrate that the model constructed using 3DCNN and Transformer encoders outperforms the model constructed using only one of the two, we first trained a standard model (3DVIT) that best matches the depth and scale of this model on the same dataset (i.e., it contains 12 attention encoding layers and 768 embedding dimensions), denoted as "3DVIT"; secondly, we also trained a model in which an MLP consisting of two fully connected layers replaces "VITE" and "MAC", denoted as "MVFF+CMSF+CN+MLP"; then we compared the two models with the combined model "MVFF+CMSF+CN+VITE", and the results showed that the performance of both models was far lower than that of the combined model, which illustrates the limited convergence of "3DVIT" on small datasets and its insufficient ability to generalize local features, and the lack of effective context modeling capability of "MVFF+CMSF+CN+MLP";

[0214] Table 3 Details of ablation experiment results

[0215]

[0216] This method proposes an early Alzheimer's disease auxiliary diagnosis model based on a multi-view hybrid attention convolutional neural network assisted by a Transformer. In this model, a feature extraction backbone network based on 3D convolution is constructed, which includes an adaptive and efficient spatial and channel-dimensional weighted hybrid attention module to fuse the feature information of sagittal, coronal and axial views of sMRI to more accurately locate the subtle atrophy areas of the brain in AD patients. At the same time, a cascaded multi-scale feature fusion strategy is adopted to reuse feature maps of different granularity levels to further enhance the feature representation of the backbone network. Secondly, a Transformer encoder network is introduced into the model to perform contextual modeling on the output feature maps of the backbone network to learn the dependencies between pathological areas across the whole brain and achieve more generalized feature modeling. In addition, the MMSE clinical diagnosis score is introduced as additional supervision information, and a hybrid loss function is designed to guide the model to learn more efficiently and accurately. In summary, the model proposed by this method is an end-to-end model based on whole-brain imaging features. The evaluation results on the ADNI dataset show that compared with existing studies, the model performs well in both AD diagnosis and MCI conversion prediction tasks.

[0217] The same or similar reference numerals correspond to the same or similar components;

[0218] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0219] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for Alzheimer's disease classification based on multi-view hybrid attention network, characterized by: The following steps are involved: S1: Acquire sMRI dataset and perform preprocessing; The sMRI data set includes three-dimensional sMRI image data of several subjects; S2: Establishing a classification model for Alzheimer's disease; The Alzheimer's disease classification model includes: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier connected in sequence; The feature extraction backbone network also includes: a feature coarse extraction subnetwork, a plurality of multi-view feature fusion modules connected in sequence, and a cascade multi-scale feature fusion subnetwork; the output of each multi-view feature fusion module is connected to the input of the cascade multi-scale feature fusion subnetwork; The multi-view feature fusion module includes parallel branches 1, 2 and 3, where branches 1, 2 and 3 correspond to sagittal, coronal and axial plane perspectives, respectively. The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence; The structure of each convolution block is the same, and includes: a 4×4×4 convolution layer, a batch normalization BN layer, a ReLU activation layer, and a 1×1×1 convolution layer connected in sequence; The structure of each hybrid attention block is the same, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the spatial domain attention branch includes: a 1×1×1 convolutional layer, a 3×3×3 convolutional layer, a 3×3×3 convolutional layer, a 1×1×1 convolutional layer and a batch normalization BN layer connected in sequence; The channel-domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence; The outputs of the spatial domain attention branch and the channel domain attention branch are added and normalized by the Sigmoid activation function to obtain the attention weighted score of the hybrid domain, the attention weighted score of the hybrid domain is weighted multiplied by the input of the hybrid attention block, and the weighted multiplication result is used as the output of the hybrid attention block; The outputs of branches 1, 2 and 3 are spliced ​​and fused as the output of the multi-view feature fusion module; S3: Input the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training to obtain a trained Alzheimer's disease classification model; S4: Obtain sMRI image data of the subject to be classified, input the sMRI image data of the subject to be classified into the trained Alzheimer's disease classification model for classification, and complete the classification of Alzheimer's disease.

2. The Alzheimer's disease classification method based on multi-view hybrid attention network according to claim 1 is characterized in that: The pre-processing in step S1 includes: The three-dimensional sMRI image data of each subject are subjected to skull stripping, spatial registration, smoothing and denoising, blank background removal, resampling, format conversion, data enhancement and data partitioning operations to complete preprocessing.

3. The Alzheimer's disease classification method based on multi-view hybrid attention network according to claim 1 is characterized in that: The feature coarse extraction subnetwork in step S2 includes: a 3D convolution block, a Convnext feature extraction block 1 and a Convnext feature extraction block 2 connected in sequence; The structure of the 3D convolutional block includes: a 4×4×4 convolutional layer, a batch normalization (BN) layer, and a ReLU activation layer connected in sequence; The structure of each Convnext feature extraction block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext feature extraction block also form a residual sum connection through a bypass.

4. The Alzheimer's disease classification method based on multi-view hybrid attention network according to claim 1, characterized in that: In step S2, a transition layer is provided before each of the multi-view feature fusion modules, and the transition layer includes: a 1×1×1 convolution layer, a batch normalization (BN) layer, a ReLU activation layer, a maximum pooling downsampling layer, and a Convnext block connected in sequence; The Convnext block includes: a 7×7×7 convolutional layer, a layer normalization LN layer, a 1×1×1 convolutional layer, a GeRU activation layer and a 1×1×1 convolutional layer connected in sequence; the input and output of the Convnext block also form a residual sum connection through a bypass.

5. The Alzheimer's disease classification method based on multi-view hybrid attention network according to claim 1 is characterized in that: In step S2, the output of each multi-view feature fusion module is recorded as: , n is a positive integer, indicating the number of multi-view feature fusion modules; In the cascaded multi-scale feature fusion sub-network, After 1×1×1 convolution layer and upsampling layer, Perform feature splicing and fusion to obtain the intermediate fusion feature map ; The intermediate fusion feature map After 1×1×1 convolution layer and upsampling layer, Perform feature splicing and fusion to obtain the intermediate fusion feature map ; Repeat the above fusion steps recursively to obtain the intermediate fusion feature map ; The intermediate fusion feature map After convolution operation and average pooling downsampling operation, it is fused with the intermediate feature map Perform secondary fusion to obtain the intermediate fusion feature map ; The intermediate fusion feature map After convolution operation and average pooling downsampling operation, it is fused with the intermediate feature map Perform secondary fusion to obtain the intermediate fusion feature map ; Repeat the above secondary fusion steps recursively to obtain the intermediate fusion feature map of the last secondary fusion , the intermediate fusion feature map and An addition operation is performed, and the addition result is used as the output of the cascaded multi-scale feature fusion sub-network.

6. The Alzheimer's disease classification method based on multi-view hybrid attention network according to claim 1, characterized in that: The Transformer encoder in step S2 includes: a flat linear layer and a plurality of Transformer blocks connected in sequence; Each of the Transformer blocks has the same structure, and includes the following connected in sequence: layer normalization LN layer 1, multi-head self-attention layer, layer normalization LN layer 2 and multi-layer perceptron layer; the input of layer normalization LN layer 1 also forms a residual sum connection with the output of the multi-head self-attention layer; the input of layer normalization LN layer 2 also forms a residual sum connection with the output of the multi-layer perceptron layer.

7. The Alzheimer's disease classification method based on multi-view hybrid attention network according to claim 1, characterized in that: The MMSE score auxiliary classifier in step S2 includes: an MMSE score prediction branch and an Alzheimer's disease classification branch arranged in parallel; The MMSE score prediction branch includes sequentially connected linear layers and linear layers ; The Alzheimer's disease classification branch includes a linear layer ; The output of the Transformer encoder is subjected to matrix slicing processing to obtain the classification projection vector c after global information mapping and the prediction projection vector m that additionally introduces the MMSE score; The predicted projection vector m is input into the MMSE score prediction branch for nonlinear mapping to obtain the predicted value of the MMSE score; the classified projection vector c is input into the Alzheimer's disease classification branch to obtain the predicted value of the class probability of Alzheimer's disease.

8. The Alzheimer's disease classification method based on a multi-view hybrid attention network according to any one of claims 1 to 7, characterized in that: The Alzheimer's disease classification model is also provided with a mixed loss function , the hybrid loss function Including MMSE score loss function And the classification loss function , specifically: in, is the total number of batch samples; are the MMSE prediction score and category prediction probability of the i-th sample, are the MMSE true score and true category label of the i-th sample, Represents the true category label of the i-th sample after smoothing; represents the label smoothing parameter; is a hyperparameter.

9. An Alzheimer's disease classification system based on a multi-view hybrid attention network, applying the Alzheimer's disease classification method based on a multi-view hybrid attention network as described in any one of claims 1 to 8, characterized in that: include: Data acquisition and preprocessing unit: used to acquire sMRI datasets and perform preprocessing; The sMRI data set includes three-dimensional sMRI image data of several subjects; Model building unit: used to build an Alzheimer's disease classification model; The Alzheimer's disease classification model includes: a feature extraction backbone network, a Transformer encoder, and an MMSE score auxiliary classifier connected in sequence; The feature extraction backbone network also includes: a feature coarse extraction subnetwork, a plurality of multi-view feature fusion modules connected in sequence, and a cascade multi-scale feature fusion subnetwork; the output of each multi-view feature fusion module is connected to the input of the cascade multi-scale feature fusion subnetwork; The multi-view feature fusion module includes parallel branches 1, 2 and 3, where branches 1, 2 and 3 correspond to sagittal, coronal and axial plane perspectives, respectively. The structure of the branch 1 includes: a convolution block 1 and a hybrid attention block 1 connected in sequence; the structure of the branch 2 includes: a convolution block 2 and a hybrid attention block 2 connected in sequence; the structure of the branch 3 includes: a convolution block 3 and a hybrid attention block 3 connected in sequence; The structure of each convolution block is the same, and includes: a 4×4×4 convolution layer, a batch normalization BN layer, a ReLU activation layer, and a 1×1×1 convolution layer connected in sequence; The structure of each hybrid attention block is the same, and includes: a spatial domain attention branch and a channel domain attention branch arranged in parallel; the spatial domain attention branch includes: a 1×1×1 convolutional layer, a 3×3×3 convolutional layer, a 3×3×3 convolutional layer, a 1×1×1 convolutional layer and a batch normalization BN layer connected in sequence; The channel-domain attention branch includes: an adaptive pooling layer and a one-dimensional convolution layer connected in sequence; The outputs of the spatial domain attention branch and the channel domain attention branch are added and normalized by the Sigmoid activation function to obtain the attention weighted score of the hybrid domain, the attention weighted score of the hybrid domain is weighted multiplied by the input of the hybrid attention block, and the weighted multiplication result is used as the output of the hybrid attention block; The outputs of branches 1, 2 and 3 are spliced ​​and fused as the output of the multi-view feature fusion module; Model training unit: used to input the preprocessed sMRI dataset into the Alzheimer's disease classification model for iterative training to obtain a trained Alzheimer's disease classification model; Alzheimer's disease classification unit: used to obtain sMRI image data of the subject to be classified, input the sMRI image data of the subject to be classified into the trained Alzheimer's disease classification model for classification, and complete the classification of Alzheimer's disease.

Images