Alzheimer disease diagnosis method based on FCNFormer network

Abstract

The invention provides an Alzheimer disease diagnosis method based on an FCNFormer network, which is used for early diagnosis and pathological area visualization of Alzheimer disease, relates to the technical field of image processing and mode recognition, introduces a three-dimensional position coding mechanism based on element spatial positions, enhances the spatial perception ability of a model to a brain structure, and improves the diagnosis accuracy of the Alzheimer disease. The model can learn the unique performance of each position of the brain in the AD pathological process; a self-attention mechanism of Transform is integrated into an FCN network, and a long-distance dependency relationship between image blocks is captured, so that the network can better extract cross-scale features, and the overall distribution rule of diseases in the brain can be more easily captured; an MLP model fused with a multi-mode attention mechanism is introduced, data of two modes of high-risk voxel points representing brain disease information and clinical information are fused through the multi-mode attention mechanism, the cooperative advantages of the high-risk voxel points and the clinical information are brought into play, the defects of the high-risk voxel points and the clinical information are made up, and the AD diagnosis accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing and pattern recognition, and in particular to an Alzheimer's disease diagnosis method based on an FCNFormer network. Background Art

[0002] Alzheimer's disease is a chronic neurodegenerative disease in which abnormal proteins accumulate in the form of amyloid plaques and neurofibrillary tangles in the brains of Alzheimer's patients, triggering neuronal loss. However, due to the subtle changes in the early stages of Alzheimer's disease, traditional radiological readouts and quantitative analysis often have difficulty accurately identifying these patterns.

[0003] With the rapid development of medical imaging technology, Magnetic Resonance Imaging (MRI) has become an important tool for studying brain lesions in Alzheimer's disease. Although MRI has a very high sensitivity for observing changes in brain structure, simple imaging features, such as the degree of hippocampal atrophy, are usually difficult to directly serve as sufficient basis for diagnosing Alzheimer's disease. Among many deep learning architectures, traditional CNN models usually view images as two-dimensional data, which has certain limitations when processing three-dimensional medical imaging data. In addition, CNN usually requires a large amount of labeled data during training and verification, and the labeling process of medical imaging data is both expensive and time-consuming, so a more advanced network structure is needed to improve model accuracy and reduce dependence on labeled data.

[0004] In addition, deep learning models have certain "black box" problems when performing disease diagnosis and image analysis. Although these models can perform disease diagnosis and image analysis with amazing accuracy, due to their complex structure and operation mode, it is often difficult to understand how the model makes a specific diagnosis or classification decision. This opacity limits the application of these models in clinical practice, so the decision-making process of the model needs to be able to provide doctors with sufficient explainability to guide clinical treatment. Summary of the invention

[0005] In order to solve the problem of the insufficiency of traditional CNN models in terms of interpretability, the present invention provides an Alzheimer's disease diagnosis method based on FCNFormer network, which is used for early diagnosis of Alzheimer's disease and visualization of pathological areas, aiming to improve the accuracy and interpretability of AD diagnosis.

[0006] A method for diagnosing Alzheimer's disease based on FCNFormer network, mainly including:

[0007] S1. Obtain a dataset for sMRI image training;

[0008] S2, perform image preprocessing on the data set;

[0009] S3, randomly sampling and obtaining image blocks combined with three-dimensional position information according to the preprocessed data set;

[0010] S4. Based on the FCN network and the Transformer attention mechanism, the FCNFormer network is obtained. The FCNFormer network integrates the MLP model of the multimodal attention mechanism to obtain an Alzheimer's disease diagnosis model.

[0011] S5. Input the sMRI image to be detected into the Alzheimer's disease diagnosis model, output the diagnosis result, and generate a disease probability map of brain disease distribution and a Matthews correlation coefficient map.

[0012] A storage device stores instructions and data for implementing an Alzheimer's disease diagnosis method based on an FCNFormer network.

[0013] An Alzheimer's disease diagnosis device based on a FCNFormer network comprises: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement an Alzheimer's disease diagnosis method based on a FCNFormer network.

[0014] The beneficial effects brought by the technical solution provided by the present invention are:

[0015] The Alzheimer's disease diagnosis method based on the FCNFormer network of the present invention organically combines three-dimensional position coding, Transformer self-attention mechanism and Fully Convolutional Network (FCN), and solves the deficiencies of the traditional CNN model in interpretability by providing more interpretable levels. The three-dimensional position coding mechanism based on the spatial position of the element is introduced to enhance the model's spatial perception of the brain structure, so that the model can learn the unique performance of each position of the brain in the AD pathological process; the Transformer self-attention mechanism is integrated into the FCN network to capture the long-distance dependency relationship between image blocks, so that the network can better extract cross-scale features, thereby more easily capturing the overall distribution of the disease in the brain; the MLP model that integrates the multimodal attention mechanism is introduced, and the high-risk voxel points representing brain disease information and clinical information are fused through the multimodal attention mechanism. The two modal data, high-risk voxel points representing brain disease information and clinical information, are integrated to give play to their synergistic advantages, make up for their respective deficiencies, improve diagnostic accuracy, and provide better interpretability and transparency for the model decision-making process, and it is expected to provide strong technical support for clinical practice and medical research. Specifically:

[0016] (1) Three-dimensional position coding and image block fusion. This enables the model to capture the spatial position information of the data and more accurately locate the pathological change areas related to Alzheimer's disease, thereby improving the accuracy of the entire model in disease diagnosis.

[0017] (2) Combining FCN network with Transformer. FCN network extracts deep features, and Transformer's self-attention mechanism can capture the ability of long-distance dependencies between image blocks. This fusion helps the network to better extract cross-scale features, making it easier to capture the overall distribution of diseases in the brain. The final network model can provide more explanation levels, solving the shortcomings of traditional models in terms of interpretability.

[0018] (3) Generation of high-resolution disease probability maps. The image blocks encoded with three-dimensional positions are input into the FCN network and converted into one-dimensional vectors. After feature aggregation by the Transformer, these enhanced feature vectors are restored to their original three-dimensional shapes and reconstructed into high-resolution disease probability maps representing the entire brain based on the initial block positions.

[0019] (4) Multimodal fusion of imaging and clinical information. Combine high-risk voxel points representing brain disease information with clinical information. Fusion of the two modal data can bring out their synergistic advantages, make up for their respective deficiencies, improve diagnostic accuracy, and provide a more comprehensive decision-making basis for disease diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0021] Figure 1 It is a flow chart of a method for diagnosing Alzheimer's disease based on the FCNFormer network in an embodiment of the present invention.

[0022] Figure 2 It is a diagram showing the effect of sMRI image preprocessing in an embodiment of the present invention.

[0023] Figure 3 It is a network structure diagram of the Alzheimer's disease diagnosis method based on the FCNFormer network in an embodiment of the present invention.

[0024] Figure 4 It is a diagram of the FCN network structure in an embodiment of the present invention.

[0025] Figure 5 is a network structure diagram of the multimodal attention mechanism in an embodiment of the present invention.

[0026] Figure 6 It is a schematic diagram of the operation of the hardware device in the embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0028] Please refer to Figure 1 , Figure 1 : is a flow chart of a method for diagnosing Alzheimer's disease based on the FCNFormer network in an embodiment of the present invention, which specifically includes:

[0029] S1. Obtain a dataset for sMRI image training;

[0030] The present invention selected three data sets: the Alzheimer's Disease Neuroimaging Initiative (ADNI); the Australian Imaging, Biomarkers and Lifestyle Study of Aging (AIBL); and the National Alzheimer's Coordinating Center (NACC) database of the United States, and based on the baseline recruitment criteria of ADNI, screened the sMRI images of eligible participants from the three public data sets of ADNI, AIBL and NACC.

[0031] S2, perform image preprocessing on the data set;

[0032] As a preferred embodiment of the present invention, refer to Figure 2 , perform standardized preprocessing operations on the sMRI images in the dataset. The specific process is as follows:

[0033] AC-PC correction: Using the two anatomical landmarks inside the brain, the anterior commissure (AC) and the posterior commissure (PC), as reference coordinates, the AC and PC points in the image are aligned with the corresponding points in the standard brain template. The present invention uses the SPM toolkit to uniformly perform AC-PC correction on all data.

[0034] Skull separation: To remove non-brain tissues such as skull, skin, fat and extracranial soft tissue from sMRI images, the present invention uses HD-BET tools for skull separation.

[0035] Affine registration: Medical images from different individuals or different imaging devices are mapped into a unified standard coordinate system. The present invention uses FSL software for affine registration to linearly align the sMRI image with the MNI152 template and resample all data at the same time. The processed data have the same spatial resolution, i.e., 1×1×1mm 3 .

[0036] Bias field correction: To eliminate non-uniform brightness changes in the image due to device hardware limitations or uneven magnetic fields, the present invention uses the N4BiasFieldCorrection tool in the ANTs software package to perform bias field correction.

[0037] S3, randomly sampling to obtain image blocks combined with three-dimensional position information;

[0038] As a preferred embodiment of the present invention, refer to Figure 3 In (a), a 3D position encoding mechanism based on the spatial position of the element is introduced in S3. A series of non-overlapping 3D image blocks are randomly sampled from the complete sMRI scan using a block-based method. The concat() function in the figure concatenates the position information with the image block. The process of obtaining the image block combined with the 3D position information is as follows:

[0039] For each training sample, let T i is the extracted image patch, L i is the corresponding three-dimensional position code (Pos.Emb), and P is obtained by concatenating the image block and the position code. i , expressed as:

[0040] P i =cat(T i ,L i )

[0041] Let V be the entire 3D MRI image voxel space. For each training sample, define the input image X as a voxel space of 181×217×181, and randomly select N volume image blocks P from V. i , where i = 1, 2, ..., N, N = 125, the size of each image block is defined as 45 × 55 × 45, and the random sampling strategy of image blocks can be expressed by the following formula:

[0042]

[0043] Among them, P(V) is the set of image patches generated from the sMRI image voxel space V, Represents each image block P i is a subset of the volume space V, It indicates that all image patches cover the entire image voxel space V.

[0044] S4. The fully convolutional network (FCN) model is combined with the Transformer attention mechanism to construct the FCNFormer network. The FCNFormer network is combined with the MLP model (MLP Model) that integrates the multimodal attention mechanism to obtain an Alzheimer's disease diagnosis model that automatically diagnoses AD and generates a high-resolution disease probability map representing the entire brain;

[0045] S5. Input the sMRI image to be detected into the Alzheimer's disease diagnosis model with fixed model parameters to obtain the diagnosis result, and output a three-dimensional stereogram of the spatial structure information of the brain image data and a disease probability map reflecting the distribution of brain diseases.

[0046] As a preferred embodiment of the present invention, refer to Figure 3 (b) in S4, the FCN network is combined with the Transformer attention mechanism and the MLP model with multimodal attention mechanism to obtain an Alzheimer's disease diagnosis model that automatically diagnoses AD and generates a high-resolution disease probability map representing the entire brain.

[0047] As a preferred embodiment of the present invention, refer to Figure 4 The process of using the FCN network to extract deep features from image input and perform data dimensionality reduction includes:

[0048] The FCN network consists of 4 consecutive convolution blocks (Conv Layer1, Conv Layer2, Conv Layer3, ConvLayer4), and the input is (4, 45, 55, 45), which means that the batch size of the input data is 4 and the size is 45×55×45. In the four convolution blocks, each block first performs a 3D convolution operation, then performs batch normalization (BathNormalization) to reduce internal covariate shift, and uses the Leaky ReLU activation function to introduce nonlinear transformations. Finally, a Dropout layer is added to reduce the risk of overfitting. For each given image block P i , the FCN network performs the above operations four times in a row, and finally outputs a feature map (1.9.11.9) with a dimension of 9×11×9. All image blocks use the same model weight W. The process can be formalized by the following formula:

[0049] F i =F(P i ; W)

[0050] Among them, F i is the FCN network for the i-th image block P iThe output feature map of F () represents the FCN network, W represents the weight of the FCN network, and the output of the entire three-dimensional sMRI image voxel space is represented as a feature map set {F1, F2, ..., F125}, each F i They are all feature maps with a resolution of 9×11×9, representing the local feature information after being processed by the FCN network.

[0051] As a preferred embodiment of the present invention, the feature map F output by the FCN network i The process of performing a linear flatten operation includes:

[0052] The feature map F output by the FCN network i Convert to a one-dimensional feature vector F' i This operation linearizes each 9×11×9 feature block and flattens it into an 891-dimensional feature vector, corresponding to 125 different image patches.

[0053]

[0054] Among them, L represents the flattening process, and F' is the feature vector after flattening, and its dimension is 125×891.

[0055] As a preferred embodiment of the present invention, a Transformer Encoder is used to output feature blocks and reassemble them into a complete 3D stereogram according to the original block positions. The specific process is as follows:

[0056] The Transformer attention mechanism in the FCNFormer network consists of a stack of 6 Transformer encoder blocks. A Transformer encoder block consists of a multi-head self-attention layer (Multi-HeadAttention, MHA) with three self-attention heads and a multi-layer perceptron (FFNN). Layer normalization (Norm) is performed before each Transformer encoder block. After each Transformer encoder block, a skip connection is used to input the flattened feature vector F' into the Transformer module. The lth block of the Transformer encoder (denoted as Z l ) can be expressed as:

[0057] Z′ l =LayerNorm(Z l-1 )

[0058] Z″ l =MHA(Z l )+Z l-1

[0059] Z l =MLP1(LayerNorm(Z″ l ))+Z″ l

[0060] Among them, Z l-1 It is used as the input of the lth Transformer encoder block. LayerNorm is the layer normalization operation on the input. MHA represents the multi-head attention layer. MLP1 represents a component MLP in the Transformer encoder block, which is used to further process the output of the self-attention mechanism. The multi-head attention layer is responsible for capturing the long-distance dependencies within the input sequence. Information in different modes is extracted through multiple parallel attention heads. Multi-layer perceptrons are used to further complicate and abstract the extracted features. Skip connections allow information to pass directly to the next layer. The first block in the Transformer encoder stack, after layer normalization, multi-head self-attention and MLP operations, outputs a representation Z that has fully mined the intrinsic structure and deep features of the input sequence. l , then the features are reassembled into a complete 3D stereogram according to the original block positions, and a disease probability map that reflects the distribution of brain diseases in the current sample is accurately generated. At the same time, the Matthews Correlation Coefficient (MCC) map representing the entire sample of this data set can be constructed based on the disease probability map of each single sample, as shown in Figure 3 As shown in (c) in .

[0061] As a preferred embodiment of the present invention, after the FCNFormer network, the process of (NC / AD) classification by the MLP model integrating the multimodal attention mechanism includes:

[0062] The feature vector Z output by the Transformer encoder block l After the global average pooling layer (Global AveragePooling, GAP), it is converted into G l , expressed as:

[0063] G l =GAP(Z l )

[0064] G l The MLP model with multimodal attention mechanism is used for classification to obtain the final predicted label T, which represents AD (disease) or NC (normal), expressed as:

[0065] T=MLP(G l )

[0066] Reference Figure 5 The multimodal attention mechanism fused by the MLP model transforms the non-imaging data of age, gender, and Mental State Examination Scale (MMSE) score into language modality features N t , which is used as the visual feature interaction of Q and HW×C, and the two are multiplied by the linear layer to obtain N t ×HW feature matrix, and then multiply it by the Linear visual feature Value through softmax to get N t ×C features, and then pass through the Linear layer to obtain the visual information dominant feature (Vison-dominated Feature). The Attention Matrix in the figure is the attention matrix, and GenericAttention is the attention mechanism. Suppose the non-visual data feature matrix is Where N represents the dimension of non-visual features. Suppose the visual data feature matrix is Where H and W represent the dimensions of the visual data in height and width respectively, and C represents the dimension of the feature. The process is shown in the following formula:

[0067] The non-visual data feature matrix Q is transformed through a linear layer f(·) to obtain

[0068] Q′=f(Q)

[0069] Adjust the shape of the visual data feature matrix V to get

[0070] Perform matrix multiplication of Q' and V' to obtain the characteristic matrix

[0071] S=Q'V' T

[0072] Applying the softmax function to the matrix S, we get

[0073] S′=softmax(S)

[0074] Perform matrix multiplication of S' and visual feature matrix V' to obtain

[0075] U=S′V′

[0076] Transform T through another linear layer g(·) to obtain the final visual dominance feature matrix

[0077] Z=g(U)

[0078] Among them, f(·) and g(·) represent the linear transformation in the multimodal attention mechanism. The parameters of the linear transformation f(·) and g(·) can be trained and updated, that is, the parameters updated by the linear layer refer to the weight matrix and bias vector.

[0079] As a preferred embodiment of the present invention, the loss function of the Alzheimer's disease diagnosis model is defined as the weighted sum of the cross entropy loss and the entropy loss. The specific calculation formula is as follows:

[0080] In the deep learning framework for Alzheimer's disease diagnosis tasks, the formula for the intersection and entropy loss is as follows:

[0081]

[0082] where y LS is a modified label with a value between [0.1, 0.9], which is used to alleviate the problem of class imbalance; β is a hyperparameter set to the inverse of the ratio of negative samples to the total number of samples; is the posterior probability predicted by the Alzheimer's disease diagnosis model.

[0083] For a probability distribution vector p = (p1, p2, ..., pn) predicted by a certain category, the calculation formula of its entropy H(p) is as follows:

[0084]

[0085] Among them, p i represents the predicted probability of the i-th category.

[0086] The total loss function is the weighted sum of the cross entropy loss and the entropy loss, and the formula is as follows:

[0087] L total =L cls +λH(p)

[0088] Among them, λ is a weight hyperparameter used to balance the contribution between classification loss and entropy loss.

[0089] As a preferred embodiment of the present invention, in S5, a single sample is input into the Alzheimer's disease diagnosis model with fixed model parameters, and a three-dimensional stereogram of the spatial structure information of the brain imaging data and a disease probability map reflecting the brain disease distribution of the current sample are output. This probability map is organically integrated with objective clinical information to achieve accurate predictive diagnosis of AD.

[0090] See also Figure 6 , Figure 64 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: an Alzheimer's disease diagnosis device 401 based on the FCNFormer network, a processor 402 and a storage device 403.

[0091] An Alzheimer's disease diagnosis device 401 based on a FCNFormer network: The Alzheimer's disease diagnosis device 401 based on a FCNFormer network implements the Alzheimer's disease diagnosis method based on a FCNFormer network.

[0092] Processor 402: The processor 402 loads and executes the instructions and data in the storage device 403 to implement the Alzheimer's disease diagnosis method based on the FCNFormer network.

[0093] Storage device 403: The storage device 403 stores instructions and data; the storage device 403 is used to implement the Alzheimer's disease diagnosis method based on the FCNFormer network.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for diagnosing Alzheimer's disease based on FCNFormer network, characterized by: include: S1. Obtain a dataset for sMRI image training; S2, perform image preprocessing on the data set; S3, randomly sampling and obtaining image blocks combined with three-dimensional position information according to the preprocessed data set; S4. Based on the FCN network and the Transformer attention mechanism, the FCNFormer network is obtained. The FCNFormer network integrates the MLP model of the multimodal attention mechanism to obtain an Alzheimer's disease diagnosis model. S5. Input the sMRI image to be detected into the Alzheimer's disease diagnosis model, output the diagnosis result, and generate a disease probability map of brain disease distribution and a Matthews correlation coefficient map.

2. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 1, characterized in that: In S2, the image preprocessing process includes: 1) AC-PC correction: Using the two anatomical landmarks inside the brain, the anterior commissure and the posterior commissure, as reference coordinates, the AC and PC points in the image were aligned with the corresponding points in the standard brain template, and the SPM toolkit was used to uniformly perform AC-PC correction on all data; 2) Skull separation: Use HD-BET tool to perform skull separation and remove non-brain tissue from sMRI images; 3) Affine registration: medical images from different individuals or different imaging devices are mapped into a unified standard coordinate system. The present invention uses FSL software for affine registration to linearly align the sMRI image with the MNI152 template and resample all data at the same time. The processed data have the same spatial resolution; 4) Bias field correction: Use the N4BiasFieldCorrection tool in the ANTs software package to perform bias field correction to eliminate non-uniform brightness changes in the image due to device hardware limitations or uneven magnetic fields.

3. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 1, characterized in that: S3 introduces a three-dimensional position encoding mechanism based on the spatial position of the element. The process of obtaining the image block combined with the three-dimensional position information is as follows: For each training sample in the dataset, let T i is the extracted image patch, L i is the corresponding three-dimensional position code. P is obtained by concatenating the image block and the position code. i : P i =cat(T i ,L i ) Among them, cat() represents the concatenation function; Let V be the entire 3D sMRI image voxel space. For each training sample, define the voxel space of the input image X and randomly select N volume image blocks P from V. i , where i = 1, 2, ..., N, and the random sampling strategy of image blocks is expressed as: Among them, P(V) is the set of image patches generated from the sMRI image voxel space V, Represents each image block P i is a subset of the voxel space V, It indicates that all image patches cover the entire image voxel space V.

4. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 1, characterized in that: In S4, the FCN network is used to extract deep features from image input and perform data dimensionality reduction: The FCN network consists of 4 consecutive convolution blocks. Each convolution block first performs a 3D convolution operation, then performs batch normalization to reduce internal covariate shift, and uses the Leaky ReLU activation function to introduce nonlinear transformations. Finally, a Dropout layer is added to reduce the risk of overfitting. For each given image block P i , the FCN network is used to output feature maps, and the process is formalized by the following formula: F i =F(P i ;W) Among them, F i It is the output feature map of the FCN network for the i-th image block Pi, F() represents the FCN network, and W represents the model weight.

5. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 4, characterized in that: In S4, the feature map F output by the FCN network i Perform a linear flattening operation: The feature map F output by the FCN network i Convert to a one-dimensional feature vector F' i , which linearizes each feature block and flattens it into a multi-dimensional feature vector corresponding to N different image blocks: Among them, L represents the flattening process, and F' is the feature vector after flattening.

6. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 5, characterized in that: In S5, the process of performing feature aggregation through the Transformer encoder to generate the disease probability map of brain disease distribution and the Matthews correlation coefficient map is as follows: The Transformer attention mechanism consists of a stack of 6 Transformer encoder blocks. A Transformer encoder block consists of a multi-head self-attention layer with three self-attention heads and a multi-layer perceptron. Layer normalization is performed before each Transformer encoder block. After each Transformer encoder block, a skip connection is used to input the flattened feature vector F' into the Transformer attention mechanism. The l-th block Z of the Transformer encoder l The output is represented as: WITH' l =LayerNorm(Z l-1 ) Z″ l =MHA(Z′ l )+Z l-1 WITH l =MLP1(LayerNorm(Z″ l ))+Z″ l Among them, Z l-1 represents the visual dominance feature matrix as the input of the lth Transformer encoder block, LayerNorm represents the layer normalization operation on the input, MHA represents the multi-head attention layer, MLP1 represents a component MLP in the Transformer encoder block, which is used to further process the output of the self-attention mechanism. The stack of the first Transformer encoder block after layer normalization, multi-head self-attention and MLP operations, Z l Represents the output feature vector; then the feature vector is reassembled into a complete 3D stereogram according to the original block position, accurately generating the disease probability map of brain disease distribution and the Matthews correlation coefficient map.

7. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 5, characterized in that: In S5, after the FCNFormer network, the MLP model integrating the multimodal attention mechanism performs (NC / AD) classification process, which includes: The feature vector Z output by the Transformer encoder block l After the global average pooling layer, it is converted to G l : G l =GAP(Z l ( Among them, GAP() represents the global average pooling layer; G l Classify through the MLP model and get the final predicted label T: T=MLP(G l ) The predicted label is AD or NC, AD means belonging to the Alzheimer's disease population, and NC means belonging to the healthy population; The multimodal attention mechanism integrated by the MLP model converts non-image data of age, gender, and MMSE score into features of language modality N t , which is used as the visual feature interaction of Q and HW×C, where H and W represent the dimensions of the visual data in height and width respectively, and C represents the dimension of the feature. The two are multiplied by the Linear layer to obtain N t ×HW feature matrix, and then multiply it by the Linear visual feature Value through softmax to get N t ×C features, and then pass through the Linear layer to obtain the features dominated by visual information. The process is shown in the following formula: The non-visual data feature matrix Q is transformed through a linear layer f(·) to obtain Q': Q' = f(Q); Adjust the shape of the visual data feature matrix V to obtain V'; Perform matrix multiplication of Q' and V' to obtain the characteristic matrix S: S = Q'V' T , the superscript T indicates transposition; Apply the softmax function to the matrix S to obtain S': S' = softmax(S); Perform matrix multiplication of S' and visual feature matrix V' to obtain U: U = S'V'; Transform U through another linear layer g(·) to obtain the final visual dominance feature matrix Z: Z = g(U); Where f(·) and g(·) represent linear transformations, and their parameters can be trained and updated.

8. The method for diagnosing Alzheimer's disease based on the FCNFormer network as claimed in claim 1, characterized in that: In S5, the calculation formula of the loss function of the Alzheimer's disease diagnosis model based on the FCNFormer network is as follows: The formula for the intersection and entropy loss is: Among them, y LS is a modified label whose value is between [0.1, 0.9], which is used to alleviate the problem of class imbalance. β is a hyperparameter. is the posterior probability predicted by the Alzheimer's disease diagnostic model; For a probability distribution vector p = (p1, p2, ..., pn) predicted by a certain category, the calculation formula for its entropy H(p) is: Among them, p i represents the predicted probability of the i-th category, and n represents the number of categories; The total loss function is the weighted sum of the cross entropy loss and the entropy loss, and the formula is as follows: L total =L cls +λH(p) Among them, λ is a weight hyperparameter used to balance the contribution between classification loss and entropy loss.

9. A storage device, characterized in that: The storage device stores instructions and data for implementing the Alzheimer's disease diagnosis method based on the FCNFormer network as described in any one of claims 1 to 8.

10. An Alzheimer's disease diagnostic device based on FCNFormer network, characterized in that: include: Processor and storage device; the processor loads and executes instructions and data in the storage device to implement the Alzheimer's disease diagnosis method based on the FCNFormer network as described in any one of claims 1 to 8.

Images