Alzheimer disease risk prediction method based on feature fusion, medium and equipment

Abstract

The invention discloses an Alzheimer disease risk prediction method based on feature fusion, a medium and equipment. The method comprises the following steps: firstly, acquiring an sMRI image, an FDG-PET image, a biomarker and intelligence score data of a target subject; then, sMRI and FDG-PET image data are respectively input into a three-dimensional convolutional neural network module, and a corresponding first image feature map and a corresponding second image feature map are extracted; secondly, inputting the two feature maps into a cross attention fusion module to obtain image fusion features; and inputting a U-shaped up-sampling network to segment a hippocampus region to obtain hippocampus features. And then, merging the image fusion features, the hippocampus features, the biomarker data and the intelligence score data, and inputting the merged data into an extreme learning machine for calculation, thereby obtaining an onset risk prediction result of the Alzheimer's disease. According to the method, by means of cross-modal data fusion, multi-modal data complementary information is extracted, and the accuracy of onset risk prediction can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and particularly to a method, medium and device for predicting the risk of Alzheimer's disease based on feature fusion. Background Art

[0002] Alzheimer's disease (AD) is a neurodegenerative disease characterized by irreversible neuronal damage. Research shows that mild cognitive impairment (MCI), as the prodromal stage of AD, has significant clinical intervention value: approximately 60% of MCI patients will convert to AD within 5 years, but the accurate discrimination of its subtypes (stable sMCI and progressive pMCI) directly affects the effectiveness of intervention strategies. Currently, the prediction methods relying on single-modal biomarkers (such as MRI gray matter atrophy) have obvious limitations: single-modal data is difficult to characterize the multi-dimensional pathological mechanisms of AD, resulting in the classification accuracy of sMCI and pMCI generally being lower than 85%.

[0003] Traditional multi-modal fusion methods (such as feature concatenation or decision voting) are difficult to coordinate the deep interactions among the structural degeneration of sMRI, the abnormal metabolic function of FDG-PET, and the molecular pathological indicators of CSF, which further affects the accuracy of predicting the risk of Alzheimer's disease. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method, medium and device for predicting the risk of Alzheimer's disease based on feature fusion, so as to solve the problem of low accuracy in predicting the risk of Alzheimer's disease by existing algorithms.

[0005] To achieve the above object, in the first aspect, the present application provides a method for predicting the risk of Alzheimer's disease based on feature fusion, and the method includes the following steps:

[0006] S1. Obtain sMRI image data, FDG-PET image data, biomarker data and corresponding intelligence score data of a target subject;

[0007] S2. Input the sMRI image data and the FDG-PET image data into a three-dimensional convolutional neural network module respectively, and extract a first image feature map corresponding to the sMRI image data and a second image feature map corresponding to the FDG-PET image data;

[0008] S3. Input the first image feature map and the second image feature map into a cross-attention fusion module to obtain an image fusion feature;

[0009] S4. Input the first image feature map and the second image feature map into a U-shaped upsampling network to segment the hippocampal region and obtain hippocampal features;

[0010] S5. Combine the image fusion features, the hippocampal features, the biomarker data, and the intelligence score data, and input the combined data into an extreme learning machine for calculation to obtain the prediction result of the onset risk of Alzheimer's disease.

[0011] Further, the biomarker data includes the data of three Alzheimer's biomarkers, namely Aβ42, p-Tau, and t-Tau in cerebrospinal fluid;

[0012] The intelligence score data includes the Mini-Mental State Examination (MMSE) and the Clinical Dementia Rating - Sum of Boxes (CDR - SB).

[0013] Further, the method further includes:

[0014] Obtain the sMRI image data and FDG - PET image of the target subject, and perform pre - processing of brain tissue extraction and rigid registration. The template used in the registration pre - processing process is MNI152.

[0015] Further, the three - dimensional convolutional neural network module includes four convolutional sub - modules, and the structures of the four convolutional sub - modules are the same;

[0016] Each convolutional sub - module includes a first convolutional layer, a second convolutional layer, a first max - pooling layer, a first batch normalization layer, and a first ReLU activation function layer connected in sequence. The convolutional kernels of the first convolutional layer and the second convolutional layer are both 3×3×3.

[0017] Further, step S3 includes the following steps:

[0018] S31. Concatenate the first image feature map and the second image feature map in the channel dimension to obtain a first concatenated feature map;

[0019] S32. Input the first concatenated feature map into the global average pooling layer and the global max - pooling layer respectively to obtain a global average feature map and a global max - feature map;

[0020] S33. Concatenate the global average feature map and the global max - feature map in the channel dimension to obtain a second concatenated feature map. Input the second concatenated feature map into the spatial attention module to obtain a spatial attention feature map, and input the global average feature map into the channel attention module to obtain a channel attention feature map;

[0021] S34. Multiply the spatial attention feature map and the channel attention feature map to obtain an attention feature map;

[0022] S35. Merge the attention feature map and the first concatenated feature map in the channel dimension, and input them into a convolutional layer with a 3×3×3 convolutional kernel. The output of this convolutional layer is then input into the Sigmoid function to obtain the weight assignment coefficient matrix. The specific expression of the weight assignment coefficient matrix P is as follows:

[0023] P = (1 + exp(-Conv3d(FS × FC))) -1 ;

[0024] where exp(·) is the exponential function with base e, Conv3d(·) is the three-dimensional convolution operation, FS is the spatial attention feature map, and FC is the channel attention feature map;

[0025] S36. Multiply the first image feature map by the weight assignment coefficient matrix P to obtain the first image weight feature map. Multiply the second image feature map by the complementary weight coefficient matrix 1 - P to obtain the second image weight feature map. Add the first image weight feature map and the second image weight feature map, and input the added data into a convolutional layer with a 1×1×1 convolutional kernel to obtain the fused feature map;

[0026] S37. Input the fused feature map into the feature extraction module to obtain the image fusion feature.

[0027] Furthermore, the feature extraction module includes a third convolutional layer, a fourth convolutional layer, a second global average pooling layer, a second ReLU activation function layer, a first fully connected layer, a Dropout layer with a dropout rate of 0.5, a third ReLU activation function layer, and a second fully connected layer connected in sequence;

[0028] After the spatial attention module concatenates the global average feature map and the global maximum feature map in the channel dimension, it inputs them into a convolutional layer with a 3×3×3 convolutional kernel to obtain the spatial attention feature map. The specific expression of the spatial attention feature map FS is as follows:

[0029] FS = Conv3d(cat(GAP(FMRI), GMP(FPET)));

[0030] In the formula, Conv3d(·) is the three-dimensional convolution operation, cat(·) is the channel dimension concatenation operation, GAP(·) is the global average pooling operation, GMP(·) is the global maximum pooling operation, FMRI is the first image feature map, and FPET is the second image feature map;

[0031] The channel attention module includes a first channel convolutional layer, a fourth ReLU activation function layer, and a second channel convolutional layer connected in sequence. The convolutional kernels of the first channel convolutional layer and the second channel convolutional layer are both 1×1×1. The specific expression of the output channel attention feature map FC is:

[0032] FS = Conv3d(ReLU(GAP(FMRI)));

[0033] Wherein, Conv3d(·) is a three-dimensional convolution operation, ReLU(·) is a ReLU activation function, and GAP(·) is a global average pooling operation.

[0034] Furthermore, the U-shaped upsampling network includes four sequentially connected upsampling transposed convolution layers. Each upsampling transposed convolution layer takes the output of the previous layer and the image feature map of the corresponding level as inputs, gradually increasing the size of the feature map. The last upsampling transposed convolution layer outputs the segmented region of the hippocampus, and the volume of the hippocampus is used as the hippocampus feature.

[0035] Furthermore, the extreme learning machine uses a Gaussian function as the kernel function, and its expression is specifically as follows

[0036] f(x) = K(x, x1), K(x, x2), …, K(x, x N )] T × (Ω + I) -1 Y;

[0037] Wherein, x is the sample to be measured, x n is the nth training sample, N is the number of training samples, Y is the one-hot label matrix corresponding to the training samples, Ω is an N×N training sample matrix, and each element of Ω is K(x i , x j ), I is the identity matrix, K(u, v) is the Gaussian kernel function, and the specific expression of K(u, v) is as follows:

[0038] K(u, v) = exp(-||u - v|| 2 × γ -1 );

[0039] Wherein, exp(·) is the exponential function with base e, and γ is an adjustable parameter.

[0040] In a second aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for predicting the risk of Alzheimer's disease based on feature fusion as described in the first aspect of the present application.

[0041] In a third aspect, the present application provides an electronic device, on which a computer program is stored, including a processor and a storage medium. The computer program is stored on the storage medium, and when the computer program is executed by the processor, it implements the method for predicting the risk of Alzheimer's disease based on feature fusion as described in the first aspect of the present application.

[0042] Different from the prior art, the above technical solution provides a method, medium and device for predicting the risk of Alzheimer's disease based on feature fusion. The method includes: obtaining sMRI image data, FDG-PET image data, biomarker data and corresponding intelligence score data of a target subject; respectively inputting the sMRI image data and the FDG-PET image data into a three-dimensional convolutional neural network module to respectively extract a first image feature map corresponding to the sMRI image data and a second image feature map corresponding to the FDG-PET image data; inputting the first image feature map and the second image feature map into a cross-attention fusion module to obtain an image fusion feature; inputting the first image feature map and the second image feature map into a U-shaped upsampling network to segment the hippocampal region to obtain hippocampal features; combining the image fusion feature, the hippocampal feature, the biomarker data and the intelligence score data, and inputting the combined data into an extreme learning machine for calculation to obtain a prediction result of the onset risk of Alzheimer's disease. The present invention extracts complementary information of multi-modal data through cross-modal data fusion, effectively improving the accuracy of predicting the onset risk of Alzheimer's disease. The evaluation results on the ADNI dataset show that the present method has good performance in predicting the onset risk of Alzheimer's disease.

[0043] The above relevant records of the invention content are only an overview of the technical solution of the present invention. In order to enable those of ordinary skill in the art to more clearly understand the technical solution of the present invention, and then to be implemented according to the content recorded in the description and the drawings, and in order to make the above objects, other objects, features and advantages of the present invention more easily understood, the following is described in conjunction with the specific embodiments and drawings of the present invention. Brief Description of the Drawings

[0044] The drawings are only used to illustrate the principles, implementation methods, applications, features and effects of the specific embodiments of the present invention and other related contents, and should not be regarded as a limitation of the present invention.

[0045] In the drawings of the description:

[0046] Figure 1 It is the first flowchart of the method for predicting the risk of Alzheimer's disease based on feature fusion involved in the specific embodiment;

[0047] Figure 2 It is the second flowchart of the method for predicting the risk of Alzheimer's disease based on feature fusion involved in the specific embodiment;

[0048] Figure 3 It is a schematic diagram of the modules of the electronic device described in the specific embodiment;

[0049] The descriptions of the reference numerals involved in the above drawings are as follows:

[0050] 10. Electronic device;

[0051] 101. Processor;

[0052] 102. Storage medium. Detailed implementation manners

[0053] To describe in detail the possible application scenarios, technical principles, specific implementable solutions, achievable objectives and effects of the present invention, etc., the following will be described in detail with reference to the specific examples listed and in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present invention, so they are only examples and cannot be used to limit the protection scope of the present invention.

[0054] Referring to "embodiments" herein means that the specific features, structures or characteristics described in connection with the embodiments may be included in at least one embodiment of the present invention. The term "embodiment" appearing in various positions in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in the present invention, as long as there is no technical contradiction or conflict, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0055] Unless otherwise defined, the meanings of the technical terms used herein are the same as those commonly understood by those skilled in the technical field to which the present invention belongs; the use of the relevant terms herein is only for describing specific embodiments and is not intended to limit the present invention.

[0056] In the description of the present invention, the phrase "and / or" is an expression used to describe the logical relationship between objects, indicating that there can be three relationships. For example, A and / or B means: there is A, there is B, and there is both A and B at the same time. In addition, the character " / " in this article generally represents an "or" logical relationship between the associated objects before and after.

[0057] In the present invention, terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary or secondary, or order relationship between these entities or operations.

[0058] Without further limitation, in the present invention, the expressions such as "including", "comprising", "having" or other similar expressions used in the statements are intended to cover non-exclusive inclusion. These expressions do not exclude that there may be other elements in the process, method or product including the said elements, so that the process, method or product including a series of elements may not only include those defined elements, but also include other elements not explicitly listed, or also include elements inherent to this process, method or product.

[0059] In the present invention, expressions such as "greater than", "less than", "exceeding", etc. are understood not to include the number itself; expressions such as "above", "below", "within", etc. are understood to include the number itself. In addition, in the description of the embodiments of the present invention, the meaning of "a plurality of" is two or more (including two), and similar expressions related to "many" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically defined.

[0060] In the description of the embodiments of the present invention, the spatially related expressions used, such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "perpendicular", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the specific embodiment or the accompanying drawings, and is only for the convenience of describing the specific embodiments of the present invention or facilitating the understanding of the reader, rather than indicating or implying that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0061] Unless otherwise clearly specified or limited, in the description of the embodiments of the present invention, the terms "installed", "connected", "connected", "fixed", "set", etc. used should be understood in a broad sense. For example, the "connection" may be a fixed connection, a detachable connection, or an integral setting; it may be a mechanical connection, an electrical connection, or a communication connection; it may be directly connected, or indirectly connected through an intermediate medium; it may be the communication inside two elements or the interaction relationship between two elements. For those skilled in the art to which the present invention pertains, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0062] As Figure 1 shown, in the first aspect, the present application provides a method for predicting the risk of Alzheimer's disease based on feature fusion, and the method includes the following steps:

[0063] S1. Obtain the sMRI image data, FDG-PET image data, biomarker data, and corresponding intelligence score data of the target subject;

[0064] S2. Input the sMRI image data and the FDG-PET image data into a three-dimensional convolutional neural network module respectively, and extract the first image feature map corresponding to the sMRI image data and the second image feature map corresponding to the FDG-PET image data respectively;

[0065] S3. Input the first image feature map and the second image feature map into a cross-attention fusion module to obtain an image fusion feature;

[0066] S4. Input the first image feature map and the second image feature map into a U-shaped upsampling network to segment the hippocampal region and obtain hippocampal features;

[0067] S5. Combine the image fusion feature, the hippocampal feature, the biomarker data, and the intelligence score data, and input the combined data into an extreme learning machine for calculation to obtain a prediction result of the onset risk of Alzheimer's disease.

[0068] In step S1, the biomarker data includes data of three Alzheimer's biomarkers, namely Aβ42, p-Tau, and t-Tau in cerebrospinal fluid; the intelligence score data includes the Mini-Mental State Examination MMSE and the Clinical Dementia Rating Sum of Boxes CDR-SB.

[0069] In this embodiment, brain tissue extraction and processing use the brain extraction tool BET in the open-source software FSL, and registration processing uses the open-source software IRTK. First, register the FDG-PET image to the sMRI image to align the two images. Subsequently, use BET to perform skull stripping on the sMRI to generate a brain tissue region template, and use this template to retain the brain tissue in the sMRI and FDG-PET images, and clear the images in the remaining space. Then register the sMRI image to the MNI152 template, and at the same time register the FDG-PET image to the sMRI image again. Finally, crop the black edge area at the edge and retain a three-dimensional image of size 128×160×128.

[0070] In this embodiment, the sMRI and FDG-PET two-modal images can be first fused using a deep neural network, and Alzheimer's disease-related features can be extracted. The samples used in the training process of the neural network model are Alzheimer's disease patients (AD) and normal control groups (NC). Since the brain structures of AD and NC are quite different, the neural network can be better trained to improve the feature extraction ability. The trained neural network is used to extract the image features of sMCI and pMCI subjects.

[0071] Before performing step S2, the method further includes: obtaining the sMRI image data and the FDG-PET image of the target subject after brain tissue extraction and rigid registration preprocessing, and the template used in the registration preprocessing process is MNI152.

[0072] In step S2, the three-dimensional convolutional neural network module includes four convolutional sub-modules, and the structures of the four convolutional sub-modules are the same;

[0073] Each convolutional sub-module includes a first convolutional layer, a second convolutional layer, a first max-pooling layer, a first batch normalization layer, and a first ReLU activation function layer connected in sequence. The convolutional kernels of the first convolutional layer and the second convolutional layer are both 3×3×3.

[0074] In this embodiment, the convolutional layers in the convolutional sub-module adopt convolutional kernels of size 3×3×3 to extract local feature patterns in three-dimensional space. The network is then connected to a 2×2×2 max-pooling layer, which effectively retains key spatial features while reducing the dimension of the feature map by performing downsampling operations. A batch normalization (BatchNorm) module is configured behind each convolutional layer. This design can not only maintain training stability but also significantly improve the model convergence efficiency. In addition, the ReLU non-linear activation function is integrated behind each convolutional layer, and by introducing a non-linear transformation mechanism, the feature representation ability of the network is further enhanced. The output feature map size of each convolutional sub-module is one-half of the input feature map size. Therefore, the feature map size after passing through four convolutional sub-modules for a three-dimensional image of size 128×160×128 is 8×10×8, and the number of channels is 128.

[0075] As Figure 2 shown, in some embodiments, step S3 includes the following steps:

[0076] S31. Concatenate the first image feature map and the second image feature map in the channel dimension to obtain a first concatenated feature map;

[0077] S32. Input the first concatenated feature map into a global average pooling layer and a global max-pooling layer respectively to obtain a global average feature map and a global max feature map;

[0078] S33. Concatenate the global average feature map and the global max feature map in the channel dimension to obtain a second concatenated feature map. Input the second concatenated feature map into a spatial attention module to obtain a spatial attention feature map, and input the global average feature map into a channel attention module to obtain a channel attention feature map;

[0079] S34. Multiply the spatial attention feature map and the channel attention feature map to obtain an attention feature map;

[0080] S35. Merge the attention feature map and the first concatenated feature map in the channel dimension, input them into a convolutional layer with a convolutional kernel of 3×3×3, and then input the output of this convolutional layer into a Sigmoid function to obtain a weight distribution coefficient matrix. The expression of the weight distribution coefficient matrix P is specifically as follows:

[0081] P = (1 + exp(-Conv3d(FS×FC))) -1 ;

[0082] Among them, exp(·) is the exponential function with base e, Conv3d(·) is the three-dimensional convolution operation, FS is the spatial attention feature map, and FC is the channel attention feature map;

[0083] S36. Multiply the first image feature map by the weight distribution coefficient matrix P to obtain the first image weight feature map, multiply the second image feature map by the complementary weight coefficient matrix 1 - P to obtain the second image weight feature map, add the first image weight feature map and the second image weight feature map, and input the added data into a convolution layer with a convolution kernel of 1×1×1 to obtain the fused feature map;

[0084] S37. Input the fused feature map into the feature extraction module to obtain the image fusion feature.

[0085] In this embodiment, the feature maps of sMRI and FDG-PET are weighted and fused after calculating the weight matrix through the cross-attention module. The fused multi-modal feature map is again subjected to a convolution operation to obtain a feature map with a size of 4×5×4 and 256 channels. After passing through the global average pooling layer, a feature vector with a length of 256 is obtained, and 16 features are output through two fully connected layers as the image fusion feature.

[0086] Further, the feature extraction module includes a third convolution layer, a fourth convolution layer, a second global average pooling layer, a second ReLU activation function layer, a first fully connected layer, a Dropout layer with a dropout rate of 0.5, a third ReLU activation function layer, and a second fully connected layer connected in sequence;

[0087] After splicing the global average feature map and the global maximum feature map in the channel dimension, the spatial attention module inputs them into a convolution layer with a convolution kernel of 3×3×3 to obtain the spatial attention feature map. The specific expression of the spatial attention feature map FS is as follows:

[0088] FS = Conv3d(cat(GAP(FMRI), GMP(FPET)));

[0089] In the formula, Conv3d(·) is the three-dimensional convolution operation, cat(·) is the channel dimension splicing operation, GAP(·) is the global average pooling operation, GMP(·) is the global maximum pooling operation, FMRI is the first image feature map, and FPET is the second image feature map;

[0090] The channel attention module includes a first channel convolution layer, a fourth ReLU activation function layer, and a second channel convolution layer connected in sequence. The convolution kernels of the first channel convolution layer and the second channel convolution layer are both 1×1×1, and the specific expression of the output channel attention feature map FC is:

[0091] FS=Conv3d(ReLU(GAP(FMRI)));

[0092] Where Conv3d(·) is a three-dimensional convolution operation, ReLU(·) is the ReLU activation function, and GAP(·) is a global average pooling operation.

[0093] In step S4, the U-shaped upsampling network includes four sequentially connected upsampling deconvolution layers. Each upsampling deconvolution layer takes the output of the previous layer and the image feature map of the corresponding layer as input, gradually increasing the size of the feature map. The last upsampling deconvolution layer outputs the segmented area of the hippocampus and uses the volume of the hippocampus as the hippocampus feature.

[0094] In this embodiment, in order to enhance the neural network's ability to extract features of Alzheimer's disease, a multi-task training method is adopted during the training process of the neural network. The neural network not only outputs the classification of the samples, but also outputs the segmentation results of the hippocampus through a U-shaped upsampling network. Because the hippocampus is an important relevant area of Alzheimer's disease, the patient's disease process is usually accompanied by atrophy of the hippocampus. Through multi-task training, the neural network can learn the morphological characteristics of the hippocampus, thereby improving the ability to extract disease features. The hippocampus segmentation training label is generated using the segmentation instruction in the open source software FSL. Finally, the neural network obtained by multi-task training not only has feature extraction capability, but also has hippocampus segmentation capability, so the hippocampus volume it outputs can also be used as a feature.

[0095] In some embodiments, in step S5, the extreme learning machine uses a Gaussian function as a kernel function, and its expression is as follows:

[0096] f(x)=K(x,x1),K(x,x2),…,K(x,x N )] T ×(Ω+I) -1 Y;

[0097] Among them, x is the sample to be tested, x n is the nth training sample, N is the number of training samples, Y is the unique label matrix corresponding to the training sample, Ω is the N×N training sample matrix, and each element of Ω is K(x i ,x j ), I is the identity matrix, K(u,v) is the Gaussian kernel function, and the specific expression of K(u,v) is as follows:

[0098] K(u,v)=exp(-||uv|| 2 ×γ -1 );

[0099] Among them, exp(·) is the exponential function with base e, and γ is an adjustable parameter.

[0100] In this embodiment, the image fusion features, hippocampal features, biomarker data, and intelligence score data of all pMCI and sMCI samples are first subjected to Z-score normalization. Then, they are input into an extreme learning machine for prediction and classification, and the value of the parameter γ of the extreme learning machine is set to 10 times the number of sample features.

[0101] In some embodiments, experimental data from the ADNI database is used as samples to verify the method involved in the present application. Among them, the number of pMCI samples is 110, and the number of sMCI samples is 153. All samples are tested through five-fold cross-validation, that is, all samples are randomly divided into 5 parts, and 1 part is selected as the test set, and the remaining 4 parts are used as the training set. This process is repeated 5 rounds, so that each part of the sample is used as the test set for testing. The comprehensive test results of 5 rounds are used as the result of one five-fold cross-validation. The five-fold cross-validation is repeated 100 times, and the obtained statistical results are used as the prediction accuracy of the present invention for Alzheimer's disease. Finally, the average accuracy, precision, recall rate, and F1 score reach 93.2%, 94.55%, 92.6%, and 93.7% respectively. These data indicate that the method proposed by the present invention can effectively distinguish sMCI and pMCI, thereby improving the accuracy of predicting the risk of Alzheimer's disease.

[0102] In a second aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for predicting the risk of Alzheimer's disease based on feature fusion as described in the first aspect of the present invention.

[0103] Among them, the computer-readable storage medium can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0104] The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD ROM); the magnetic surface memory may be a disk memory or a tape memory.

[0105] The volatile memory may be a random access memory (RAM) which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as a static random access memory (SRAM), a synchronous static random access memory (SSRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a sync link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The computer-readable storage medium described in embodiments of the present invention is intended to include these and any other suitable types of memory.

[0106] As Figure 3 shown, in a third aspect, the present invention provides an electronic device 10, including a processor 101 and a storage medium 102, where a computer program is stored on the storage medium, and when the computer program is executed by the processor, it implements the method for predicting the risk of Alzheimer's disease based on feature fusion as described in the first aspect of the present invention.

[0107] In some embodiments, the processor may be implemented by software, hardware, firmware, or a combination thereof, and may use a circuit, a single or multiple application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), central processing units (CPUs), controllers, microcontrollers, microprocessors, etc., at least one of which enables the processor to execute some steps, all steps, or any combination of steps in the method for predicting the risk of Alzheimer's disease based on feature fusion described in each embodiment of the present application.

[0108] Finally, it should be noted that although the above embodiments have been described in the text and drawings of the specification of the present application, the patent protection scope of the present application cannot be limited thereby. Any technical solutions generated by equivalent structure or equivalent process substitution or modification using the content recorded in the text and drawings of the specification of the present application based on the substantial concept of the present application, as well as those directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, etc., are included in the patent protection scope of the present application.

Claims

1. A method for predicting the risk of Alzheimer's disease based on feature fusion, characterized in that, The method includes the following steps: S1. Obtain the sMRI image data, FDG-PET image data, biomarker data, and corresponding intelligence score data of the target subject; S2. Input the sMRI image data and FDG-PET image data into a three-dimensional convolutional neural network module respectively, and extract the first image feature map corresponding to the sMRI image data and the second image feature map corresponding to the FDG-PET image data respectively; S3. Input the first image feature map and the second image feature map into a cross-attention fusion module to obtain an image fusion feature; S4. Input the first image feature map and the second image feature map into a U-shaped upsampling network to segment the hippocampal region and obtain hippocampal features; S5. Combine the image fusion feature, the hippocampal feature, the biomarker data, and the intelligence score data, and input the combined data into an extreme learning machine for calculation to obtain the prediction result of the onset risk of Alzheimer's disease.

2. The method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 1, wherein: The biomarker data includes three Alzheimer's biomarker data of Aβ42, p-Tau, and t-Tau in cerebrospinal fluid; The intelligence score data includes the Mini-Mental State Examination MMSE and the Clinical Dementia Rating Sum of Boxes CDR-SB.

3. The method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 1, wherein, The method further includes: Obtain the sMRI image data and FDG-PET image of the target subject after preprocessing of brain tissue extraction and rigid registration, and the template used in the registration preprocessing process is MNI152.

4. The method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 1, wherein: The three-dimensional convolutional neural network module includes four convolutional sub-modules, and the structures of the four convolutional sub-modules are the same; Each convolutional sub-module includes a first convolutional layer, a second convolutional layer, a first max pooling layer, a first batch normalization layer, and a first ReLU activation function layer connected in sequence. The convolutional kernels of the first convolutional layer and the second convolutional layer are both 3×3×3.

5. A method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 1, characterized in that Step S3 includes the following steps: S31. Concatenate the first image feature map and the second image feature map in the channel dimension to obtain a first concatenated feature map; S32. Input the first concatenated feature map into a global average pooling layer and a global max pooling layer respectively to obtain a global average feature map and a global max feature map; S33. Concatenate the global average feature map and the global max feature map in the channel dimension to obtain a second concatenated feature map, input the second concatenated feature map into a spatial attention module to obtain a spatial attention feature map, and input the global average feature map into a channel attention module to obtain a channel attention feature map; S34. Multiply the spatial attention feature map and the channel attention feature map to obtain an attention feature map; S35. Merge the attention feature map and the first spliced feature map in the channel dimension, input them into a convolutional layer with a convolution kernel of 3×3×3, and then input the output of this convolutional layer into the Sigmoid function to obtain the weight assignment coefficient matrix. The specific expression of the weight assignment coefficient matrix P is as follows: P = (1 + exp(-Conv3d(FS × FC))) -1 ; where exp(·) is the exponential function with base e, Conv3d(·) is the three-dimensional convolution operation, FS is the spatial attention feature map, and FC is the channel attention feature map; S36. Multiply the first image feature map by the weight assignment coefficient matrix P to obtain the first image weight feature map, multiply the second image feature map by the complementary weight coefficient matrix 1 - P to obtain the second image weight feature map, add the first image weight feature map and the second image weight feature map, and input the added data into a convolutional layer with a convolution kernel of 1×1×1 to obtain the fused feature map; S37. Input the fused feature map into the feature extraction module to obtain the image fusion feature.

6. The method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 5, wherein: The feature extraction module includes a third convolutional layer, a fourth convolutional layer, a second global average pooling layer, a second ReLU activation function layer, a first fully connected layer, a Dropout layer with a dropout rate of 0.5, a third ReLU activation function layer, and a second fully connected layer connected in sequence; After splicing the global average feature map and the global maximum feature map in the channel dimension, the spatial attention module inputs them into a convolutional layer with a convolution kernel of 3×3×3 to obtain the spatial attention feature map. The specific expression of the spatial attention feature map FS is as follows: FS = Conv3d(cat(GAP(FMRI), GMP(FPET))); In the formula, Conv3d(·) is the three-dimensional convolution operation, cat(·) is the channel dimension splicing operation, GAP(·) is the global average pooling operation, GMP(·) is the global maximum pooling operation, FMRI is the first image feature map, and FPET is the second image feature map; The channel attention module includes a first channel convolutional layer, a fourth ReLU activation function layer, and a second channel convolutional layer connected in sequence. The convolution kernels of the first channel convolutional layer and the second channel convolutional layer are both 1×1×1. The specific expression of the output channel attention feature map FC is: FS = Conv3d(ReLU(GAP(FMRI))); In the formula, Conv3d(·) is the three-dimensional convolution operation, ReLU(·) is the ReLU activation function, and GAP(·) is the global average pooling operation.

7. The method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 1, wherein: The U-shaped upsampling network includes four upsampling transposed convolutional layers connected in sequence. Each upsampling transposed convolutional layer takes the output of the previous layer and the image feature map of the corresponding level as inputs, gradually increasing the size of the feature map. The last upsampling transposed convolutional layer outputs the segmented area of the hippocampus, and the volume of the hippocampus is used as the hippocampus feature.

8. A method for predicting the risk of Alzheimer's disease based on feature fusion according to claim 1, wherein: The extreme learning machine uses a Gaussian function as the kernel function, and its expression is specifically as follows: f(x) = K(x, x1), K(x, x2), …, K(x, x N )] T ×(Ω + I) -1 Y; where \(x\) is the sample to be measured, \(x\) n is the \(n\)-th training sample, \(N\) is the number of training samples, \(Y\) is the one-hot label matrix corresponding to the training samples, \(\Omega\) is the training sample matrix of \(N\times N\), and each element of \(\Omega\) is \(K(x i ,x j ), \(I\) is the identity matrix, \(K(u, v)\) is the Gaussian kernel function, and the specific expression of \(K(u, v)\) is as follows: K(u,v) = exp(-||u - v|| 2 ×γ -1 ); where exp(·) is the exponential function with base e, and γ is an adjustable parameter.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method for predicting the risk of Alzheimer's disease based on feature fusion according to any one of claims 1 to 8.

10. An electronic device on which a computer program is stored, characterized in that, It includes a processor and a storage medium. A computer program is stored on the storage medium. When the computer program is executed by the processor, it implements the method for predicting the risk of Alzheimer's disease based on feature fusion according to any one of claims 1 to 8.

Images