A method and apparatus for predicting early alzheimer's disease in combination with cognitive features

Abstract

The embodiment of the present application relates to a kind of method and device for predicting early Alzheimer disease in combination with cognitive characteristics, the method comprises: the deep learning model of early Alzheimer disease three classification prediction based on pathological cognitive multi-modal feature is constructed to obtain the first prediction model;By collecting and analyzing the pathological cognitive multi-modal information of three kinds of crowd to construct the first data set;First prediction model is trained based on the first data set;After training, the pathological cognitive multi-modal information of any individual input by user is received as multi-modal feature input first prediction model and is processed to obtain corresponding prediction vector, and the classification type corresponding to the maximum prediction probability in vector is regarded as the prediction result of current individual and is fed back to current user.The present application can improve screening accuracy, reduce the misdiagnosis rate of cognitive resilience individual.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for predicting early Alzheimer's disease by combining cognitive features. Background Technology

[0002] Pathological features for early diagnosis of Alzheimer's disease (AD) include markers of β-amyloid deposition, tau protein deposition, and white matter hypersignaling. Among these, the β-amyloid and tau protein deposition markers are the standardized uptake value ratios (SUVRs) associated with the deposition of these two types of proteins, denoted as SUVR. β SUVR Tau The white matter hyperintensity index is the white matter hyperintensity volume (WMHV). If an individual's pathological characteristics fall within the specified early AD characteristic range, that individual is considered an early AD pathological individual (referred to as a pathological individual). Numerous studies have shown that not all pathological individuals will enter the cognitive decline stage; some pathological individuals do not exhibit significant cognitive decline compared to healthy individuals of the same age, and some may never even develop AD. Neurological research refers to these individuals as individuals with cognitive resilience.

[0003] Based on the aforementioned prior knowledge, experienced medical experts, when conducting early AD screening for individuals, will comprehensively assess their pathological and cognitive characteristics and provide relevant recommendations based on the assessment results. This assessment-recommendation process can be simply summarized as follows: 1) Obtaining three types of pathological features through positron emission tomography (PET) and magnetic resonance imaging (MRI) examinations: SUVR β SUVR Tau1) WMHV; 2) Using the Mini-Mental State Examination (MMSE), Alzheimer's Disease Assessment Scale-Cognitive Subscale (ADAS-Cog), and Clinical Dementia Rating Scale-Sum of... Boxes (CDR-SB) assessment yields three cognitive scores: MMSE score, ADAS-Cog score, and CDR score; 3) A comprehensive analysis is conducted combining the three pathological features and the three cognitive scores: if the three pathological features do not clearly fall within the early AD feature range, the individual is considered healthy; if some or all of the three pathological features fall within the early AD feature range but the three cognitive scores do not show obvious signs of cognitive decline, the individual is considered cognitively resilient; if the three pathological features fall within the early AD feature range and the three cognitive scores all show signs of cognitive decline, the individual is considered an early AD individual; 4) For cognitively resilient individuals, experts generally conduct follow-up observations but do not recommend immediate active treatment; for early AD individuals, experts recommend immediate active treatment.

[0004] Currently, the aforementioned comprehensive assessment methods are mainly completed through manual evaluation. Due to limitations imposed by human experience, the stability of the assessment quality of this traditional mechanism is difficult to guarantee. Doctors with different levels of experience may arrive at completely different assessment results for the same individual, leading to an increased rate of misdiagnosis of cognitively resilient individuals during early AD screening. Furthermore, recent neurological research has further discovered that the system segregation (SyS) of functional connectivity (FC) and the structural network efficiency (SNE) of structural connectivity (SC) are both significantly positively correlated with an individual's cognitive resilience.

[0005] If the two types of neurological features (SyS, SNE) can be integrated into the comprehensive assessment of early AD screening, and a technical solution can be designed to automatically screen based on three types of pathological features, three types of cognitive scores, and two types of neurological features, then not only can stable assessment quality be output, but screening accuracy can also be further improved and the misdiagnosis rate of cognitively resilient individuals can be reduced. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, device, electronic device, and computer-readable storage medium for predicting early Alzheimer's disease by combining cognitive features. This invention first constructs a deep learning model, i.e., the first prediction model, for three-class classification prediction of early Alzheimer's disease based on pathological cognitive multimodal features. The model input parameters, i.e., the multimodal features X, consist of pathological feature components x1, cognitive score components x2, and neurological feature components x3, wherein x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The model employs a multimodal approach to predict AD, with x1 including white matter high signal intensity (WMHV), x2 including MMSE score, ADAS-Cog score, and CDR score, and x3 including system separation degree (SyS) and structural network efficiency (SNE). The model output parameter, the prediction vector Y, consists of three prediction probabilities: y1, y2, and y3, representing the classification types of individuals in the early stages of AD, those with cognitive resilience, and healthy individuals, respectively. A first dataset is constructed by collecting and analyzing multimodal information on pathological cognition from these three groups (early AD individuals, those with cognitive resilience, and healthy individuals). The first prediction model is then trained based on this dataset. After training, the first prediction model is used to predict based on the multimodal features X of any individual, and the classification type corresponding to the highest prediction probability in the prediction vector Y is taken as the prediction result for that individual. This invention provides an end-to-end prediction model capable of automatic screening based on three types of pathological features, three types of cognitive scores, and two types of neurological features. Applying this model to early AD screening helps to output stable assessment quality, improve screening accuracy, and reduce the misdiagnosis rate of individuals with cognitive resilience.

[0007] To achieve the above objectives, a first aspect of the present invention provides a method for predicting early Alzheimer's disease by combining cognitive features, the method comprising: A first prediction model is constructed for a deep learning model that performs three-class classification prediction of early Alzheimer's disease based on pathological cognitive multimodal features. This first prediction model performs three-class classification prediction based on the multimodal features X input to the model and outputs a corresponding prediction vector Y. The multimodal features X include pathological feature components x1, cognitive score components x2, and neurological feature components x3. The pathological feature component x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR TauThe white matter high signal intensity index (WMHV) is used; the cognitive score component x2 includes MMSE score, ADAS-Cog score, and CDR score; the neurological feature component x3 includes system separation degree (SyS) and structural network efficiency (SNE); the prediction vector Y consists of three prediction probabilities y1, y2, and y3; the classification types of the three prediction probabilities y1, y2, and y3 are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; The first dataset was constructed by collecting and analyzing multimodal information on pathological cognition from three groups of people: those in the early stages of Alzheimer's disease (AD), those with cognitive resilience, and healthy individuals. The first prediction model is trained based on the first dataset; After model training, the system receives multimodal information on pathological cognition of any individual from the user; generates corresponding multimodal features X based on the pathological cognition multimodal information, inputs them into the first prediction model for processing, and obtains the corresponding prediction vector Y; and feeds back the classification type corresponding to the highest prediction probability in the prediction vector Y as the prediction result for the current individual to the current user; wherein, the multimodal information on pathological cognition includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau White matter high signal intensity index (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation degree (SyS), and structural network efficiency (SNE).

[0008] Preferably, the first, second, and third model input terminals of the first prediction model are respectively used to receive the pathological feature component x1, the cognitive rating component x2, and the neurological feature component x3 of the multimodal feature X, and the model output terminal is used to output the corresponding prediction vector Y; The first prediction model includes a first MLP model, a second MLP model, a third MLP model, a pathological gate signal layer, a multimodal attention weighting layer, a fusion feature gate layer, a classification feature mapping layer, and a Softmax function layer; The inputs of the first, second, and third MLP models are connected to the inputs of the first, second, and third models, respectively, and their outputs are connected to the first, second, and third inputs of the multimodal attention weighting layer, respectively. The output of the first MLP model is also connected to the input of the pathology gating signal layer. The outputs of the pathology gating signal layer and the multimodal attention weighting layer are connected to the first and second inputs of the fusion feature gating layer, respectively. The output of the fusion feature gating layer is connected to the input of the classification feature mapping layer. The output of the classification feature mapping layer is connected to the input of the Softmax function layer. The output of the Softmax function layer is connected to the model output. The first MLP model is used to encode the pathological feature component x1 to obtain the corresponding feature vector H1, which is then sent to the pathological gating signal layer and the multimodal attention weighting layer. The encoding method of the feature vector H1 is as follows: , The pathological feature component x1 has a shape of 3×1; W1, W2, and W3 are the first, second, and third weight matrices of the model, with shapes of D1×3, D2×D1, and D3×D2, respectively; B1, B2, and B3 are the first, second, and third bias vectors of the model, with shapes of D1×1, D2×1, and D3×1, respectively; D1, D1, and D3 are the first, second, and third feature dimensions of the model; ReLU() is the ReLU activation function; the feature vector H1 has a shape of D3×1. The second MLP model is used to encode the cognitive rating component x2 to obtain the corresponding feature vector H2, which is then sent to the multimodal attention weighting layer. The encoding method of the feature vector H2 is as follows: , The cognitive rating component x2 has a shape of 3×1; W4, W5, and W6 are the fourth, fifth, and sixth weight matrices of the model, with shapes of D4×3, D5×D4, and D6×D5, respectively; B4, B5, and B6 are the fourth, fifth, and sixth bias vectors of the model, with shapes of D4×1, D5×1, and D6×1, respectively; D4, D5, and D6 are the fourth, fifth, and sixth feature dimensions of the model, with D6=D3; the feature vector H2 has a shape of D6×1. The third MLP model is used to encode the neurological feature component x3 to obtain the corresponding feature vector H3, which is then sent to the multimodal attention weighting layer. The encoding method of the feature vector H3 is as follows: , The shape of the neurological feature component x3 is 2×1; W7, W8, and W9 are the seventh, eighth, and ninth weight matrices of the model, with shapes D7×2, D8×D7, and D9×D8, respectively; B7, B8, and B9 are the seventh, eighth, and ninth bias vectors of the model, with shapes D7×1, D8×1, and D9×1, respectively; D7, D8, and D9 are the seventh, eighth, and ninth feature dimensions of the model, with D9=D3; the shape of the feature vector H3 is D9×1. The pathological gating signal layer is used to extract the pathological gating signal features based on the feature vector H1 to obtain the corresponding gating feature vector H4 and send it to the fused feature gating layer. The gated feature vector H4 is extracted in the following way: ; W G Let D be the gating weight matrix of the model. G ×D3;B G Let D be the gating bias vector of the model. G ×1;D G D is the gating feature dimension of the model. G =D3; Sigmoid() is the Sigmoid activation function; the shape of the gated feature vector H4 is D. G ×1; The multimodal attention weighting layer is used to perform feature fusion on feature vectors H1, H2, and H3 according to the feature channel concatenation method to obtain a fused feature vector R; and to calculate attention weights based on the fused feature vector R to obtain a weight vector A; and to perform attention weighted summation on the feature vectors H1, H2, and H3 based on the weight vector A to obtain the corresponding feature vector H5, which is then sent to the fused feature gating layer. The weight vector A is calculated as follows: ; The shape of the fused feature vector R is (D3+D6+D9)×1, that is, 3D3×1; W a1 W a2 Here are the first and second attention weight matrices of the model, with shapes D and D respectively. a1 ×(D3+D6+D9), 3×D a1 B a1 B a2 These are the first and second attention bias vectors of the model, with shapes D and D, respectively. a1 ×1, 3×1; tanh() is the tanh activation function; the weight vector A has a shape of 3×1 and is composed of three weight scalars a1, a2, a3; The weighted summation of the feature vector H5 is calculated as follows: ; The shape of the feature vector H5 is D3×1; The fusion feature gating layer is used to perform feature enhancement processing on the feature vector H5 using the gating feature vector H4 to obtain the corresponding feature vector H6, which is then sent to the classification feature mapping layer. The feature enhancement method for feature vector H6 is as follows: ; W 10 This is the tenth weight matrix of the model, with shape D. 10×D3;B 10 This is the tenth bias vector of the model, with shape D. 10 ×1;D 10 D is the tenth feature dimension of the model. 10 =D3; I is a preset all-1 vector with a shape of D3×1; the shape of the feature vector H6 is D3×1; The classification feature mapping layer is used to perform three-class classification feature vector mapping processing on the feature vector H6 to obtain the corresponding feature vector H7 and send it to the Softmax function layer; The feature vector H7 is calculated as follows: ; W 11 This is the eleventh weight matrix of the model, with a shape of 3×D. 10 B 11 The eleventh bias vector of the model has a shape of 3×1; the feature vector H7 has a shape of 3×1 and consists of three scalar data. , , composition; The Softmax function layer is used to calculate the corresponding three-class prediction probabilities y1, y2, and y3 based on the feature vector H7 using the Softmax function; and to form the corresponding prediction vector Y from the three-class prediction probabilities y1, y2, and y3 and output it. The calculation methods for the three types of prediction probabilities y1, y2, and y3 are as follows: , , .

[0009] Preferably, the first dataset includes multiple first data records; each first data record includes a first training feature vector and a first label vector; the first training feature vector consists of a set of corresponding pathological feature components x1, cognitive score components x2, and neurological feature components x3; the first label vector consists of three types of label probabilities. , , Composition; the probabilities of the three types of tags , , The classification types are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; the probabilities of the three types of labels are... , , There is one and only one type of label with a probability of 1, while the probabilities of the other two types are both 0.

[0010] Preferably, the step of constructing the first dataset by collecting and analyzing multimodal information on pathological cognition from three groups of people specifically includes: Step 4-1: Collect the original medical images and cognitive assessment information of multiple individuals in the early stage of AD from the public dataset to form the first volume dataset, and collect the original medical images and cognitive assessment information of multiple healthy individuals to form the second volume dataset. The publicly available dataset includes the ADNI dataset; The original medical images include PET images, T2-weighted images, fMRI images, and DTI images; The cognitive assessment information includes MMSE score, ADAS-Cog score, and CDR score; The first individual dataset includes multiple first individual records; each first individual record includes the original medical images and the cognitive assessment information. The second individual dataset includes multiple second individual records; each second individual record includes the original medical images and the cognitive assessment information. Step 4-2: Recruit volunteers who have previously undergone early AD screening to form a corresponding volunteer group; and with the authorization of each volunteer and their family, collect data from the original medical images and cognitive assessment information generated by each volunteer during their early AD screening to generate a corresponding third individual dataset. The third-body dataset includes multiple third-body records; the third-body records include the original medical images and the cognitive assessment information. Step 4-3: Organize an expert team to comprehensively evaluate whether each of the third-person records meets the individual conditions for cognitive resilience, and delete the third-person records that do not meet the conditions from the third-person dataset; The expert team consists of several experts in psychology and cognitive science, neuroscience, and clinical medicine in the field of Alzheimer's disease research. Step 4-4: Take each of the first individual records in the first individual dataset, or each of the second individual records in the second individual dataset, or each of the third individual records in the third individual dataset as the corresponding current individual record; Steps 4-5: Based on the PET images and T2-weighted images recorded by the current individual, perform analysis on β-amyloid deposition indices, Tau protein deposition indices, and white matter high signal indices to obtain the corresponding β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The pathological feature component x1 is composed of the white matter high signal index WMHV; Steps 4-6: Based on the fMRI images recorded by the current individual, the system separation degree of brain structural connectivity is analyzed to obtain the corresponding system separation degree SyS; and based on the DTI images recorded by the current individual, the structural network efficiency of brain functional connectivity is analyzed to obtain the corresponding structural network efficiency SNE; and the obtained system separation degree SyS and structural network efficiency SNE are combined to form the corresponding neurological feature component x3. Steps 4-7: The cognitive score component x2 is composed of the MMSE score, ADAS-Cog score, and CDR score recorded by the current individual. Steps 4-8: Identify the individual type corresponding to the current individual record; if the individual type is an early-stage AD individual, then set the corresponding three-category label probabilities. , , The values ​​are 1, 0, and 0; if the individual type is a healthy individual, then the corresponding three-category label probabilities are set. , , The values ​​are 0, 0, and 1; if the individual type is a cognitively resilient individual, then the corresponding three types of label probabilities are set. , , 0, 1, 0; Steps 4-9: The first training feature vector is composed of the pathological feature component x1, the cognitive score component x2, and the neurological feature component x3 corresponding to the current individual record; and the three-category label probabilities corresponding to the current individual record are... , , The first label vector is formed accordingly; and the first training feature vector and the first label vector corresponding to the current individual record are combined to form a corresponding first data record; Steps 4-10: The first dataset is composed of all the first data records obtained.

[0011] Preferably, training the first prediction model based on the first dataset specifically includes: Step 51: Based on a preset first segmentation ratio, the first dataset is divided into two sub-datasets, denoted as the first training set and the first evaluation set; Both the first training set and the first evaluation set consist of multiple first data records; the ratio of the total number of records N1 in the first training set to the total number of records N2 in the first evaluation set satisfies the first segmentation ratio; in the first training set, the record ratios corresponding to individuals with early-stage AD, cognitively resilient individuals, and healthy individuals are denoted as the corresponding... , , ; Where 1 ≤ type index j ≤ 3; Step 52: Take each of the first data records in the first training set as the corresponding current training record; and input the first training feature vector of the current training record as the current multimodal feature X into the first prediction model for processing, and record the prediction vector Y obtained in this processing as the corresponding prediction vector. And record the first label vector of the current training record as the corresponding label vector. ; and by the prediction vector and the label vector Form the corresponding first prediction-label pair ( , ); Where 1 ≤ sample index i ≤ N1; The prediction vector The three types of prediction probabilities y1, y2, and y3 are denoted as the corresponding prediction probabilities. , , The label vector The three types of label probabilities , , Let be the corresponding predicted probability. , , ; Step 53, the obtained N1 first prediction-label pairs ( , Substitute the preset model loss function L M The corresponding first loss value is obtained through calculation; Wherein, the model loss function L M Implemented based on the Focal Loss function, specifically as follows: , , ; w j Let w be the weight coefficient for class j, and w be the weight coefficient for all three classes. j The sum is 3; γ is the focusing parameter, γ≥0; Step 54: Identify whether the first loss value meets the preset first loss value range; if it does, proceed to step 55; if not, based on the preset first model optimizer, move towards making the model loss function L... MThe direction that reaches the minimum value modulates the model parameters of the first prediction model in one round, and returns to step 52 when the modulation ends; The first model optimizer includes the Adam optimizer and the SGD optimizer. Step 55: Take each of the first data records in the first evaluation set as the corresponding current evaluation record; and take the first training feature vector of the current evaluation record as the current multimodal feature X as the input to the first prediction model for processing, and form a corresponding second prediction-label pair with the prediction vector Y obtained in this processing and the first label vector of the current evaluation record. Step 56: Based on the obtained N2 second prediction-label pairs, evaluate the F1 value corresponding to each individual classification to obtain the corresponding first, second, and third F1 values; and take the average of the first, second, and third F1 values ​​as the corresponding overall F1 value; and identify the first, second, and third F1 values ​​and the overall F1 value; if the first F1 value does not meet the preset first F1 value range, or the second F1 value does not meet the preset second F1 value range, or the third F1 value does not meet the preset third F1 value range, or the overall F1 value does not meet the preset overall F1 value range, then return to step 51 to continue training; if the first F1 value meets the first F1 value range, and the second F1 value meets the second F1 value range, and the third F1 value meets the third F1 value range, and the overall F1 value meets the overall F1 value range, then stop training and confirm that the training of the first prediction model has ended.

[0012] A second aspect of the present invention provides an apparatus for implementing the method for predicting early Alzheimer's disease by combining cognitive features as described in the first aspect above. The apparatus includes: a model building module, a dataset preparation module, a model training module, and a model application module. The model building module constructs a first prediction model for a deep learning model that performs three-class classification prediction of early Alzheimer's disease based on pathological cognitive multimodal features. The first prediction model performs three-class classification prediction based on the multimodal features X input to the model and outputs a corresponding prediction vector Y. The multimodal features X include pathological feature components x1, cognitive score components x2, and neurological feature components x3. The pathological feature component x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR TauThe white matter high signal intensity index (WMHV) is used; the cognitive score component x2 includes MMSE score, ADAS-Cog score, and CDR score; the neurological feature component x3 includes system separation degree (SyS) and structural network efficiency (SNE); the prediction vector Y consists of three prediction probabilities y1, y2, and y3; the classification types of the three prediction probabilities y1, y2, and y3 are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; The dataset preparation module is used to construct the first dataset by collecting and analyzing pathological cognitive multimodal information from three groups of people; wherein, the three groups of people include people in the early stage of AD, people with cognitive resilience, and healthy people; The model training module trains the first prediction model based on the first dataset; The model application module is used to receive pathological cognitive multimodal information of any individual input by the user after model training; generate corresponding multimodal features X based on the pathological cognitive multimodal information, input them into the first prediction model for processing to obtain the corresponding prediction vector Y; and feed back the classification type corresponding to the highest prediction probability in the prediction vector Y as the prediction result for the current individual to the current user; wherein, the pathological cognitive multimodal information includes the β-amyloid deposition index SUVR β Tau protein deposition index SUVR Tau White matter high signal intensity index (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation degree (SyS), and structural network efficiency (SNE).

[0013] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver; The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0015] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for predicting early Alzheimer's disease by combining cognitive features. As described above, this invention constructs a deep learning model, or first prediction model, for three-category prediction of early Alzheimer's disease based on pathological cognitive multimodal features. The model input parameters, i.e., the multimodal features X, consist of pathological feature component x1, cognitive score component x2, and neurological feature component x3, wherein x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The model uses three parameters: 1) White matter high signal intensity (WMHV); 2) MMSE score, ADAS-Cog score, and CDR score; and 3) System separation degree (SyS) and structural network efficiency (SNE). The model output parameter, i.e., the prediction vector Y, consists of three prediction probabilities: y1, y2, and y3, representing the classification types of individuals in the early stages of AD, those with cognitive resilience, and healthy individuals, respectively. A first dataset was constructed by collecting and analyzing multimodal information on pathological cognition from these three groups (early AD individuals, cognitively resilient individuals, and healthy individuals). The first prediction model was then trained based on this dataset. After training, the first prediction model was used to predict based on the multimodal features X of any individual, and the classification type corresponding to the highest prediction probability in the prediction vector Y was taken as the prediction result for the current individual. This invention provides an end-to-end prediction model capable of automatic screening based on three types of pathological features, three types of cognitive scores, and two types of neurological features. Applying this model to early AD screening improves the stability of assessment quality, increases screening accuracy, and reduces the misdiagnosis rate for cognitively resilient individuals. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a method for predicting early Alzheimer's disease by combining cognitive features, provided in Embodiment 1 of the present invention. Figure 2 A schematic diagram of the modules of the first prediction model provided in Embodiment 1 of the present invention; Figure 3 This is a module structure diagram of a device for predicting early Alzheimer's disease by combining cognitive features, provided in Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0018] Embodiment 1 of the present invention provides a method for predicting early Alzheimer's disease by combining cognitive characteristics, such as... Figure 1 The schematic diagram of a method for predicting early Alzheimer's disease by combining cognitive features provided in Embodiment 1 of the present invention is shown. The method mainly includes the following steps: Step 1: Construct the deep learning model for early Alzheimer's disease three-classification prediction based on pathological cognitive multimodal features to obtain the corresponding first prediction model.

[0019] Here, the pathological cognitive multimodal features in this embodiment of the invention refer to those including the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau Multimodal characteristics of white matter high signal intensity index (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation degree (SyS), and structured network efficiency (SNE).

[0020] SUVR, a marker of β-amyloid deposition β SUVR is used to quantitatively measure the density of β-amyloid plaques in the brain. β Elevated levels indicate worsening β-amyloid plaque deposition and an increased probability of early AD events. SUVR β The imaging is based on individual PET images. (The last part, "Obtaining SUVR," appears to be an unrelated fragment and is left untranslated.) β In this process, multiple regions of interest (ROIs) for Aβ deposition are marked on PET images, such as the frontal lobe, parietal lobe, and posterior cingulate cortex—areas prone to Aβ deposition. Reference regions are then determined, such as the cerebellar gray matter. Finally, the average radioactivity (also known as average pixel brightness) of each Aβ-depositing ROI is calculated. ROIβ The average radioactivity r of the reference region REFβ Perform calculations and for all r ROIβ The average value r averβ Perform the calculation, and then use r averβ With r REFβ The ratio of SUVR β .

[0021] Tau protein deposition index SUVR Tau SUVR is used to quantitatively measure the density of Tau protein neurofibrillary tangles in the brain.Tau Elevated levels indicate worsening Tau pathology and an increased probability of early AD events. SUVR Tau The imaging is based on individual PET images. (The last part, "Obtaining SUVR," appears to be an unrelated fragment and is left untranslated.) Tau At that time, multiple Tau deposition ROIs, such as the entorhinal cortex and hippocampus, were marked on PET images; reference regions, such as the cerebellar gray matter, were then determined; and the average radioactivity r of each Tau deposition ROI was then calculated. ROITau The average radioactivity r of the reference region REFTau Perform calculations and for all r ROITau The average value r averTau Perform the calculation, and then use r averTau With r REFTau The ratio of SUVR Tau .

[0022] White matter high signal volume (WMHV) refers to the total volume of white matter damaged by small vessel disease. Elevated WMHV indicates worsening small vessel disease and an increased probability of early Alzheimer's disease (AD) events. WMHV is based on individual T2-weighted images. To obtain WMHV, mature image analysis tools such as SPM, FreeSurfer, and FSL are used to sum the high signal volumes across all white matter regions on T2-weighted images, and then sum the total volume of all white matter regions. The ratio of the summed high signal volume to the summed white matter volume is then used as the WMHV.

[0023] The MMSE score is a score obtained after cognitive assessment based on the Mini-Mental State Examination, ranging from 0 to 30 points; a higher score indicates better cognitive function. The ADAS-Cog score is a score obtained after cognitive assessment based on the Alzheimer's Disease Assessment Scale - Cognitive Subscale, ranging from 0 to 70 points; a lower score indicates better cognitive function. The CDR score is a score obtained after cognitive assessment based on the Clinical Dementia Rating Scale-Extended, calculated by summing scores from six domains; the CDR score ranges from 0 to 18 points, with a lower score indicating better cognitive function. For individuals in the early stages of AD, these three score types show significant differences compared to healthy individuals. For individuals with cognitive resilience, these three score types do not show significant differences compared to healthy individuals.

[0024] System Separation Score (SyS) is used to quantify the relative strength of intra-system connectivity and inter-system connectivity within brain functional systems. A higher SyS score indicates a higher degree of specialization within a functional system. In the early stages of Alzheimer's disease (AD), a decreased SyS score indicates abnormal crosstalk between functional systems, loss of system separation, and reduced cognitive function. The functional systems mentioned here include at least the Default Mode Network (DMN), Sensorimotor Network (SMN), Visual Network (VN), Executive Control Network (ECN), and Dorsal Attention Network (DAN). The imaging basis for SyS is individual fMRI images. When obtaining SyS, an N×N brain region matrix is ​​first constructed from all brain regions of DMN, SMN, VN, ECN, and DAN. Then, based on fMRI images, the functional connectivity strength between every two brain regions in the brain region matrix is ​​calculated, and an N×N functional connectivity matrix, commonly known as the FC matrix, is obtained based on the calculation results. Next, based on the FC matrix, the mean of the functional connectivity strength between all internal brain regions of each functional system is calculated, and the result is used as the corresponding system internal connectivity strength. Then, based on the FC matrix, the mean of the functional connectivity strength between all cross-system brain regions between every two functional systems is calculated, and the result is used as the corresponding system external connectivity strength. For each functional system, the difference between the system internal connectivity strength and the external connectivity strength of each system is used as a corresponding system internal-external strength difference, and the average of all system internal-external strength differences for the current functional system is used as the corresponding single-system separation degree. Finally, the average of all single-system separation degrees is used as the system separation degree SyS in this embodiment of the invention. For individuals in the early stages of AD, the system separation degree SyS is significantly different from that of healthy individuals; for individuals with cognitive resilience, the system separation degree SyS is not significantly different from that of healthy individuals.

[0025] Structural network efficiency (SNE) is used to quantify the global transmission efficiency of the brain's structural networks. A higher SNE indicates that information can be transmitted through the network via shorter paths. In the early stages of Alzheimer's disease (AD), a decreased SNE indicates damage to white matter fibers, reduced brain communication efficiency, and impaired cognitive function. The imaging basis for SNE is an individual's digital intravascular coagulation (DTI) images. When acquiring the SNE, an N×N brain region matrix is ​​first constructed from all brain regions of DMN, SMN, VN, ECN, and DAN. Then, based on DTI images, the number or density of white matter fibers between every two brain regions in the brain region matrix is ​​statistically analyzed, and the corresponding structural connectivity strength is generated based on the statistical results. An N×N structural connectivity matrix, commonly known as the SC matrix, is then generated based on all the obtained structural connectivity strengths. Next, according to the correspondence that connectivity distance = 1 / structural connectivity strength, an N×N connectivity distance matrix, commonly known as the D matrix, is generated based on the SC matrix. Then, using a shortest path search algorithm, such as Dijkstra's algorithm, the shortest path length between every two brain regions is calculated based on the D matrix to obtain an N×N path length matrix L. Then, according to the correspondence that transmission efficiency = 1 / path length, an N×N transmission efficiency matrix E is generated based on the path length matrix L. Finally, the mean of the N×(N-1) transmission efficiencies in the transmission efficiency matrix E (excluding the diagonal) is calculated, and the calculation result is used as the SNE of this embodiment of the invention. For individuals in the early stages of Alzheimer's disease (AD), the structural network efficiency (SNE) differs significantly from that of healthy individuals; for individuals with cognitive resilience, the SNE does not differ significantly from that of healthy individuals.

[0026] The first prediction model in this embodiment of the invention is used to perform three-class classification prediction based on the multimodal features X input to the model and output the corresponding prediction vector Y. The multimodal features X include pathological feature component x1, cognitive score component x2, and neurological feature component x3; the pathological feature component x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The white matter high signal intensity index (WMHV) is used; the cognitive score component x2 includes MMSE score, ADAS-Cog score, and CDR score; the neurological feature component x3 includes system separation degree (SyS) and structural network efficiency (SNE). The prediction vector Y consists of three classes of prediction probabilities y1, y2, and y3; the classification types of the three classes of prediction probabilities y1, y2, and y3 are respectively early AD individuals, cognitively resilient individuals, and healthy individuals.

[0027] like Figure 2As shown in the schematic diagram of the first prediction model provided in Embodiment 1 of the present invention, the first, second and third model input terminals of the first prediction model of the present invention are used to receive the pathological feature component x1, the cognitive score component x2 and the neurological feature component x3 of the multimodal feature X, respectively, and the model output terminal is used to output the corresponding prediction vector Y.

[0028] like Figure 2 As shown, the model components of the first prediction model include: a first MLP model, a second MLP model, a third MLP model, a pathological gating signal layer, a multimodal attention weighting layer, a fusion feature gating layer, a classification feature mapping layer, and a Softmax function layer.

[0029] like Figure 2 As shown, the connection relationships of the model components of the first prediction model are as follows: the inputs of the first, second, and third MLP models are connected to the inputs of the first, second, and third models, respectively, and their outputs are connected to the first, second, and third inputs of the multimodal attention weighting layer, respectively; the output of the first MLP model is also connected to the input of the pathology gating signal layer; the outputs of the pathology gating signal layer and the multimodal attention weighting layer are connected to the first and second inputs of the fusion feature gating layer, respectively; the output of the fusion feature gating layer is connected to the input of the classification feature mapping layer; the output of the classification feature mapping layer is connected to the input of the Softmax function layer; and the output of the Softmax function layer is connected to the model output.

[0030] The functions of the model components of the first prediction model in this embodiment of the invention are as follows.

[0031] 1) First MLP model: The first MLP model in this embodiment of the invention is used to encode the pathological feature component x1 to obtain the corresponding feature vector H1, which is then sent to the pathological gating signal layer and the multimodal attention weighting layer.

[0032] Here, the encoding method of feature vector H1 in this embodiment of the invention is as follows: ; Wherein, the pathological feature component x1 has a shape of 3×1; W1, W2, and W3 are the first, second, and third weight matrices of the model, with shapes of D1×3, D2×D1, and D3×D2, respectively; B1, B2, and B3 are the first, second, and third bias vectors of the model, with shapes of D1×1, D2×1, and D3×1, respectively; D1, D1, and D3 are the first, second, and third feature dimensions of the model; ReLU() is the ReLU activation function; and the feature vector H1 has a shape of D3×1.

[0033] 2) Second MLP model: The second MLP model in this embodiment of the invention is used to encode the cognitive rating component x2 to obtain the corresponding feature vector H2 and send it to the multimodal attention weighting layer.

[0034] Here, the encoding method of feature vector H2 in this embodiment of the invention is as follows: ; Among them, the cognitive rating component x2 has a shape of 3×1; W4, W5, and W6 are the fourth, fifth, and sixth weight matrices of the model, with shapes of D4×3, D5×D4, and D6×D5, respectively; B4, B5, and B6 are the fourth, fifth, and sixth bias vectors of the model, with shapes of D4×1, D5×1, and D6×1, respectively; D4, D5, and D6 are the fourth, fifth, and sixth feature dimensions of the model, with D6=D3; and the feature vector H2 has a shape of D6×1.

[0035] 3) Third MLP model: The third MLP model in this embodiment of the invention is used to encode the neurological feature component x3 to obtain the corresponding feature vector H3 and send it to the multimodal attention weighting layer.

[0036] Here, the encoding method of feature vector H3 in this embodiment of the invention is as follows: ; Among them, the shape of the neurological feature component x3 is 2×1; W7, W8, and W9 are the seventh, eighth, and ninth weight matrices of the model, with shapes D7×2, D8×D7, and D9×D8, respectively; B7, B8, and B9 are the seventh, eighth, and ninth bias vectors of the model, with shapes D7×1, D8×1, and D9×1, respectively; D7, D8, and D9 are the seventh, eighth, and ninth feature dimensions of the model, with D9=D3; and the shape of the feature vector H3 is D9×1.

[0037] 4) Pathological gating signal layer: In this embodiment of the invention, the pathological gating signal layer is used to extract the pathological gating signal features based on the feature vector H1 to obtain the corresponding gating feature vector H4, which is then sent to the fusion feature gating layer.

[0038] Here, the method for extracting the gated feature vector H4 in this embodiment of the invention is as follows: ; Among them, W G Let D be the gating weight matrix of the model. G ×D3;B G Let D be the gating bias vector of the model. G ×1;D G D is the gating feature dimension of the model. G=D3; Sigmoid() is the Sigmoid activation function; the shape of the gated feature vector H4 is D. G ×1.

[0039] 5) Multimodal attention weighted layer: The multimodal attention weighting layer in this embodiment of the invention is used to perform feature fusion on feature vectors H1, H2, and H3 according to the feature channel splicing method to obtain a fused feature vector R; and to calculate the attention weights based on the fused feature vector R to obtain a weight vector A; and to perform attention weighted summation on feature vectors H1, H2, and H3 based on the weight vector A to obtain the corresponding feature vector H5, which is then sent to the fused feature gating layer.

[0040] Here, the weight vector A in this embodiment of the invention is calculated as follows: ; The shape of the fused feature vector R is (D3+D6+D9)×1, or 3D3×1; W a1 W a2 Here are the first and second attention weight matrices of the model, with shapes D and D respectively. a1 ×(D3+D6+D9), 3×D a1 B a1 B a2 These are the first and second attention bias vectors of the model, with shapes D and D, respectively. a1 ×1, 3×1; tanh() is the tanh activation function; the weight vector A has a shape of 3×1 and consists of three weight scalars a1, a2, and a3.

[0041] The weighted summation calculation method for feature vector H5 in this embodiment of the invention is as follows: ; The shape of the feature vector H5 is D3×1.

[0042] 6) Feature-gating layer: In this embodiment of the invention, the fusion feature gating layer is used to perform feature enhancement processing on feature vector H5 using gating feature vector H4 to obtain the corresponding feature vector H6, which is then sent to the classification feature mapping layer.

[0043] Here, the feature enhancement method of feature vector H6 in this embodiment of the invention is as follows: ; Among them, W 10 This is the tenth weight matrix of the model, with shape D. 10 ×D3;B 10 This is the tenth bias vector of the model, with shape D. 10 ×1;D 10D is the tenth feature dimension of the model. 10 =D3; I is a preset all-1 vector with a shape of D3×1; the shape of the feature vector H6 is D3×1.

[0044] It should be noted that when the current individual is in the early stage of AD, the fusion feature gating layer can unilaterally enhance the pathological features in feature vector H6 through gating feature vector H4. The vector can unilaterally weaken the comprehensive cognitive features in feature vector H6 that are unrelated to pathological features; when the current individual is a cognitively resilient individual, it can be achieved by gating feature vector H4 and its corresponding... The feature vector H6 can enhance both the pathological features and the comprehensive cognitive features in the feature vector H6; when the current individual is healthy, the pathological features in the feature vector H6 can be weakened unilaterally by gating the feature vector H4. The vector can unilaterally enhance the comprehensive cognitive features in feature vector H6 that are unrelated to pathological features. In other words, this embodiment of the invention can achieve dynamic feature enhancement processing of feature vector H6 by fusing feature gating layers, thereby improving the distinction between the three types of features and improving prediction accuracy.

[0045] 7) Classification Feature Mapping Layer: In this embodiment of the invention, the classification feature mapping layer is used to perform three-class classification feature vector mapping processing on feature vector H6 to obtain the corresponding feature vector H7, which is then sent to the Softmax function layer.

[0046] Here, the feature vector H7 in this embodiment of the invention is calculated as follows: ; Among them, W 11 This is the eleventh weight matrix of the model, with a shape of 3×D. 10 B 11 The eleventh bias vector of the model has a shape of 3×1; the feature vector H7 has a shape of 3×1 and consists of three scalar data. , , composition.

[0047] 8) Softmax function layer: The Softmax function layer in this embodiment of the invention is used to calculate the corresponding three-class prediction probabilities y1, y2, and y3 based on the feature vector H7 using the Softmax function; and the corresponding prediction vector Y is composed of the three-class prediction probabilities y1, y2, and y3 and output.

[0048] Here, the calculation method for the three types of prediction probabilities y1, y2, and y3 in this embodiment of the invention is as follows: , , .

[0049] Step 2: Construct the first dataset by collecting and analyzing multimodal information on pathological cognition from three groups of people; Among them, the three groups include people in the early stages of Alzheimer's disease (AD), people with cognitive resilience, and healthy people; Specifically, it includes: Step 2-1, collecting original medical images and cognitive assessment information of multiple individuals in the early stage of AD from public datasets to form the first volume dataset, and collecting original medical images and cognitive assessment information of multiple healthy individuals to form the second volume dataset. Here, the publicly available datasets in this embodiment of the invention include the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset; The original medical images in this embodiment of the invention include PET images, T2-weighted images, fMRI images, and DTI images; the cognitive assessment information in this embodiment of the invention includes MMSE scores, ADAS-Cog scores, and CDR scores; the first volume dataset in this embodiment of the invention includes multiple first volume records; each first volume record includes original medical images and cognitive assessment information; the second volume dataset in this embodiment of the invention includes multiple second volume records; each second volume record includes original medical images and cognitive assessment information. Step 2-2: Recruit volunteers who have previously undergone early AD screening to form a corresponding volunteer group; and with the authorization of each volunteer and their family, collect data from the original medical images and cognitive assessment information generated by each individual volunteer during their early AD screening to generate a corresponding third individual dataset. Here, the third volume dataset of this embodiment of the invention includes multiple third volume records; the third volume records include original medical images and cognitive assessment information; Steps 2-3: Organize an expert team to conduct a comprehensive evaluation of whether each third-person record meets the individual conditions for cognitive resilience, and delete the third-person records that do not meet the conditions from the third-person dataset; Here, the expert team in this embodiment of the invention consists of multiple experts in psychology and cognitive science, neuroscience, and clinical medicine in the field of Alzheimer's disease research. Steps 2-4: Take each first individual record of the first individual dataset, or each second individual record of the second individual dataset, or each third individual record of the third individual dataset as the corresponding current individual record; Steps 2-5 involve analyzing β-amyloid deposition indices, Tau protein deposition indices, and white matter high signal indices based on the currently recorded PET and T2-weighted images of the individual to obtain the corresponding β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The pathological feature component x1 is composed of the white matter high signal index WMHV; Steps 2-6: Analyze the system separation degree of brain structural connectivity based on the fMRI images recorded by the current individual to obtain the corresponding system separation degree SyS; and analyze the structural network efficiency of brain functional connectivity based on the DTI images recorded by the current individual to obtain the corresponding structural network efficiency SNE; and combine the obtained system separation degree SyS and structural network efficiency SNE to form the corresponding neurological feature component x3. Steps 2-7: The corresponding cognitive score component is composed of the MMSE score, ADAS-Cog score, and CDR score recorded by the current individual. Steps 2-8 involve identifying the individual type corresponding to the current individual record; if the individual type is an early stage AD individual, then setting the corresponding three-category label probabilities. , , The values ​​are 1, 0, and 0; if the individual type is a healthy individual, then the corresponding three-category label probabilities are set. , , The values ​​are 0, 0, and 1; if the individual type is a cognitively resilient individual, then the corresponding three-category label probabilities are set. , , 0, 1, 0; Steps 2-9: The first training feature vector is composed of the pathological feature component x1, the cognitive score component x2, and the neurological feature component x3 corresponding to the current individual record; and the three-class label probabilities corresponding to the current individual record are also included. , , The first label vector is formed by the first training feature vector and the first label vector corresponding to the current individual record; and a corresponding first data record is formed by the first training feature vector and the first label vector corresponding to the current individual record. Steps 2-10 consist of all the first data records obtained, forming the corresponding first dataset.

[0050] Here, the first dataset in this embodiment of the invention includes multiple first data records; each first data record includes a first training feature vector and a first label vector; the first training feature vector consists of a set of corresponding pathological feature components x1, cognitive score components x2, and neurological feature components x3; the first label vector consists of three types of label probabilities. , , Composition; Probability of three types of labels , , The classification types are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; the three label probabilities , , There is one and only one type of label with a probability of 1, while the probabilities of the other two types are both 0.

[0051] Step 3: Train the first prediction model based on the first dataset; Specifically, it includes: Step 31, dividing the first dataset into two subsets based on a preset first segmentation ratio, denoted as the first training set and the first evaluation set; Here, the first segmentation ratio in this embodiment of the invention is a preset ratio parameter, such as 8:2; both the first training set and the first evaluation set are composed of multiple first data records; the ratio of the total number of records N1 in the first training set to the total number of records N2 in the first evaluation set satisfies the first segmentation ratio; It should be noted that the recorded proportions of individuals with early-stage AD, cognitively resilient individuals, and healthy individuals in the first training set are denoted as corresponding proportions. , , Where 1 ≤ type index j ≤ 3; Step 32: Take each first data record of the first training set as the corresponding current training record; and take the first training feature vector of the current training record as the current multimodal feature X and input it into the first prediction model for processing, and record the prediction vector Y obtained from this processing as the corresponding prediction vector. And record the first label vector of the current training record as the corresponding label vector. ; and by the prediction vector and label vector Form the corresponding first prediction-label pair ( , ); Where 1 ≤ sample index i ≤ N1; prediction vector The three prediction probabilities y1, y2, and y3 are denoted as the corresponding prediction probabilities. , , ; tag vector The probability of three types of labels , , Let be the corresponding predicted probability. , , ; Step 33, the obtained N1 first prediction-label pairs ( , Substitute the preset model loss function L M The corresponding first loss value is obtained through calculation; Here, the model loss function L in this embodiment of the invention M Implemented based on the Focal Loss function, specifically as follows: , , ; Among them, w j Let w be the weight coefficient for class j, and w be the weight coefficient for all three classes. j The sum is 3; γ is the focusing parameter in the Focal Loss function, γ≥0; Step 34: Identify whether the first loss value meets the preset first loss value range; if it does, proceed to step 35; if not, based on the preset first model optimizer, move towards making the model loss function L... M The direction that reaches the minimum value modulates the model parameters of the first prediction model in one round, and returns to step 32 when the modulation ends; Here, the first loss value range in this embodiment of the invention is a pre-set numerical range; the first model optimizer includes the Adam optimizer and the SGD optimizer; Step 35: Take each first data record of the first evaluation set as the corresponding current evaluation record; and take the first training feature vector of the current evaluation record as the current multimodal feature X as the input to the first prediction model for processing, and form a corresponding second prediction-label pair with the prediction vector Y obtained by this processing and the first label vector of the current evaluation record. Step 36: Based on the obtained N2 second prediction-label pairs, evaluate the F1 value corresponding to each individual classification to obtain the corresponding first, second, and third F1 values; and take the average of the first, second, and third F1 values ​​as the corresponding overall F1 value; and identify the first, second, and third F1 values ​​and the overall F1 value; if the first F1 value does not meet the preset first F1 value range, or the second F1 value does not meet the preset second F1 value range, or the third F1 value does not meet the preset third F1 value range, or the overall F1 value does not meet the preset overall F1 value range, then return to step 31 to continue training; if the first F1 value meets the first F1 value range, and the second F1 value meets the second F1 value range, and the third F1 value meets the third F1 value range, and the overall F1 value meets the overall F1 value range, then stop training and confirm that the training of the first prediction model has ended.

[0052] Here, the first F1 value range, the second F1 value range, the third F1 value range, and the overall F1 value range in this embodiment of the invention are four preset numerical ranges.

[0053] Step 4: After the model training is completed, the system receives multimodal information on pathological cognition of any individual from the user input; generates corresponding multimodal features X based on the multimodal information on pathological cognition, inputs them into the first prediction model for processing to obtain the corresponding prediction vector Y; and feeds back the classification type corresponding to the highest prediction probability in the prediction vector Y as the prediction result for the current individual to the current user.

[0054] Here, the pathological cognitive multimodal information of any individual in the embodiments of the present invention includes the β-amyloid protein deposition index SUVR. β Tau protein deposition index SUVR Tau White matter high signal intensity index (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation degree (SyS), and structural network efficiency (SNE).

[0055] Figure 3 This is a module structure diagram of a device for predicting early Alzheimer's disease by combining cognitive features, provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes: a model building module 201, a dataset preparation module 202, a model training module 203, and a model application module 204.

[0056] The model construction module 201 constructs a first prediction model for a deep learning model that performs three-class classification prediction of early Alzheimer's disease based on multimodal features of pathological cognition. The first prediction model performs three-class classification prediction based on the multimodal features X input to the model and outputs the corresponding prediction vector Y. The multimodal features X include pathological feature components x1, cognitive score components x2, and neurological feature components x3. The pathological feature component x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The white matter high signal intensity index (WMHV) is used; the cognitive score component x2 includes MMSE score, ADAS-Cog score, and CDR score; the neurological feature component x3 includes system separation degree (SyS) and structural network efficiency (SNE); the prediction vector Y consists of three types of prediction probabilities y1, y2, and y3; the classification types of the three types of prediction probabilities y1, y2, and y3 are respectively early AD individuals, cognitively resilient individuals, and healthy individuals.

[0057] The dataset preparation module 202 is used to construct the first dataset by collecting and analyzing pathological cognitive multimodal information of three groups of people; the three groups of people include people in the early stage of AD, people with cognitive resilience, and healthy people.

[0058] The model training module 203 trains the first prediction model based on the first dataset.

[0059] The model application module 204 is used to receive pathological cognitive multimodal information of any individual input by the user after model training; generate corresponding multimodal features X based on the pathological cognitive multimodal information, input them into the first prediction model for processing to obtain the corresponding prediction vector Y; and use the classification type corresponding to the highest prediction probability in the prediction vector Y as the prediction result for the current individual and feed it back to the current user; wherein, the pathological cognitive multimodal information includes the β-amyloid protein deposition index SUVR β Tau protein deposition index SUVR Tau White matter high signal intensity index (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation degree (SyS), and structural network efficiency (SNE).

[0060] The device for predicting early Alzheimer's disease by combining cognitive features provided in this embodiment of the invention can perform the method steps in the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0061] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing elements; they can be fully implemented in hardware; or some modules can be implemented by processing elements calling software, while others are implemented in hardware. For example, the model building module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0062] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0063] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0064] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 4As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0065] exist Figure 4 The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include Non-Volatile Memory, such as at least one disk storage device.

[0066] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0067] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0068] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for predicting early Alzheimer's disease by combining cognitive features. As described above, this invention constructs a deep learning model, or first prediction model, for three-category prediction of early Alzheimer's disease based on pathological cognitive multimodal features. The model input parameters, i.e., the multimodal features X, consist of pathological feature component x1, cognitive score component x2, and neurological feature component x3, wherein x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The model uses three parameters: 1) White matter high signal intensity (WMHV); 2) MMSE score, ADAS-Cog score, and CDR score; and 3) System separation degree (SyS) and structural network efficiency (SNE). The model output parameter, i.e., the prediction vector Y, consists of three prediction probabilities: y1, y2, and y3, representing the classification types of individuals in the early stages of AD, those with cognitive resilience, and healthy individuals, respectively. A first dataset was constructed by collecting and analyzing multimodal information on pathological cognition from these three groups (early AD individuals, cognitively resilient individuals, and healthy individuals). The first prediction model was then trained based on this dataset. After training, the first prediction model was used to predict based on the multimodal features X of any individual, and the classification type corresponding to the highest prediction probability in the prediction vector Y was taken as the prediction result for the current individual. This invention provides an end-to-end prediction model capable of automatic screening based on three types of pathological features, three types of cognitive scores, and two types of neurological features. Applying this model to early AD screening improves the stability of assessment quality, increases screening accuracy, and reduces the misdiagnosis rate for cognitively resilient individuals.

[0069] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0070] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting early Alzheimer's disease by combining cognitive characteristics, characterized in that, The method includes: A first prediction model is constructed for a deep learning model that performs three-class classification prediction of early Alzheimer's disease based on pathological cognitive multimodal features. This first prediction model performs three-class classification prediction based on the multimodal features X input to the model and outputs a corresponding prediction vector Y. The multimodal features X include pathological feature components x1, cognitive score components x2, and neurological feature components x3. The pathological feature component x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The white matter high signal intensity index (WMHV) is used; the cognitive score component x2 includes MMSE score, ADAS-Cog score, and CDR score; the neurological feature component x3 includes system separation degree (SyS) and structural network efficiency (SNE); the prediction vector Y consists of three prediction probabilities y1, y2, and y3; the classification types of the three prediction probabilities y1, y2, and y3 are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; The first dataset was constructed by collecting and analyzing multimodal information on pathological cognition from three groups of people: those in the early stages of Alzheimer's disease (AD), those with cognitive resilience, and healthy individuals. The first prediction model is trained based on the first dataset; After model training, the system receives multimodal information on pathological cognition of any individual from the user; generates corresponding multimodal features X based on the pathological cognition multimodal information, inputs them into the first prediction model for processing, and obtains the corresponding prediction vector Y; and feeds back the classification type corresponding to the highest prediction probability in the prediction vector Y as the prediction result for the current individual to the current user; wherein, the multimodal information on pathological cognition includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau White matter high signal intensity indicators (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation (SyS), and structural network efficiency (SNE); Wherein, the first, second and third model input terminals of the first prediction model are respectively used to receive the pathological feature component x1, the cognitive rating component x2 and the neurological feature component x3 of the multimodal feature X, and the model output terminal is used to output the corresponding prediction vector Y; The first prediction model includes a first MLP model, a second MLP model, a third MLP model, a pathological gate signal layer, a multimodal attention weighting layer, a fusion feature gate layer, a classification feature mapping layer, and a Softmax function layer; The inputs of the first, second, and third MLP models are connected to the inputs of the first, second, and third models, respectively, and their outputs are connected to the first, second, and third inputs of the multimodal attention weighting layer, respectively. The output of the first MLP model is also connected to the input of the pathology gating signal layer. The outputs of the pathology gating signal layer and the multimodal attention weighting layer are connected to the first and second inputs of the fusion feature gating layer, respectively. The output of the fusion feature gating layer is connected to the input of the classification feature mapping layer. The output of the classification feature mapping layer is connected to the input of the Softmax function layer. The output of the Softmax function layer is connected to the model output. The first MLP model is used to encode the pathological feature component x1 to obtain the corresponding feature vector H1, which is then sent to the pathological gating signal layer and the multimodal attention weighting layer. The encoding method of the feature vector H1 is as follows: ， The pathological feature component x1 has a shape of 3×1; W1, W2, and W3 are the first, second, and third weight matrices of the model, with shapes of D1×3, D2×D1, and D3×D2, respectively; B1, B2, and B3 are the first, second, and third bias vectors of the model, with shapes of D1×1, D2×1, and D3×1, respectively; D1, D1, and D3 are the first, second, and third feature dimensions of the model; ReLU() is the ReLU activation function; the feature vector H1 has a shape of D3×1. The second MLP model is used to encode the cognitive rating component x2 to obtain the corresponding feature vector H2, which is then sent to the multimodal attention weighting layer. The encoding method of the feature vector H2 is as follows: ， The cognitive rating component x2 has a shape of 3×1; W4, W5, and W6 are the fourth, fifth, and sixth weight matrices of the model, with shapes of D4×3, D5×D4, and D6×D5, respectively; B4, B5, and B6 are the fourth, fifth, and sixth bias vectors of the model, with shapes of D4×1, D5×1, and D6×1, respectively; D4, D5, and D6 are the fourth, fifth, and sixth feature dimensions of the model, with D6=D3; the feature vector H2 has a shape of D6×1. The third MLP model is used to encode the neurological feature component x3 to obtain the corresponding feature vector H3, which is then sent to the multimodal attention weighting layer. The encoding method of the feature vector H3 is as follows: ， The shape of the neurological feature component x3 is 2×1; W7, W8, and W9 are the seventh, eighth, and ninth weight matrices of the model, with shapes D7×2, D8×D7, and D9×D8, respectively; B7, B8, and B9 are the seventh, eighth, and ninth bias vectors of the model, with shapes D7×1, D8×1, and D9×1, respectively; D7, D8, and D9 are the seventh, eighth, and ninth feature dimensions of the model, with D9=D3; the shape of the feature vector H3 is D9×1. The pathological gating signal layer is used to extract the pathological gating signal features based on the feature vector H1 to obtain the corresponding gating feature vector H4 and send it to the fused feature gating layer. The gated feature vector H4 is extracted in the following way: ； W G Let D be the gating weight matrix of the model. G ×D3;B G Let D be the gating bias vector of the model. G ×1;D G D is the gating feature dimension of the model. G =D3; Sigmoid() is the Sigmoid activation function; the shape of the gated feature vector H4 is D. G ×1; The multimodal attention weighting layer is used to perform feature fusion on feature vectors H1, H2, and H3 according to the feature channel concatenation method to obtain a fused feature vector R; and to calculate attention weights based on the fused feature vector R to obtain a weight vector A; and to perform attention weighted summation on the feature vectors H1, H2, and H3 based on the weight vector A to obtain the corresponding feature vector H5, which is then sent to the fused feature gating layer. The weight vector A is calculated as follows: ； The shape of the fused feature vector R is (D3+D6+D9)×1, that is, 3D3×1; W a1 W a2 Here are the first and second attention weight matrices of the model, with shapes D and D respectively. a1 ×(D3+D6+D9), 3×D a1 B a1 B a2 These are the first and second attention bias vectors of the model, with shapes D and D, respectively. a1 ×1, 3×1; tanh() is the tanh activation function; the weight vector A has a shape of 3×1 and is composed of three weight scalars a1, a2, a3; The weighted summation of the feature vector H5 is calculated as follows: ； The shape of the feature vector H5 is D3×1; The fusion feature gating layer is used to perform feature enhancement processing on the feature vector H5 using the gating feature vector H4 to obtain the corresponding feature vector H6, which is then sent to the classification feature mapping layer. The feature enhancement method for feature vector H6 is as follows: ； W 10 This is the tenth weight matrix of the model, with shape D. 10 ×D3;B 10 This is the tenth bias vector of the model, with shape D. 10 ×1;D 10 D is the tenth feature dimension of the model. 10 =D3; I is a preset all-1 vector with a shape of D3×1; the shape of the feature vector H6 is D3×1; The classification feature mapping layer is used to perform three-class classification feature vector mapping processing on the feature vector H6 to obtain the corresponding feature vector H7 and send it to the Softmax function layer; The feature vector H7 is calculated as follows: ； W 11 This is the eleventh weight matrix of the model, with a shape of 3×D. 10 B 11 The eleventh bias vector of the model has a shape of 3×1; the feature vector H7 has a shape of 3×1 and consists of three scalar data. , , composition; The Softmax function layer is used to calculate the corresponding three-class prediction probabilities y1, y2, and y3 based on the feature vector H7 using the Softmax function; and to form the corresponding prediction vector Y from the three-class prediction probabilities y1, y2, and y3 and output it. The calculation methods for the three types of prediction probabilities y1, y2, and y3 are as follows: ， ， 。 2. The method for predicting early Alzheimer's disease by combining cognitive characteristics according to claim 1, characterized in that, The first dataset includes multiple first data records; each first data record includes a first training feature vector and a first label vector; the first training feature vector consists of a set of corresponding pathological feature components x1, cognitive score components x2, and neurological feature components x3; the first label vector consists of three types of label probabilities. , , Composition; the probabilities of the three types of tags , , The classification types are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; the probabilities of the three types of labels are... , , There is one and only one type of label with a probability of 1, while the probabilities of the other two types are both 0.

3. The method for predicting early Alzheimer's disease by combining cognitive characteristics according to claim 2, characterized in that, The first dataset is constructed by collecting and analyzing multimodal information on pathological cognition from three groups of people, specifically including: Step 4-1: Collect the original medical images and cognitive assessment information of multiple individuals in the early stage of AD from the public dataset to form the first volume dataset, and collect the original medical images and cognitive assessment information of multiple healthy individuals to form the second volume dataset. The publicly available dataset includes the ADNI dataset; The original medical images include PET images, T2-weighted images, fMRI images, and DTI images; The cognitive assessment information includes MMSE score, ADAS-Cog score, and CDR score; The first individual dataset includes multiple first individual records; each first individual record includes the original medical images and the cognitive assessment information. The second individual dataset includes multiple second individual records; each second individual record includes the original medical images and the cognitive assessment information. Step 4-2: Recruit volunteers who have previously undergone early AD screening to form a corresponding volunteer group; and with the authorization of each volunteer and their family, collect data from the original medical images and cognitive assessment information generated by each volunteer during their early AD screening to generate a corresponding third individual dataset. The third-body dataset includes multiple third-body records; the third-body records include the original medical images and the cognitive assessment information. Step 4-3: Organize an expert team to comprehensively evaluate whether each of the third-person records meets the individual conditions for cognitive resilience, and delete the third-person records that do not meet the conditions from the third-person dataset; The expert team consists of several experts in psychology and cognitive science, neuroscience, and clinical medicine in the field of Alzheimer's disease research. Step 4-4: Take each of the first individual records in the first individual dataset, or each of the second individual records in the second individual dataset, or each of the third individual records in the third individual dataset as the corresponding current individual record; Steps 4-5: Based on the PET images and T2-weighted images recorded by the current individual, perform analysis on β-amyloid deposition indices, Tau protein deposition indices, and white matter high signal indices to obtain the corresponding β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The pathological feature component x1 is composed of the white matter high signal index WMHV; Steps 4-6: Based on the fMRI images recorded by the current individual, the system separation degree of brain structural connectivity is analyzed to obtain the corresponding system separation degree SyS; and based on the DTI images recorded by the current individual, the structural network efficiency of brain functional connectivity is analyzed to obtain the corresponding structural network efficiency SNE; and the obtained system separation degree SyS and structural network efficiency SNE are combined to form the corresponding neurological feature component x3. Steps 4-7: The cognitive score component x2 is composed of the MMSE score, ADAS-Cog score, and CDR score recorded by the current individual. Steps 4-8: Identify the individual type corresponding to the current individual record; if the individual type is an early-stage AD individual, then set the corresponding three-category label probabilities. , , The values ​​are 1, 0, and 0; if the individual type is a healthy individual, then the corresponding three-category label probabilities are set. , , The values ​​are 0, 0, and 1; if the individual type is a cognitively resilient individual, then the corresponding three types of label probabilities are set. , , 0, 1, 0; Steps 4-9 involve constructing the first training feature vector from the pathological feature component x1, the cognitive score component x2, and the neurological feature component x3 corresponding to the current individual record; and then constructing the three-category label probabilities corresponding to the current individual record. , , The first label vector is formed accordingly; and the first training feature vector and the first label vector corresponding to the current individual record are combined to form a corresponding first data record; Steps 4-10: The first dataset is composed of all the first data records obtained.

4. The method for predicting early Alzheimer's disease by combining cognitive characteristics according to claim 2, characterized in that, The step of training the first prediction model based on the first dataset specifically includes: Step 51: Based on a preset first segmentation ratio, the first dataset is divided into two sub-datasets, denoted as the first training set and the first evaluation set; Both the first training set and the first evaluation set consist of multiple first data records; the ratio of the total number of records N1 in the first training set to the total number of records N2 in the first evaluation set satisfies the first segmentation ratio; in the first training set, the record ratios corresponding to individuals with early-stage AD, cognitively resilient individuals, and healthy individuals are denoted as the corresponding... , , ; Where 1 ≤ type index j ≤ 3; Step 52: Take each of the first data records in the first training set as the corresponding current training record; and input the first training feature vector of the current training record as the current multimodal feature X into the first prediction model for processing, and record the prediction vector Y obtained in this processing as the corresponding prediction vector. And record the first label vector of the current training record as the corresponding label vector. ; and by the prediction vector and the label vector Form the corresponding first prediction-label pair ( , ); Where 1 ≤ sample index i ≤ N1; The prediction vector The three types of prediction probabilities y1, y2, and y3 are denoted as the corresponding prediction probabilities. , , The label vector The three types of label probabilities , , Let be the corresponding predicted probability. , , ; Step 53, the obtained N1 first prediction-label pairs ( , Substitute the preset model loss function L M The corresponding first loss value is obtained through calculation; Wherein, the model loss function L M Implemented based on the Focal Loss function, specifically as follows: ， ， ； w j Let w be the weight coefficient for class j, and w be the weight coefficient for all three classes. j The sum is 3; γ is the focusing parameter, γ≥0; Step 54: Identify whether the first loss value meets the preset first loss value range; if it does, proceed to step 55; if not, based on the preset first model optimizer, move towards making the model loss function L... M The direction that reaches the minimum value modulates the model parameters of the first prediction model in one round, and returns to step 52 when the modulation ends; The first model optimizer includes the Adam optimizer and the SGD optimizer. Step 55: Take each of the first data records in the first evaluation set as the corresponding current evaluation record; and take the first training feature vector of the current evaluation record as the current multimodal feature X as the input to the first prediction model for processing, and form a corresponding second prediction-label pair with the prediction vector Y obtained in this processing and the first label vector of the current evaluation record. Step 56: Based on the obtained N2 second prediction-label pairs, evaluate the F1 value corresponding to each individual classification to obtain the corresponding first, second, and third F1 values; and take the average of the first, second, and third F1 values ​​as the corresponding overall F1 value; and identify the first, second, and third F1 values ​​and the overall F1 value; if the first F1 value does not meet the preset first F1 value range, or the second F1 value does not meet the preset second F1 value range, or the third F1 value does not meet the preset third F1 value range, or the overall F1 value does not meet the preset overall F1 value range, then return to step 51 to continue training; if the first F1 value meets the first F1 value range, and the second F1 value meets the second F1 value range, and the third F1 value meets the third F1 value range, and the overall F1 value meets the overall F1 value range, then stop training and confirm that the training of the first prediction model has ended.

5. An apparatus for performing the method of predicting early Alzheimer's disease by combining cognitive features as described in any one of claims 1-4, characterized in that, The device includes: a model building module, a dataset preparation module, a model training module, and a model application module; The model building module constructs a first prediction model for a deep learning model that performs three-class classification prediction of early Alzheimer's disease based on pathological cognitive multimodal features. The first prediction model performs three-class classification prediction based on the multimodal features X input to the model and outputs a corresponding prediction vector Y. The multimodal features X include pathological feature components x1, cognitive score components x2, and neurological feature components x3. The pathological feature component x1 includes the β-amyloid deposition index SUVR. β Tau protein deposition index SUVR Tau The white matter high signal intensity index (WMHV) is used; the cognitive score component x2 includes MMSE score, ADAS-Cog score, and CDR score; the neurological feature component x3 includes system separation degree (SyS) and structural network efficiency (SNE); the prediction vector Y consists of three prediction probabilities y1, y2, and y3; the classification types of the three prediction probabilities y1, y2, and y3 are respectively early AD individuals, cognitively resilient individuals, and healthy individuals; The dataset preparation module is used to construct the first dataset by collecting and analyzing pathological cognitive multimodal information from three groups of people; wherein, the three groups of people include people in the early stage of AD, people with cognitive resilience, and healthy people; The model training module trains the first prediction model based on the first dataset; The model application module is used to receive pathological cognitive multimodal information of any individual input by the user after model training; generate corresponding multimodal features X based on the pathological cognitive multimodal information, input them into the first prediction model for processing to obtain the corresponding prediction vector Y; and feed back the classification type corresponding to the highest prediction probability in the prediction vector Y as the prediction result for the current individual to the current user; wherein, the pathological cognitive multimodal information includes the β-amyloid deposition index SUVR β Tau protein deposition index SUVR Tau White matter high signal intensity index (WMHV), MMSE score, ADAS-Cog score, CDR score, system separation degree (SyS), and structural network efficiency (SNE).

6. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-4; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-4.

Images