Alzheimer's disease multi-modal feature processing method, device and processing equipment

Abstract

The invention provides a multi-modal feature processing method, device and processing equipment for an Alzheimer's disease, a multi-modal analysis framework integrating sMRI, resting state fMRI, fundus OCTA and cognitive testing is created, and a coupling mechanism among brain, eye and cognition is systematically depicted, so that deep and accurate recognition of the Alzheimer's disease is obtained, and the recognition efficiency of the Alzheimer's disease is improved. The method is beneficial for promoting early recognition and individualized risk assessment, so that timely intervention is realized, and the method has a wide application prospect clinically.

Description

Technical Field

[0001] The present application relates to the field of information, and specifically to a multimodal feature processing method, apparatus, and processing equipment for Alzheimer's disease. Background Art

[0002] Alzheimer's disease (AD) is a common neurodegenerative disease. Depending on the severity, it can be divided into three stages: early, middle, and late. Early symptoms include short-term memory loss (e.g., forgetting recent conversations), difficulty speaking, and impaired judgment. Middle symptoms include long-term memory loss, disorientation (getting lost), mood swings (irritability or depression), and the need for assistance. Late symptoms include loss of self-care, inability to recognize friends and family, difficulty swallowing, organ failure, and ultimately, life-threatening conditions.

[0003] Therefore, in actual situations, accurate identification of Alzheimer's disease, especially early Alzheimer's disease, can be responded to as early as possible, which is of practical significance.

[0004] However, the inventors of the present application have found that for Alzheimer's disease, the current mainstream diagnostic methods mostly focus on single brain imaging or neuropsychological assessment, which are more suitable for the diagnosis of mid- and late-stage Alzheimer's disease, but are difficult to capture subtle and complex pathological changes in the early stages of the disease. Summary of the Invention

[0005] This application provides a multimodal feature processing method, device and processing equipment for Alzheimer's disease, creating a multimodal analysis framework that integrates sMRI, resting-state fMRI, fundus OCTA and cognitive testing, systematically depicting the coupling mechanism between brain-eye-cognition, thereby obtaining in-depth and accurate identification of Alzheimer's disease, helping to promote its early identification and individualized risk assessment, and thus achieving more timely intervention, which has broad application prospects in clinical practice.

[0006] In a first aspect, the present application provides a multimodal feature processing method for Alzheimer's disease, the method comprising:

[0007] Obtaining sample data of sample Alzheimer's disease patients, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data, the head MRI image data includes sample sMRI image data and sample resting-state fMRI image data, and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B;

[0008] Performing a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data;

[0009] performing a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data;

[0010] Based on sample MRI image feature data, sample retinal OCTA image feature data, and sample neuropsychological cognitive test evaluation data, the relationship between the three is analyzed using a multivariate regression model to obtain the corresponding multimodal features of Alzheimer's disease. Among them, the multimodal features of Alzheimer's disease are used to evaluate the different stages of Alzheimer's disease in users corresponding to the input parameters to be processed.

[0011] In a second aspect, the present application provides a multimodal feature processing device for Alzheimer's disease, the device comprising:

[0012] a sample acquisition unit, configured to acquire sample data of a sample Alzheimer's disease patient, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data; the head MRI image data includes sample sMRI image data and sample MRI image data; and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B;

[0013] A first feature extraction unit is used to perform a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data;

[0014] a second feature extraction unit, configured to perform a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data;

[0015] The analysis unit is used to analyze the relationship between the sample MRI image feature data, the sample retinal OCTA image feature data and the sample neuropsychological cognitive test evaluation data based on a multivariate regression model to obtain the corresponding multimodal features of Alzheimer's disease, wherein the multimodal features of Alzheimer's disease are used to evaluate the different stages of the Alzheimer's disease of the user corresponding to the input parameters to be processed.

[0016] In a third aspect, the present application provides a processing device comprising a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method provided in the first aspect of the present application or any possible implementation of the first aspect of the present application is executed.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the method provided in the first aspect of the present application or any possible implementation of the first aspect of the present application.

[0018] From the above content, it can be concluded that this application has the following beneficial effects:

[0019] Focusing on the goal of early identification of Alzheimer's disease, this application has created a multimodal analysis framework that integrates resting-state fMRI, sMRI, fundus OCTA and cognitive tests, systematically characterizing the coupling mechanism between brain, eye and cognition, thereby obtaining in-depth and accurate identification of Alzheimer's disease, which will help promote its early identification and individualized risk assessment, and thus achieve more timely intervention, and has broad application prospects in clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a multimodal feature processing method for Alzheimer's disease in this application;

[0022] Figure 2 A logical diagram of a multimodal feature processing method for Alzheimer's disease in this application;

[0023] Figure 3 This is a schematic structural diagram of a multimodal feature processing device for Alzheimer's disease in this application;

[0024] Figure 4 This is a structural diagram of the processing equipment for this application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0027] The division of modules in this application is a logical division. In actual application, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. Moreover, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed into multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application.

[0028] Before introducing the multimodal feature processing method for Alzheimer's disease provided by this application, the background content involved in this application is first introduced.

[0029] The multimodal feature processing method, device and computer-readable storage medium for Alzheimer's disease provided in this application can be applied to processing equipment to create a multimodal analysis framework that integrates resting-state fMRI, sMRI, fundus OCTA and cognitive testing. It systematically depicts the coupling mechanism between brain-eye-cognition, thereby obtaining in-depth and accurate identification of Alzheimer's disease, which helps promote its early identification and individualized risk assessment, and thus achieve more timely intervention, and has broad application prospects in clinical practice.

[0030] The multimodal feature processing method for Alzheimer's disease mentioned in this application can be executed by a multimodal feature processing device for Alzheimer's disease, or a server, physical host, or user equipment (UE) of different types of processing devices that integrate the multimodal feature processing device for Alzheimer's disease. The multimodal processing device for Alzheimer's disease can be implemented in hardware or software, and the UE can specifically be a terminal device such as a smartphone, tablet computer, laptop computer, desktop computer, or personal digital assistant (PDA). The processing device can be set up in a device cluster.

[0031] Among them, it can be understood that the solution of the present application is mainly data processing and analysis based on existing data. Therefore, the processing equipment that executes the multimodal feature processing method of Alzheimer's disease of the present application or is equipped with the corresponding application service of the multimodal feature processing method of Alzheimer's disease of the present application usually only needs to meet the required data processing capabilities, and its specific device type and device deployment form are relatively flexible.

[0032] If data acquisition capabilities directly related to MRI images, retinal OCTA images or neuropsychological cognitive test assessments are also required, it obviously means that the processing equipment needs to be further adaptively configured in terms of software and hardware.

[0033] For example, compared to directly linking an external MRI device to use its MRI image acquisition function or retrieving pre- / real-time acquired MRI images from the system, the MRI device can be incorporated into the device cluster of the processing device or even directly integrated into the device itself, thereby directly having MRI image acquisition capabilities.

[0034] In addition, when it comes to device clusters, the device composition of the processing equipment can further involve the application of remote services, so that more flexible solution application effects can be achieved in actual situations, thereby meeting the diverse user needs in actual situations.

[0035] As an example, the core solution processing of the multimodal features of Alzheimer's disease in this application can be performed in the laboratory background or even on the equipment of a third party (not the user of the multimodal features of Alzheimer's disease), while the application of the processed multimodal features of Alzheimer's disease can be performed on the equipment at the hands of medical staff in the clinical field.

[0036] Next, we will introduce the multimodal feature processing method for Alzheimer's disease provided by this application.

[0037] First, refer to Figure 1 A flow chart of the multimodal feature processing method for Alzheimer's disease of the present invention is shown and Figure 2 A logic diagram of the multimodal feature processing method for Alzheimer's disease provided by the present application is shown. The multimodal feature processing method for Alzheimer's disease provided by the present application may specifically include the following steps S101 to S104:

[0038] Step S101, obtaining sample data of a sample Alzheimer's disease patient, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data, the head MRI image data includes sample sMRI image data and sample resting-state fMRI image data, and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B;

[0039] It can be understood that the multimodal characteristics of Alzheimer's disease that reflect the coupling relationship between brain, eye and cognition to be constructed in this application involve three major aspects of data input, namely, head MRI images, sample retinal OCTA images and sample neuropsychological cognitive test evaluation data. For the convenience of explanation, the three aspects of data obtained at this time are recorded as sample head MRI image data, sample retinal OCTA image data and sample neuropsychological cognitive test evaluation data.

[0040] Among them, MRI stands for Nuclear Magnetic Resonance Imaging; OCTA stands for Optical Coherence Tomography Angiography; sMRI stands for Structural Magnetic Resonance Imaging; fMRI stands for Functional Magnetic Resonance Imaging; MMSE stands for Mini-Mental State Examination; MoCA stands for Montreal Cognitive Assessment; BNT stands for Boston Naming Test; CDT stands for Clock Drawing Test; BDST stands for Brief Dementia Screening Test; TMT-A stands for Trail Making Test Part A; TMT-B stands for Trail Making Test Part B.

[0041] Further details include:

[0042] 1) Sample head MRI image data

[0043] In the present application, the sample head MRI image data specifically includes sample sMRI image data and sample resting-state fMRI image data.

[0044] Specifically, sMRI imaging can be understood as a basic type of MRI imaging. It is a method of non-invasive three-dimensional imaging of the human brain using magnetic resonance imaging technology. It can provide high-precision anatomical information, such as the location, size, and shape of structures such as gray matter, white matter, and ventricles in the brain, and explain information on the structural composition and developmental process of the brain.

[0045] fMRI imaging is an important tool that helps to non-invasively examine, locate and explore the brain's language, memory and other functions. In recent years, the focus of neuroscience research has clearly shifted to the study of the brain in the "resting state", focusing on the intrinsic activities within the brain in the absence of any sensory or cognitive stimulation. The analysis of the functional connectivity of the brain in the resting state reveals different resting-state networks, which describe specific functions and different spatial topological structures. Therefore, in this application, resting-state fMRI is specifically used.

[0046] 2) Sample retinal OCTA image data

[0047] OCTA technology is known for its non-invasive and efficient blood flow detection capabilities. It can clearly display the blood flow morphology and dynamic changes of the retina and choroid, and has many advantages of non-invasive, three-dimensional imaging and high resolution.

[0048] 3) Sample neuropsychological cognitive test assessment data

[0049] In this application, the sample neuropsychological cognitive test assessment data specifically includes the test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A and TMT-B, providing high-quality cognitive assessment samples through diversified and comprehensive neuropsychological cognitive test assessment data.

[0050] In addition, it should be understood that the above three types of data, namely sample head MRI image data, sample retinal OCTA image data and sample neuropsychological cognitive test assessment data, are obtained from sample Alzheimer's patients (or subjects). The sample Alzheimer's patients can be real patients (whose data have the corresponding personal privacy parts removed), or virtual patients modified and processed based on real patients, or completely virtual virtual patients. These are all possible in actual situations and correspond to diverse and rich sample needs.

[0051] Step S102, performing a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data;

[0052] It can be understood that after obtaining the initial sample MRI image data, the corresponding image features can be extracted from the image level to lay the foundation for subsequent data processing.

[0053] As an exemplary embodiment, the first feature extraction process is performed on the sample MRI image data to obtain the sample MRI image feature data, which may specifically include:

[0054] The fractional low-frequency amplitude features and local consistency features were calculated for the sample sMRI image data, and the cortical areas were segmented and the corresponding cerebral cortical thickness was extracted for the sample resting-state fMRI image data. Both were used as the sample MRI image feature data.

[0055] Among them, the fractional low-frequency amplitude is Fraction Amplitude of Low Frequency Fluctuation, FALFF, and the local consistency is Regional Homogeneity, ReHo. Therefore, the fractional low-frequency amplitude feature and the local consistency feature can also be referred to as FALFF feature and ReHo feature.

[0056] In terms of specific calculation, the fractional low-frequency amplitude is the relative contribution rate of the low-frequency amplitude (ALFF) in the total energy range (Total Power), that is, the signal power in the entire frequency range.

[0057] Local consistency measures the consistency of the temporal signal of a voxel with its surrounding voxels, where the local neighborhood is typically 27 voxels.

[0058] It is understandable that the fractional low-frequency amplitude feature and the local consistency feature are indicators among many mature MRI analysis indicators. Considering that the concepts and calculation methods themselves can be directly referred to the existing technology, this application does not elaborate on them here. Similarly, the thickness of the cerebral cortex itself is one of the common indicators in MRI image processing.

[0059] It can be understood that in this setting, the focus of this application is not how to calculate the fractional low-frequency amplitude characteristics, local consistency characteristics, and cerebral cortical thickness, but rather to focus on selecting these indicators and incorporating them into the multimodal analysis framework of this application that integrates sMRI, resting-state fMRI, fundus OCTA, and cognitive testing, which has better practical significance.

[0060] In addition, before the feature extraction processing of the sample MRI image data, the present application may also involve relevant preprocessing to further improve the data quality and thus improve the subsequent data processing effect.

[0061] In this regard, the present application also provides a specific implementation scheme applicable to the present application. Specifically, as an exemplary embodiment, before performing a first feature extraction process on the sample MRI image data to obtain the sample MRI image feature data, the method of the present application may further include:

[0062] The sample MRI image data is preprocessed, wherein the preprocessing includes:

[0063] 1) Remove the first 10 time points, perform head motion correction, and perform spatial normalization to the MNI template;

[0064] This step involves the removal of invalid / ineffective time points that are prone to exist in the previous step, and also involves further data quality enhancement operations such as head correction and spatial normalization.

[0065] Among them, MNI stands for Montreal Neurological Institute, and the MNI template is a standard MRI image template.

[0066] 2) Regression of 24 head motion parameters and white matter and cerebrospinal fluid signals;

[0067] This step achieves deep and highly targeted data enhancement through regression operations on 24 head movement parameters and white matter and cerebrospinal fluid signals.

[0068] 3) Extract data within the 0.01-0.1 Hz frequency band for use in the first feature extraction process.

[0069] This step involves filtering operations based on preset frequency band ranges, further filtering out invalid / insignificant data points.

[0070] In this way, through three stages of data enhancement, high-quality data input is provided for subsequent MRI image feature extraction.

[0071] At the same time, for the sample retinal OCTA image data and sample neuropsychological cognitive test evaluation data, corresponding preprocessing operations can also be involved in the specific operation to further improve the data quality.

[0072] Step S103, performing a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data;

[0073] It can be understood that the execution order of the second feature extraction processing for the sample retinal OCTA image data and the first feature extraction processing for the sample MRI image data is not fixed, but independent of each other. They can be performed simultaneously or successively (the order can also be swapped), and can be configured according to actual needs.

[0074] It can be understood that the specific retinal OCTA image features to be extracted by the second feature extraction process here are adaptively configured according to the multimodal analysis framework that integrates sMRI, resting-state fMRI, fundus OCTA and cognitive testing to be implemented in the subsequent application.

[0075] Specifically, in a preferred solution that can be adopted in the present application, as an exemplary embodiment, the second feature extraction process is performed on the sample retinal OCTA image data to obtain the sample retinal OCTA image feature data, which may include:

[0076] 1) Using a deep learning-based image segmentation model to segment the foveal avascular area and vascular network in the sample retinal OCTA image data;

[0077] The foveal avascular zone (FAZ) is specifically the foveal avascular zone of the macula.

[0078] Here, we mainly locate the two elements of the foveal avascular area and the vascular network contained in the image content of the retinal OCTA image from the perspective of image recognition.

[0079] 2) Extract the morphological features of the foveal avascular zone and vascular network, and calculate the retinal thickness corresponding to the foveal avascular zone in a hierarchical manner. The morphological features include the area of ​​the foveal avascular zone, the perimeter of the foveal avascular zone, the vascular density, and the vascular curvature.

[0080] It can be understood that after locating the foveal avascular zone and vascular network in the image in the previous stage, the specific morphological features of the two can be extracted as subsequent data input, specifically involving the area and perimeter of the foveal avascular zone and the retinal thickness calculated by layering, as well as the density and curvature of the blood vessels (both of which can also be recorded as vascular distribution parameters).

[0081] In terms of specific operations, it can be understood that the feature extraction processing of the sample retinal OCTA image feature data here can be implemented by the corresponding target detection algorithm.

[0082] As an example, the FARGO model built on PyTorch can be used. PyTorch is an open source deep learning framework for machine learning and deep learning. FARGO is a joint framework for FAZ and RV segmentation from OCTA images. The FARGO model is an existing medical model suitable for this application.

[0083] Similar to the above, the several indicators involved here are actually existing concepts, but the focus of this application is to select these indicators and incorporate them into the multimodal analysis framework of this application that integrates sMRI, resting-state fMRI, fundus OCTA and cognitive tests, which has better practical significance.

[0084] Step S104, based on the sample MRI image feature data, the sample retinal OCTA image feature data and the sample neuropsychological cognitive test evaluation data, a multivariate regression model is used to analyze the relationship between the three to obtain the corresponding multimodal features of Alzheimer's disease, wherein the multimodal features of Alzheimer's disease are used to evaluate the different stages of the Alzheimer's disease of the user corresponding to the input parameters to be processed.

[0085] After obtaining the sample MRI image feature data and sample retinal OCTA image feature data through the previous data processing, the sample neuropsychological cognitive test evaluation data obtained at the beginning can be combined to analyze the relationship between the three through a multivariate regression model to obtain the multimodal characteristics of Alzheimer's disease under the multimodal analysis framework of this application integrating sMRI, resting-state fMRI, fundus OCTA and cognitive testing. It is mainly used to evaluate the specific stage of the Alzheimer's disease course of the corresponding user (mainly three major stages, and secondly, further subdivision or further evaluation can be carried out).

[0086] In layman's terms, the multimodal characteristics of Alzheimer's disease are based on the characteristics of three aspects: MRI image characteristics, retinal OCTA image characteristics and neuropsychological cognitive test evaluation, and represent the characteristics of the gradual evolution of the Alzheimer's disease process from subjective cognitive decline, mild cognitive impairment to dementia.

[0087] In this process, it is understandable that it relies on statistical analysis and processing, and specifically involves the analysis and processing of multivariate regression models. In short, the multivariate regression model is a model used to establish the relationship between multiple independent variables and dependent variables. In the field of machine learning, multiple linear regression is a common multivariate regression model used for predictive analysis, time series models, and discovering causal relationships between variables.

[0088] In practice, a mathematical model is constructed that represents the relationship between a dependent variable (Y) and one or more independent variables (X1, X2, ..., Xn). The general form of the model is [Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + ... + \beta_nX_n + \epsilon], where (\beta_0) is the intercept, (\beta_1, \beta_2, ..., \beta_n) are the coefficients, and (\epsilon) is the error term. The coefficients (\beta_i) are estimated by minimizing the sum of squares of the error terms, resulting in the best fitting model.

[0089] Furthermore, based on the sample MRI image feature data, the sample retinal OCTA image feature data, and the sample neuropsychological cognitive test assessment data, a multivariate regression model is used to analyze the relationship between the three, and the corresponding multimodal features of Alzheimer's disease are obtained, which may specifically include:

[0090] Based on sample MRI image feature data, sample retinal OCTA image feature data, and sample neuropsychological cognitive test assessment data, after correcting for the effects of age, gender, years of education, and intracranial volume, a two-stage multivariate regression model was used to analyze the relationship between the three and obtain the corresponding multimodal characteristics of Alzheimer's disease. In the two-stage regression process, the first stage screened significant brain regions significantly correlated with cognitive assessment through multivariate regression processing across the whole brain, and the second stage analyzed the statistical relationship with retinal parameters within the significant brain regions.

[0091] For the initial correction process, age, gender, and years of education are included in the sample neuropsychological cognitive test assessment data. As can be understood, the sample neuropsychological cognitive test assessment data involves cognitive test assessments across multiple dimensions / systems, and age, gender, and years of education are among the key elements that this application considers necessary for verification and correction. Furthermore, correction for the influence of intracranial volume can also be performed on the sample MRI image feature data corresponding to the brain.

[0092] In the following specific analysis and processing of the multivariate regression model, it can be seen that the present application specifically designs a two-stage regression method. The first stage screens brain areas that are significantly related to cognition, and realizes the preliminary construction of the mapping relationship between the sample MRI image feature data and the sample neuropsychological cognitive test evaluation data. The second stage analyzes the statistical relationship between the significant brain areas and the retinal parameters. While continuing to introduce the sample retinal OCTA image feature data, the multimodal characteristics of Alzheimer's disease are finally constructed by integrating the multimodal analysis results of sMRI, resting-state fMRI, fundus OCTA and cognitive tests.

[0093] Among them, the first stage involves significant brain areas. In the specific operation, the overall error rate (FWE) correction may also be involved. The core purpose of FWE correction is to control the FWE of multiple hypothesis testing and ensure that the probability of at least one false positive result in all statistical comparisons does not exceed the preset threshold. The preset threshold can be denoted as p. As an example, it can be set to 5%.

[0094] Furthermore, as an exemplary embodiment, the quantitative formula involved in the multivariate regression processing in the first stage may specifically include:

[0095] Y = β0 + β1 × brain imaging characteristics + β2 × age + β3 × sex + β4 × education + β5 × brain volume,

[0096] Among them, Y is the cognitive test score, β0, β1, β2, β3, β4 and β5 are different adjustment coefficients, and the brain imaging features include the fractional low-frequency amplitude features and local consistency features extracted from the sample sMRI image data, and also include the cerebral cortex thickness extracted from the sample resting-state fMRI image data (the specific features here correspond to the processing results of the feature extraction processing of the previous exemplary embodiment).

[0097] After completing the multimodal feature analysis of Alzheimer's disease, the easy-to-understand features can be put into practical application and used in the evaluation and analysis of specific known / potential Alzheimer's patients to better serve the diagnosis and intervention of Alzheimer's disease.

[0098] Correspondingly, as an exemplary embodiment, the method of the present application may further include:

[0099] Get the input parameters of the user to be evaluated;

[0100] Based on the input parameters of the user to be evaluated, the multimodal features of Alzheimer's disease are used to evaluate the different stages of the Alzheimer's disease of the user to be evaluated;

[0101] Output the evaluation results of the different stages of Alzheimer's disease of the user to be evaluated.

[0102] It can be understood that the input parameters of the user to be evaluated involved here can be either the relevant key parameters directly involved in the multimodal characteristics of Alzheimer's disease, or the parameter data collected corresponding to the previous sample MRI image feature data, sample retinal OCTA image feature data and sample neuropsychological cognitive test evaluation data, or the parameter data can be configured and collected in other forms.

[0103] In specific operations, the multimodal characteristics of Alzheimer's disease can be applied in the form of data models or in the form of machine learning algorithms built in combination with relevant artificial intelligence (AI) technologies.

[0104] Taking the above input parameters of the user to be evaluated as an example, which are parameter data collected corresponding to the previous sample MRI image feature data, sample retinal OCTA image feature data and sample neuropsychological cognitive test evaluation data, if the relevant key parameters involved in the multimodal characteristics of Alzheimer's disease are to be obtained, the corresponding key parameter extraction processing may be involved, and this part of the data processing can be completed efficiently and accurately by machine learning algorithms.

[0105] In addition, in actual situations, the application of the multimodal characteristics of Alzheimer's disease may also involve other aspects of data processing with a certain degree of complexity. This can also be completed efficiently and accurately by machine learning algorithms. The core is that it is equipped with the multimodal characteristics of Alzheimer's disease obtained by pre-analysis of this application.

[0106] After the evaluation results of the Alzheimer's disease course stage of the user to be evaluated in the current practical application link are completed by combining the application of the multimodal characteristics of Alzheimer's disease, they can be output.

[0107] It is easy to understand that, similar to the situation of obtaining the multimodal characteristics of Alzheimer's disease, in terms of output, local storage, remote storage, prompts for output completion processing, result display, result push, or further data analysis can be performed. Obviously, this can be adaptively configured according to specific data application rules that are pre-configured and configured in real time.

[0108] Finally, regarding the above program content, in general, focusing on the goal of early identification of Alzheimer's disease, this application has created a multimodal analysis framework that integrates sMRI, resting-state fMRI, fundus OCTA and cognitive tests, and systematically portrays the coupling mechanism between brain-eye and cognition. It has created a multimodal analysis framework that integrates sMRI, resting-state fMRI, fundus OCTA and cognitive tests, and systematically portrays the coupling mechanism between brain-eye and cognition, thereby obtaining in-depth and accurate identification of Alzheimer's disease, which will help promote its early identification and individualized risk assessment, and thus achieve more timely intervention, and has broad application prospects in clinical practice.

[0109] In specific aspects, it can have the following beneficial effects:

[0110] 1. Be the first to achieve unified quantification and systematic modeling of brain, eye, and cognition characteristics;

[0111] 2. It can identify subtle but coordinated structural and functional changes in the early stages of Alzheimer's disease (such as subjective cognitive decline);

[0112] 3. Improve the ability to stratify and assess populations at risk for Alzheimer's disease;

[0113] 4. It has good scalability and is suitable for multi-center data research and clinical early screening of the elderly population.

[0114] Of course, it should be understood that although the present application is highly suitable for the early identification of Alzheimer's disease compared to the existing technology, or in other words, it can complete the identification of early Alzheimer's disease with high precision, it can also perform well in the identification of mid-term / late-stage Alzheimer's disease.

[0115] The above is an introduction to the multimodal feature processing method for Alzheimer's disease provided by this application. In order to facilitate better implementation of the multimodal feature processing method for Alzheimer's disease provided by this application, this application also provides a multimodal feature processing device for Alzheimer's disease from the perspective of functional modules.

[0116] See Figure 3 , Figure 3 This is a schematic diagram of the structure of a multimodal feature processing device for Alzheimer's disease in this application. In this application, the multimodal feature processing device 300 for Alzheimer's disease may specifically include the following structure:

[0117] The sample acquisition unit 301 is used to acquire sample data of a sample Alzheimer's disease patient, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data, the head MRI image data includes sample sMRI image data and sample resting-state fMRI image data, and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B;

[0118] A first feature extraction unit 302 is configured to perform a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data;

[0119] A second feature extraction unit 303 is configured to perform a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data;

[0120] The analysis unit 304 is used to analyze the relationship between the sample MRI image feature data, the sample retinal OCTA image feature data, and the sample neuropsychological cognitive test evaluation data based on a multivariate regression model to obtain corresponding multimodal features of Alzheimer's disease, wherein the multimodal features of Alzheimer's disease are used to evaluate different stages of the Alzheimer's disease of the user corresponding to the input parameters to be processed.

[0121] In an exemplary embodiment, the first feature extraction unit 302 is specifically configured to:

[0122] The fractional low-frequency amplitude features and local consistency features were calculated for the sample sMRI image data, and the cortical areas were segmented and the corresponding cerebral cortical thickness was extracted for the sample resting-state fMRI image data. Both were used as the sample MRI image feature data.

[0123] In another exemplary embodiment, the first feature extraction unit 302 is further configured to:

[0124] The sample MRI image data is preprocessed, wherein the preprocessing includes:

[0125] The first 10 time points were removed, head motion correction was performed, and spatial normalization was performed to the MNI template;

[0126] Regress 24 head movement parameters and white matter and cerebrospinal fluid signals;

[0127] Data within the 0.01-0.1 Hz frequency band is extracted for use in the first feature extraction process.

[0128] In another exemplary embodiment, the second feature extraction unit 303 is specifically configured to:

[0129] The sample retinal OCTA image data was segmented into the foveal avascular area and vascular network using a deep learning-based image segmentation model.

[0130] The morphological features of the foveal avascular zone and vascular network were extracted, and the retinal thickness corresponding to the foveal avascular zone was calculated hierarchically. The morphological features included the area of ​​the foveal avascular zone, the perimeter of the foveal avascular zone, the vascular density, and the vascular curvature.

[0131] In another exemplary embodiment, the analyzing unit 304 is specifically configured to:

[0132] Based on sample MRI image feature data, sample retinal OCTA image feature data, and sample neuropsychological cognitive test assessment data, after correcting for the effects of age, gender, years of education, and intracranial volume, a two-stage multivariate regression model was used to analyze the relationship between the three and obtain the corresponding multimodal characteristics of Alzheimer's disease. In the two-stage regression process, the first stage screened significant brain regions significantly correlated with cognitive assessment through multivariate regression processing across the whole brain, and the second stage analyzed the statistical relationship with retinal parameters within the significant brain regions.

[0133] In another exemplary embodiment, the quantitative formula involved in the multivariate regression process includes:

[0134] Y = β0 + β1 × brain imaging characteristics + β2 × age + β3 × sex + β4 × education + β5 × brain volume,

[0135] Among them, Y is the cognitive test score, β0, β1, β2, β3, β4 and β5 are different adjustment coefficients, and the brain imaging features include the fractional low-frequency amplitude features and local consistency features extracted from the sample sMRI image data, as well as the cerebral cortical thickness extracted from the sample resting-state fMRI image data.

[0136] In another exemplary embodiment, the apparatus further includes an application unit 305, configured to:

[0137] Get the input parameters of the user to be evaluated;

[0138] Based on the input parameters of the user to be evaluated, different stages of Alzheimer's disease of the user to be evaluated are evaluated using multimodal features of Alzheimer's disease;

[0139] Output the evaluation results of the different stages of Alzheimer's disease of the user to be evaluated.

[0140] This application also provides a processing device from the perspective of hardware structure, see Figure 4 , Figure 4 The schematic diagram of the structure of the processing device of the present application is shown. Specifically, the processing device of the present application may include a processor 401, a memory 402 and an input / output device 403. The processor 401 is used to execute the computer program stored in the memory 402 to implement the following Figure 1 Each step of the multimodal feature processing method for Alzheimer's disease in the corresponding embodiment; or, when the processor 401 is used to execute the computer program stored in the memory 402, the following is implemented Figure 3 The memory 402 is used to store the functions of each unit in the embodiment, and the processor 401 executes the above Figure 1 The computer program required for the multimodal feature processing method for Alzheimer's disease in the corresponding embodiment.

[0141] For example, the computer program may be divided into one or more modules / units, one or more of which are stored in memory 402 and executed by processor 401 to implement the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in a computer device.

[0142] The processing device may include, but is not limited to, a processor 401, a memory 402, and an input / output device 403. Those skilled in the art will appreciate that the illustrations are merely examples of processing devices and do not limit the processing device. The processing device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the processing device may also include a network access device, a bus, etc., and the processor 401, the memory 402, the input / output device 403, etc. are connected via the bus.

[0143] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the processing device and connects various parts of the entire device using various interfaces and lines.

[0144] Memory 402 can be used to store computer programs and / or modules. Processor 401 implements various functions of the computer device by running or executing computer programs and / or modules stored in memory 402 and accessing data stored in memory 402. Memory 402 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the like; the data storage area may store data created based on the use of the processing device. Furthermore, memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0145] When the processor 401 is used to execute the computer program stored in the memory 402, it can specifically implement the following functions:

[0146] Obtaining sample data of sample Alzheimer's disease patients, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data, the head MRI image data includes sample sMRI image data and sample resting-state fMRI image data, and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B;

[0147] Performing a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data;

[0148] performing a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data;

[0149] Based on sample MRI image feature data, sample retinal OCTA image feature data, and sample neuropsychological cognitive test evaluation data, the relationship between the three is analyzed using a multivariate regression model to obtain the corresponding multimodal features of Alzheimer's disease. Among them, the multimodal features of Alzheimer's disease are used to evaluate the different stages of Alzheimer's disease in users corresponding to the input parameters to be processed.

[0150] Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working process of the multimodal feature processing device, processing equipment and corresponding units of Alzheimer's disease described above can refer to the following. Figure 1 The description of the multimodal feature processing method for Alzheimer's disease in the corresponding embodiment will not be repeated here.

[0151] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0152] To this end, the present application provides a computer-readable storage medium, which stores a plurality of instructions, which can be loaded by a processor to execute the present application as follows: Figure 1 The steps of the multimodal feature processing method for Alzheimer's disease in the corresponding embodiment, the specific operations can be referred to as follows Figure 1 The description of the multimodal data processing method for Alzheimer's disease in the corresponding embodiment will not be repeated here.

[0153] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0154] Due to the instructions stored in the computer readable storage medium, the present application can be executed as follows: Figure 1 The steps of the multimodal feature processing method for Alzheimer's disease in the corresponding embodiment, therefore, the present application can be realized as follows Figure 1 The beneficial effects that can be achieved by the multimodal feature processing method for Alzheimer's disease in the corresponding embodiment are detailed in the previous description and will not be repeated here.

[0155] The above is a detailed introduction to the multimodal feature processing method, device, processing equipment and computer-readable storage medium for Alzheimer's disease provided by this application. Specific examples are used in this article to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the core idea of ​​this application; at the same time, for technical personnel in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A multimodal feature processing method for Alzheimer's disease, characterized in that: The method comprises: Acquiring sample data of a sample Alzheimer's disease patient, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data, the head MRI image data includes sample sMRI image data and sample resting-state fMRI image data, and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B; Performing a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data; performing a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data; Based on the sample MRI image feature data, the sample retinal OCTA image feature data and the sample neuropsychological cognitive test assessment data, the relationship between the three is analyzed in combination with a multivariate regression model to obtain corresponding multimodal features of Alzheimer's disease, wherein the multimodal features of Alzheimer's disease are used to evaluate different stages of the Alzheimer's disease of the user corresponding to the input parameters to be processed.

2. The method according to claim 1, characterized in that The performing a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data includes: The fractional low-frequency amplitude feature and local consistency feature are calculated for the sample resting-state fMRI image data, and the cortical area is segmented and the corresponding cerebral cortex thickness is extracted for the sample sMRI image data, both of which are used as the sample MRI image feature data.

3. The method according to claim 1, characterized in that Before performing the first feature extraction process on the sample MRI image data to obtain the sample MRI image feature data, the method further includes: Preprocessing the sample MRI image data, wherein the preprocessing includes: The first 10 time points were removed, head motion correction was performed, and spatial normalization was performed to the MNI template; Regress 24 head movement parameters and white matter and cerebrospinal fluid signals; Data within the 0.01-0.1 Hz frequency band is extracted for use in the first feature extraction process.

4. The method according to claim 1, wherein The performing a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data includes: Using a deep learning-based image segmentation model to segment the sample retinal OCTA image data into a foveal avascular area and a vascular network; The morphological features of the foveal avascular zone and the vascular network are extracted, and the retinal thickness corresponding to the foveal avascular zone is calculated in layers, wherein the morphological features include the area of ​​the foveal avascular zone, the perimeter of the foveal avascular zone, the vascular density, and the vascular curvature.

5. The method according to claim 1, wherein Based on the sample MRI image feature data, the sample retinal OCTA image feature data, and the sample neuropsychological cognitive test assessment data, a multivariate regression model is used to analyze the relationship between the three to obtain corresponding multimodal features of Alzheimer's disease, including: Based on the sample MRI image feature data, the sample retinal OCTA image feature data and the sample neuropsychological cognitive test assessment data, after correcting the effects of age, gender, years of education and intracranial volume, a two-stage regression multivariate regression model is used to analyze the relationship between the three to obtain the corresponding multimodal characteristics of Alzheimer's disease. In the processing process of the two-stage regression method, the first stage uses multivariate regression processing to screen significant brain areas that are significantly correlated with cognitive assessment within the whole brain, and the second stage analyzes the statistical relationship with retinal parameters within the significant brain areas.

6. The method according to claim 1, characterized in that The quantitative formulas involved in the multivariate regression process include: Y = β0 + β1 × brain imaging characteristics + β2 × age + β3 × sex + β4 × education + β5 × brain volume, Among them, Y is the cognitive test score, β0, β1, β2, β3, β4 and β5 are different adjustment coefficients, and the brain imaging features include the fractional low-frequency amplitude features and local consistency features extracted from the sample resting-state fMRI image data, and also include the cerebral cortex thickness extracted from the sample sMRI image data.

7. The method according to claim 1, characterized in that The method further comprises: Get the input parameters of the user to be evaluated; Based on the input parameters of the user to be evaluated, evaluating different stages of Alzheimer's disease of the user to be evaluated using the multimodal features of Alzheimer's disease; Output the evaluation results of the user to be evaluated at different stages of Alzheimer's disease.

8. A multimodal feature processing device for Alzheimer's disease, characterized in that: The device comprises: a sample acquisition unit, configured to acquire sample data of a sample Alzheimer's disease patient, wherein the sample data includes sample head MRI image data, sample retinal OCTA image data, and sample neuropsychological cognitive test assessment data, the head MRI image data includes sample sMRI image data and sample MRI image data, and the sample neuropsychological cognitive test assessment data includes test assessment results of MMSE, MoCA, BNT, CDT, BDST, TMT-A, and TMT-B; a first feature extraction unit, configured to perform a first feature extraction process on the sample MRI image data to obtain sample MRI image feature data; a second feature extraction unit, configured to perform a second feature extraction process on the sample retinal OCTA image data to obtain sample retinal OCTA image feature data; An analysis unit is configured to analyze the relationship between the sample MRI image feature data, the sample retinal OCTA image feature data, and the sample neuropsychological cognitive test assessment data in combination with a multivariate regression model to obtain corresponding multimodal features of Alzheimer's disease, wherein the multimodal features of Alzheimer's disease are used to evaluate different stages of the Alzheimer's disease of the user corresponding to the input parameters to be processed.

9. A processing device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 7.

Images