A cognitive enhancement method and system based on multi-modal data

Abstract

The application discloses a kind of cognitive promotion method and system based on multi-modal data.The cognitive promotion method includes: obtaining the first multi-modal data before treatment of patient;Amyloid load grouping is carried out to patient based on the amyloid load in the brain of patient, to obtain the initial cognitive training scheme corresponding;Cognitive training is carried out based on initial cognitive training scheme during the treatment of patient, and the second multi-modal data during treatment is collected;Based on multi-modal data before and after treatment of patient, in combination with brain network group atlas, obtain the multi-modal change data of patient;The multi-modal change data of patient is input into preset model, to output the cognitive promotion scheme that cognitive index of patient is most improved, to push to patient and carry out cognitive training.The method will be integrated by A beta-PET data, functional nuclear magnetic resonance imaging data and event-related potential data, real-time evaluation of the neuroplasticity change of patient, to dynamically generate personalized cognitive promotion scheme.

Description

Technical Field

[0001] This invention relates to a cognitive enhancement method based on multimodal data, and also to a corresponding cognitive enhancement system, belonging to the field of cognitive training technology. Background Technology

[0002] Alzheimer's disease (AD) is a neurodegenerative disease characterized by cognitive decline, primarily manifested by amyloid protein deposition. In recent years, the FDA-approved two anti-amyloid monoclonal antibody drugs—lencanetumab and donepemab—represent a major breakthrough in this field, becoming the first drugs capable of slowing the progression of AD. However, while these monoclonal antibody drugs can slow disease progression (assessed through activities of daily living), their improvement in cognitive function is very limited. For example, after 18 months of lencanetumab treatment, the difference in cognitive function observed on the ADAS-Cog14 scale (out of 90) between the treatment and placebo groups was only 1.44 points, almost clinically unobservable. Therefore, further improving patients' cognitive function in addition to monoclonal antibody therapy is a pressing clinical challenge.

[0003] In clinical trials of monoclonal antibody drugs, patients with Alzheimer's disease (AD)-related mild cognitive impairment were treated with anti-amyloid monoclonal antibodies alone, while patients with mild dementia routinely used cholinesterase inhibitors, the first-line treatment for AD, to improve cognitive function. Neither group of patients received non-pharmacological interventions (such as cognitive training) to improve cognitive function. Previously developed cognitive training programs were used within the natural course of the disease, and the underlying basis of cognitive training—changes in neuroplasticity—was not subject to external intervention. However, monoclonal antibody drugs alter the disease course by rapidly clearing amyloid protein from the brains of AD patients, and amyloid protein deposition is closely related to impaired neuroplasticity. Therefore, after using anti-amyloid monoclonal antibody drugs, new cognitive training programs and models need to be designed to adapt to changes in neural plasticity, thereby better improving patients' cognitive function.

[0004] Most existing non-pharmacological interventions (such as cognitive training) are not designed to address the dynamic changes in neuroplasticity after anti-amyloid drug treatment, and lack integrated programs that fully utilize multimodal biomarkers for dynamic monitoring of neuroplasticity. The pathological changes in Alzheimer's disease (AD) are closely related to the reorganization of brain functional networks, including functional magnetic resonance imaging (fMRI) revealing functional abnormalities in the default mode network (DMN) and task-related networks, as well as cognitive processing impairment reflected by event-related potentials (ERPs). These changes provide objective indicators for assessing neuroplasticity. Furthermore, amyloid PET imaging (Aβ-PET) offers the possibility of quantitatively measuring amyloid load in the brain. However, currently, there are no dynamic monitoring technologies based on these modalities in cognitive training for patients after antibody treatment. Therefore, there is an urgent need for a program that integrates fMRI, ERP, and Aβ-PET data to personalize the content and dosage of cognitive training, adapting in real time to changes in the patient's nervous system and maximizing the effectiveness of cognitive training. Summary of the Invention

[0005] The primary technical problem to be solved by this invention is to provide a cognitive enhancement method based on multimodal data.

[0006] Another technical problem to be solved by the present invention is to provide a cognitive enhancement system based on multimodal data.

[0007] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0008] According to a first aspect of the present invention, a cognitive enhancement method based on multimodal data is provided, comprising the following steps:

[0009] Obtain the patient's first multimodal data before treatment;

[0010] Based on the patient's first multimodal data before treatment, the patient's amyloid protein load is obtained, and the patient is grouped according to a preset value;

[0011] Based on the grouping results of the patients, corresponding initial cognitive training plans are obtained; wherein, different grouping results correspond to different initial cognitive training plans, and the training dose and adaptive coefficient of each initial cognitive training plan are different;

[0012] During the patient's treatment, the initial cognitive training program is pushed to the patient for cognitive training, and second multimodal data is collected during the patient's treatment.

[0013] Based on the patient's first multimodal data before treatment and the patient's second multimodal data during treatment, combined with brain network atlases, the differences in the patient's baseline amyloid load, functional connectivity strength, and cognitive ability assessment before and after treatment were obtained.

[0014] The differences in baseline amyloid load, functional connectivity strength, and cognitive ability assessment before and after treatment of the patient are input into a preset model to output a cognitive improvement scheme that maximizes the improvement of the patient's cognitive index.

[0015] The cognitive enhancement program is pushed to the patient for cognitive training to improve the patient's cognitive index;

[0016] The first multimodal data and the second multimodal data include Aβ-PET data, fMRI data, and ERP data, respectively.

[0017] Preferably, the difference in baseline amyloid load before and after treatment for the patient is obtained through the following method:

[0018] Based on the patient's Aβ-PET data before treatment, the patient's baseline amyloid load before treatment was obtained and expressed as a centiloid value;

[0019] Brain region distribution is obtained based on the brain network atlas, and combined with the patient's baseline amyloid load before treatment, the first brain region amyloid distribution matrix of the patient is obtained; wherein, the number of columns of the first brain region amyloid distribution matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is 1, and the matrix elements are the patient's centiloid values ​​before treatment.

[0020] Based on the Aβ-PET data of the patient during treatment, the patient's baseline amyloid load during treatment was obtained and expressed as a centiloid value;

[0021] Brain region distribution is obtained based on the brain network atlas, and combined with the patient's baseline amyloid load during treatment, a second brain region amyloid distribution matrix is ​​obtained for the patient; wherein, the number of columns in the second brain region amyloid distribution matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is at least 1, and the matrix elements are the patient's centiloid values ​​during treatment.

[0022] By comparing the amyloid distribution matrix of the first brain region with the amyloid distribution matrix of the second brain region, the amyloid deposition change matrix in the patient's brain is obtained; wherein, the number of columns in the amyloid deposition change matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is 1, and the matrix elements are the difference in centiloid values ​​before and after treatment.

[0023] Preferably, the difference in functional connectivity strength before and after treatment is obtained by the following method:

[0024] Based on the patient's fMRI data before treatment, the functional connectivity strength of the patient's pre-set brain network before treatment was obtained;

[0025] The brain region distribution is obtained based on the brain network atlas, and the first functional connectivity matrix of the patient is obtained by combining the functional connectivity strength of the patient's brain network before treatment. The number of rows and columns of the first functional connectivity matrix are the number of brain regions included in the brain network atlas, and the elements of the matrix are the functional connectivity strength between the brain regions represented by the rows and columns before treatment.

[0026] Based on the fMRI data of the patient during treatment, the functional connectivity strength of the preset brain network of the patient during treatment is obtained;

[0027] The brain region distribution is obtained based on the brain network atlas, and the second functional connectivity matrix of the patient is obtained by combining the functional connectivity strength of the preset brain network during the treatment period. The number of rows and columns of the second functional connectivity matrix are the number of brain regions included in the brain network atlas, and the elements of the matrix are the functional connectivity strength between the brain regions represented by the rows and columns during the patient's treatment period.

[0028] By comparing the first functional connectivity matrix with the second functional connectivity matrix, the functional connectivity change matrix of the patient is obtained; wherein, the number of rows and columns of the functional connectivity change matrix are the number of brain regions included in the brain network atlas, and the elements of the matrix are the difference in functional connectivity strength between the brain regions represented by the rows and columns before and after the patient's treatment.

[0029] Preferably, the difference in cognitive ability assessment before and after treatment is obtained through the following method:

[0030] Based on the patient's ERP data before treatment, the patient's EEG activity for cognitive training tasks before treatment was recorded, and preset key indicators were extracted based on the EEG activity.

[0031] Based on the preset key indicators of the patient before treatment, the user's multiple cognitive abilities are assessed to obtain a first assessment result;

[0032] Based on the patient's ERP data during treatment, the patient's EEG activity during cognitive training tasks was recorded, and preset key indicators were extracted based on the EEG activity.

[0033] Based on the preset key indicators during the patient's treatment, the user's multiple cognitive abilities are reassessed to obtain a second assessment result;

[0034] By comparing the first assessment result with the second assessment result, the changes in the patient's cognitive ability are obtained;

[0035] The results of the changes in the patient's cognitive ability are combined with the corresponding preset key indicators to convert them into an ERP plasticity improvement matrix; wherein, the number of columns in the ERP plasticity improvement matrix is ​​the number of preset key indicators; the number of rows is at least 2, representing potential amplitude and occurrence time, respectively.

[0036] Preferably, the preset model is constructed in the following manner:

[0037] Acquire a large amount of multimodal data on historical users before and after treatment;

[0038] For each historical user, a multimodal change matrix is ​​obtained based on the multimodal data of the historical user before and after treatment.

[0039] Obtain the cognitive index of each historical user at different time points;

[0040] Using the cognitive index as a reference variable, the multimodal change matrices of all historical users are input into the neural network model to output the combination of cognitive training tasks that improves the cognitive index the most, thereby forming a mapping relationship from the multimodal change matrix to the combination of cognitive training tasks, and constructing the preset model.

[0041] Preferably, after the patient undergoes cognitive training based on the cognitive enhancement program, the patient's latest multimodal data is reacquired.

[0042] Based on the patient's latest multimodal data, the weights of different types of cognitive training tasks in the cognitive training task combination output by the preset model are adjusted to personalize the preset model.

[0043] Preferably, when the patient undergoes an Aβ-PET examination, the patient's plasma is simultaneously collected for the detection of AD biomarkers to obtain the corresponding plasma test results.

[0044] Based on the centiloid values ​​of the patient's multiple Aβ-PET examinations and the plasma test results corresponding to each Aβ-PET examination, the data are input into a preset model framework for model training to output the mapping relationship from the plasma test results to the centiloid values ​​of the Aβ-PET examinations, thereby forming a predictive model for predicting the total amyloid load of the patient.

[0045] The prediction model is used to predict the patient's next baseline amyloid load.

[0046] Preferably, when any modality is missing in the first multimodal data and / or the second multimodal data input into the preset model, it is supplemented by weighting the data of other modalities.

[0047] According to a second aspect of the present invention, a cognitive enhancement system based on multimodal data is provided, comprising:

[0048] The data acquisition unit is used to acquire first multimodal data of the patient before treatment and / or second multimodal data during treatment; wherein the first multimodal data and the second multimodal data respectively include Aβ-PET data, fMRI data and ERP data;

[0049] An initial training program push unit is connected to the data acquisition unit to obtain the patient's amyloid load based on the patient's first multimodal data before treatment, and to group the patients according to a preset value; and to obtain the corresponding initial cognitive training program based on the grouping results of the patients; wherein, different grouping results correspond to different initial cognitive training programs, and the training dose and adaptive coefficient of each initial cognitive training program are different;

[0050] A data processing unit, connected to the data acquisition unit and the initial training scheme push unit, obtains the patient's multimodal change matrix based on the patient's first multimodal data before treatment and the patient's second multimodal data during treatment, combined with brain network atlases; wherein, the multimodal change matrix includes at least: a brain amyloid deposition change matrix, a functional connectivity change matrix, and an ERP plasticity improvement matrix;

[0051] A cognitive enhancement program push unit is connected to the data processing unit and has a preset model to output a cognitive enhancement program that maximizes the improvement of the patient's cognitive index based on the patient's multimodal change matrix.

[0052] An interactive unit is connected to the cognitive enhancement program push unit to push the cognitive enhancement program to the patient for cognitive training, thereby improving the patient's cognitive index.

[0053] According to a third aspect of the present invention, a cognitive enhancement system based on multimodal data is provided, including a processor and a memory, wherein the processor reads a computer program in the memory for executing the above-described cognitive enhancement method based on multimodal data.

[0054] Compared with existing technologies, this invention utilizes three modalities of data—amyloid load (detected by Aβ-PET imaging), fMRI, and ERP—before and after medication to form a multimodal change matrix tailored to the patient. This fully leverages the guiding role of multimodal data to assess changes in the patient's neuroplasticity in real time and dynamically generate personalized computerized cognitive training programs. Based on this, a pre-built neuroplasticity prediction model using historical user data is used to adjust the weights of different types of cognitive training tasks in the preset model's output cognitive training task combination according to the patient's latest multimodal data, thereby achieving personalized adjustments to the preset model. This method, combining drug intervention and personalized cognitive training, can significantly improve the patient's cognitive enhancement. Attached Figure Description

[0055] Figure 1 This is an overall flowchart of a cognitive enhancement method based on multimodal data provided in the first embodiment of the present invention;

[0056] Figure 2 A structural diagram of a cognitive enhancement system based on multimodal data provided in the second embodiment of the present invention;

[0057] Figure 3 This is a structural diagram of a cognitive enhancement system based on multimodal data, provided in the third embodiment of the present invention. Detailed Implementation

[0058] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0059] The technical concept of this invention is to integrate Aβ-PET data, functional magnetic resonance imaging (fMRI) data, and event-related potential (ERP) data to assess changes in the patient's neuroplasticity in real time and dynamically generate personalized computerized cognitive training programs. This allows for the improvement of cognitive function in AD patients receiving anti-amyloid monoclonal antibody (such as lencanezumab) treatment, in addition to drug therapy and personalized cognitive training.

[0060] It's important to note that early changes in Alzheimer's disease (AD) often involve dysfunction of the default mode network (DMN), which primarily includes areas such as the posterior cingulate cortex, medial prefrontal cortex, and hippocampus. Studies have shown that a significant reduction in DMN functional connectivity is closely associated with cognitive decline. Furthermore, overactivation of task-related networks (such as attention networks) is a manifestation of the brain's attempt to compensate for lost function. These changes reflect insufficient neural plasticity, and anti-amyloid drugs may indirectly affect the functional recovery of these networks by reducing pathological burden.

[0061] Amyloid PET imaging uses radioactive tracers (such as ^18F-Florbetapir) to bind to amyloid plaques in the brain, and assesses the patient's pathological burden through quantitative analysis (centiloid value). The centiloid value can accurately reflect the level of amyloid accumulation in the patient's brain and decreases rapidly after antibody treatment. This dynamic change can serve as an indirect marker of improved neuroplasticity.

[0062] In one embodiment of the present invention, a brain network atlas is used to assess the distribution of amyloid protein. This brain network atlas is referenced from one published in Cerebral Cortex in 2016 by the team of Jiang Tianzai at the Institute of Automation, Chinese Academy of Sciences. This atlas divides the human brain into 246 finely defined regions. The brain network atlas contains detailed structures of cortical regions and subcortical nuclei, and quantitatively depicts the anatomical and functional connectivity patterns of different brain subregions in vivo, providing a detailed functional description of each subregion. Applying this atlas to PET data helps to accurately characterize the deposition patterns of amyloid protein.

[0063] Event-related potentials (ERPs) are the time-locked responses of EEG signals to specific stimuli. The P300 wave is a commonly used ERP indicator, reflecting the processes of attention allocation and working memory updating. Studies have shown that increased P300 wave amplitude and shortened latency are directly related to improved neuroplasticity. Therefore, ERP can serve as a sensitive indicator for assessing the effectiveness of cognitive training.

[0064] like Figure 1 As shown, the first embodiment of the present invention provides a cognitive enhancement method based on multimodal data, which includes at least the following steps:

[0065] S1: Obtain the patient's first multimodal data before treatment.

[0066] In one embodiment of the present invention, the multimodal data includes at least Aβ-PET data, fMRI data, and ERP data, but is not limited to specific data types. It should be noted that in other embodiments, other types of data may be selected for multimodal data fusion as needed.

[0067] Specifically, for patients with mild cognitive impairment of AD origin, the baseline amyloid burden x is determined by Aβ-PET examination before treatment and calculated as a centiloid value.

[0068] Furthermore, functional magnetic resonance imaging (fMRI) data of patients were collected before the start of monoclonal antibody drug treatment to assess the functional connectivity characteristics of the patients' brain networks.

[0069] In addition, before starting monoclonal antibody drug treatment, ERP data is collected from the patient using event-related potential (ERP) technology to record the patient's EEG activity for cognitive tasks and extract key indicators such as P300 amplitude and latency.

[0070] S2: Based on the patient's first multimodal data before treatment, obtain the patient's brain amyloid load, and group the patients according to preset values.

[0071] Specifically, the amyloid burden in the brain of patients is quantified based on Aβ-PET data and expressed as a centiloid value. Patients are then divided into high, medium, and low burden groups according to preset values. In one embodiment of the invention, the groups are as follows: 12 < centiloid ≤ 30: low burden group; 30 < centiloid ≤ 50: medium burden group; centiloid > 50: high burden group. It is understood that in other embodiments, the preset values ​​can be adjusted as needed, and other methods can be used to group patients; no specific limitations are made here.

[0072] Furthermore, preferably, when a patient undergoes an Aβ-PET scan, plasma is simultaneously collected for AD biomarker detection to obtain corresponding plasma test results. Then, based on the centiloid values ​​from multiple Aβ-PET scans and the corresponding plasma test results from each Aβ-PET scan, the data is input into a pre-defined model framework for model training. This trains the model to output a mapping relationship from plasma test results to centiloid values ​​from Aβ-PET scans, thereby forming a predictive model for predicting the total amyloid burden in patients.

[0073] Once the predictive model is developed, it can be used to predict the patient's next amyloid baseline load, thus saving the patient the trouble of going to the hospital for an Aβ-PET scan and improving the convenience of data collection (i.e., collection of amyloid baseline load).

[0074] S3: Based on the patient grouping results, obtain the corresponding initial cognitive training plan.

[0075] Specifically, after the patients are grouped based on step S2, the corresponding initial cognitive training plan is obtained according to the grouping results, thereby providing computerized cognitive training for the patients.

[0076] In one embodiment of the present invention, the initial cognitive training scheme is based on adaptive cognitive training, and selects three training doses K(x) of high, medium and low according to the grouping results, and three basic adaptive coefficients P(x), which correspond to the high load group, medium load group and low load group respectively.

[0077] S4: During the patient's treatment, push the initial cognitive training program to the patient for cognitive training, and collect the second multimodal data of the patient during the treatment period.

[0078] Understandably, the type of multimodal data in step S4 is the same as that in step S1. The difference lies in the fact that the data collection time in step S4 is during the patient's treatment period, and cognitive training is also required during the patient's treatment period based on the initial cognitive training plan obtained in step S3.

[0079] S5: Based on the patient's first multimodal data before treatment and the patient's second multimodal data during treatment, combined with brain network atlases, the differences in baseline amyloid load, functional connectivity strength, and cognitive ability assessment before and after treatment were obtained.

[0080] In one embodiment of the present invention, a multimodal change matrix is ​​formed using the differences in baseline amyloid load, functional connectivity strength, and cognitive ability assessment before and after treatment. This multimodal change matrix includes at least: a matrix of changes in brain amyloid deposition (corresponding to step S51), a matrix of changes in functional connectivity (corresponding to step S52), and a matrix of improvements in ERP plasticity (corresponding to step S53). It is understood that in one embodiment of the present invention, only a matrix is ​​used as an example for illustration, and the representation of the differences in baseline amyloid load, functional connectivity strength, and cognitive ability assessment before and after treatment using a matrix is ​​not limited to this specific form.

[0081] The following sections will explain in detail the process of obtaining each matrix:

[0082] S51: Obtain the matrix of changes in amyloid protein deposition in the brain.

[0083] Specifically, steps S511 to S515 are included:

[0084] S511: Based on the patient's pre-treatment Aβ-PET data, obtain the patient's baseline amyloid load before treatment and express it as a centiloid value.

[0085] S512: Obtain the brain region distribution based on the brain network atlas and, combined with the patient's baseline amyloid load before treatment, obtain the amyloid distribution matrix of the patient's first brain region. Specifically, the number of columns in this first brain region amyloid distribution matrix is ​​equal to the number of brain regions included in the brain network atlas, the number of rows is 1, and the matrix elements are the patient's centiloid values ​​before treatment.

[0086] S513: Based on Aβ-PET data during the patient's treatment, obtain the patient's baseline amyloid load during treatment and express it as a centiloid value.

[0087] S514: Obtain the brain region distribution based on the brain network atlas and, combined with the patient's baseline amyloid load during treatment, obtain the patient's second brain region amyloid distribution matrix. Specifically, the number of columns in this second brain region amyloid distribution matrix is ​​equal to the number of brain regions included in the brain network atlas, the number of rows is 1, and the matrix elements are the patient's centiloid values ​​during treatment.

[0088] S515: Compare the amyloid distribution matrix of the first brain region with the amyloid distribution matrix of the second brain region to obtain the matrix of changes in amyloid deposition in the patient's brain.

[0089] After obtaining the amyloid distribution matrix of the first brain region and the amyloid distribution matrix of the second brain region based on steps S512 and S514 respectively, the difference between the corresponding matrix elements in the two matrices is calculated to form the amyloid deposition change matrix in the brain. That is, the number of columns in this amyloid deposition change matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is at least 1, and the matrix elements are the difference in centiloid values ​​before and after the patient's treatment.

[0090] S52: Obtain the function connection change matrix.

[0091] Specifically, steps S521 to S525 are included:

[0092] S521: Based on the patient's pre-treatment fMRI data, obtain the functional connectivity strength of the patient's pre-defined brain network. This functional connectivity strength can be obtained by calculating the Pearson correlation index, a conventional technique in this field, and will not be elaborated further here.

[0093] S522: Obtain the brain region distribution based on the brain network atlas and, combined with the patient's pre-set brain network functional connectivity strength, obtain the patient's first functional connectivity matrix. Specifically, the number of rows and columns in this first functional connectivity matrix is ​​the number of brain regions included in the brain network atlas, and the elements of the matrix are the functional connectivity strengths of the brain regions represented by the rows and columns before the patient's treatment.

[0094] S523: Based on fMRI data during the patient's treatment, obtain the functional connectivity strength of the patient's pre-defined brain network during the treatment period.

[0095] S524: Obtain the brain region distribution based on the brain network atlas and, combined with the pre-defined functional connectivity strength of the patient's brain network during treatment, obtain the patient's second functional connectivity matrix. Specifically, the number of rows and columns in this second functional connectivity matrix is ​​the number of brain regions included in the brain network atlas, and the elements of the matrix are the functional connectivity strength between the brain regions represented by the rows and columns during the patient's treatment.

[0096] S525: Compare the first functional connectivity matrix with the second functional connectivity matrix to obtain the patient's functional connectivity change matrix.

[0097] Specifically, after obtaining the first functional connectivity matrix and the second functional connectivity matrix based on steps S512 and S514 respectively, the difference between the corresponding matrix elements in the two matrices is calculated to obtain the functional connectivity change matrix. That is, the number of rows and columns in the functional connectivity change matrix is ​​the number of brain regions included in the brain network atlas, and the elements of the matrix are the difference in functional connectivity strength between the brain regions represented by the rows and columns before and after the patient's treatment.

[0098] S53: Obtain the ERP flexibility improvement matrix.

[0099] Specifically, steps S531 to S536 are included:

[0100] S531: Based on the patient's ERP data before treatment, record the patient's EEG activity for cognitive training tasks before treatment, and extract preset key indicators based on the EEG activity.

[0101] In one embodiment of the present invention, the preset key indicators include: P1, N1, P2, N2, and P3. ERP-related event correlation is a response to stimuli presented in the form of electroencephalogram (EEG). Generally, the main components are five: P1, N1, P2, N2, and P3. The numbers represent the time after stimulus presentation, and P and N represent whether the wave is positive or negative. Therefore, P1 (or P100) is a positive wave 100ms after stimulus presentation, while N2 is a negative wave 200ms after stimulus presentation, and so on.

[0102] Furthermore, in chronological order, the first three (i.e., P1, N1, and P2) are called exogenous components, which are influenced by physical stimuli and are only related to the physical characteristics of the stimuli. The latter two (i.e., N2 and P3) are endogenous components, which reflect psychological activities, including but not limited to attention, discrimination, and working memory.

[0103] S532: Based on the patient's pre-set key indicators, assess the user's multiple cognitive abilities to obtain the first assessment result;

[0104] S533: Based on the patient's ERP data during treatment, record the patient's EEG activity for cognitive training tasks during treatment, and extract preset key indicators based on the EEG activity.

[0105] S534: Based on the preset key indicators during the patient's treatment, reassess the user's multiple cognitive abilities to obtain a second assessment result;

[0106] S535: Compare the results of the first assessment with the results of the second assessment to obtain the results of changes in the patient's cognitive abilities;

[0107] S536: Combine the patient's cognitive ability changes with the corresponding preset key indicators to convert them into an ERP plasticity improvement matrix to reflect the real-time changes in the patient's neuroplasticity.

[0108] In one embodiment of the present invention, the ERP plasticity improvement matrix has 5 columns, corresponding to P1, N1, P2, N2, and P3 respectively, and at least 2 rows, representing the potential amplitude and occurrence time respectively.

[0109] S6: Input the patient's multimodal change matrix into the preset model to output the cognitive improvement plan that maximizes the improvement of the patient's cognitive index.

[0110] In one embodiment of the present invention, a model needs to be pre-built, so as to use the patient's multimodal change matrix as the model input, and then output the cognitive improvement scheme that maximizes the improvement of the patient's cognitive index through the model.

[0111] Specifically, the preset model is constructed in the following way:

[0112] ① Obtain a large amount of multimodal data on historical users before and after treatment.

[0113] It is understandable that the multimodal data of the historical user before and after treatment correspond to the data collected in steps S1 and S4 above, respectively.

[0114] ② For each historical user, obtain the multimodal change matrix of each historical user based on the multimodal data before and after treatment.

[0115] It is understandable that the multimodal change matrix of this historical user is obtained through the same process as obtaining the multimodal change matrix in step S5 above.

[0116] ③ Obtain the cognitive index of each historical user at different time points.

[0117] It is understandable that for each historical user, the cognitive index (CI) will be different at different points in time, which can reflect the differences in the effectiveness of cognitive training.

[0118] ④ Construct a pre-defined model.

[0119] Specifically, using the cognitive index (CI) as a reference variable, the multimodal change matrices of all historical users are input into the neural network model to output the combination of cognitive training tasks that maximizes the improvement of the cognitive index. This establishes a mapping relationship from the multimodal change matrix to the combination of cognitive training tasks, thus constructing the pre-defined model. Furthermore, the loss function of this neural network model can be the cross-entropy loss function, or other loss functions; no specific limitation is made here.

[0120] S7: Push cognitive enhancement programs to patients for cognitive training to improve their cognitive index.

[0121] In one embodiment of the present invention, preferably, after the patient has undergone cognitive training based on the cognitive enhancement program, the patient's latest multimodal data is reacquired. Furthermore, based on the patient's latest multimodal data, the weights of different types of cognitive training tasks in the cognitive training task combination output by the preset model are adjusted, thereby personalizing the preset model.

[0122] Even better, when data from any modality is missing in the multimodal data input into the preset model, it is supplemented by weighting data from other modalities to ensure the stability and continuity of the cognitive enhancement program.

[0123] like Figure 2 As shown, based on the first embodiment described above, the second embodiment of the present invention provides a cognitive enhancement system based on multimodal data, including a data acquisition unit 1, an initial training scheme push unit 2, a data processing unit 3, a cognitive enhancement scheme push unit 4, and an interaction unit 5.

[0124] Specifically, the data acquisition unit 1 is used to acquire second multimodal data of the patient before or during treatment. The initial training plan push unit 2 is connected to the data acquisition unit 1 to push the initial training plan to the user. The initial training plan push unit 2 pre-stores three different initial training plans, with adaptive cognitive training as the base version. Based on the patient's grouping results, three training doses K(x) are selected: high, medium, and low, and three basic adaptive coefficients P(x) are corresponding to the high-load group, medium-load group, and low-load group, respectively.

[0125] The data processing unit 3 is connected to the data acquisition unit 1 and the initial training program push unit 2 to obtain the patient's multimodal change matrix. In one embodiment of the present invention, the multimodal change matrix includes at least: a brain amyloid deposition change matrix, a functional connectivity change matrix, and an ERP plasticity improvement matrix. Accordingly, the data processing unit 3 includes at least a brain amyloid deposition assessment module 31, a functional connectivity assessment module 32, and an ERP plasticity assessment module 33 to respectively acquire the above three matrices.

[0126] The cognitive enhancement program delivery unit 4 is connected to the data processing unit 3 and has a preset model (which is constructed in the manner described in step S6 above). Using this preset model, and based on the multimodal change matrix obtained by the data processing unit 3, it outputs the cognitive enhancement program that maximizes the improvement of the patient's cognitive index.

[0127] Interaction unit 5 is connected to cognitive enhancement program push unit 4 to push the cognitive enhancement program to the patient for cognitive training in order to improve the patient's cognitive index.

[0128] It is understood that the above-mentioned functional units are functional modules that implement the steps in the first embodiment, but are not limited to the module type or connection form. In other embodiments, other types of modules can be selected as needed to implement the steps in the first embodiment.

[0129] like Figure 3 As shown, based on the first embodiment described above, the present invention further provides a cognitive enhancement system based on multimodal data. This cognitive enhancement system includes one or more processors 21 and a memory 22. The memory 22 is coupled to the processor 21 and is used to store one or more programs. When the programs are executed by the processor 21, the processor 21 implements the cognitive enhancement method based on multimodal data as described in the above embodiment.

[0130] The processor 21 controls the overall operation of the cognitive enhancement system to complete all or part of the steps of the aforementioned cognitive enhancement method based on multimodal data. The processor 21 can be a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processing (DSP) chip, etc. The memory 22 stores various types of data to support the operation of the cognitive enhancement system. This data may include, for example, instructions for any application or method operating on the cognitive enhancement system, as well as application-related data. The memory 22 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, etc.

[0131] In one exemplary embodiment, the cognitive enhancement system may be implemented by a computer chip or physical entity, or by a product with certain functions, to execute the aforementioned cognitive enhancement method based on multimodal data and achieve the same technical effect as described above. A typical embodiment is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0132] In another exemplary embodiment, the present invention also provides a computer-readable storage medium including program instructions that, when executed by a processor, implement the steps of the cognitive enhancement method based on multimodal data in any of the above embodiments. For example, the computer-readable storage medium may be the memory including the program instructions described above, which can be executed by the processor of the cognitive enhancement system to complete the cognitive enhancement method based on multimodal data described above and achieve the same technical effects as the method described above.

[0133] In summary, the cognitive enhancement method and system based on multimodal data provided in this invention utilizes amyloid load (detected by Aβ-PET imaging), fMRI, and ERP data before and after medication to form a multimodal change matrix tailored to the patient. This fully leverages the guiding role of multimodal data to assess changes in the patient's neuroplasticity in real time and dynamically generate personalized computerized cognitive training programs. Furthermore, by using a pre-built neuroplasticity prediction model based on historical user groups, the weights of different types of cognitive training tasks in the cognitive training task combinations output by the preset model are adjusted according to the patient's latest multimodal data, thereby achieving personalized adjustments to the preset model. This method, combining drug intervention and personalized cognitive training, can significantly improve the patient's cognitive enhancement effect.

[0134] It should be noted that the above embodiments are merely illustrative examples. The technical solutions of each embodiment can be combined, and all are within the protection scope of this invention.

[0135] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0136] The foregoing has provided a detailed description of the cognitive enhancement method and system based on multimodal data provided by this invention. Any obvious modifications made by those skilled in the art without departing from the essential content of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. A cognitive enhancement method based on multimodal data for improving the cognitive function of Alzheimer's disease patients undergoing anti-amyloid monoclonal antibody therapy, characterized in that... Includes the following steps: Acquire first multimodal data of patients before treatment to assess changes in patients' neuroplasticity in real time; Based on the patient's first multimodal data before treatment, the patient's amyloid protein load is obtained, and the patient is grouped according to a preset value; Based on the grouping results of the patients, corresponding initial cognitive training plans are obtained; wherein, different grouping results correspond to different initial cognitive training plans, and the training dose and adaptive coefficient of each initial cognitive training plan are different; During the patient's treatment, the initial cognitive training program is pushed to the patient for cognitive training, and second multimodal data is collected during the patient's treatment. Based on the patient's first multimodal data before treatment and the patient's second multimodal data during treatment, combined with brain network atlases, the differences in the patient's baseline amyloid load, functional connectivity strength, and cognitive ability assessment before and after treatment were obtained. The patient's multimodal change matrix was formed by using the difference in baseline amyloid load, difference in functional connectivity strength, and difference in cognitive ability assessment before and after treatment. The patient's multimodal change matrix includes at least: a matrix of changes in brain amyloid deposition, a matrix of changes in functional connectivity, and a matrix of improvement in ERP plasticity. The patient's multimodal change matrix is ​​input into a preset model to output a cognitive improvement scheme that maximizes the improvement of the patient's cognitive index. The preset model is constructed as follows: acquiring a large amount of multimodal data of historical users before and after treatment; for each historical user, acquiring a multimodal change matrix based on the multimodal data before and after treatment; acquiring the cognitive index of each historical user at different time points; using the cognitive index as a reference variable, inputting the multimodal change matrices of all historical users into a transformer algorithm model to output a combination of cognitive training tasks that maximizes the improvement of the cognitive index, thereby forming a mapping relationship from the multimodal change matrix to the combination of cognitive training tasks, and thus constructing the preset model. The cognitive enhancement program is pushed to the patient for cognitive training to improve the patient's cognitive index; Based on the patient's latest multimodal data, the weights of different types of cognitive training tasks in the combination of cognitive training tasks output by the preset model are adjusted to personalize the preset model; and when data of any modality is missing in the multimodal data input into the preset model, it is supplemented by weighting data of other modalities. The first multimodal data and the second multimodal data include Aβ-PET data, fMRI data and ERP data, respectively; the data acquisition time of the second multimodal data is during the patient's treatment period, and cognitive training is also required during the patient's treatment period based on the acquired cognitive training task combination.

2. The cognitive enhancement method as described in claim 1, characterized in that... The matrix of changes in amyloid protein deposition in the brain was obtained through the following method: Based on the patient's Aβ-PET data before treatment, the patient's baseline amyloid load before treatment was obtained and expressed as a centiloid value; Brain region distribution is obtained based on the brain network atlas, and combined with the patient's baseline amyloid load before treatment, the first brain region amyloid distribution matrix of the patient is obtained; wherein, the number of columns of the first brain region amyloid distribution matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is 1, and the matrix elements are the patient's centiloid values ​​before treatment. Based on the Aβ-PET data of the patient during treatment, the patient's baseline amyloid load during treatment was obtained and expressed as a centiloid value; Brain region distribution is obtained based on the brain network atlas, and combined with the patient's baseline amyloid load during treatment, a second brain region amyloid distribution matrix is ​​obtained for the patient; wherein, the number of columns in the second brain region amyloid distribution matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is at least 1, and the matrix elements are the patient's centiloid values ​​during treatment. By comparing the amyloid distribution matrix of the first brain region with the amyloid distribution matrix of the second brain region, the amyloid deposition change matrix in the patient's brain is obtained; wherein, the number of columns in the amyloid deposition change matrix is ​​the number of brain regions included in the brain network atlas, the number of rows is 1, and the matrix elements are the difference in centiloid values ​​before and after treatment.

3. The cognitive enhancement method as described in claim 2, characterized in that... The functional connection change matrix is ​​obtained in the following way: Based on the patient's fMRI data before treatment, the functional connectivity strength of the patient's pre-set brain network before treatment was obtained; The brain region distribution is obtained based on the brain network atlas, and the first functional connectivity matrix of the patient is obtained by combining the functional connectivity strength of the patient's brain network before treatment. The number of rows and columns of the first functional connectivity matrix are the number of brain regions included in the brain network atlas, and the elements of the matrix are the functional connectivity strength between the brain regions represented by the rows and columns before treatment. Based on the fMRI data of the patient during treatment, the functional connectivity strength of the preset brain network of the patient during treatment is obtained; The brain region distribution is obtained based on the brain network atlas, and the second functional connectivity matrix of the patient is obtained by combining the functional connectivity strength of the preset brain network during the treatment period. The number of rows and columns of the second functional connectivity matrix are the number of brain regions included in the brain network atlas, and the elements of the matrix are the functional connectivity strength between the brain regions represented by the rows and columns during the patient's treatment period. By comparing the first functional connectivity matrix with the second functional connectivity matrix, the functional connectivity change matrix of the patient is obtained; wherein, the number of rows and columns of the functional connectivity change matrix are the number of brain regions included in the brain network atlas, and the elements of the matrix are the difference in functional connectivity strength between the brain regions represented by the rows and columns before and after the patient's treatment.

4. The cognitive enhancement method as described in claim 3, characterized in that... The ERP plasticity improvement matrix was obtained through the following methods: Based on the patient's ERP data before treatment, the patient's EEG activity for cognitive training tasks before treatment was recorded, and preset key indicators were extracted based on the EEG activity. Based on the preset key indicators of the patient before treatment, the user's multiple cognitive abilities are assessed to obtain a first assessment result; Based on the patient's ERP data during treatment, the patient's EEG activity during cognitive training tasks was recorded, and preset key indicators were extracted based on the EEG activity. Based on the preset key indicators during the patient's treatment, the user's multiple cognitive abilities are reassessed to obtain a second assessment result; By comparing the first assessment result with the second assessment result, the changes in the patient's cognitive ability are obtained; The results of the changes in the patient's cognitive ability are combined with the corresponding preset key indicators to convert them into an ERP plasticity improvement matrix; wherein, the number of columns in the ERP plasticity improvement matrix is ​​the number of the preset key indicators; the number of rows is at least 2, representing potential amplitude and occurrence time, respectively.

5. The cognitive enhancement method as described in claim 2, characterized in that: When the patient undergoes an Aβ-PET scan, plasma is simultaneously collected from the patient to detect AD biomarkers in order to obtain the corresponding plasma test results. Based on the centiloid values ​​of the patient's multiple Aβ-PET examinations and the plasma test results corresponding to each Aβ-PET examination, the data are input into a preset model framework for model training to output the mapping relationship from the plasma test results to the centiloid values ​​of the Aβ-PET examinations, thereby forming a predictive model for predicting the total amyloid load of the patient. The prediction model is used to predict the patient's next baseline amyloid load.

6. A cognitive enhancement system based on multimodal data, used to implement the cognitive enhancement method according to any one of claims 1 to 5, characterized in that... include: A data acquisition unit is used to acquire multimodal data of patients before or during treatment; wherein the multimodal data includes at least Aβ-PET data, fMRI data, and ERP data; An initial training program push unit is connected to the data acquisition unit to obtain the patient's amyloid load based on the patient's pre-treatment multimodal data, and to group the patient according to a preset value; and to obtain a corresponding initial cognitive training program based on the patient's grouping results; wherein, different grouping results correspond to different initial cognitive training programs, and the training dose and adaptive coefficient of each initial cognitive training program are different; A data processing unit, connected to the data acquisition unit and the initial training scheme push unit, obtains the patient's multimodal change matrix based on the patient's pre-treatment multimodal data and the patient's multimodal data during treatment, combined with brain network atlases; wherein, the multimodal change matrix includes at least: a brain amyloid deposition change matrix, a functional connectivity change matrix, and an ERP plasticity improvement matrix; A cognitive enhancement program push unit is connected to the data processing unit and has a preset model to output a cognitive enhancement program that maximizes the improvement of the patient's cognitive index based on the patient's multimodal change matrix. An interactive unit is connected to the cognitive enhancement program push unit to push the cognitive enhancement program to the patient for cognitive training, in order to improve the patient's cognitive index.

7. A cognitive enhancement system based on multimodal data, characterized in that... It includes a processor and a memory, wherein the processor reads a computer program from the memory for implementing the cognitive enhancement method according to any one of claims 1 to 5.

Images