An alzheimer's disease early cognitive impairment assessment system based on a multi-modal large model
Patent Information
- Application Number
- CN202610902291.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-23
- Publication Date
- 2026-09-15
Smart Images
Figure CN122762272A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer-aided medical technology, and in particular to an assessment system for early cognitive impairment in Alzheimer's disease based on a multimodal large model. Background Technology
[0002] With the increasing global aging population, the number of Alzheimer's disease (AD) patients is constantly rising. Expert consensus points out that shifting the window for AD diagnosis and intervention earlier, focusing on the pre-dementia stage—prodromal AD (pAD)—is currently a focus of clinical and medical research. In the pAD stage, clinical symptoms are not yet severe, but neuropathological changes have already begun to form. Timely intervention at this stage may stop or slow the progression of AD to dementia. Therefore, early diagnosis is crucial for delaying and postponing the course of AD. Early cognitive assessment of AD includes seven domains: memory function, language function, orientation, application ability, attention, perception, and executive function. Currently, clinical assessment scales have indispensable diagnostic value in the analysis of cognitive abilities and are an important component of the system for localizing and characterizing cognitive impairment. However, current clinical scales—the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA)—have problems such as high repetition rate of test items, high false negative rates, susceptibility to subjective factors of the test taker, high cost, and low accessibility.
[0003] Therefore, the current assessment of early cognitive impairment in Alzheimer's disease is hampered by uneven distribution of medical resources, difficulty in large-scale hospital screening, high medical resource consumption, heavy personal financial burden, and problems with clinical screening scales such as single static question banks, stringent testing conditions, and susceptibility to subjective factors of physicians. Summary of the Invention
[0004] The purpose of this invention is to provide an early cognitive impairment assessment system for Alzheimer's disease based on a multimodal large model, aiming to solve or improve at least one of the above-mentioned technical problems.
[0005] To achieve the above objectives, the present invention provides the following solution: An assessment system for early cognitive impairment in Alzheimer's disease based on a multimodal large model, comprising: The interaction layer includes a question display module, a question answering interaction module, a result display module, and a history record module. It is used to interact with users in a multimodal manner and to display the question-generating process, the question answering process, the result output, and the history record. The multimodal information interaction includes text interaction, voice interaction, and image interaction. The API gateway connects to each module in the interaction layer to uniformly manage and route requests from the interaction layer, and to perform authentication and access control. The core service layer includes a test question service module, a question answer service module, a scoring and scheduling service module, a speech synthesis service module, a speech transcription service module, an AI dialogue engine module, a result feedback module, and a history record service module. Each module is connected to the API gateway and is used to execute corresponding business processes and task scheduling based on the transmitted data. The AI capability layer is connected to the modules in the core service layer. It is used to execute the evaluation process using the preset multi-agent and Spark voice big model, and generate personalized cognitive evaluation reports based on the scoring results of each agent. The preset multi-agent includes a doctor agent, a cognitive reasoning agent, a voice semantic agent, and a visual analysis agent. The data layer, connected to the core service layer and the AI capability layer, is used for persistent storage of question bank data, answer process data, answer result data, and file data. The evaluation process is as follows: The doctor's intelligent agent interacts with the user in multiple rounds through the AI dialogue engine module, and records the user's answers to each screening question during the interaction. The scoring and scheduling service module distributes user answers to the cognitive reasoning agent, the speech and semantic agent, or the visual analysis agent for scoring based on the question type. The scoring results output by each intelligent agent are aggregated by the scoring scheduling service module and then transmitted to the doctor intelligent agent; The doctor's intelligent agent, combined with the user's background information, generates a personalized cognitive assessment report, which is then presented to the user through the results feedback module.
[0006] Optionally, the test item service module constructs a dynamic question bank based on the clinical MMSE scale and the MoCA scale, and the dynamic question bank is used to generate 4.2 × 10 19 Different sets of questionnaires.
[0007] Optionally, the doctor agent guides the user to complete the screening through a multi-turn dialogue strategy, and dynamically calls the speech synthesis service module to read the questions aloud throughout the dialogue process. At the same time, it supports the user to input voice through the speech transcription service module or text through the question-answering interaction module.
[0008] Optionally, the context processing logic of the scoring and scheduling service module is as follows: When the question type is orientation, memory, or calculation, the user's answer is distributed to the cognitive reasoning agent; when the question type is language ability related, the user's answer is distributed to the speech and semantic agent; when the question type is visual drawing or executive function, the user's answer is distributed to the visual analysis agent.
[0009] Optionally, the visual analysis agent performs structured understanding of the user-uploaded drawing images based on the Gemini large model, analyzes the elements, spatial structure and expressive intent in the images, and outputs the drawing question score after comparing it with the standard answer. The drawing question score is returned to the doctor agent through the scoring scheduling service module.
[0010] Optionally, the history record module can call the user's historical answer records and evaluation results stored in the data layer through the history record service module, enabling users to track the trend of cognitive function changes across time periods.
[0011] Optionally, the personalized cognitive assessment report generated by the doctor's intelligent agent includes the total screening score, the score distribution of each cognitive domain, the risk assessment level, and clear guidance on whether to recommend further medical examination.
[0012] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention discloses an early cognitive impairment assessment system for Alzheimer's disease based on a multimodal large model. The system includes an interaction layer, an API gateway, an AI capability layer, and a data layer. It assesses early cognitive impairment in Alzheimer's disease through a pre-defined assessment process. This process involves: a doctor's intelligent agent interacting with the user in multiple rounds via an AI dialogue engine and recording the responses; a scoring scheduling service distributing questions to corresponding intelligent agents for scoring based on question type; and after summarizing the scores, the doctor's intelligent agent generating a personalized cognitive assessment report based on the user's background, which is then presented through a results feedback module. This invention significantly improves scoring accuracy and system stability through intelligent agent interaction logic and continuous iteration using massive amounts of test data. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a structural diagram of the Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model, as described in this invention. Figure 2 This is the model interaction diagram in this embodiment. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] The purpose of this invention is to provide an early cognitive impairment assessment system for Alzheimer's disease based on a multimodal large model, aiming to solve or improve at least one of the above-mentioned technical problems.
[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0018] like Figure 1 As shown, this invention provides an early cognitive impairment assessment system for Alzheimer's disease based on a multimodal large model, comprising: The interaction layer includes a question display module, a question answering interaction module, a result display module, and a history record module. It is used to interact with users in a multimodal manner and to display the question-generating process, the question answering process, the result output, and the history record. The multimodal information interaction includes text interaction, voice interaction, and image interaction. The API gateway connects to each module in the interaction layer to uniformly manage and route requests from the interaction layer, and to perform authentication and access control. The core service layer includes a test question service module, a question answer service module, a scoring and scheduling service module, a speech synthesis service module, a speech transcription service module, an AI dialogue engine module, a result feedback module, and a history record service module. Each module is connected to the API gateway and is used to execute corresponding business processes and task scheduling based on the transmitted data. The AI capability layer is connected to the modules in the core service layer. It is used to execute the evaluation process using the preset multi-agent and Spark voice big model, and generate personalized cognitive evaluation reports based on the scoring results of each agent. The preset multi-agent includes a doctor agent, a cognitive reasoning agent, a voice semantic agent, and a visual analysis agent. The data layer, connected to the core service layer and the AI capability layer, is used for persistent storage of question bank data, answer process data, answer result data, and file data. The evaluation process is as follows: The doctor's intelligent agent interacts with the user in multiple rounds through the AI dialogue engine module, and records the user's answers to each screening question during the interaction. The scoring and scheduling service module distributes user answers to the cognitive reasoning agent, the speech and semantic agent, or the visual analysis agent for scoring based on the question type. The scoring results output by each intelligent agent are aggregated by the scoring scheduling service module and then transmitted to the doctor intelligent agent; The doctor's intelligent agent, combined with the user's background information, generates a personalized cognitive assessment report, which is then presented to the user through the results feedback module.
[0019] As one specific implementation method, the test item service module constructs a dynamic question bank based on the clinical MMSE scale and the MoCA scale. This dynamic question bank is used to generate 4.2 × 10⁻⁶ questions. 19 Different sets of questionnaires.
[0020] As one specific implementation, the doctor's intelligent agent guides the user to complete the screening through a multi-turn dialogue strategy, and dynamically calls the speech synthesis service module to read the questions aloud throughout the dialogue process. At the same time, it supports the user to input voice through the speech transcription service module or text through the question-answering interaction module.
[0021] As a specific implementation method, the context processing logic of the scoring and scheduling service module is as follows: When the question type is orientation, memory, or calculation, the user's answer is distributed to the cognitive reasoning agent; when the question type is language ability related, the user's answer is distributed to the speech and semantic agent; when the question type is visual drawing or executive function, the user's answer is distributed to the visual analysis agent.
[0022] As a specific implementation method, the visual analysis agent performs structured understanding on the user-uploaded drawing images based on the Gemini large model, analyzes the elements, spatial structure and expressive intent in the images, and outputs the drawing question score after comparing it with the standard answer. The drawing question score is returned to the doctor agent through the scoring scheduling service module.
[0023] As a specific implementation, the history recording module calls the user's historical answer records and evaluation results stored in the data layer through the history recording service module, supporting users to track the trend of cognitive function changes across time periods.
[0024] As one specific implementation, the personalized cognitive assessment report generated by the doctor's intelligent agent includes the total screening score, the score distribution of each cognitive domain, the risk assessment level, and clear guidance on whether to recommend further medical examination.
[0025] Based on the above technical solution, the following embodiments are provided.
[0026] In this embodiment, the assessment system is built on a layered architecture, comprising an interaction layer, an API gateway, a core service layer, an AI capability layer, and a data layer. Users initiate assessment requests through the front-end interface of the interaction layer (supporting text, voice, and image input). All requests are first managed and routed by the API gateway. The API gateway is responsible for distributing requests to the corresponding modules in the core service layer, while also performing authentication and access control to ensure session security. For registered users, the history module retrieves past assessment results from the data layer through the history service, allowing users to understand their cognitive function trends before the assessment. After user authentication, the system assigns a unique identifier to the current session and initiates various business processes in the core service layer, preparing for subsequent rounds of interaction.
[0027] During the session initialization phase, the core service layer invokes the doctor's intelligent agent located in the AI capability layer and loads user background information (such as age, education level, and past medical history). The doctor's intelligent agent, acting as the overall commander of the assessment process, is responsible for formulating the interaction strategy. Simultaneously, the question service module constructs a dynamic question bank based on the widely used MMSE and MoCA scales, which theoretically can generate approximately 4.2 × 10⁻⁶ questions. 19 Different questionnaires are used to effectively avoid the practice effect. The system extracts a combination of questions suitable for the current user from the question bank to form a personalized screening questionnaire. Subsequently, the doctor's intelligent agent establishes a multi-turn dialogue connection with the user through the AI dialogue engine module, and can optionally enable the speech synthesis service module to read the questions aloud throughout the process. At the same time, the speech transcription service module is ready to convert the user's voice answers into text at any time, ensuring seamless integration of multimodal interaction.
[0028] The evaluation process employs a cyclical question-and-answer mechanism, with each round comprising five sub-steps: question generation, answering, type determination, agent scoring, and result summarization. The question display module presents questions in text, image, or audio format, and the doctor's agent decides whether to add voice narration based on a preset strategy. Users submit answers through the answer interaction module, supporting keyboard input, voice input, or image upload. After user voice input, the voice transcription service module performs real-time speech-to-text conversion to ensure the accuracy of subsequent analysis.
[0029] The key lies in the context processing logic of the scoring and scheduling service module: it intelligently distributes user answers to different specialized agents based on the cognitive domain to which the question belongs. Specifically, questions involving orientation, memory, or calculation ability are sent to the cognitive reasoning agent; language ability-related questions (such as naming, repetition, and word fluency) are sent to the speech and semantic agent; and visual drawing or executive function questions (such as clock drawing tests and graphic copying) are handled by the visual analysis agent. The visual analysis agent uses the Gemini large model to perform structured understanding of the drawn images, analyzing the completeness of elements, spatial relationships, and expressive intent, and outputs a score after comparing it with the standard answer. After each agent completes the scoring in parallel, it returns the scoring results and confidence levels to the scoring and scheduling service module. This module writes the score for the current question and the user's answer into the data layer, and simultaneously synchronizes the results to the doctor agent. Based on this, the doctor's AI agent dynamically adjusts the difficulty or prompting strategy of subsequent questions. For example, if a user answers incorrectly in a certain area, the system can appropriately simplify the description of similar questions or proactively provide auxiliary guidance to maintain user engagement and reduce frustration.
[0030] Once all preset questions have been answered, the question-answering service module sends an "assessment complete" event to the doctor's AI agent. At this point, the doctor's AI agent aggregates all scoring results from the cognitive reasoning AI agent, the speech and semantic AI agent, and the visual analysis AI agent, and performs comprehensive weighting and normalization processing based on user background information (age, years of education, past cognitive status, etc.) to generate a personalized cognitive assessment report. This report typically includes: a total screening score based on the MMSE / MoCA scale, score distribution charts for each cognitive domain (memory function, executive function, language ability, visuospatial ability, etc.), a cognitive impairment risk assessment level (low, medium, high risk), and clear behavioral guidelines, such as: "It is recommended to visit a neurologist for further consultation within 3 months" or "Regular follow-up observation." The results feedback module pushes this report to the results display module in the interaction layer via the API gateway, presenting it to the user in a graphical and textual format, and supporting download as a PDF document. Simultaneously, the history service module adds key indicators of this assessment (time, total score, domain score, report summary) to the history table in the data layer, facilitating users to track changes in their cognitive function over time periods.
[0031] After the report is delivered, the system enters the session termination phase. The doctor's AI agent proactively closes the connection with the AI dialogue engine module, releasing the computing resources occupied by the multiple agents. The API gateway records the user's logout behavior and updates the session state to the data layer. If the system is configured with a follow-up reminder function, it can automatically generate a reminder plan for the next assessment based on the risk level of this assessment (e.g., every 3 months for high-risk users, and every 6 months for medium-risk users). In addition, the system has a comprehensive exception handling and retry mechanism: when voice transcription fails, the interaction layer will prompt the user to manually input or re-record; if any agent does not respond for more than 10 seconds in scoring, the scoring scheduling service module will attempt to retry once, and if it still fails, the question will be marked as "awaiting manual review"; in case of unexpected situations such as network interruption, the front end will locally cache the user's current answer progress and automatically resume transmission after the connection is restored, ensuring the integrity of the assessment data and the continuity of the user experience.
[0032] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0033] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A multimodal large model-based assessment system for early cognitive impairment in Alzheimer's disease, characterized in that, include: The interaction layer includes a question display module, a question answering interaction module, a result display module, and a history record module. It is used to interact with users in a multimodal manner and to display the question-generating process, the question answering process, the result output, and the history record. The multimodal information interaction includes text interaction, voice interaction, and image interaction. The API gateway connects to each module in the interaction layer to uniformly manage and route requests from the interaction layer, and to perform authentication and access control. The core service layer includes a test question service module, a question answer service module, a scoring and scheduling service module, a speech synthesis service module, a speech transcription service module, an AI dialogue engine module, a result feedback module, and a history record service module. Each module is connected to the API gateway and is used to execute corresponding business processes and task scheduling based on the transmitted data. The AI capability layer is connected to the modules in the core service layer. It is used to perform the evaluation process using the preset multi-agent and Spark voice big model, and generate personalized cognitive evaluation reports based on the scoring results of each agent. The preset multi-agent system includes a doctor agent, a cognitive reasoning agent, a speech and semantic agent, and a visual analysis agent. The data layer, connected to the core service layer and the AI capability layer, is used for persistent storage of question bank data, answer process data, answer result data, and file data. The evaluation process is as follows: The doctor's intelligent agent interacts with the user in multiple rounds through the AI dialogue engine module, and records the user's answers to each screening question during the interaction. The scoring and scheduling service module distributes user answers to the cognitive reasoning agent, the speech and semantic agent, or the visual analysis agent for scoring based on the question type. The scoring results output by each intelligent agent are aggregated by the scoring scheduling service module and then transmitted to the doctor intelligent agent; The doctor's intelligent agent, combined with the user's background information, generates a personalized cognitive assessment report, which is then presented to the user through the results feedback module.
2. The Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model according to claim 1, characterized in that, The test question service module constructs a dynamic question bank based on a clinical MMSE scale and a MoCA scale, and the dynamic question bank is used to generate 4.2x10 19 different questionnaires.
3. The Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model according to claim 1, characterized in that, The doctor's intelligent agent guides the user to complete the screening through a multi-turn dialogue strategy, and dynamically calls the speech synthesis service module to read the questions aloud throughout the dialogue. At the same time, it supports the user to input voice through the speech transcription service module or text through the question-answering interaction module.
4. The Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model according to claim 1, characterized in that, The context processing logic of the scoring and scheduling service module is as follows: When the question type is orientation, memory, or calculation, the user's answer is distributed to the cognitive reasoning agent; when the question type is language ability related, the user's answer is distributed to the speech and semantic agent; when the question type is visual drawing or executive function, the user's answer is distributed to the visual analysis agent.
5. The Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model according to claim 1, characterized in that, The visual analysis agent performs structured understanding of the user-uploaded drawing images based on the Gemini large model, analyzes the elements, spatial structure and expressive intent in the images, and outputs the drawing question score after comparing it with the standard answer. The drawing question score is returned to the doctor agent through the scoring and scheduling service module.
6. The Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model according to claim 1, characterized in that, The history record module calls the user's historical answer records and evaluation results stored in the data layer through the history record service module, supporting users to track the trend of cognitive function changes across time periods.
7. The Alzheimer's disease early cognitive impairment assessment system based on a multimodal large model according to claim 1, characterized in that, The personalized cognitive assessment report generated by the doctor's AI agent includes the total screening score, the score distribution of each cognitive domain, the risk assessment level, and clear guidance on whether to recommend further medical examination.