Cognitive impairment patient multi-modal accompanying system based on ai technology

Abstract

This invention relates to the field of cognitive health companionship technology, specifically to a multimodal companionship system for patients with cognitive impairment based on AI technology. The invention receives memory materials from a family member's terminal, constructs an interactive virtual scene, configures guiding questions, records the patient's answers to all guiding questions, sets the initial difficulty level of cognitive training based on an assessed memory recall integrity score, and collects performance data after the patient completes cognitive training tasks to determine if the initial difficulty level needs adjustment. This achieves precise matching of cognitive training difficulty to the patient's current cognitive level. Simultaneously, based on the performance data of cognitive training tasks within a subsequently set training period, a cognitive ability report is generated and compared with historical cognitive ability reports in multiple dimensions to assess the patient's cognitive ability improvement. This reflects the dynamic changes and improvement trajectory of the patient's cognitive ability, effectively improving the targeting and effectiveness of cognitive training.

Description

Technical Field

[0001] This invention relates to the field of cognitive health companionship technology, and specifically to a multimodal companionship system for patients with cognitive impairment based on AI technology. Background Technology

[0002] Cognitive impairment is a group of diseases characterized by impaired cognitive functions such as memory, thinking, and attention. Patients need continuous cognitive stimulation and emotional support to slow functional decline and maintain basic living abilities. With the increasing aging of the population, the patient group with cognitive impairment is constantly expanding, and the demand for professional and personalized companionship and cognitive training services is becoming increasingly urgent. However, traditional companionship and training models are no longer able to meet the individualized needs and dynamically changing cognitive states of patients.

[0003] The existing technology has many shortcomings, as follows: (1) The existing technology uses generalized cognitive training scenarios and content, without combining the patient's personalized memory materials to construct training scenarios. This results in a lack of targeted training scenarios, which makes it impossible to effectively awaken the patient's memory reserves, reduce the patient's sense of identity and enthusiasm for cognitive training, and thus affect the overall effect of cognitive training, making it difficult to achieve the expected goal of delaying cognitive decline.

[0004] (2) Existing technologies mostly set fixed levels or static settings based on experience for the difficulty of cognitive training. They do not dynamically adjust based on the patient's real-time memory evoked status and training performance data. There is a problem that the difficulty does not match the patient's cognitive level. If the training difficulty is too high, it is easy to damage the patient's self-confidence and cause the patient to resist training. If the difficulty is too low, it cannot effectively stimulate the patient's cognitive function and cannot achieve cognitive ability improvement.

[0005] (3) Existing technologies for assessing patients’ cognitive abilities are mostly single static assessments, focusing only on the immediate results of a single training session. They lack analysis of the patient’s recent cognitive state trends, resulting in incomplete and unobjective assessments. This makes it difficult for medical staff and family members to accurately grasp the dynamic changes and improvement trajectory of the patient’s cognitive abilities, and to formulate precise and appropriate follow-up training and rehabilitation plans, thus affecting the scientificity and effectiveness of the rehabilitation plan. Summary of the Invention

[0006] To address the problems of lack of personalization in cognitive training scenarios, inability to dynamically adapt training difficulty, and insufficient comprehensiveness in cognitive ability assessment in existing technologies, this invention provides a multimodal companionship system for patients with cognitive impairment based on AI technology. This system enables personalized scenario construction, dynamic difficulty adjustment, and multidimensional cognitive assessment, thereby improving the relevance and effectiveness of cognitive training.

[0007] The technical solution adopted by the present invention to solve its technical problem is: a multimodal companion system for patients with cognitive impairment based on AI technology, including an interactive virtual scene generation module, a difficulty level determination module, a difficulty level adjustment judgment module, a cognitive ability report generation module, and a cognitive ability assessment module.

[0008] The connections between the modules are as follows: the interactive virtual scene generation module communicates with the difficulty level determination module; the difficulty level adjustment judgment module communicates with both the difficulty level determination module and the cognitive ability report generation module; and the cognitive ability assessment module communicates with the cognitive ability report generation module.

[0009] The interactive virtual scene generation module receives memory materials from family members' terminals, generates interactive virtual scenes based on the memory materials, and configures guiding questions.

[0010] The difficulty level determination module records the patient's answers to all guiding questions, assesses the patient's memory arousal integrity score, and sets the initial difficulty level of cognitive training based on the memory arousal integrity score.

[0011] The difficulty level adjustment judgment module presents patients with cognitive training tasks of the corresponding level based on the initial difficulty level, collects performance data of patients after completing the cognitive training tasks in real time, and determines whether the initial difficulty level needs to be adjusted based on the performance data.

[0012] The cognitive ability report generation module dynamically adjusts the difficulty level of subsequent cognitive training tasks when the initial difficulty level needs to be adjusted, statistically analyzes the performance data of cognitive training tasks within the subsequent set training period, and generates a cognitive ability report for the subsequent set training period.

[0013] The cognitive ability assessment module is used to compare the cognitive ability report with the patient's historical cognitive ability reports over a recent set period in multiple dimensions to assess the degree of improvement in the patient's cognitive ability.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) By receiving memory materials from family members' terminals, constructing interactive virtual scenes based on the memory materials and configuring guiding questions, the present invention can accurately awaken the patient's exclusive memories, strengthen the emotional connection between the patient and the training scene, improve the patient's acceptance of cognitive training and participation, and provide a memory basis for the effective development of cognitive training.

[0015] (2) The present invention sets the initial difficulty level of cognitive training based on the memory waking integrity score, and collects the performance data of patients after completing the cognitive training task to determine whether the initial difficulty level needs to be adjusted, so as to achieve accurate matching between the difficulty of cognitive training and the patient's current cognitive level, avoid the problem of poor training effect caused by fixed difficulty, ensure that the training is challenging but does not exceed the patient's tolerance, and effectively improve the pertinence and effectiveness of cognitive training.

[0016] (3) Based on the performance data of cognitive training tasks within the subsequent set training period, the present invention generates a cognitive ability report and compares it with the patient's historical cognitive ability reports in the recent set period in multiple dimensions to assess the degree of improvement of the patient's cognitive ability, thereby comprehensively reflecting the dynamic changes and improvement trajectory of the patient's cognitive ability, providing accurate data support for medical staff and family members to formulate personalized rehabilitation plans, solving the deficiency of traditional static assessment that cannot reflect long-term changes, and helping to scientifically optimize rehabilitation plans. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the system module connections of the present invention.

[0019] Figure 2 This is a schematic diagram illustrating the specific content and flow of the difficulty level adjustment and determination module in this invention.

[0020] Figure 3 This is a schematic diagram of the steps in this invention to generate a cognitive ability report for a subsequent training period. Detailed Implementation

[0021] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention. Furthermore, it should be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale.

[0022] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification.

[0023] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0024] Please see Figure 1 As shown, the present invention provides a multimodal companionship system for patients with cognitive impairment based on AI technology, including an interactive virtual scene generation module, a difficulty level determination module, a difficulty level adjustment judgment module, a cognitive ability report generation module, and a cognitive ability assessment module.

[0025] The connections between the modules are as follows: the interactive virtual scene generation module communicates with the difficulty level determination module; the difficulty level adjustment judgment module communicates with both the difficulty level determination module and the cognitive ability report generation module; and the cognitive ability assessment module communicates with the cognitive ability report generation module.

[0026] The interactive virtual scene generation module receives memory materials from family members' terminals, generates interactive virtual scenes based on the memory materials, and configures guiding questions.

[0027] In one embodiment of the present invention, the memory materials uploaded by the family member terminal are personalized materials related to the patient, including but not limited to images, videos, voice and text. The memory materials are related to the patient's life experience, important events, and interactions with relatives, avoiding irrelevant and redundant information.

[0028] Specifically, the interactive virtual scene generation module performs the following steps: First, it standardizes the format of the memory materials uploaded by family members' terminals, extracts the core characters, scenes, and event memory elements from the memory materials, and establishes the temporal relationship of the memory elements.

[0029] The standardization of formats resolves compatibility issues among different types and specifications of materials, providing a stable and unified data foundation for subsequent AI 3D modeling and preventing modeling failures or scene distortions due to format confusion. The establishment of the temporal relationship between memory elements is centered on time cues. Combining the timestamps of the footage shooting and time statements mentioned in audio or text, a timeline modeling method is used to associate core characters, scenes, and event elements in chronological order, making the virtual scene conform to the development of events and providing patients with natural recall cues.

[0030] Then, based on the core characters, scenes, and event memory elements and their temporal relationships, an interactive virtual scene is constructed using AI 3D modeling technology, and guiding questions corresponding to the memory elements are set in the interactive virtual scene.

[0031] Finally, the constructed interactive virtual scene and guiding questions are pushed to the family member's terminal. Based on the correction instructions from the family member's terminal, the interactive virtual scene and guiding questions are optimized, and the final interactive virtual scene and guiding questions are output.

[0032] It should be noted that the core individuals are identified by using object detection algorithms to identify people in images or videos, and combined with the identity information marked by family members, those whose identity information is that of relatives are identified as core individuals.

[0033] The scenes are divided into family scenes, outdoor scenes, and special event scenes, among which special event scenes include weddings, birthday parties, etc. Key objects in the scene are extracted simultaneously, such as furniture, green plants, and commemorative items.

[0034] Events can be extracted from speech-to-text and text materials, including the time, location, and actions involved, such as a family trip or a birthday celebration.

[0035] Prior to this, in specific embodiments of the present invention, AI 3D modeling technology is already an existing technology and will not be described in detail. For example, the interactive virtual scene is a scene where relatives celebrate a patient's birthday. The guiding questions are designed in a gradient, from basic recognition to detailed mining and then to logical association, gradually awakening the patient's shallow and deep memories, avoiding the blindness of memory awakening, and are divided into three levels of gradient.

[0036] The first level focuses on identifying and confirming key figures and scene memory elements, such as "Who is sitting opposite you in the photo?" or "Is this scene your old family courtyard?"

[0037] The second level involves guiding patients to recall details of the event's memory elements, such as "What gifts did you receive at your birthday party?" or "Who sang you a birthday song at your birthday party?"

[0038] The third level: such as "Which child came back first before the birthday party?" "What did you do at the birthday party?", strengthens the logical coherence of chronological memory elements.

[0039] This invention receives memory materials from family members' terminals, constructs interactive virtual scenes based on these materials, and configures guiding questions. This allows for the precise recall of patients' unique memories, strengthens the emotional connection between patients and the training scenario, and enhances patients' acceptance and participation in cognitive training, thus providing a memory foundation for the effective implementation of cognitive training.

[0040] The difficulty level determination module records the patient's answers to all guiding questions, assesses the patient's memory arousal integrity score, and sets the initial difficulty level of cognitive training based on the memory arousal integrity score.

[0041] In one embodiment of the present invention, the method for setting the initial difficulty level of cognitive training is as follows: S1. When the patient enters the interactive virtual scene, the patient's answers to all guiding questions are recorded in real time, and the answers to all guiding questions are semantically recognized and matched with keywords to obtain each keyword in the answer content and the corresponding keyword logical sequence.

[0042] Specifically, the semantic recognition and keyword matching technologies mentioned are existing technologies, and will not be described in detail in this invention.

[0043] S2. Compare each keyword and its corresponding logical sequence in the answers to each guiding question with all standard keywords and their corresponding standard logical sequences in the corresponding standard content to assess the patient's memory arousal integrity score.

[0044] S3. Compare the patient's memory arousal integrity score with the preset scoring threshold. If the patient's memory arousal integrity score is greater than or equal to the preset scoring threshold, set the initial difficulty level of cognitive training to normal difficulty; otherwise, set the initial difficulty level of cognitive training to easy difficulty.

[0045] It should be noted that the determination of the preset scoring threshold should refer to clinical data. A large number of memory arousal integrity scores from patients with varying degrees of cognitive impairment should be collected. For example, a preset scoring threshold of 70% of the average score for mild patients should be used. This threshold ensures that mild patients are likely to enter normal difficulty training, while moderate and severe patients enter easy difficulty training. Implementers can adjust the preset scoring threshold according to the application scenario.

[0046] Preferably, the evaluation method for the patient's memory recall integrity score is as follows: S21, compare each keyword in the answer to each guiding question with all standard keywords in the corresponding standard content, filter the number of identical keywords, and use the ratio of the number of identical keywords to the number of standard keywords in the standard content as the memory accuracy rate.

[0047] S22. Based on the order of all identical keywords in the answer content of each guiding question in the corresponding keyword logical sequence, determine the positional relationship vector between each identical keyword, perform positional analysis on it and the standard positional relationship vector between each identical keyword in the standard logical sequence of the corresponding keyword, obtain the positional relationship vector similarity, and use it as the memory coherence rate.

[0048] In one specific embodiment of the present invention, the keywords need to be related to the memory elements corresponding to the guiding question. For example, if the guiding question is "What did you do at your birthday party?" and the answer is "I cut the cake first, then blew out the candles, and finally received gifts from my family", then the keywords are "cut the cake", "blow out the candles", "family" and "gifts", and the logical sequence of the keywords is "cut the cake → blow out the candles → family → gifts".

[0049] The standard content is "sing the birthday song first, then blow out the candles, then cut the cake, and finally give gifts to family members." Therefore, all the standard keywords are "birthday song," "blowing out candles," "cutting the cake," "family members," and "gifts." The standard logical sequence of keywords is: birthday song → blowing out candles → cutting the cake → family members → gifts.

[0050] The positional relationship vector between each identical keyword is (cutting cake, blowing out candles, family, gift), and the standard positional relationship vector between each identical keyword is (blowing out candles, cutting cake, family, gift). Therefore, the number of identical keywords between the positional relationship vector and the standard positional relationship vector is 2, and the similarity of the positional relationship vectors is [value missing]. .

[0051] S23. Integrate the memory accuracy and memory coherence of each guiding question, and use the fusion result as the patient's memory evoked integrity score.

[0052] As an example, a weighted average method is used to combine memory accuracy and memory coherence. The formula for this combination is: .

[0053] In the formula Assess the patient's memory recall integrity score. In a preferred embodiment of the invention, the overall score for memory recall integrity is, for example... , These represent the accuracy and coherence of recalling the i-th guiding question, respectively. , The total number of guiding questions. and These are the weights for memory accuracy and memory coherence, respectively. In a preferred embodiment of the invention, for example... and All values ​​are set to 0.5, which is suitable for most patients with cognitive impairment; for patients with mild cognitive impairment, the value can be... Adjusting it to 0.6 emphasizes assessing the coherence of memory logic; for patients with severe cognitive impairment, it can be... The value was adjusted to 0.6 to focus on assessing the accuracy of memory of core information. Implementers can flexibly adjust this value according to the characteristics of the patient group.

[0054] This invention compares the patient's memory recall integrity score with a preset scoring threshold to automatically determine the initial difficulty level, ensuring that the difficulty matches the patient's current memory recall level. This avoids the problem of excessive difficulty damaging the patient's confidence or insufficient difficulty failing to provide effective stimulation, and provides initial data support for subsequent dynamic adjustment of difficulty, ensuring the continuity of cognitive training.

[0055] The difficulty level adjustment judgment module presents patients with cognitive training tasks of the corresponding level based on the initial difficulty level, collects performance data of patients after completing the cognitive training tasks in real time, and determines whether the initial difficulty level needs to be adjusted based on the performance data.

[0056] In one embodiment of the present invention, the cognitive training task library is stored in categories according to difficulty levels. Each difficulty level includes, but is not limited to, easy difficulty, normal difficulty, intermediate difficulty and advanced difficulty. Each difficulty level includes cognitive training tasks such as memory, attention, visuospatial ability, language ability, executive ability and sensorimotor function.

[0057] For example, memory and cognitive training tasks can improve patients' short-term and long-term memory abilities through games such as memory gardens and N-backs.

[0058] Attention and cognitive training tasks improve patients' focus and reaction speed through games such as Schulte Grid and whack-a-mole.

[0059] Visual-spatial cognitive training tasks use games such as jigsaw puzzles, spatial rotation, and cube projection to train patients' visual-spatial perception and spatial imagination abilities.

[0060] Language ability cognitive training tasks improve patients' language comprehension and expression abilities through games such as word association and story relay.

[0061] The cognitive training task for executive function uses games such as Spider Solitaire to train patients' planning, organization, and decision-making abilities.

[0062] Cognitive training tasks for sensory-motor function: Through games such as hand-eye coordination and motion-sensory fruit cutting, patients' sensory-motor coordination ability is improved.

[0063] Preferably, such as Figure 2 As shown, the specific content of the difficulty level adjustment and determination module is as follows: First, based on the initial difficulty level, select cognitive training tasks of the corresponding difficulty level from the cognitive training task library and present the cognitive training tasks to the patient terminal.

[0064] Then, the performance data of patients after completing various cognitive training tasks are collected in real time. The performance data includes task accuracy, average reaction time, task completion rate and task completion time.

[0065] Finally, the performance data of patients after completing various cognitive training tasks are matched with the corresponding preset difficulty level adjustment criteria. If the performance data of patients after completing a certain cognitive training task meets the preset difficulty level adjustment criteria, the initial difficulty level is determined to need to be adjusted; otherwise, the initial difficulty level is determined not to need to be adjusted.

[0066] As an example, task accuracy is the ratio of the number of sub-items that a patient correctly responded to in completing a task to the total number of sub-items.

[0067] Average reaction time: Record the time from task presentation to patient response for each sub-item, and take the average reaction time of all sub-items.

[0068] Task completion rate: Statistics on the percentage of sub-items actually completed.

[0069] Task completion time: The total time from when the patient clicks to start the cognitive training task to when the cognitive training task is submitted.

[0070] The preset difficulty level adjustment criteria adopt the principle of simultaneous satisfaction of performance data to avoid misjudgment caused by fluctuations in a single performance data point and ensure the rigor of adjustment decisions. In a preferred embodiment of the present invention, the preset difficulty level adjustment criteria are as follows: when the task accuracy rate is greater than the reference task accuracy rate, the average reaction time is less than the reference reaction time, the task completion rate is greater than the reference task completion rate, and the task completion time is less than the reference task completion time, then the difficulty level is determined to need adjustment.

[0071] It should be noted that the reference task accuracy, reference reaction time, reference task completion rate, and reference task completion time are obtained based on historical data from the cognitive training database. The specific steps are illustrated in the following example: First, retrieve the historical performance data of each cognitive training task at each difficulty level from the cognitive training database, and obtain the average historical performance data of each cognitive training task at each difficulty level.

[0072] The second step is to select historical cognitive training with a historical task accuracy rate greater than the average historical task accuracy rate and use them as labeled historical cognitive training. The historical task accuracy rate of each labeled historical cognitive training is statistically analyzed. Based on the distribution characteristics of the historical task accuracy rate, the reference task accuracy rate of each cognitive training task in each difficulty level is determined.

[0073] In one specific embodiment of the present invention, the median is used as the reference task accuracy rate based on the distribution characteristics of historical task accuracy rates. In other embodiments, the implementer may use the distribution characteristics instead of the median; if the distribution characteristics of historical task accuracy rates follow a perfect normal distribution, the mean may be used as the reference task accuracy rate.

[0074] The third step is to similarly filter historical cognitive training data that have historical average reaction times less than the average historical average reaction time, historical task completion rates greater than the average historical task completion rates, and historical task completion times less than the average historical task completion times, and determine the reference reaction time, reference task completion rate, and reference task completion time for each cognitive training task in each difficulty level.

[0075] This invention sets the initial difficulty level of cognitive training based on the memory recall integrity score and collects the patient's performance data after completing the cognitive training task to determine whether the initial difficulty level needs to be adjusted. This achieves a precise match between the difficulty of cognitive training and the patient's current cognitive level, avoiding the problem of poor training effect caused by fixed difficulty. It ensures that the training is challenging but does not exceed the patient's tolerance, effectively improving the pertinence and effectiveness of cognitive training.

[0076] The cognitive ability report generation module dynamically adjusts the difficulty level of subsequent cognitive training tasks when the initial difficulty level needs to be adjusted, statistically analyzes the performance data of cognitive training tasks within the subsequent set training period, and generates a cognitive ability report for the subsequent set training period.

[0077] In a specific embodiment of the present invention, such as Figure 3 As shown, the process of generating a cognitive ability report for the subsequent training period is as follows: First, obtain the highest difficulty level of each cognitive training task within the subsequent training period, and extract the performance data of each cognitive training task at the highest difficulty level for each cognitive training session.

[0078] The second step is to filter out the maximum and minimum performance data of each cognitive training session in each cognitive training task, and process them to obtain the normalized performance data corresponding to each cognitive training session.

[0079] It should be noted that the normalization method described above is the min-max normalization technique, which is an existing technology and will not be elaborated further.

[0080] The third step involves combining the influence weights of all performance data corresponding to each cognitive training task with the set weighted average to calculate the current cognitive ability index for each cognitive training task.

[0081] It should be noted that the influence weights of all performance data for each cognitive training task are set based on the correlation between the assessment points of the cognitive training task and the performance data. The sum of the influence weights of all performance data is 1, ensuring that the assessment focuses on the core cognitive ability of the task, while taking into account both data science and practicality.

[0082] For example, memory training assesses the ability to store, retain, and accurately retrieve information, emphasizing the accuracy and completeness of memory. Task accuracy directly reflects the precision of information retrieval and is strongly correlated with the core assessment points; task completion reflects the completeness of memory coverage and is also strongly correlated with the core assessment points; average reaction time only reflects the speed of memory retrieval and has a relatively small impact on the core assessment of memory ability; task completion time is affected by retrieval speed and operational proficiency and has a weaker correlation with core memory ability. Therefore, the weights for task accuracy, task completion, average reaction time, and task completion time are 0.15 and 0.35 respectively.

[0083] In other examples, implementers may also adjust the impact weights of all performance data themselves.

[0084] The fourth step is to combine the current highest difficulty level of each cognitive training task to generate a cognitive ability report for the subsequent set training period.

[0085] This invention generates cognitive ability reports for subsequent training periods, which can clearly show the shortcomings of cognitive ability indicators for each cognitive training task. This provides accurate data support for medical staff to formulate targeted rehabilitation plans and for family members to adjust their companionship training strategies, effectively improving the practical value and rehabilitation effect of cognitive training services.

[0086] The cognitive ability assessment module is used to compare the cognitive ability report with the patient's historical cognitive ability reports over a recent set period in multiple dimensions to assess the degree of improvement in the patient's cognitive ability.

[0087] In one embodiment of the present invention, the recent set time period is set to 3 months based on the clinical rehabilitation assessment cycle. The implementer can adjust it to 2-6 months according to the patient's rehabilitation progress to ensure that enough training cycles are covered to reflect trend changes.

[0088] Specifically, the assessment method for the patient's cognitive ability improvement is as follows: First, retrieve the patient's historical cognitive ability reports for each recent set period from the cognitive training database.

[0089] Secondly, if the current highest difficulty level of a certain cognitive training task is greater than the highest difficulty level of the corresponding cognitive training task in each historical cognitive ability report, then the improvement in ability for that cognitive training task is recorded as the set value.

[0090] It should be noted that, based on the quantitative standard for clinical cognitive rehabilitation effects, the set value is uniformly 0.2, which represents a significant improvement in cognitive ability for the task. Implementers can make minor adjustments according to the type of task, with an adjustment range of 0.15-0.25, to ensure that the improvement is quantified uniformly.

[0091] Conversely, select historical cognitive ability reports that are at the same highest difficulty level as the current cognitive training task, compare their trends with the current cognitive ability indicators of the cognitive training task, and calculate the improvement in ability for the cognitive training task.

[0092] Finally, the improvement rate of each cognitive training task was statistically analyzed, and the average value was used as the degree of improvement in the patient's cognitive ability.

[0093] As an example, the method for calculating the improvement of the cognitive training task is as follows: based on the cognitive ability indicators of the cognitive training task and the current cognitive ability indicators in the selected historical cognitive ability reports, a time series set of historical cognitive ability indicators is constructed.

[0094] By comparing the cognitive ability indicators of adjacent dates in the historical cognitive ability indicator time series set, the improvement rate of the cognitive ability indicator of each adjacent date is obtained, and the improvement of the cognitive training task is obtained by averaging the results.

[0095] The improvement rate of cognitive ability indicators is obtained by the ratio of the difference in cognitive ability indicators between adjacent dates to the cognitive ability indicators of the previous adjacent date.

[0096] This invention generates a cognitive ability report based on the performance data of cognitive training tasks within a subsequently set training period, and compares it with the patient's historical cognitive ability reports within a recent set period in multiple dimensions to assess the degree of improvement in the patient's cognitive ability. This comprehensively reflects the dynamic changes and improvement trajectory of the patient's cognitive ability, providing accurate data support for medical staff and families to develop personalized rehabilitation plans. It solves the shortcomings of traditional static assessments that cannot reflect long-term changes and helps to scientifically optimize rehabilitation plans.

[0097] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0098] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0100] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0101] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal companionship system for patients with cognitive impairment based on AI technology, characterized in that: include: The interactive virtual scene generation module receives memory materials from family members' terminals, generates interactive virtual scenes based on the memory materials, and configures guiding questions. The difficulty level determination module records the patient's answers to all guiding questions, assesses the patient's memory arousal integrity score, and sets the initial difficulty level of cognitive training based on the memory arousal integrity score. The difficulty level adjustment judgment module presents the corresponding level of cognitive training tasks to the patient based on the initial difficulty level, collects the patient's performance data after completing the cognitive training tasks in real time, and determines whether the initial difficulty level needs to be adjusted based on the performance data. The cognitive ability report generation module dynamically adjusts the difficulty level of subsequent cognitive training tasks when the initial difficulty level needs to be adjusted, statistically analyzes the performance data of cognitive training tasks within the subsequent set training period, and generates a cognitive ability report for the subsequent set training period. The cognitive ability assessment module is used to compare the cognitive ability report with the patient's historical cognitive ability reports over a recent set period in multiple dimensions to assess the degree of improvement in the patient's cognitive ability.

2. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 1, characterized in that: The specific content of the interactive virtual scene generation module is as follows: The memory materials uploaded by family members are standardized in format, the core characters, scenes and event memory elements in the memory materials are extracted, and the temporal relationship of the memory elements is established. Based on the core characters, scenes, and event memory elements and the temporal relationship of memory elements, an interactive virtual scene is constructed using AI 3D modeling technology, and guiding questions corresponding to memory elements are set in the interactive virtual scene; The constructed interactive virtual scene and guiding questions are pushed to the family member's terminal. Based on the correction instructions from the family member's terminal, the interactive virtual scene and guiding questions are optimized, and the final interactive virtual scene and guiding questions are output.

3. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 1, characterized in that: The method for setting the initial difficulty level of cognitive training is as follows: When a patient enters an interactive virtual scene, the system records the patient's answers to all guided questions in real time. The system performs semantic recognition and keyword matching on the answers to all guided questions to obtain each keyword in the answer and the corresponding keyword logical sequence. The patient's memory arousal integrity score was assessed by comparing each keyword and its corresponding logical sequence in the answers to each guiding question with all standard keywords and their corresponding standard logical sequences in the corresponding standard content. The patient's memory arousal integrity score is compared with a preset scoring threshold. If the patient's memory arousal integrity score is greater than or equal to the preset scoring threshold, the initial difficulty level of cognitive training is set to normal difficulty; otherwise, the initial difficulty level of cognitive training is set to easy difficulty.

4. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 3, characterized in that: The assessment method for the patient's memory arousal integrity score is as follows: The keywords in the answers to each guiding question are compared with all the standard keywords in the corresponding standard content. The number of identical keywords is selected, and the ratio of the number of identical keywords to the number of standard keywords in the standard content is used as the memory accuracy rate. Based on the order of all identical keywords in the corresponding keyword logical sequence in the answers to each guiding question, the positional relationship vector between each identical keyword is determined. This vector is then compared with the standard positional relationship vector between each identical keyword in the standard logical sequence of the corresponding keyword to obtain the positional relationship vector similarity, which is used as the memory coherence rate. The accuracy and coherence of memory for each guiding question were combined, and the combined result was used as the patient's memory recall integrity score.

5. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 1, characterized in that: The specific content of the difficulty level adjustment determination module is as follows: Based on the initial difficulty level, cognitive training tasks of the corresponding difficulty level are selected from the cognitive training task library and presented to the patient's terminal. Real-time collection of patients' performance data after completing various cognitive training tasks, including task accuracy, average reaction time, task completion rate, and task completion time; The performance data of patients after completing various cognitive training tasks is matched with the corresponding preset difficulty level adjustment criteria. If the performance data of patients after completing a certain cognitive training task meets the preset difficulty level adjustment criteria, it is determined that the initial difficulty level needs to be adjusted; otherwise, it is determined that the initial difficulty level does not need to be adjusted.

6. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 5, characterized in that: The preset difficulty level adjustment judgment condition is: If the task accuracy rate is greater than the reference task accuracy rate, the average reaction time is less than the reference reaction time, the task completion rate is greater than the reference task completion rate, and the task completion time is less than the reference task completion time, then the difficulty level needs to be adjusted.

7. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 6, characterized in that: The reference task accuracy, reference reaction time, reference task completion rate, and reference task completion time are obtained in the following ways: Retrieve historical performance data of each cognitive training task at each difficulty level from the cognitive training database, and obtain the average historical performance data of each cognitive training task at each difficulty level. Historical cognitive training with a historical task accuracy greater than the average historical task accuracy is selected and used as labeled historical cognitive training. The historical task accuracy of each labeled historical cognitive training is statistically analyzed. Based on the distribution characteristics of historical task accuracy, the reference task accuracy of each cognitive training task in each difficulty level is determined. Similarly, historical cognitive training with historical average reaction time less than the average historical average reaction time, historical task completion rate greater than the average historical task completion rate, and historical task completion time less than the average historical task completion time are selected to determine the reference reaction time, reference task completion rate, and reference task completion time for each cognitive training task in each difficulty level.

8. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 1, characterized in that: The cognitive ability report generated for subsequent training periods is as follows: Obtain the highest difficulty level of each cognitive training task within the subsequent set training period, and extract the performance data of each cognitive training task at the highest difficulty level for each cognitive training session. The maximum and minimum performance data of each cognitive training session in each cognitive training task were selected and processed to obtain the normalized performance data corresponding to each cognitive training session. By combining the influence weights of all performance data corresponding to each cognitive training task, the current cognitive ability index of each cognitive training task is calculated by linear weighted average. Based on the current highest difficulty level of each cognitive training task, a cognitive ability report is generated for the subsequent set training period.

9. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 8, characterized in that: The assessment method for the degree of improvement in the patient's cognitive ability is as follows: Retrieve the patient's historical cognitive ability reports from recent set time periods from the cognitive training database; If the current highest difficulty level of a certain cognitive training task is greater than the highest difficulty level of the corresponding cognitive training task in each historical cognitive ability report, then the improvement of the ability of that cognitive training task is recorded as the set value. Conversely, select historical cognitive ability reports that are at the same highest difficulty level as the current cognitive training task, compare their trends with the current cognitive ability indicators of the cognitive training task, and calculate the improvement in ability for the cognitive training task. The improvement rate of each cognitive training task was statistically analyzed, and the average value was used as the degree of improvement in the patient's cognitive ability.

10. The AI-based multimodal companionship system for patients with cognitive impairment according to claim 9, characterized in that: The method for calculating the improvement in ability for this cognitive training task is as follows: Based on the cognitive ability indicators and current cognitive ability indicators of the cognitive training task from each screened historical cognitive ability report, a time series set of historical cognitive ability indicators is constructed. By comparing the cognitive ability indicators of adjacent dates in the historical cognitive ability indicator time series set, the improvement rate of the cognitive ability indicator of each adjacent date is obtained, and the improvement of the cognitive training task is obtained by averaging the results.

Images