Ageing-suitable intelligent interaction system and method for cognitive impairment and semi-disabled old people

Abstract

The invention relates to the technical field of aged intelligent interaction, and discloses an aged intelligent interaction system and method for cognitive impairment and semi-disabled old people, a multi-modal information acquisition unit cooperatively works through voice, vision, touch, physiology, micro-action and environment perception units, and awakens and acquires multi-modal data as required; the adaptive processing unit performs multi-modal data verification, adaptive adjustment of an instruction guide mode and an intention recognition mode, adjusts an interaction mode according to a scene, and provides sound-visual-touch multi-mode feedback output; the cloud server interacts preset data with the adaptive processing unit, the remote collaboration unit synchronizes key data in the preset data to the remote terminal, triggers remote collaboration with the remote terminal and interface / scene sharing, and triggers remote alarm to the remote terminal and / or an emergency center when abnormity occurs. Potential risks are warned through the behavior baseline, and a remote terminal is triggered to limit care. According to the invention, an interaction system which is really adaptive to the physiological and psychological characteristics of the special group is constructed.

Description

Technical Field

[0001] This invention relates to the field of age-friendly intelligent interaction technology, specifically to age-friendly intelligent interaction systems and methods for elderly people with cognitive impairment and partial disability. Background Technology

[0002] With the increasing aging of the population, the demand for age-friendly smart devices is growing. Although existing age-friendly smart interactive systems have solved basic interaction problems, they still have significant shortcomings in meeting the specific needs of elderly people with cognitive impairment (memory decline, weak logical comprehension) and semi-disabled elderly people (limited physical activity). (1) Lack of cognitive adaptation: The existing system relies on fixed operation logic (such as the "instruction-response" pattern) and does not take into account the problem that elderly people with cognitive impairment are prone to forgetting operation steps and have difficulty understanding abstract instructions (such as being unable to understand abstract instructions such as "click the icon"). (2) Limitations of physical interaction: Existing multimodal interactions (such as gestures and touch) require a certain level of physical activity, which is difficult for semi-disabled elderly people (such as those with hand tremors or limited upper limb movement) to complete, resulting in a situation where "there are interactive functions but they cannot be used"; (3) Insufficient scene adaptability: In high-frequency activity scenarios for the elderly (such as bedroom at night, cooking in the kitchen, and washing up in the bathroom), the existing system is not optimized for environmental characteristics (such as low light at night, oil stains in the kitchen, and slippery bathroom), which poses a risk of interaction failure (such as accidental touch when the touch screen is oily, and failure of visual acquisition in low light). (4) Lack of proactive care: Most existing remote collaboration is "passive response to help requests" and does not proactively warn of potential needs based on the elderly's behavior habits (e.g., elderly people living alone may have health risks if they do not use the "go out" function for 3 consecutive days).

[0003] Therefore, there is an urgent need for an intelligent interactive system that is designed for specific elderly groups and adapted to high-frequency scenarios. Summary of the Invention

[0004] The present invention aims to provide an age-friendly intelligent interactive system and method for elderly people with cognitive impairment and partial disability, in order to solve the technical problem that existing age-friendly intelligent systems cannot meet the needs of special elderly groups.

[0005] The basic solution provided by this invention is: an age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability, comprising: It includes an electrically connected multimodal information acquisition unit, an adaptive processing unit, a cloud server, a remote collaboration unit, and a remote terminal; The multimodal information acquisition unit includes a voice acquisition subunit, a vision acquisition subunit, a touch acquisition subunit, a physiological information acquisition subunit, a micro-motion acquisition subunit, and an environmental perception subunit. Each subunit is activated according to a preset wake-up strategy to acquire multimodal data and send it synchronously to the adaptive processing unit. The multimodal data includes the raw data acquired by each subunit and the corresponding instructions or information obtained through parsing and processing. The adaptive processing unit includes a data fusion subunit for multimodal data verification, a personalized adaptation subunit for adapting and adjusting intent recognition mode and instruction guidance mode, a scene response subunit for adjusting interaction mode based on multimodal data representing scene information, and a feedback execution subunit for providing multi-mode feedback output based on different interactive inputs (audio, visual, and tactile). The cloud server is used to interact with the adaptive processing unit using preset data; and a scene database is set up to store the corresponding scene interaction strategies and user data according to different scenes. The remote collaboration unit includes an information synchronization subunit for synchronizing key data from preset data to a remote terminal, a one-click collaboration subunit for triggering remote collaboration with the remote terminal and sharing of interfaces / scenes, an emergency response subunit for triggering remote alarms to the remote terminal and / or emergency center in case of anomalies, and a behavior analysis subunit for warning of potential risks through behavioral baselines and triggering remote terminal-limited care.

[0006] This invention also provides an age-friendly intelligent interaction method for elderly people with cognitive impairment and partial disability, utilizing an age-friendly intelligent interaction system for elderly people with cognitive impairment and partial disability; the method includes: S1, the multimodal information acquisition unit collects the elderly’s interactive commands, physiological state and environmental information through its sub-units, and transmits the corresponding multimodal data to the adaptive processing unit in real time; S2, the data fusion subunit of the adaptive processing unit verifies the multimodal data; the personalized adaptation subunit adapts and adjusts the instruction guidance mode and intent recognition mode; the scene response subunit adjusts the interaction mode based on the multimodal data representing scene information; the feedback execution subunit outputs the operation results based on different interactive inputs through audio, visual and tactile methods. S3: The cloud server and the adaptive processing unit perform preset data interaction and store the corresponding scene interaction strategies and user data in the scene database according to the scene. S4, the information synchronization subunit of the remote collaboration unit pushes key data from the preset data to the remote terminal in real time; the one-click collaboration subunit triggers remote collaboration with the remote terminal and interface / scene sharing; the emergency response subunit triggers a remote alarm to the remote terminal and / or emergency center in case of an anomaly; the behavior analysis subunit warns of potential risks through behavior baselines and triggers remote terminal-limited care.

[0007] The working principle and advantages of this invention are as follows: This solution is specifically designed for elderly people with cognitive impairment and partial disability, achieving a significant breakthrough for this special population based on existing age-friendly technologies. Compared to general-purpose intelligent interactive systems, its advantage lies in using cognitive decline and limited physical function as core design constraints to construct a closed-loop interactive system truly adapted to the physiological and psychological characteristics of this group. Traditional systems are mostly designed for able-bodied users, relying on clear voice, accurate touch control, or proactive assistance. However, this solution, through micro-motion sensing technology, enables even partially disabled elderly people with only slight limb movements to effectively operate the device, solving the fundamental obstacle of wanting to use it but being unable to. At the same time, combined with step-by-step guidance and semantic simplification mechanisms from cognitive psychology, it significantly reduces the user's memory and comprehension burden, allowing elderly people with mild cognitive impairment to complete complex operations without the assistance of a guardian, avoiding anxiety and withdrawal caused by operational failures. In terms of safety protection, it breaks through the passive alarm mode, achieving early warning of abnormal activities (such as wandering around at night without reason, or prolonged stay in dangerous areas) through behavioral baseline modeling, proactively triggering remote limited care, transforming post-event rescue into pre-event intervention. Furthermore, through multimodal fusion and adaptive adjustment, the system effectively addresses real-world issues such as unclear pronunciation, slow response, and unstable operation among the elderly, significantly improving interaction robustness. The overall design not only overcomes the shortcomings of existing technologies that are unusable, undesirable, or unacceptable to specific elderly groups, but also, through interdisciplinary integration, promotes a substantial leap from functional implementation to human-centered adaptation in age-friendly intelligent systems.

[0008] The specific advantages are: 1) More comprehensive multimodal acquisition: Integrating voice, vision, touch, and physiological information to overcome the limitations of single input methods (such as improved dialect recognition accuracy).

[0009] 2) Smarter Adaptive Processing: Based on real-time data, the interaction parameters are dynamically adjusted to adapt to individual differences among the elderly (such as font, speech speed, and interface complexity can change with the state, reducing the error rate).

[0010] 3) Enhanced scene adaptability: Reduced accidental touch rate in kitchen scenes with oily environments; shortened command recognition response time in low light conditions at night; and triggered fall prevention warnings in bathroom scenes (existing systems rely on post-event alarms).

[0011] 4) More efficient remote collaboration / proactive care: Low information synchronization latency, one-click calling and interface sharing simplify the assistance process, and the emergency response time is shortened to within 30 seconds (the existing system averages 3 minutes); through behavioral baseline analysis, potential risks are warned in advance, reducing the number of times passive assistance is needed.

[0012] 5) Balancing safety and ease of use: Physical buttons and simplified operation adapt to the habits of elderly people with special needs, while physiological monitoring and emergency response enhance safety, making it particularly suitable for various special elderly groups such as those living alone, those with cognitive impairment, and those who are partially disabled. Through "simplified semantics" and "step-by-step guidance," the accuracy rate of operation is improved and the forgetting rate is reduced for elderly people with cognitive impairment; through "micro-motion interaction," the need for physical activity is reduced from "large-range movements" to "millimeter-level micro-movements," improving the effectiveness and simplicity of the operating system for semi-disabled elderly people.

[0013] 6) Multidisciplinary technology integration: Combining "cognitive psychology" (memory enhancement), "environmental engineering" (scene adaptation) with artificial intelligence interaction technology solves the industry pain point of "special elderly people being unable to use smart devices". Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the structure of the age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability provided in an embodiment of the present invention. Figure 1 Figure 2 This is a schematic diagram of the structure of the age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability provided in an embodiment of the present invention. Figure 2 . Detailed Implementation

[0015] The following detailed explanation illustrates the specific implementation methods: The basic implementation examples are as follows: Figure 1 and Figure 2 As shown: An age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability, including a multimodal information acquisition unit for data interaction, an adaptive processing unit, a cloud server, a remote collaboration unit, and a remote terminal.

[0016] The multimodal information acquisition unit includes a voice acquisition subunit, a vision acquisition subunit, a touch acquisition subunit, a physiological information acquisition subunit, a micro-motion acquisition subunit, and an environmental perception subunit. Each subunit is activated according to a preset wake-up strategy to acquire multimodal data and send it synchronously to the adaptive processing unit. The multimodal data includes the raw data acquired by each subunit and the corresponding instructions or information obtained through parsing and processing.

[0017] Specifically: The data collection devices are deployed in the elderly activity space according to a preset method, enabling them to collect relevant multimodal data from any location within the activity space / scene. Each unit and module uses machine learning or deep learning algorithms to analyze and predict elderly behavior based on the multimodal data collected over a long period of time, and performs corresponding interactions.

[0018] The voice acquisition subunit includes a microphone array (supporting long-distance sound pickup within 3 meters and with noise reduction function) and a dialect / accent recognition module (covering 8 mainstream dialects such as Northern dialect and Cantonese), which is used to acquire sound through the microphone array and parse it to obtain voice commands and emotional tone information (such as anxiety and confusion).

[0019] The visual acquisition subunit includes a low-power camera (supporting infrared night vision to capture elderly movements), a motion recognition module (such as recognizing gesture commands, "waving to wake up" and "pointing to select"), a physiological feature analysis module (such as judging vision status through eye tracking and judging emotions through facial expressions), and a micro-expression recognition module with an integrated infrared depth camera (used to collect head micro-movements and micro-expressions and parse them to obtain corresponding instructions or information, such as recognizing head micro-movements (such as nodding = confirming, shaking head = canceling) even in the absence of light at night, and judging operational confusion through micro-expressions (such as frowning).

[0020] The touch acquisition subunit includes a large-size (≥10 inches) anti-accidental touch display module with an integrated pressure-sensing layer (supports operation while wearing gloves); the pressure-sensing layer is used to distinguish between unintentional touches and valid operations.

[0021] The physiological information acquisition subunit includes a wristband sensor (to collect heart rate and blood oxygen) and a seat pressure sensor (to determine whether the person has been sitting for a long time or has fallen).

[0022] The micro-motion acquisition subunit is equipped with millimeter-wave radar (non-contact type). Based on the data acquired by the millimeter-wave radar, it identifies subtle limb movements (such as the trajectory of finger tremors or slight shoulder swaying) and converts them into corresponding instructions or information (such as the control instruction "sway shoulder to the right = turn page").

[0023] The environmental perception subunit deploys environmental perception sensors such as temperature and humidity sensors, oil stain sensors, and ground flatness sensors. Based on the data obtained by the environmental perception sensors, it identifies the scene type (e.g., kitchen: oil stains + high temperature; bathroom: high humidity + tiled floor). The acquired data can be used for environmental labeling in scene sharing.

[0024] The wake-up strategy can be based on data collected from any one sub-unit, which then wakes up other sub-units in sequence; it can also be based on the space / scene where the elderly person is located, which then wakes up the devices in that space / scene; or it can be based on long-term data collection to predict the elderly person's behavioral habits and then wake up the corresponding devices based on those habits.

[0025] The adaptive processing unit includes a data fusion subunit for multimodal data verification, a personalized adaptation subunit for adapting and adjusting intent recognition mode and instruction guidance mode, a scene response subunit for adjusting interaction mode based on multimodal data representing scene information, and a feedback execution subunit for providing multi-mode feedback output based on different interactive inputs (audio, visual, and tactile).

[0026] Specifically: It also includes a data preprocessing and feature extraction subunit. After the acquisition unit outputs the raw signal, this unit immediately performs preprocessing to provide "clean" feature data for subsequent calculations.

[0027] The data fusion subunit is used to verify the collected multimodal data (such as improving the credibility of the command to over 95% when the voice "turn on the TV" is synchronized with the gesture pointing to the TV), thereby improving the system's response accuracy.

[0028] The personalized adaptation subunit includes a cognitive adaptation module and a limb adaptation module. The cognitive adaptation module incorporates a memory reinforcement model to provide repeated feedback on key steps in different ways for elderly individuals with cognitive impairment, offering step-by-step guidance (e.g., voice + text step-by-step guidance such as "Press the red button first, then wait for the green light to turn on"). It also records frequently used operations to create one-click interactive strategies (e.g., automatically activating the "measure blood pressure" function upon waking). The limb adaptation module establishes a deviation-allowing calculation strategy between collected multimodal data and core intent recognition for semi-disabled elderly individuals, lowering the operational threshold from "precise movements" to "error-tolerant movements" (e.g., allowing trajectory deviations during finger tremors, with algorithms recognizing the core intent).

[0029] The personalized adaptation subunit also includes a learning optimization module. Through long-term data accumulation, it dynamically optimizes the calculation models of each module within the personalized adaptation subunit, reducing the interactive learning cost for elderly people with cognitive impairment. For example, it records the user's feedback on instructions (such as refusal to execute or repeating instructions) and updates the weight parameters of the decision model regularly (such as weekly) (if the elderly person has recently relied more on gestures than voice, then the weight of gesture instructions is increased), truly achieving dynamic adaptation.

[0030] The scene response subunit performs scene perception based on multimodal data representing scene information to determine the scene in which the elderly are located and adjust the interaction mode accordingly (e.g., in the kitchen scene, it automatically switches to "voice + infrared gesture" and disables touch control; in the bathroom scene, it triggers "fall prevention warning" and prompts "Do you need help?" every 30 seconds).

[0031] The feedback execution subunit analyzes and processes the output results from other subunits of the adaptive processing unit and multimodal data using a pre-integrated computational model. It then outputs corresponding feedback through multiple methods, including voice (with adjustable speech rate: normal / slow), subtitles (scalable font), lights (red / green indicators of operation results), and vibration (haptic feedback). It also includes a semantic simplification module to translate complex feedback instructions into colloquial expressions, such as "open the health code" becoming "retrieve the code used for medical treatment," enabling elderly users to correctly understand the system's guidance instructions.

[0032] The cloud server is used to interact with the adaptive processing unit to pre-set data and store it according to different scenarios. The cloud server has a scenario database, which stores the corresponding scenario interaction strategies and user data according to different scenarios. That is, the interaction strategies and user data of all scenarios or high-frequency scenarios such as kitchen and bathroom are ultimately stored in the scenario database, forming a complete closed loop of "collection-processing-feedback-collaboration-storage", ensuring that the system continuously adapts to the needs of the elderly.

[0033] The remote collaboration unit includes an information synchronization subunit for synchronizing key data from preset data to a remote terminal, a one-click collaboration subunit for triggering remote collaboration with the remote terminal and sharing of interfaces / scenes, an emergency response subunit for triggering remote alarms to the remote terminal and / or emergency center in case of anomalies, and a behavior analysis subunit for warning of potential risks through behavioral baselines and triggering remote terminal-limited care.

[0034] Specifically: The information synchronization subunit synchronizes key data (such as elderly operation records, system status (such as battery level, network status), and physiological data) to the remote terminals (mobile APP, tablet) of children / caregivers via a cloud server (latency ≤ 500ms).

[0035] The one-click collaboration subunit allows seniors to trigger a one-click sharing of their current user interface with a remote terminal. The remote terminal provides guidance and / or voice instructions on the shared interface, enabling remote collaboration. Seniors can initiate a video call by long-pressing the physical button on the touch-sensitive data acquisition subunit or by saying "find my children." Once connected, the system automatically shares the senior's user interface with the remote terminal. Children can annotate operation steps and provide voice guidance, allowing for remote collaboration. Simultaneously, the one-click collaboration subunit supports "scene sharing." It also receives scene sharing requests from remote terminals and shares the senior's current scene interface and environmental annotations (calculated by the feedback execution subunit) with the remote terminal. Children can see the senior's current scene interface and environmental annotations (e.g., "The kitchen countertop is oily; voice guidance is recommended," "Bathroom washing (real-time fall prevention monitoring + voice reminder 'Get up slowly')," "Kitchen cooking (voice control of heat in oily environments)") on the remote terminal, enabling remote collaboration based on the shared information.

[0036] The emergency response subunit automatically sends an alarm (including location) to a preset contact and dials an emergency number when abnormal physiological information is collected (such as heart rate > 120 beats / min for 5 minutes) or when a fall is visually detected.

[0037] The behavior analysis subunit establishes a baseline based on behavioral data over a preset time period (e.g., two weeks) (e.g., an elderly person's habit of going out at 10 AM daily). When the behavior deviates from the baseline (e.g., not going out for two consecutive days), it automatically sends a non-urgent "mild care reminder" (just a notification to pay attention) to a remote terminal (e.g., a child's family member). It is also used to automatically send a non-urgent mild care reminder to a remote terminal if the elderly person does not take corresponding action within a preset time after the system outputs feedback. Understandably, this limited care mechanism intelligently assesses risks beyond emergency responses and sends non-intrusive notifications to remote terminals when necessary. This avoids frequent phone or video calls triggered by routine warnings, which could disrupt the normal work and life rhythm of remote terminals, achieving more precise and user-friendly interaction that respects the usage scenario.

[0038] In practical use: Module connection relationship: Each sub-unit of the multimodal information acquisition unit (voice, vision, touch, physiological, etc.) is connected to the main control chip (such as ARM Cortex-A72) of the adaptive processing unit via USB or serial port to transmit data in real time; the adaptive processing unit communicates with the cloud server of the remote collaboration unit via wireless network (such as Wi-Fi) to realize data synchronization and command interaction, and dynamically update the guidance strategy.

[0039] Key parameter settings: Voice acquisition: sampling rate 16kHz, noise reduction level adjustable (default high noise reduction, suitable for noisy environments); Visual recognition: frame rate 30fps, facial recognition distance 0.5-3 meters; Touch screen: resolution 1920×1080, pressure sensing threshold 50g (avoid accidental triggering with light touches); Micro-motion recognition: millimeter-wave radar sampling rate 30Hz, capable of recognizing limb displacement ≥0.5mm; Scene response: environment type recognition latency ≤0.3 seconds, interaction mode switching time ≤0.5 seconds; Cognitive guidance: step repetition count adjustable (default 3 times, supports remote adjustment by children); Cloud synchronization: encrypted data transmission (AES-256 algorithm), data stored in the scene database.

[0040] This embodiment also provides an age-friendly intelligent interaction method for elderly people with cognitive impairment and partial disability. Utilizing an age-friendly intelligent interaction system for elderly people with cognitive impairment and partial disability, the method includes: S1, the multimodal information acquisition unit collects the elderly’s interactive commands (such as voice, head micro-movements), physiological state (such as heart rate) and environmental information (such as kitchen grease) in an all-round way through its sub-units (voice, vision, micro-movements, environment, etc.), and transmits the corresponding multimodal data to the adaptive processing unit in real time. S2, the data fusion subunit of the adaptive processing unit verifies multimodal data (such as consistency verification between voice commands and gestures); the personalized adaptation subunit adapts and adjusts the command guidance mode and intent recognition mode, initiating "memory enhancement" (step-by-step guided operation) for elderly people with cognitive impairment, and lowering the operation threshold for semi-disabled elderly people (error-tolerant handling of limb micro-movements); the scene response subunit adjusts the interaction mode based on multimodal data representing scene information (such as disabling touch control in the kitchen and enabling voice); the feedback execution subunit provides feedback on the operation results through multiple methods such as sound, sight, and touch based on different interactive inputs; S3: The cloud server interacts with the adaptive processing unit to exchange preset data (such as operation records and physiological data), and stores the corresponding scene interaction strategies and user data in the scene database according to the scene. S4, the information synchronization subunit of the remote collaboration unit pushes key data from the preset data to the remote terminal in real time; the one-click collaboration subunit triggers remote collaboration with the remote terminal and interface / scene sharing; the emergency response subunit triggers a remote alarm to the remote terminal and / or emergency center in case of an anomaly; the behavior analysis subunit warns of potential risks through behavior baselines and triggers remote terminal-limited care.

[0041] Example of a workflow in a typical scenario: Scenario 1: The elderly need to "open their health code". 1) Information collection phase: When the elderly person approaches the system (the visual acquisition subunit recognizes the face), the voice acquisition subunit automatically wakes up the microphone from low power consumption (reducing standby power consumption), and at the same time, the touch acquisition subunit automatically wakes up the touch screen and lights it up (initially displaying a large font time interface); the elderly person issues a voice command "Open health code" and points their hand to the touch screen. At this time, the voice acquisition subunit collects the sound and converts it into the text "Open health code", and the visual acquisition subunit simultaneously recognizes the hand pointing to the touch screen.

[0042] 2) Adaptive Processing Phase: The data fusion subunit verifies the command. The voice command "Open Health Code" is given while the user points to the touchscreen, double-verifying the command's validity. After confirmation, the personalized adaptation unit detects the elderly user's previous operation record indicating a "high contrast preference" and automatically adjusts the health code interface to black background with white text and enlarges the font to 28pt. The feedback execution unit then announces "Opening Health Code" via voice (slow speed), while a progress bar and green light indicator appear on the screen.

[0043] 3) Remote Collaboration Phase: If an elderly person accidentally closes their health code three times consecutively, the system automatically triggers an "Assistance Reminder" and sends the interface status to the remote terminal (e.g., a mobile app) of a designated person (children) via the information synchronization subunit. After the designated person (children) clicks "Remote Assistance" on their remote terminal (e.g., the mobile app), they can mark "Click the 'Save' button in the lower right corner" on the app, and the marked content is synchronized to the elderly person's touchscreen in real time. If the physiological information collection submodule detects a sudden increase in the elderly person's heart rate, the emergency response subunit immediately calls the designated person (children) and sends their location to their remote terminal (e.g., the mobile app).

[0044] Scenario 2: Elderly people with cognitive impairment retrieving medication from the kitchen. The system analyzes and predicts, based on long-term collected multimodal data of "kitchen + medicine packaging," that the elderly will retrieve their medicine at a certain location in the kitchen at a preset time. After the preset time is reached, the scene response subunit switches to "voice + text guidance" based on the kitchen scene in "kitchen + medicine packaging." The feedback execution subunit first outputs a simplified voice instruction: "First find the red medicine box and press the button on the box." If the visual acquisition submodule recognizes that the elderly person is frowning (confused), the cognitive adaptation module automatically repeats the voice instruction under the effect of the memory reinforcement model and adds image guidance (close-up of the medicine box + finger tap animation), or adapts to recent habitual interaction methods under the effect of the learning optimization module.

[0045] Scenario 3: A semi-disabled elderly person getting up at night Infrared depth cameras detect subtle head movements (nodding), triggering the night light to turn on when the micro-expression recognition module is activated; millimeter-wave radar detects a rightward shoulder movement, automatically triggering the "drinking water" function (linking with a smart water bottle) when the micro-motion acquisition subunit is activated; and through the behavior analysis subunit, if no activity is detected in the elderly person within a preset time (e.g., 10 minutes), a "mild reminder" is sent to their children.

[0046] It is understandable that the above system can fully execute the above method, with the same process and effect, so it will not be described in detail here.

[0047] Example 2 Unlike Embodiment 1, the personalized adaptation subunit also includes a personalized cognitive training module, which provides dynamically adapted training tasks for elderly people with cognitive impairment and partial disability through a data-driven approach.

[0048] Specific functions and usage include: Upon first use, an initial ability assessment is conducted, pushing a set of standardized cognitive test tasks: a simple orientation test ("What day is it today?"), a short-term memory task (reciting 3 words), and a visual matching game (finding matching patterns). Multimodal data acquisition units record: answer accuracy, reaction time (voice / touch), micro-motion stability (hand tremor level), and physiological indicators (heart rate variability (HRV), reflecting cognitive load). A cognitive ability assessment model is introduced, outputting an initial cognitive ability profile, including a cognitive ability score and scores for each dimension (memory, attention, executive ability), used to update the user profile. For example, a cognitive ability score of 61.5 indicates below-average performance, with memory and judgment abilities at an average level and executive ability at a weak level.

[0049] The cognitive ability assessment model is as follows:

[0050] In the formula, Cognitive Score represents the cognitive ability score, which comprehensively reflects the user's current cognitive state. The higher the score, the better the performance. Accuracy represents the task answer accuracy rate, which is the number of correctly answered questions divided by the total number of questions, measuring memory and judgment ability. Response Time represents the average reaction time, which is the average time from the appearance of the prompt to the user's response, reflecting processing speed. The shorter the time, the better. Therefore, its reciprocal is taken to avoid slow reaction time lowering the overall score, measuring execution ability. HRV Stability represents the stability of heart rate variability. Based on the HRV (Heart Rate Variability) data obtained by the physiological information acquisition subunit, the low-frequency to high-frequency power ratio (LF / HF) is calculated, reflecting the autonomic nervous system regulation ability. The more stable the value (closer to the normal range), the more moderate the cognitive load and the more focused the attention, measuring attention ability. w1, w2, and w3 are weighting coefficients, and their sum is 1.

[0051] Training tasks are generated and pushed automatically, with 1-2 cognitive training tasks pushed daily. The types include: memory tasks (picture recall), word association tasks (time / location judgment), and execution tasks (simple sorting and finding differences). A task difficulty adjustment model is introduced to match the initial difficulty with the initial evaluation results.

[0052] The task difficulty adjustment model based on the Q-Learning reinforcement learning model is as follows: Define State: User's current ability rating; Define Action: Increase / maintain / decrease task difficulty; Define Reward: Successful completion with moderate workload → +1, Failure but multiple attempts → 0, Consecutive failures or abandonment → -1.

[0053] Update Q value:

[0054] In the formula, Q(s,a) represents the state-action value function; s represents the current state, the user's current cognitive ability score range (low / medium / high); a represents the current action, increasing difficulty, maintaining difficulty, or decreasing difficulty; α represents the learning rate, ranging from 0 to 1, controlling the weight of new information on the old Q value update, for example, it means updating the Q value with 10% new error each time to avoid drastic fluctuations; R represents the immediate reward, the feedback value given by the system, such as +1 for successful completion of the task and -1 for failure; γ represents the discount factor, the degree of importance attached to future rewards, for example, γ=0.9 means that the system values ​​long-term performance rather than just the current result; This represents the maximum Q value in the next state.

[0055] The process of updating the Q value can be understood as: new Q value = old Q value + learning rate × (actual return - expected return).

[0056] Employing the ε-greedy approach from Q-Learning, this method balances exploration and exploitation, ensuring the difficulty gradually approaches the user's optimal challenge range. Specifically, an exploration probability ε is defined (e.g., ε = 0.1, or 10%). At each decision point: with probability ε, an action is randomly selected (exploration); with probability 1 - ε, the action with the highest Q-value is selected (exploitation). If ε = 0.1, the system spends 90% of its time choosing the current optimal difficulty and 10% of its time trying higher or lower difficulties to determine if a better fit is desired for the user.

[0057] The training process is interactive, allowing seniors to complete tasks via voice, touch, or micro-movements. The system collects data in real time: operation path (e.g., touch trajectory), error type (confusion, forgetting, accidental touch), completion time, and number of attempts. The feedback execution subunit provides multimodal positive feedback: upon success, voice encouragement and gentle light flashing; upon failure, simplified prompts and step-by-step guidance (e.g., "Think again, the first word is apple"). A training content generation rule engine is introduced, based on a "cognitive training template library" in a cloud-based scenario database, combining user life experiences (e.g., occupation, interests) to generate personalized content examples: if the user was a teacher, the training questions are integrated into a "grading homework" scenario; if the user enjoys Peking Opera, Peking Opera excerpts are used as memory cues to enhance participation and emotional resonance, thereby improving training effectiveness.

[0058] Training effect analysis and model update: After each training session, the system calls the ability assessment model to recalculate the cognitive score; updates the user profile and inputs it into the Q-Learning model for the next round of task strategy optimization; long-term data is uploaded to the cloud to support the behavior analysis subunit in providing trend warnings (such as a continuous decline in cognitive ability).

[0059] This embodiment provides an age-friendly intelligent interactive system and method for elderly people with cognitive impairment and partial disability. For the first time, it combines reinforcement learning (Q-Learning) with a multimodal physiological behavior fusion assessment model to achieve dynamic adaptive adjustment of cognitive training difficulty. It introduces HRV stability as a cognitive load indicator to improve assessment objectivity. The training content is deeply integrated with the user's life experience, enhancing emotional connection and engagement. By using the Q-Learning deep learning model and the ε-greedy strategy for adaptive cognitive training difficulty in the elderly, the system actively explores the optimal challenge range, avoiding a stalemate of being "always too difficult" or "always too easy." Combining physiological data (HRV) as the reward function input makes the exploration safer (avoiding anxiety caused by excessive difficulty). Overall, it forms an interpretable, convergent, and personalized intelligent training mechanism.

[0060] The entire process is based on changes in user capabilities, adjusting training content, difficulty, and feedback methods in real time to avoid frustration or ineffective repetition, achieving cognitive intervention that is "personalized and adaptable to different times," which can significantly improve the personalization and effectiveness of cognitive intervention.

[0061] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. An age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability, characterized in that: It includes an electrically connected multimodal information acquisition unit, an adaptive processing unit, a cloud server, a remote collaboration unit, and a remote terminal; The multimodal information acquisition unit includes a voice acquisition subunit, a vision acquisition subunit, a touch acquisition subunit, a physiological information acquisition subunit, a micro-motion acquisition subunit, and an environmental perception subunit. Each subunit is activated according to a preset wake-up strategy to acquire multimodal data and send it synchronously to the adaptive processing unit. The multimodal data includes the raw data acquired by each subunit and the corresponding instructions or information obtained through parsing and processing. The adaptive processing unit includes a data fusion subunit for multimodal data verification, a personalized adaptation subunit for adapting and adjusting intent recognition mode and instruction guidance mode, a scene response subunit for adjusting interaction mode based on multimodal data representing scene information, and a feedback execution subunit for providing multi-mode feedback output based on different interactive inputs (audio, visual, and tactile). A cloud server is used to interact with the adaptive processing unit using preset data. It also has a scene database for storing corresponding scene interaction strategies and user data by scene; The remote collaboration unit includes an information synchronization subunit for synchronizing key data from preset data to a remote terminal, a one-click collaboration subunit for triggering remote collaboration with the remote terminal and sharing of interfaces / scenes, an emergency response subunit for triggering remote alarms to the remote terminal and / or emergency center in case of anomalies, and a behavior analysis subunit for warning of potential risks through behavioral baselines and triggering remote terminal-limited care.

2. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The visual acquisition subunit includes a micro-expression recognition module with an integrated infrared depth camera, used to capture head micro-movements and micro-expressions and parse them to obtain corresponding instructions or information.

3. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The micro-motion acquisition subunit is equipped with a non-contact millimeter-wave radar. Based on the data acquired by the millimeter-wave radar, it identifies subtle limb movements and analyzes them to obtain corresponding instructions or information. Subtle limb movements include the trajectory of slight finger tremors and small shoulder swings.

4. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The personalized adaptation subunit includes a cognitive adaptation module, which has a built-in memory reinforcement model to provide repeated feedback on key steps in different ways for elderly people with cognitive impairment and to provide step-by-step guidance; it is also used to record frequently used operations to form a one-click direct interaction strategy.

5. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The personalized adaptation subunit, including the limb adaptation module, is used to establish a deviation-allowing computational interaction strategy between the collected multimodal data and core intent recognition for semi-disabled elderly people.

6. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The feedback execution subunit includes a simplified semantic module, which is used to transform complex instructions for output feedback into colloquial expressions.

7. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The one-click collaboration subunit allows the elderly to trigger a one-click sharing of their current user interface with a remote terminal. The remote terminal provides guidance, annotations, and / or voice guidance on the shared user interface to facilitate remote collaboration in the elderly's operation.

8. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The one-click collaboration subunit is used to receive scene sharing requests from remote terminals and share the elderly person's current scene interface and environment annotations to the remote terminal.

9. The age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in claim 1, characterized in that, The behavior analysis subunit is used to establish a baseline based on behavior data over a preset time period. When the behavior deviates from the baseline, it automatically sends a non-urgent, mild care reminder to the remote terminal.

10. An age-friendly intelligent interaction method for elderly people with cognitive impairment and partial disability, characterized in that: The method utilizes the age-friendly intelligent interactive system for elderly people with cognitive impairment and partial disability as described in any one of claims 1-9; the method includes: S1, the multimodal information acquisition unit collects the elderly’s interactive commands, physiological state and environmental information through its sub-units, and transmits the corresponding multimodal data to the adaptive processing unit in real time; S2, the data fusion subunit of the adaptive processing unit verifies the multimodal data; the personalized adaptation subunit adapts and adjusts the instruction guidance mode and intent recognition mode; the scene response subunit adjusts the interaction mode based on the multimodal data representing scene information; the feedback execution subunit outputs the operation results based on different interactive inputs through audio, visual and tactile methods. S3: The cloud server and the adaptive processing unit perform preset data interaction and store the corresponding scene interaction strategies and user data in the scene database according to the scene. S4, the information synchronization subunit of the remote collaboration unit pushes key data from the preset data to the remote terminal in real time; the one-click collaboration subunit triggers remote collaboration with the remote terminal and interface / scene sharing; the emergency response subunit triggers a remote alarm to the remote terminal and / or emergency center in case of an anomaly; the behavior analysis subunit warns of potential risks through behavior baselines and triggers remote terminal-limited care.

Images