Virtual reality-based multi-platform early ad rehabilitation system and method for the elderly

Abstract

This invention relates to a multi-platform, age-friendly early AD rehabilitation system based on virtual reality, aiming to provide a comprehensive rehabilitation experience. The system includes a user login unit, a virtual reality interaction unit, a multidimensional data analysis unit, and an information management unit. The user login unit is used for selecting, inputting, and verifying the subject's identity; the virtual reality interaction unit provides immersive cognitive training and operational interaction for the subject within a virtual reality environment; the multidimensional data analysis unit processes and analyzes the subject's multidimensional data; and the information management unit collects, stores, and retrieves multidimensional user information. This solution improves the visibility and ease of operation for elderly users in the virtual environment, reduces learning costs, and thus enhances the system's usability and adaptability among the elderly population.

Description

Technical Field

[0001] This invention relates to the field of early AD diagnosis and rehabilitation, and in particular to a multi-platform age-friendly early AD rehabilitation system and method based on virtual reality. Background Technology

[0002] Dementia is a brain disease that affects cognitive function. Alzheimer's disease (AD) is the most common form of dementia, currently prevalent among the elderly. The total number of AD patients worldwide continues to rise, but with timely intervention and assessment, early-stage AD patients still have a chance of being cured. The government also pays close attention to and supports cognitive intervention and treatment for early-stage AD patients. Although current treatments for early-stage AD mostly involve medication, these methods suffer from significant side effects, limited long-term efficacy, drug interactions leading to reduced efficacy, high patient dependence, high cost, and individual variability. Existing technologies lack effective objective diagnostic methods for early-stage AD. Furthermore, early-stage AD patients often behave indistinguishably from normal individuals, limiting current diagnostic methods: firstly, the technology is not yet mature, many methods are still experimental, and clinical standards have not been established; secondly, early-stage AD patients often show subtle changes, making effective detection difficult; and thirdly, current testing for early-stage AD is costly, requiring expensive medical equipment, making it impractical for implementation in primary care hospitals.

[0003] In recent years, researchers have conducted extensive studies on digital therapy (VR) for treating cognitive impairment. Evidence-based digital therapies for cognitive impairment, especially digital cognitive training, have shown significant efficacy in patients with mild cognitive impairment. According to the "Chinese Guidelines for Cognitive Training (2022 Edition)," cognitive training has an improving effect on the cognitive function of cognitively normal older adults, especially multi-cognitive domain cognitive training, which can improve the cognitive function of healthy older adults, slow the decline of daily living abilities, reduce the risk of mobility impairment, and also improve mood.

[0004] Virtual Reality (VR), as an emerging and advanced technology, can provide a fully controlled experimental environment, facilitating real-time observation of users' memory, emotions, and other aspects, and enabling motor control. Currently, virtual reality is increasingly being used in areas such as Alzheimer's disease (AD) screening, cognitive training, and caregiver support.

[0005] However, traditional single-terminal VR systems still suffer from strong equipment dependence and inconvenience in scenarios such as large-scale screening, daily follow-up, and home intervention, making it difficult to meet the screening needs of early Alzheimer's disease patients, and the system's controllability is insufficient. At the same time, the elderly often have problems such as unfamiliarity with operation and poor adaptability when using immersive devices. If there is a lack of external auxiliary terminals to supplement them, it is easy to affect the usability and promotion of the system. As an auxiliary device, tablet terminals have multiple requirements such as lightweight, age-friendly, and convenient. They can not only achieve rapid initial screening and high-frequency follow-up, but also solve the problems of 3D dizziness and convenience for the elderly. Summary of the Invention

[0006] In view of the current state of technology for diagnosing early-stage Alzheimer's disease (AD), and to address the problems of high treatment costs and the need for expensive medical equipment for early-stage AD patients, as well as the difficulty in diagnosing early-stage AD patients and the difficulty in detecting symptoms, this invention employs the following technical solution:

[0007] On one hand, the present invention provides a multi-platform age-friendly early AD rehabilitation system based on virtual reality, comprising:

[0008] The user login unit is used to select, input, and verify the identity of the subject. After successful login, it retrieves and associates the corresponding individual profile and historical training data to support the subsequent evaluation, training, and result display process.

[0009] The virtual reality interactive unit primarily provides subjects with immersive cognitive training and operational interaction based on a virtual reality environment. It enables task scheduling, device coordination, and adaptive training control according to the subject's state. Cognitive rehabilitation training tasks and activities are presented through the system's interactive functions on a virtual reality operating interface or computer. It integrates diagnostic and rehabilitation training methods for subjects who may have early-stage Alzheimer's disease into virtual reality scenario games, allowing appropriate home scenarios to be selected for rehabilitation training based on different cognitive domains.

[0010] Furthermore, the virtual reality interaction unit includes:

[0011] The adaptation module uses a meditation scenario and a parent-child companionship scenario before training begins. It detects the subject's eye movement signals in the device module and uses the eye movement data acquisition and analysis subsystem to extract features from them, obtaining four types of feature indicators: fixation point coordinate sequence, saccade speed, blink frequency, and pupil diameter. Through indicator analysis, it adaptively selects either the virtual reality VR terminal rehabilitation training mode or the tablet terminal rehabilitation training mode.

[0012] Furthermore, the eye-tracking data acquisition and analysis subsystem acquires the subject's eye-tracking signals in real time through the eye-tracking sensor built into the VR headset. Specifically, the virtual reality device includes a headset, an infrared eye-tracking submodule, and a data processing submodule. The infrared module consists of an infrared emitting unit and an imaging unit, emitting invisible infrared light and acquiring eye images.

[0013] The task scheduling module, as the central mode of the system, has two modes: a free mode, in which the subject can freely choose the task they want to perform based on the cognitive domain labels; and a story mode, in which the task scheduling module schedules different tasks in the virtual reality scene according to the pre-set story flow, so that the subject can complete the training tasks corresponding to the five cognitive domains in a preset order.

[0014] Furthermore, the system embeds cognitive training tasks such as memory, attention, executive function, language, and visuospatial ability into continuous virtual scenario plots, and progresses step by step according to a preset process, so that the subjects can train each cognitive domain in turn while completing the plot tasks.

[0015] The equipment module manages various devices required by the system, including virtual reality devices, detection devices, auxiliary devices, and tablet devices. Furthermore, the virtual reality devices are head-mounted virtual reality terminal devices that support gesture recognition interaction and eye-tracking signal detection, allowing users to interact using natural gestures without the need for external controllers. The detection devices mainly include EEG straps for detecting electroencephalogram (EEG) signals and ECG patches for detecting electrocardiogram (ECG) signals. The auxiliary devices include a monitor, keyboard, and mouse for external detection and control of the virtual reality training system. The tablet devices mainly include various types of tablet computers.

[0016] The cognitive rehabilitation training module manages various scenario tasks required for the rehabilitation of early-stage Alzheimer's disease patients, including tablet scenarios, home cooking scenarios, living room entertaining scenarios, and fruit platter making scenarios.

[0017] A multidimensional data analysis unit. This unit is configured to process the multidimensional data collected by the information management unit, perform multidimensional cognitive assessments of the subjects using a pre-trained model, and generate suggestions tailored to the subjects' individual circumstances based on a large language model. The unit includes a model training module, a cognitive assessment module, and an intelligent suggestion generation module.

[0018] Multidimensional data includes the following categories:

[0019] First, the personal information of the participants: age, gender, and education level.

[0020] Secondly, the MMSE scores of the subjects before they underwent cognitive rehabilitation training.

[0021] Thirdly: the subject's task performance: accuracy, total time, number of prompts requested, difficulty of selection, etc.

[0022] Fourth: the physiological data of the subjects, including electrocardiogram and electroencephalogram data during resting state and task execution.

[0023] The model training module uses pre-collected and processed multidimensional subject data for model training. Before training begins, subjects with MMSE scores less than 28 are identified as having cognitive impairment, and their data is labeled as having cognitive impairment. Other subjects with scores greater than 28 are labeled as not having cognitive impairment. Based on this, other multidimensional data are used as multidimensional features in the training of the cognitive assessment model.

[0024] The cognitive assessment model mentioned above is a deep learning-based cognitive impairment probability assessment model. The specific model framework is as follows:

[0025] Firstly, a multimodal feature extraction subsystem is configured to extract and preprocess features from three types of input data (subject personal information, subject task performance, and subject physiological data). The following are the different preprocessing branches for the three types of data:

[0026] 1) Personal background information branch. Categorical variables (e.g., gender) are one-hot encoded, and numerical variables (age, height) are standardized to transform them into background prior vectors. .

[0027] 2) Task Performance Branch. Data including subjects' accuracy, total time, number of prompts requested, and selection difficulty during cognitive training are processed using Min-Max standardization to transform them into behavioral feature vectors reflecting the subjects' real-time cognitive load. .

[0028] 3) Physiological Feature Branch. The physiological feature branch employs a deep convolutional neural network with a self-attention mechanism to perform feature mapping on the original real-time electrophysiological time-series signals of electroencephalography (EEG) and electrocardiography (ECG). In a preferred embodiment, the physiological feature branch extracts frequency features by covering the complete physiological signal cycle with convolutional kernels, and uses the self-attention mechanism to assign correlation weights in the time dimension, while strengthening weak EEG / ECG waveform features related to abnormal cognitive load, thereby outputting a fixed-dimensional physiological feature vector. .

[0029] Secondly: The cascaded fusion subsystem will integrate the extracted data... , and Linear concatenation is performed along the feature dimension to form a globally fused feature vector. The specific integration formula is as follows:

[0030]

[0031] in This represents a vector concatenation operation. This indicates a vectorized construction.

[0032] Thirdly, the probability output and decision system employs a meta-classifier consisting of a 3-layer fully connected network, with the top layer using the Sigmoid activation function to fuse features. Mapped to The interval outputs the probability value of the subject having cognitive impairment. .

[0033] The cognitive assessment module is configured to input the multidimensional features of the subjects collected by the information management unit into the trained cognitive assessment model and output the probability value of the patient having cognitive impairment, thereby realizing the detection and early warning of early AD patients; at the same time, it combines past data to monitor the degree of cognitive impairment of the subjects during long-term training.

[0034] The intelligent suggestion generation module takes the cognitive impairment probability value, the historical assessment difference threshold (such as the difference between the current probability value of 0.68 and the historical probability value of 0.75), and the multidimensional physiological feature vector as input parameters, and calls the large language model to generate structured intervention suggestions. The structured intervention suggestions include: risk trend assessment indicators (such as the positive trend corresponding to the decrease in probability value), physiological feature feedback status (such as the attention status corresponding to the EEG feature vector), and adaptive training parameter adjustment instructions.

[0035] As a preferred approach, this multidimensional feature-based cognitive assessment model addresses the lack of specificity of single physiological features in diagnosing early-stage Alzheimer's disease (AD). By introducing MMSE scores and task performance as key factors, it enhances the robustness of the assessment. Simultaneously, its cascaded fusion architecture can process both high-dimensional continuous signals (such as EEG and ECG signals) and low-dimensional discrete indicators (such as the subject's gender) without altering the physical meaning of the original signals. Furthermore, the model's output probability values ​​can be directly integrated with large language models to generate personalized rehabilitation suggestions based on the subject's actual physiological characteristics.

[0036] The information management unit is configured to collect, store, and quickly retrieve multi-dimensional user information during use. The unit includes an information collection module, an information storage module, and an information retrieval module.

[0037] The information collection module collects user information. This information includes personal details, performance data during virtual testing tasks, and physiological signal data monitored during the testing process. Personal details include gender and age. Performance data across various task scenarios primarily includes objective behavioral characteristics such as total time spent completing tasks, scores, and accuracy. Physiological signal data includes EEG and ECG data monitored during task completion. Personal details are collected upon initial login and can be modified subsequently. Task performance data is collected after each task completion, temporarily cached locally, and then written to a cloud database after all tasks are completed. As an implementation, all task performance data is transmitted to the information collection module in real-time via a custom event listener interface. The information collection module uses `java.io.FileWriter` to write data line by line to a CSV file, stored in the device's internal storage or external storage. The file name includes the subject ID and a timestamp. EEG and ECG data are collected by monitoring devices during task training. After the subjects complete all task training, all physiological data collected by the monitoring devices are saved to a cloud database for subsequent model training modules to perform unified data processing and obtain model evaluation results.

[0038] The information storage module is used to store multidimensional user information. In addition to the multidimensional data mentioned above, it also includes the subject's previous assessment results and data, which are stored through the cloud platform database.

[0039] The information retrieval module connects the underlying cloud data warehouse and the upper-level multidimensional data analysis unit, responsible for the efficient retrieval of multimodal data and the closed-loop storage of evaluation results. This module is built on a MySQL cloud database and uses standardized SQL to achieve unified management of heterogeneous data. In practical applications, this module performs efficient index queries in the MySQL database using key fields such as subject name, subject number, or collection date. When the multidimensional data analysis unit initiates the evaluation process, the retrieval module uses SQL to extract the subject's static information (such as age and gender), dynamic task performance data (such as total time and accuracy), and physiological signals (EEG and ECG) from the cloud database. This design ensures that the model training module and the cognitive evaluation module can quickly obtain the necessary data for subsequent model training or analysis.

[0040] On the other hand, the present invention provides a multi-platform rehabilitation method based on virtual reality, comprising the following steps:

[0041] S1: Medical staff record the initial identity of the subject through the user login module and load the cognitive-related data and rehabilitation training data of the user previously;

[0042] S2: Medical staff select an appropriate cognitive assessment scale through the information collection module to conduct an initial assessment of the patient and record the cognitive information of the subject;

[0043] S3: Subjects first undergo adaptation training, adapting to parent-child companionship scenarios and meditation scenarios through the adaptation module. The adaptation module uses eye-tracking data to infer whether to choose tablet-based environmental cognitive rehabilitation training or early AD cognitive rehabilitation training in a virtual reality environment.

[0044] S4: The task scheduling module selects and loads a specific rehabilitation training scenario for the subject based on the cognitive domain score of the previous assessment module;

[0045] S5: Before starting rehabilitation training, the subject will undergo a corresponding adaptation procedure to understand the complete rules of the training. When starting rehabilitation training, the subject will use the ECG strap and EEG patch from the equipment module to assist in the corresponding virtual reality rehabilitation training.

[0046] S6: After completing a period of rehabilitation training, the medical staff will use the information collection module to assess the subjects again and record the relevant data.

[0047] Furthermore, in step S2, the cognitive assessment scale is preferably the Mini-Mental State Examination (MMSE), although the Moca scale can also be used as an alternative. After acquiring the subject's MMSE assessment data, the information collection module sends it to the task scheduling module for threshold comparison and logical branch determination. The specific determination control rules are as follows:

[0048] Branch 1: If the subject's total score on the MMSE mini-scale is >28, their objective cognitive function is considered normal. The task scheduling module automatically switches to the "regular story mode branch" and loads a global rehabilitation training task covering the five cognitive domains (audiovisual memory, attention and problem-solving, working memory, logical abstraction, and spatial pathfinding) onto the subject according to the system's preset standard training map.

[0049] Branch Two: If the subject's total score on the Mini-MMSE is ≤28 (i.e., they are identified as a potential early-stage AD patient), the task scheduling module will perform targeted deficit assessment, breaking down the subject's sub-item scores in each independent cognitive domain and comparing them with thresholds to generate cognitive domain impairment labels.

[0050] 1) Visual and auditory domain impairment test: Extract the scores of the corresponding test questions. If the score is <6 points (out of 6), the subject will be labeled with "visual and auditory memory impairment".

[0051] 2) Attention and problem-solving disorder test: Extract the scores of the corresponding test questions. If the score is <7 points (out of 8), the subject will be labeled with "attention and executive function disorder".

[0052] 3) Working memory impairment test: Extract the scores of the corresponding test questions. If the score is <3 points (out of 3), then the subject will be labeled with "working memory impairment".

[0053] 4) Logical Abstraction Disorder Test: Extract the score of the corresponding test question. If the score is <2 points (out of 2), then the subject will be labeled with "Logical Abstraction Disorder".

[0054] 5) Spatial Pathfinding Obstacle Test: Extract the score of the corresponding test question. If the score is <11 points (out of 11), then the subject will be assigned the "Spatial Pathfinding Obstacle" label.

[0055] After the fine-grained breakdown is completed through the built-in analysis script, the task scheduling module uses a built-in dynamic rule engine to perform weighted control scheduling based on the set of obstacle tags triggered in the subject's running object. For subjects with specific cognitive domain obstacle tags, the system's underlying event delegation mechanism, Event Delegate, will trigger a priority loading instruction, calling Unity's SceneManager to prioritize and asynchronously load TaskPrefabs of a specific virtual reality rehabilitation training task type that match the subject's defect tag.

[0056] Furthermore, in step S5, before rehabilitation training, the subject first needs to wear physiological signal acquisition devices, including an EEG patch for collecting EEG signals and an ECG strap for collecting ECG signals. The information acquisition module synchronously collects the subject's EEG and ECG signals during the virtual reality rehabilitation training process to assess the subject's cognitive state and physiological response. After completing the wearing of the physiological signal acquisition devices, the subject must undergo a rehabilitation training task adaptation phase before entering each cognitive rehabilitation training session. Before each rehabilitation training task officially begins, the system will provide the subject with corresponding adaptive guidance procedures to help the subject understand the operating rules and interaction methods of the current training program.

[0057] Furthermore, the system first demonstrates two typical operation methods related to the actions and interactions involved in the current training task. For example, it uses a virtual hand model to demonstrate basic gesture operations such as grasping, clicking, and dragging, guiding the subject to become familiar with the interaction logic in the virtual reality environment. By allowing the subject to try, the system demonstrates the standard execution method of the actions.

[0058] The above one or more technical solutions have the following beneficial effects:

[0059] 1. This invention provides a multi-platform age-friendly early AD rehabilitation system and method based on virtual reality. It constructs rehabilitation training scenarios involving five cognitive domains through virtual reality technology to expand the virtual reality scenarios. The virtual reality training interface adopts age-friendly design, such as large font size display, large interactive buttons, simplified operation interface and voice prompts, to improve the visibility and operation convenience of elderly users in the virtual environment, reduce the learning cost, and thus improve the usability and adaptability of the system in the elderly population.

[0060] 2. This invention introduces a gesture recognition-based natural interaction method into a virtual reality rehabilitation training system. By using virtual reality devices to scan and track the user's hands in real time, a virtual hand model corresponding to the user's hand movements is constructed in the virtual environment. This allows users to perform interactive behaviors such as selection, grasping, and manipulation through natural gestures, thus eliminating the need for traditional virtual reality controllers. Compared to existing technologies, this method reduces the operational complexity for elderly users, lowers the barrier to entry for device use, and enhances the user's immersion and naturalness of interaction in the virtual reality environment through the synchronous mapping of virtual and real hand movements. It also reduces the sense of separation between the virtual and real environments, thereby making it more conducive for subjects to complete cognitive rehabilitation training tasks.

[0061] 3. This invention constructs a multi-platform system architecture that integrates virtual reality (VR) terminals and tablet terminals. The VR terminal provides immersive cognitive rehabilitation training scenarios, while the tablet terminal serves as a supplementary and alternative platform for VR training. When some users experience dizziness or discomfort while using VR devices, or when prolonged use of the headset is not suitable, they can continue to complete the corresponding training tasks through the tablet terminal, thus ensuring the continuity of the rehabilitation training process. Compared to systems that rely solely on VR devices, this invention effectively expands the applicable user base and improves the system's flexibility and applicability in different usage scenarios by introducing a tablet terminal alternative.

[0062] 4. This invention incorporates a multi-dimensional data analysis unit within the system. By collecting and storing behavioral data and task completion information of the subjects during training, and combining this data with a pre-trained classification model, the system analyzes and processes the relevant data to comprehensively assess the subjects' cognitive state. Based on this, the system can automatically generate corresponding training suggestions according to the analysis results, achieving self-optimization of rehabilitation training. Compared to traditional systems with fixed training programs, this invention can dynamically adjust according to individual differences in the subjects, thereby improving the targeting and effectiveness of rehabilitation training.

[0063] 5. This invention employs training tasks covering five cognitive domains when constructing virtual reality rehabilitation training scenarios: memory, attention, executive function, language ability, and visuospatial ability. By designing different training tasks for each cognitive domain, subjects can receive targeted rehabilitation training, focusing on improving their cognitive function in specific areas. This method not only improves subjects' abilities in a single cognitive domain but also enhances their overall brain function through multi-dimensional training, thereby achieving a more comprehensive and long-lasting brain function rehabilitation effect, helping early-stage Alzheimer's disease patients improve cognitive function and slow disease progression. Attached Figure Description

[0064] Figure 1 This is a diagram illustrating the overall architecture of the multi-platform age-friendly early AD rehabilitation system based on virtual reality, as described in this invention.

[0065] Figure 2 This is a flowchart illustrating the implementation of the multi-platform age-friendly early AD rehabilitation system based on virtual reality according to the present invention;

[0066] Figure 3 This is a detailed diagram of the VR adaptability detection module. Detailed Implementation

[0067] Example 1:

[0068] To help participants adapt to virtual reality devices more quickly and identify those who may suffer from motion sickness, the system designed a planar training task as an adaptation pre-screening process. This process is executed before the formal cognitive training task begins and includes two sub-scenes: a meditation task and a parent-child interaction task. These two sub-scenes will be presented sequentially, guiding participants to adapt to the VR environment, screening for motion sickness risk, and providing emotional relief before training.

[0069] The adaptation block integrates an eye-tracking data acquisition and analysis subsystem. During the subjects' experience of meditation and parent-child companionship scenarios, it collects the subjects' eye-tracking characteristic data in real time and evaluates the subjects' VR adaptation status according to preset abnormal judgment criteria. If the eye-tracking data triggers an abnormal threshold, the system determines that the subjects are not suitable to continue receiving VR training and automatically recommends switching to a tablet-based cognitive training program.

[0070] The meditation task scenario is used to provide subjects with a multi-sensory immersive relaxation experience before the formal training begins, effectively relieving the subjects' tension when they first enter the VR system, and at the same time providing the first stage of observation for eye-tracking data collection.

[0071] Meditation scenarios include seated meditation, music meditation, and outdoor nature meditation:

[0072] The indoor meditation scenario constructs a brightly lit virtual indoor environment, configures multiple meditation cushions as interactive elements, and a virtual TV plays meditation guidance videos in a loop, providing subjects with static meditation training mainly guided by visual means.

[0073] The indoor music meditation scene constructs a virtual quiet indoor environment with dim lighting and drawn curtains, equipped with a reclining lounge chair, a ceiling covered with a starry sky and universe-themed visual texture, and a background audio system playing soothing meditation music to achieve a multi-sensory fusion relaxation training that combines auditory stimulation and visual immersion.

[0074] The outdoor natural meditation scene constructs a natural environment outside a virtual home building, equipped with a leisure swing chair, and an environmental sound system that outputs natural insect chirping sounds. Subjects can choose to interact with a virtual wooden fish by tapping it or sit quietly, achieving multi-sensory natural immersive relaxation training with visual, auditory, and potential tactile stimulation.

[0075] The parent-child companionship task scenario is used to conduct light, everyday conversations with the subject through a virtual granddaughter character after the meditation scenario, further eliminating the subject's unfamiliarity and anxiety with the VR environment; it also provides a second stage of observation for eye-tracking data collection.

[0076] The virtual granddaughter character is driven by a 3D virtual human model and interacts with other children through dialogues on everyday themes (cooking) related to cognitive training content, creating a warm parent-child interaction atmosphere.

[0077] The parent-child companionship task scenario includes a multimodal intelligent feedback mechanism that can dynamically generate feedback based on the real-time response performance of the subject during the dialogue interaction process. The feedback types include: verbal feedback (praise when the subject successfully completes the dialogue question and answer, prompts and encouragement when there is no response for a long time, and comfort and guidance after the wrong response), facial expression feedback (the virtual human's facial expressions change dynamically with the interactive situation), and action feedback (the virtual human's body movements respond in conjunction with the interactive situation, such as cheering and comforting actions).

[0078] like Figure 3 As shown, the eye-tracking data acquisition and analysis subsystem acquires the subject's raw eye-tracking signals in real time through the eye-tracking sensor built into the VR headset. Specifically, the virtual reality device includes a head-mounted display submodule, an infrared eye-tracking submodule, and a data processing submodule, wherein the infrared eye-tracking submodule consists of an infrared emitting unit and an infrared imaging acquisition unit. The infrared emitting unit emits invisible infrared light towards the subject's eye area, and the infrared imaging acquisition unit acquires images of the corneal reflection point and the center position of the pupil using a high-speed camera, and calculates the real-time gaze direction and fixation point coordinates based on the pupil-corneal reflection (PCCR) eye-tracking algorithm. The acquired eye-tracking image data is preprocessed by the device's built-in processor, including image denoising, pupil edge detection, and corneal reflection point localization, and further converted into time-series eye-tracking signal data.

[0079] After obtaining the raw eye movement signals, the eye movement data acquisition and analysis subsystem extracts features from them to obtain four types of feature indicators: fixation point coordinate sequence, saccade velocity, blink frequency, and pupil diameter. The following composite judgment criteria are then used to quantitatively evaluate the VR adaptation status:

[0080] (a) Saccade Velocity Index (SVI): The calculation formula is as follows:

[0081]

[0082] in The mean of the current saccade speed sequence of the subjects. This serves as a reference value for baseline saccade speed in healthy subjects of the same age group; when or At that time, the scanning speed was determined to be abnormal;

[0083] (ii) Fixation Stability Index (FSI): The calculation formula is as follows:

[0084]

[0085] in and These are the normalized x and y coordinate sequences of the fixation point; when

[0086] At that time, it was determined that gaze stability was abnormal. It is a unit of measurement, meaning the square of a pixel;

[0087] (iii) Blink Rate Index (BRI): The number of blinks per unit time is used as the reference range. The normal reference range is 10 to 25 times / minute. When the BRI is < 5 times / minute or BRI > 40 times / minute, the blink rate is considered abnormal.

[0088] (iv) Pupil Diameter Change Rate (PDCR): The calculation formula is as follows:

[0089] ,

[0090] in This represents the average pupil diameter within the current sampling window. The baseline pupil diameter was measured at rest before the subject entered the scene; when PDCR > 30%, abnormal pupil diameter change was considered.

[0091] Comprehensive judgment logic: During the experience of meditation and parent-child companionship scenarios, if any two or more of the above four indicators trigger an abnormal judgment at the same time, the system will output an assessment result of "insufficient VR adaptability" and automatically prompt the operator or subject to switch to the tablet training program; if only one indicator triggers an abnormality, the system will record a warning mark and extend the adaptation observation time; if all indicators are within the normal range, the system will output an assessment result of "good VR adaptability" and allow the subject to enter the formal VR cognitive training task sequence;

[0092] Both sub-scenes report the subject's eye movement characteristics, dwell time, interaction behavior, and emotional response data to the data acquisition module through a data recording interface. This data is used for both VR adaptability assessment and subsequent cognitive function analysis.

[0093] Example 2:

[0094] To address the issues of insufficient portability and high operational complexity for some elderly users of VR head-mounted display devices, this invention develops lightweight cognitive training tasks on a tablet platform as an auxiliary rehabilitation training terminal.

[0095] By leveraging the portability and ease of use of tablet devices, daily cognitive training and stress relief tasks can be deployed across platforms, enabling early-stage Alzheimer's disease patients to independently conduct rehabilitation training and self-assess their cognitive functions in their daily lives.

[0096] The tablet scenario includes two sub-tasks: a counting sheep task to assess and train the subject's short-term and long-term memory, and a jigsaw puzzle task to test and train the subject's spatial cognition and reaction abilities.

[0097] The following are the instructions for the counting sheep task:

[0098] This mission features four levels of increasing difficulty:

[0099] The first to third difficulty levels involve corresponding stimuli (lambs) randomly appearing and disappearing in a 2×2, 3×3, and 4×4 task grid matrix, respectively. Participants have 30 seconds to select the square where the lamb previously appeared via touch.

[0100] The fourth difficulty level: This difficulty level is executed after the subject has completed the first three difficulty levels. It is configured to recall the memory task of the first difficulty level (2×2 matrix). Subjects are required to select the location of the sheep in this level within 30 seconds by touch without re-observing the sheep appear.

[0101] From the first to the third difficulty level, the task difficulty increases progressively with the size of the grid matrix (i.e., the probability of different positions). The fourth difficulty level is used to examine the subject's long-term memory recall ability. This level design allows the task to quantitatively assess the subject's spatial working memory ability.

[0102] At the system implementation level, the memory training submodule (Counting Sheep game) on the tablet is developed using Java and implemented with the Android SDK. It controls multiple difficulty levels through dynamically generated grid layouts and random event scheduling. The task scenario is hosted in an Android Activity, and the grid layout is generated by dynamically adding Buttons or ImageView controls. The system creates grid objects at runtime based on the current difficulty parameters.

[0103] The subject's click events are handled in the onClick callback. After each click, the system displays the square containing the stimulus, shows the comparison results, and ends the level. In the first difficulty level, the system additionally records the ID of the square containing the stimulus for reuse in the fourth difficulty level.

[0104] The timing module is implemented using CountDownTimer. Each difficulty level has a total time limit, and the countdown is updated in real time in the UI using a TextView. If the user completes the click within 30 seconds, the system records the "correct / incorrect" status and the remaining time; if the time limit expires before completion, the system automatically terminates the current level, records the "incomplete" status, and locks the interaction.

[0105] The difficulty switching logic is controlled by the state machine within the Activity. The fourth difficulty level reuses the grid creation method and the grid where the stimulus is located in the first difficulty, but the stimulus presentation is prohibited, and only blank grids are displayed, allowing the test taker to click based on memory.

[0106] Here is how to complete the jigsaw puzzle task:

[0107] This mission features two levels with increasing difficulty:

[0108] The first level of difficulty: the puzzle is divided into 2×2 pieces, and the test taker has a 7-second reaction time limit.

[0109] The second difficulty level: the puzzle is divided into 3×3 pieces, and the test taker has a 15-second reaction time limit.

[0110] The difficulty of the levels is reflected in the number of puzzle pieces and the reaction time limit, which is used to assess and train subjects' spatial cognition and graphic processing abilities under time pressure.

[0111] At the system implementation level, the spatial cognition training submodule (puzzle game) on the tablet is developed using Java and the Android SDK. It controls multiple difficulty levels through dynamically generated puzzle grids and timed scheduling. The task scenario is carried out by an Android Activity, and the puzzle area is dynamically generated using GridLayout. The system calculates the number of rows and columns of the grid based on the current difficulty parameter (2×2 or 3×3), and evenly cuts the preset source image into the corresponding number of tiles using Bitmap.createBitmap. All tiles are then shuffled and filled into the grid.

[0112] The interaction logic employs a click-and-swap mechanism: when a participant clicks on two tiles, the system swaps their positions within the grid and updates the corresponding display content. The system continuously monitors whether the current grid tile order matches the original order; if they do, the puzzle is considered complete.

[0113] The timing module is implemented using CountDownTimer. Each difficulty level has a corresponding reaction time limit, and the countdown is updated in real time in the UI using a TextView. If the subject completes the puzzle within the time limit, the system records the "complete" status and the remaining time; if the time limit expires before completion, the system automatically terminates the current level, records the "incomplete" status, and locks the interaction.

[0114] The difficulty switching logic is controlled by a state machine within the Activity. After completing the first difficulty level, the system automatically proceeds to the second difficulty level. If the test subject fails (times out) at either difficulty level, the system is configured to terminate the task.

[0115] Example 3:

[0116] In this embodiment, the living room scenario is primarily used to test comprehension, decision-making, attention, executive function, and spatial cognitive abilities. The assessment of the subject's cognitive abilities based on audiovisual memory and executive function tests provides a reliable basis for subsequent rehabilitation training. In this embodiment, the virtual testing scenario is the living room, and the training task is cleaning.

[0117] At the system implementation level, the cleaning training task is developed based on the Unity engine, with core logic written in C#. The task scene is hosted in Unity's 3D environment, and dynamic trajectory patterns are generated in real time through a script-controlled LineRenderer component. The system uses a random algorithm to call preset spiral, triangle, line segment, and letter combination shape data, dynamically creating and refreshing GameObject target objects at runtime.

[0118] The subject's interactive behavior is determined in real time using Unity's OnTriggerEnter collision detection mechanism. When the virtual rag or broom controlled by the subject completely overlaps with the target pattern collider, the system executes the Shader masking and erasing effect and the model gradient fading logic, and simultaneously triggers AudioSource to play feedback sound effects. The task's timing and calculation logic is implemented by Unity's coroutines, and the score is accumulated in real time by global variables in the script. When the cumulative number of patterns cleared in a single task reaches the preset level duration, the system immediately executes the TaskEnd function, automatically terminating the current process and locking user interaction.

[0119] The following is the process of carrying out the cleaning task:

[0120] (1) Task initialization and dynamic pattern generation stage: The system starts the living room test scene according to the preset parameters, which is mainly used to test the subject's comprehension, decision-making, attention, execution and spatial cognition abilities. At the start of the task, the system randomly generates dynamic patterns such as spirals, triangles and line segments on the desktop or ground scene, and requires the player to control the virtual rag or broom to move along the path according to the arrow guidance.

[0121] (2) Execution and Interaction Stage: The subject manipulates the tool to overlap with the pattern, causing the pattern to disappear in real time. The design uses gradual visual feedback (trajectory fading effect) and sound prompts to enhance the intuitiveness of the operation. If the subject exceeds the time limit or deviates from the path during the operation, recalibration is required.

[0122] (3) Tool switching and comprehensive training stage: The system combines diverse patterns including complex shapes such as letter combinations and introduces a tool switching mechanism (such as wiping the table with a rag and sweeping the floor with a broom) to construct a cognitive-motor comprehensive training module in a life-like scenario.

[0123] (4) Data statistics and analysis stage: Each subject accumulates 1 point when they completely clear the pattern in one go. The task ends when the accumulated score reaches 10 points. After the task is completed, the system automatically calculates the total time spent by the subject in completing the task, the score and the accuracy rate, and other objective behavioral characteristics data. It also records the subject's real-time physiological data during the execution process, including electrocardiogram and electroencephalogram indicators.

[0124] (5) Assessment Report Generation Stage: The system writes the statistical classification scores, task time, and real-time physiological data into the cloud database. Based on this multimodal data, combined with the multidimensional data analysis unit mentioned above, the probability of the subject having cognitive impairment is assessed, and compared with previous assessment data. Finally, the large language model automatically generates targeted intervention suggestions.

[0125] Example 4:

[0126] In this embodiment, the fruit platter-making scenario is primarily used to test executive function, logical abstraction, and audiovisual memory. It assesses the subject's cognitive abilities based on their attention, execution, comprehension, and decision-making capabilities, and is highly reliable for subsequent rehabilitation training. To enhance the system's age-friendly interactive feedback, the virtual test task in this embodiment is set in a kitchen refrigerator and stove. Users need to take a specified variety and quantity of fruit from the refrigerator and place them in a fruit platter to complete the task. The system scores the user's task completion and times the completion time. Different difficulty levels are set for subjects with varying cognitive abilities; higher difficulty tasks require more and more complex fruit varieties.

[0127] At the system implementation level, the fruit platter making training task is developed based on the Unity engine, with core logic written in C#. The task scene is hosted in Unity's 3D virtual kitchen environment, including interactive facilities such as refrigerators and stoves. The system uses a random algorithm to call a preset fruit resource database and dynamically generates target fruit combinations at runtime based on the level progress (2 types for the first level, 3 types for subsequent levels, etc.).

[0128] The subject's interactive behavior is determined using Unity's OnTriggerEnter collision detection. When the subject uses a virtual hand to grab fruit from the refrigerator and place it in the collision zone of the fruit plate, the system performs pose binding and logic verification. The timing and calculation of the task and the display logic of the dynamic tooltip (which automatically hides after an initial 5-second display) are implemented using Unity's coroutines. After the subject submits the plating result and it passes system verification, the TaskEnd function is executed to lock the interaction and unlock the next stage.

[0129] The following is the process of making the fruit platter:

[0130] (1) Task initialization and goal setting stage: The system starts the kitchen scene according to the preset parameters. This scene is mainly used to test the subject's execution ability, logical abstraction ability and audiovisual memory ability. When the task starts, the system randomly assigns the target fruit combination according to the level progress and displays the initial requirements on the built-in dynamic prompt board on the interface. After 5 seconds, it is automatically hidden to reduce the memory load.

[0131] (2) Execution and Interaction Phase: Subjects need to grab the specified variety and quantity of fruit from the refrigerator according to the target requirements (ensuring 1-2 pieces of each type) and place them in a fruit platter. After completing the platter, the player needs to submit it to the system for verification. If it meets the rules, the fruit platter will be sent to the living room and the next phase will be unlocked; if there are missing varieties or quantity deviations, real-time error correction feedback will be triggered.

[0132] (3) Stage Switching and Comprehensive Training Stage: To enhance ease of use and fun, the system adopts a tiered difficulty upgrade mechanism. As the levels increase, the types of fruits required become more complex. The design combines error-tolerant adjustment with intuitive interaction, aiming to enhance the age-friendly experience and help elderly users achieve cognitive training in gamified tasks.

[0133] (4) Data Statistics and Analysis Stage: After the task is completed, the system will automatically score the user's task completion and simultaneously record objective behavioral characteristic data such as the total time spent completing the task and the accuracy rate. At the same time, the system will collect the subject's physiological data during the execution of this task in real time through monitoring equipment, including key indicators such as electrocardiogram and electroencephalogram.

[0134] (5) Assessment Report Generation Stage: The system writes the statistical classification scores, task time, and real-time physiological data into the cloud database. Based on this multimodal data, combined with the multidimensional data analysis unit mentioned above, the probability of the subject having cognitive impairment is assessed, and compared with previous assessment data. Finally, the large language model automatically generates targeted intervention suggestions.

[0135] Example 5:

[0136] The tomato and egg stir-fry cooking scenario in this embodiment is primarily used to test the subjects' executive function, process comprehension, and audiovisual memory. By observing the subjects' operational sequence, task completion, and reaction time in the multi-step cooking task, the system comprehensively assesses their attentional execution ability, comprehension and decision-making ability, and short-term memory, which can be used for cognitive rehabilitation training in subsequent training phases. To enhance the task's operability and interest, the virtual test scenario in this embodiment is a kitchen environment, including functional modules such as a refrigerator area, a food preparation workbench area, and a stove cooking area. Subjects need to complete the cooking process of tomato and egg stir-fry under the guidance of system prompts. During the task execution, the system records and scores the user's operational process correctness, task completion rate, and completion time. Based on subjects with different cognitive levels, this embodiment sets tasks of varying difficulty levels. Higher-difficulty tasks will have fewer prompts or more operational steps to further examine the subjects' process memory and executive function.

[0137] At the system implementation level, the virtual kitchen environment is built on Unity's 3D scene system. Objects such as refrigerators, worktops, stoves, and cooking utensils are loaded as GameObjects and visualized and interacted with using Mesh Render and Collider components. Food objects such as eggs, tomatoes, knives, pots, and spoons are set as interactive objects and bound to RigidBody physics components to support grabbing, moving, and placing operations.

[0138] (1) Task initialization and goal setting stage:

[0139] The system activates a virtual kitchen scenario based on preset parameters. This scenario includes interactive areas such as a refrigerator, work surface, and stove, designed to simulate a real cooking environment and test the subject's executive function and audiovisual memory. At the start of the task, the system displays the task objective—to prepare scrambled eggs with tomatoes—to the subject via a virtual cue board. The cue board displays the required ingredients and basic preparation process, including the preparation of eggs and tomatoes, ingredient handling, and cooking steps. The cue information is displayed for 5 seconds and then automatically hides to assess the subject's ability to remember the task process.

[0140] (2) Execution and Interaction Phase:

[0141] Participants are required to complete the preparation and preliminary processing of ingredients in a virtual kitchen environment based on the prompts provided.

[0142] Furthermore, participants first needed to take the required ingredients, including eggs and tomatoes, from the refrigerator and place them on the work surface. Then, participants used virtual tools to process the ingredients, such as using a spoon to crack and whisk the eggs to form an egg mixture, and using a knife to cut the tomatoes into pieces.

[0143] (3) Cooking process execution stage:

[0144] After preparing the ingredients, participants were required to perform cooking operations at the stove area. Specific steps included adding a suitable amount of cooking oil to a pan, then pouring in the prepared tomatoes and beaten eggs in sequence and stir-frying to complete the initial cooking process. During this process, the system monitored the participant's operational sequence and whether key steps conformed to preset procedural rules. After the training task concluded, the system used the recorded operational data to count the number of correct actions performed by the participant during the task and calculated the proportion of correct actions to all actions, generating a task accuracy index.

[0145] (4) Task completion and result statistics stage:

[0146] After the cooking process is complete, participants can selectively add ingredients such as chopped green onions or cilantro before the dish is served to finish. Once these steps are completed, the system determines the task is finished. Statistical data includes behavioral characteristics such as task flow accuracy, total task time, and the number of erroneous operations.

[0147] (5) Data collection and evaluation analysis stage:

[0148] During task execution, the system collects real-time physiological data from the subjects using monitoring devices, including key indicators such as electroencephalogram (EEG) and electrocardiogram (ECG) signals. After the task is completed, the system records the task completion score, operation time, and real-time physiological data into its database, and combines this with a multi-dimensional data analysis unit to comprehensively assess the subjects' cognitive state. Through the joint analysis of behavioral and physiological data, the system can calculate the probability of cognitive impairment in the subjects and generate corresponding assessment results, providing a basis for subsequent cognitive training and intervention.

[0149] After completing multiple rehabilitation training tasks, the system will enter the multi-dimensional data in-depth analysis and feedback intervention stage. This embodiment mainly details the complete closed-loop process of how the system uniformly uploads and stores data, assesses cognitive status, and generates personalized intervention suggestions after the subject completes all or part of the aforementioned rehabilitation training sequence.

[0150] Example 6:

[0151] (1) During task execution, the system temporarily stores task performance (time consumption, accuracy, etc.) locally and collects EEG and ECG signals simultaneously. After all training is completed, the information storage module starts the "caching + batch writing" mechanism to upload behavioral data and physiological data to the cloud database in a unified manner.

[0152] (2) After the data is uploaded, the model inference is started and three types of data are processed in parallel: one-hot encoding and standardization of personal information to generate background prior vectors; Min-Max standardization of task performance to generate behavioral feature vectors; and deep convolutional networks with self-attention mechanism are used to process brain / ECG signals, filter out noise and extract fixed-dimensional physiological feature vectors.

[0153] (3) The three vectors are linearly concatenated along the feature dimension to form a global fusion feature vector. This vector is then input into a fully connected meta-classifier with a sigmoid top layer, mapped to the [0,1] interval, and outputs the probability value of cognitive impairment. This fusion mechanism effectively solves the problem of insufficient specificity of a single physiological feature, achieving accurate early warning.

[0154] (4) Using the current probability value of cognitive impairment, the historical assessment difference threshold, and the multidimensional physiological feature vector as joint inputs, the large language model is invoked. The model automatically generates structured intervention suggestions that include risk trends, physiological feedback, and adaptive adjustment instructions for training parameters.

[0155] (5) The assessment report and recommendations are sent to the terminal for medical staff and family members to review, and are also archived as past data. In subsequent rehabilitation cycles, the changes in the patient's cognitive state are compared through reassessment (step S6) to achieve dynamic adjustment of system parameters and self-optimization of rehabilitation training in a closed loop.

[0156] This invention provides a multi-platform, age-friendly early AD rehabilitation system based on virtual reality. By combining virtual reality gesture interaction technology with multiple platforms and model generation, it effectively solves the problems of traditional virtual reality systems, such as high operational barriers, insufficient age-friendliness, the tendency of single immersive devices to cause motion sickness and limited usage scenarios, and the lack of personalized dynamic adjustments in traditional rehabilitation programs. By capturing user gestures, it reduces the learning cost for the elderly; by utilizing the multi-platform collaboration between virtual reality and tablets, it effectively expands the system's applicable population and ensures the continuity of rehabilitation training; simultaneously, through the cascading fusion of multimodal features and intelligent analysis of large language models, it achieves accurate early warning of early AD tendencies in subjects and adaptive rehabilitation guidance. This invention constructs a complete technical system from "multi-dimensional data acquisition" to "state assessment" and then to "personalized closed-loop intervention," significantly enhancing the system's clinical practical value and its prospects for age-friendly promotion.

[0157] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-platform age-friendly early AD rehabilitation system based on virtual reality, characterized in that: It includes a user login unit, a virtual reality interaction unit, a multidimensional data analysis unit, and an information management unit; wherein, the user login unit is used for selecting, inputting, and verifying the identity of the subject; the virtual reality interaction unit is used for providing the subject with immersive cognitive training and operational interaction based on a virtual reality environment; the multidimensional data analysis unit is used for processing and analyzing the multidimensional data of the subject; and the information management unit is used for collecting, storing, and retrieving multidimensional user information.

2. The multi-platform age-friendly early AD rehabilitation system based on virtual reality according to claim 1, characterized in that, The user login unit is further configured to retrieve and associate the corresponding individual profile and historical training data after successful login, in order to support subsequent evaluation, training and result display processes.

3. The multi-platform age-friendly early AD rehabilitation system based on virtual reality according to claim 2, characterized in that, The virtual reality interaction unit includes an adaptation module, a task scheduling module, a device module, and a cognitive rehabilitation training module. The adaptation module detects eye movement signals in meditation and parent-child companionship scenarios and uses the eye movement data acquisition and analysis subsystem to adaptively select the virtual reality or tablet training mode based on four characteristic indicators: converted fixation point coordinate sequence, saccade speed, blink frequency, and pupil diameter. The task scheduling module provides free mode and story mode, allowing selection of different cognitive domain labeled tasks according to needs. The device module manages virtual reality, detection, assistive, and tablet devices. The cognitive rehabilitation training module manages various rehabilitation scenario tasks.

4. The multi-platform age-friendly early AD rehabilitation system based on virtual reality according to claim 3, characterized in that, The multidimensional data analysis unit includes a model training module, a cognitive assessment module, and an intelligent suggestion generation module. The model training module uses the subjects' multidimensional data (including personal information, MMSE scores, task performance, and physiological data) to train the cognitive assessment model, labeling those with MMSE scores less than 28 as having cognitive impairment and those with scores greater than or equal to 28 as having no impairment. The cognitive assessment module inputs multidimensional features and outputs the probability of cognitive impairment for early AD detection. The intelligent suggestion generation module generates intervention suggestions based on the probability of cognitive impairment, historical assessment differences, and physiological characteristics.

5. The multi-platform age-friendly early AD rehabilitation system based on virtual reality according to claim 4, characterized in that, The information management unit includes an information acquisition module, an information storage module, and an information retrieval module. The information acquisition module is configured to collect multi-dimensional user information, including user personal information, task performance data, and physiological signal data. User personal information is collected when the user logs in for the first time, task performance data is collected and temporarily stored locally after each task is completed, and physiological signal data is collected by monitoring devices during task execution. All data is written to the cloud database after the subject completes all training tasks. The information storage module is configured to store multi-dimensional user information, including subject personal information, task performance data, physiological data and previous evaluation results, which are then uploaded to the cloud database after the subject completes all training tasks. The information retrieval module is configured to connect cloud data storage and multidimensional data analysis unit.

6. A multi-platform age-friendly early AD rehabilitation method based on virtual reality, applied to the system described in claims 1 to 5, characterized in that, The method includes user login, virtual reality interaction, multidimensional data analysis, and information management steps; wherein, the login step verifies identity and associates with individual profiles; the interaction step provides immersive cognitive training; the analysis step processes data and outputs cognitive assessments and suggestions; and the information management step collects, stores, and retrieves data.

7. The multi-platform age-friendly early AD rehabilitation method based on virtual reality according to claim 6, characterized in that, The virtual reality interaction steps include adaptation steps, task scheduling steps, device management steps, and cognitive rehabilitation training steps. The adaptation step involves detecting the subject's electroencephalogram (EEG) and electrocardiogram (ECG) signals through meditation and parent-child companionship scenarios before training begins, and adaptively selecting either virtual reality rehabilitation training mode or tablet-based rehabilitation training mode based on the detection results. The task scheduling step provides two task scheduling methods: free mode and scenario mode. In the free mode, subjects can freely select tasks through cognitive domain labels. In the scenario mode, subjects' current cognitive load is assessed based on their EEG and ECG signals, and tasks with different difficulty levels and different cognitive domain labels are dynamically selected. The equipment management steps manage various devices required by the system, including virtual reality devices, detection devices, auxiliary devices, and tablet devices; The cognitive rehabilitation training steps provide a variety of scenario tasks for the rehabilitation of early AD patients, including tablet scenarios, home cooking scenarios, living room entertaining scenarios, and fruit platter making scenarios.

8. The multi-platform age-friendly early AD rehabilitation method based on virtual reality according to claim 7, characterized in that, The multidimensional data analysis steps include a model training step, a cognitive assessment step, and an intelligent suggestion generation step. The model training step uses collected and processed multidimensional subject data to train the cognitive assessment model. This multidimensional data includes subject personal information, MMSE scale scores, task performance, and physiological data. During training, the MMSE scale is used to generate corresponding data, and other multidimensional data are used as multidimensional features in the training. The cognitive assessment step inputs the multidimensional features of the subjects collected in the information management step into the trained cognitive assessment model, outputting a probability value of cognitive impairment in the subject, enabling early detection and warning of AD patients, and monitoring the degree of cognitive impairment in subjects during long-term training based on past data. The intelligent suggestion generation step uses the probability value of cognitive impairment, historical assessment difference thresholds, and multidimensional physiological feature vectors as input parameters to call a large language model to generate structured intervention suggestions.

9. The multi-platform age-friendly early AD rehabilitation method based on virtual reality according to claim 8, characterized in that, The cognitive assessment model is a deep learning-based cognitive impairment probability assessment model, and its processing includes three steps. The first step involves parallel feature extraction and preprocessing of three types of input data—personal information of the subject, subject's task performance, and subject's physiological data—using a multimodal feature extraction subsystem to generate background prior vectors, behavioral feature vectors, and physiological feature vectors, respectively. The second step involves linearly cascading the background prior vectors, behavioral feature vectors, and physiological feature vectors along the feature dimension using a cascaded fusion subsystem to form a global fusion feature vector. The third step involves using a meta-classifier composed of a fully connected network in the probability output and decision system to map the fusion features to the [0,1] interval and output the probability value of the subject having cognitive impairment.

10. The multi-platform age-friendly early AD rehabilitation method based on virtual reality according to claim 9, characterized in that, The information management steps include information collection, information storage, and information retrieval. The information collection step collects user information, including personal information, task performance data, and physiological signal data. Personal information is collected when the user logs in for the first time, task performance data is collected and temporarily stored locally after each task is completed, and physiological signal data is collected by the monitoring device during task execution. All data is written to the cloud database after the user completes all tasks. The information storage step stores multi-dimensional user information, including subject personal information, task performance data, physiological data, and previous evaluation results. It adopts an asynchronous persistence mechanism of "caching and batch writing". The structured data is first stored in the memory buffer and then uploaded to the cloud database after the subject finishes all training tasks. The information retrieval step is based on a MySQL cloud database and performs index queries through key fields to provide the necessary data for the model training and cognitive evaluation steps.

Images