Alzheimer risk home monitoring method and system based on multi-modal behavior analysis

Abstract

The invention belongs to the technical field of health monitoring, and particularly relates to a senile dementia risk home monitoring method based on multi-modal behavior analysis, which comprises the following specific steps: monitoring human body postures in real time, analyzing repeated actions, synchronously early warning abnormal movement, and carrying out voice interaction operation through a built-in AI dialogue terminal; performing behavior, emotion and physiology correlation modeling, executing cognitive decline dynamic evaluation through cognitive function self-test, and judging a risk level according to an analysis and evaluation result; executing a multi-level early warning mechanism based on the risk level information, and automatically taking corresponding emergency measures; the threshold value is dynamically calibrated by integrating user habit data, a closed-loop feedback mechanism is designed to realize false alarm suppression, and a monthly health report is synchronously generated to assist a doctor in adjusting an intervention scheme. The problems that in a traditional monitoring mode, tiny behavior changes are difficult to capture, rescue is delayed due to subjective evaluation hysteresis and passive response, the screening frequency is low, and the scene is limited can be solved.

Description

Technical Field

[0001] This invention belongs to the field of health monitoring technology, specifically relating to a home monitoring method and system for Alzheimer's disease risk based on multimodal behavior analysis. Background Technology

[0002] Home-based risk monitoring technology for Alzheimer's disease mainly focuses on dimensions such as behavioral abnormalities, physiological indicators, and environmental risks. It achieves early warning through non-invasive sensing and intelligent analysis technologies. Mature solutions that have been implemented cover multiple aspects such as behavior recognition, location tracking, and ECG / sleep monitoring. They do not require active operation by patients and are adapted to the cognitive decline characteristics of Alzheimer's patients. They enable real-time warnings and long-term behavioral data tracking to assist family members and doctors in assessing the condition.

[0003] Problems with existing technology: Subtle behavioral changes are difficult to detect: Traditional monitoring is limited by relying on manual observation or simple cameras, making it difficult to identify early symptoms such as repetitive actions (e.g., repeatedly adding salt) and disorientation (getting lost), resulting in a high rate of missed detection. Subjective assessment lag: Traditional monitoring is limited by its reliance on subjective feedback from family members, making it difficult to capture psychological changes such as paranoia and emotional detachment in a timely manner; Passive response leads to rescue delays: Traditional monitoring is limited by the fact that the emergency button needs to be actively triggered, and cannot be operated after a fall; single sensors are easily affected by environmental interference; Low screening frequency and limited scenarios: Traditional monitoring is limited by the need for medical staff to come to the door or to the hospital for testing, which takes a long time and the results are affected by environmental stress. Summary of the Invention

[0004] The purpose of this invention is to provide a home-based monitoring method and system for Alzheimer's disease risk based on multimodal behavioral analysis, which can solve the problems of traditional monitoring methods, such as difficulty in capturing subtle behavioral changes, lag in subjective assessment, delays in rescue due to passive response, low screening frequency, and limited scenarios.

[0005] The specific technical solution adopted by this invention is as follows: The specific steps of the home-based monitoring method for Alzheimer's disease risk based on multimodal behavioral analysis are as follows: It can monitor human posture in real time, analyze repetitive movements, orientation disorders and decline in daily living skills, and simultaneously issue early warnings for abnormal movements, monitor gait stability and heart rate variability. It also continuously collects blood oxygen saturation and skin conductance response, and can also perform voice interaction operations through a built-in AI dialogue terminal. After initial screening at the edge, the data undergoes multimodal fusion analysis in the cloud, and behavioral, emotional, and physiological correlation models are constructed. In addition, a cognitive decline dynamic assessment is performed through cognitive function self-testing, and the risk level is determined based on the analysis and assessment results. Based on risk level information, a multi-level early warning mechanism is implemented to automatically take corresponding emergency measures. In addition, an active intervention mechanism is implemented to assist in cognitive function self-testing and home safety monitoring. By integrating user habit data to dynamically calibrate thresholds and designing a closed-loop feedback mechanism to suppress false alarms, monthly health reports are generated simultaneously to assist doctors in adjusting intervention plans.

[0006] When using the aforementioned monitoring method to perform daily behavioral anomaly monitoring, the specific steps are as follows: It integrates visual sensing, environmental assistance, and physiological parameters to collect multi-source data, including capturing three-dimensional spatial movements, tracking human posture in real time, combining smart home device logs to help judge spatial orientation anomalies, and cross-validating with wearable devices and video stream data. Analyze action sequences to detect abnormal loop patterns, track movement trajectories to identify "repeated wandering" behavior, construct a "human-space" relationship model to determine orientation impairment, analyze object misuse behavior to detect life skills decline; extract dynamic risk features and dynamically update risk probabilities; A multi-level early warning mechanism is adopted to conduct risk assessments, determine low, medium, and high risk levels, and take emergency measures accordingly. Integrate user habit data to perform personalized threshold adjustments; generate monthly behavior reports and mark abnormal frequency trend lines.

[0007] When using the aforementioned monitoring method to perform daily behavioral anomaly monitoring, the specific steps are as follows: First, multi-source data collection is carried out by integrating voice analysis, facial expression recognition and physiological parameters. Specifically, this includes locking the target user to capture the dialogue content, collecting facial micro-expressions, and simultaneously monitoring heart rate variability and skin conductance response through wearable devices to capture physiological stress signals during emotional fluctuations. Voice features are input into an NLP model to identify emotional states, and physiological and behavioral data are cross-validated to verify the authenticity of emotions. Abnormal patterns are identified by constructing a "behavior-time-space" relationship graph. Smart home logs and UWB positioning data are analyzed simultaneously to quantify social withdrawal. Abnormal behaviors are matched by comparing them with typical symptom features.

[0008] Risk assessment is conducted based on the aforementioned abnormal behaviors, including low risk when there is a single instance of low mood, medium risk when there is social withdrawal and anxiety index > threshold for 3 consecutive days, and high risk when there is aggressive behavior and self-harm tendencies. The alarm threshold is dynamically calibrated based on baseline personality, and the probability is dynamically updated to generate a monthly mood fluctuation heatmap, marking personality change nodes.

[0009] The specific steps for using the aforementioned monitoring method to perform home safety monitoring are as follows: It tracks the position of the human body's center of gravity, movement trajectory and posture changes in real time, combines environmental perception to detect slippery ground and obstacles, and marks high-risk areas. Wearable devices are used to monitor the center of gravity shift and gait imbalance precursors, and physiological parameters are combined to capture the physiological stress response at the moment of fall. It identifies "sudden fall and stillness" actions, and simultaneously uses wearable devices to detect weightlessness acceleration and abnormal heart rate to trigger a secondary verification mechanism, while simultaneously using algorithms to eliminate false alarm scenarios.

[0010] The home safety monitoring system indicates low risk when an environmental warning is issued, medium risk when abnormal posture occurs but physiological parameters remain stable, and high risk when sudden posture changes, physiological stress, or no response occurs. Simultaneously execute local intervention operations such as voice prompts and emergency lighting; Adaptive optimization is achieved based on user habits, and multi-user data is integrated to improve the recognition accuracy in complex scenes.

[0011] The specific steps for using the aforementioned monitoring method to perform a self-test of cognitive function are as follows: The system identifies target users and collects their voice information. It also combines visual devices to provide feedback on users' micro-expressions and body movements, and simultaneously analyzes the degree of attention distraction. Execute core test projects, integrate three-word recall error rate, calculate number of interruptions, response delay characteristics, output cognitive risk level, update risk probability, and mark key indicator changes based on historical monthly cognitive ability trajectory reports; Low risk is indicated by missing ≤1 word recall but no calculation errors; medium risk is indicated by missing ≥2 words or making ≥2 calculation errors; high risk is indicated by being unable to complete any test item; local intervention operations such as simultaneous voice encouragement and push brain training games are performed. The test difficulty is dynamically adjusted based on baseline ability, and a dynamic error tolerance mechanism is set up; a cognitive ability curve is generated every month, and abnormal fluctuation nodes are marked.

[0012] The core test items include: The three-word recall test is conducted as follows: The system first reads three unrelated words at a slow pace and asks the elderly to repeat them immediately. The accuracy of the repetition and the response delay are analyzed. The elderly are then guided to engage in other activities for 5 minutes. After 5 minutes, the elderly are asked to repeat the words. The system records the number of omissions and incorrect substitutions, detects and assesses the degree of memory decline, and analyzes the completeness of the first repetition. The test involved subtracting 7 consecutively from 100. The elderly person answered verbally, and the speech recognition engine transcribed the answers into text in real time and verified the answers. Abnormal behavior was captured simultaneously.

[0013] A home-based monitoring system for Alzheimer's disease risk based on multimodal behavior analysis, comprising four layers: perception layer, edge computing layer, platform layer, and application layer. The sensing layer is used for data acquisition and includes a millimeter-wave radar array deployed on the ceiling / corner of the wall, a multispectral camera with infrared illumination, a wearable device with a built-in IMU sensor, a smart ring that measures blood oxygen and skin conductance, and environmental sensors. The edge computing layer is used for real-time data processing and is equipped with a lightweight AI model and a multimodal data fusion engine. The platform layer is used for data analysis and storage, and it is deployed with multi-source databases and intelligent analysis engines. The application layer provides functional services and has four core functional modules: First, a daily behavior monitoring module, which identifies repetitive actions / life skill decline through a behavior time series analyzer; second, an emotion and personality analysis module, which detects paranoia and emotional apathy through a multimodal emotion computing engine; third, a home safety monitoring module, which uses a fall prediction model and an active intervention unit, combined with millimeter-wave radar for real-time early warning, triggering voice reminders / automatic lighting; and fourth, a cognitive self-testing interaction module, which uses an integrated NLP dialogue robot to perform three-word recall tests and continuous subtraction of 7 operations to assess memory decline. Furthermore, a hierarchical response system and an adaptive threshold management system have also been developed.

[0014] The multi-source database stores corresponding action frequency and path trajectory content according to the data type of behavior time sequence, which is used to identify repetitive actions / orientation disorders; it stores corresponding HRV, GSR, and sleep quality content according to the data type of physiological parameters, which is used to identify emotional apathy and anxiety analysis; and it stores corresponding water and electricity usage and location records according to the data type of environmental logs, which is used to assist in the analysis of abnormal behavior. The intelligent analysis engine analyzes abnormal behavior sequences using an LSTM+Transformer model, constructs "human-object-space" relationships using a GNN graph model, detects misuse of items, and updates disease probability based on historical data using a Bayesian dynamic network.

[0015] The technical effects achieved by this invention are as follows: This invention employs multimodal perception fusion, combining depth cameras (skeletal joint tracking), UWB positioning (spatial path analysis), and smart home logs (item misuse detection) to achieve accurate identification of abnormal behavior. It can also dynamically adjust alarm thresholds and automatically optimize alarm rules based on user habits (such as increasing the judgment duration for slow walking speed) to reduce false alarms.

[0016] By using voice analysis (dialect NLP model to identify repetitive questions / semantic confusion), infrared cameras to capture micro-expressions (drooping corners of the mouth / dull eyes), and wearable devices to monitor physiological indicators (heart rate variability reflects anxiety), multi-dimensional emotion quantification is achieved. In addition, by integrating smart access control records and community activity data, the system can identify trends in declining social frequency and provide early warnings for social withdrawal.

[0017] This invention employs millimeter-wave radar to penetrate clothing / obstacles and monitor sudden changes in the center of gravity in real time, lidar to construct a 3D environmental map, an environmental sensor array to identify slippery ground / obstacles, multi-source cross-verification to prevent false alarms, and also achieves automated risk warning, thus shortening rescue delays.

[0018] This invention also features a natural conversational assessment function, including a three-word recall test and continuous subtraction of 7 mental arithmetic. By analyzing error rates and response delays in real time, it achieves high-frequency automatic screening without any scene or time limit. In addition, it can generate cognitive ability trend charts based on historical test data to identify minor declines. Attached Figure Description

[0019] Figure 1 This is a flowchart of the monitoring method provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of the monitoring system provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0021] like Figure 1 As shown, the home-based monitoring method for Alzheimer's disease risk based on multimodal behavioral analysis includes the following specific steps: As part of the multi-source data collaborative acquisition and preprocessing stage in step S01 of this method, millimeter-wave radar and lidar deployed in high-risk areas such as bathrooms and corridors are used to generate 3D point cloud data in real time to monitor human posture and identify signs of impending falls. RGB-D cameras are deployed to capture the coordinates of skeletal joints and analyze repetitive movements (such as repeatedly opening and closing the refrigerator in the kitchen), orientation disorders (wandering in the living room for a long time), and decline in daily living skills (misuse of objects). Furthermore, it combines UWB / Bluetooth beacons to locate the elderly person's position in real time, and uses electronic fence technology to warn of abnormal movement (such as leaving home at night); it also uses wearable devices, such as IMU sensors (wristbands / badges), to monitor gait stability and heart rate variability (HRV) to help verify fall events and emotional fluctuations. It also includes wearable smart rings to continuously collect blood oxygen saturation and skin conductance response (GSR) and associate them with emotional stress states (such as a sudden increase in GSR during anxiety). Furthermore, at the user interaction device layer, a microphone array is used to collect voice content, identify repetitive questions or aggressive language, and can also use voiceprint to lock onto target users. The dialect database is adapted to regional differences. The built-in AI dialogue terminal has a cognitive self-test module (three-word recall, continuous subtraction of 7 test) to reduce the operation threshold through voice interaction.

[0022] As part of the edge-cloud intelligent analysis and risk assessment stage in step S02 of this method, multi-source data is first screened in real time at the edge. Among them, fall detection related data uses millimeter-wave radar data to trigger an early warning within 10 seconds through the TinyLSTM model and intervenes through local voice reminders. Repetitive action recognition data is analyzed by LSTM to detect the frequency of action sequences (such as opening the refrigerator ≥ 5 times within 10 minutes) and local alarms are pushed in real time. Furthermore, cloud-based multimodal fusion analysis is performed, prioritizing behavioral-emotional-physiological correlation modeling. For example, after fusing features such as voice analysis ("suspected theft" keyword), abnormal object interaction (repeatedly hiding the remote control), and sudden increase in heart rate, it is determined to be the risk type of "persecution complex"; after fusing features such as facial expression recognition (frequency of drooping corners of the mouth), social data (no phone calls for 3 consecutive days), and reduced activity range, it is determined to be the risk type of "emotional apathy"; after fusing features such as falls, disorientation, and emotional anxiety, it is determined to be a complex high-risk event, triggering a level-three emergency response. Furthermore, a dynamic assessment of cognitive decline was performed using a three-word recall test. When the vocabulary omission rate in speech recognition retelling was >30%, the risk of memory decline was marked. Additionally, a continuous subtraction of 7 test was used. When the error rate was >50% or the time taken was >2 minutes, the probability of associated prefrontal cortex function decline was marked. Furthermore, a Bayesian network dynamic scoring method is adopted, which specifically requires integrating historical behavioral data to output a daily risk index.

[0023] As the graded response and active intervention stage in step S03 of this method, a multi-level early warning mechanism is mainly adopted, specifically divided into: when a single abnormality occurs (such as accidentally adding salt once), it is judged as low risk, and measures such as device voice reminders and playing soothing music are taken; when the average number of abnormalities per day is ≥3 (such as repeated inquiries), it is judged as medium risk, and measures such as APP push notifications to family members and telephone follow-ups by community doctors are taken; when abnormalities such as unresponsiveness to falls, delusions of persecution, and disorientation occur, it is judged as high risk, and measures such as calling emergency contacts and coordinating with the community to visit the home are taken. Furthermore, a cognitive training proactive intervention mechanism is adopted, which pushes personalized brain training games (such as number matching and memory cards) through the system when self-testing reveals a decline in computing ability. It also includes using AI robots to initiate virtual social scenarios (such as VR recall therapy) when emotion analysis indicates depressive tendencies.

[0024] As the system adaptive optimization and long-term management stage in step S04 of this method, it is mainly based on the federated learning framework and integrates user habit data for dynamic calibration of thresholds. For example, for elderly people with slow walking speed, the gait abnormality judgment time can be extended from 5 seconds to 8 seconds; for elderly people with introverted personalities, the social frequency threshold can be lowered by 30%. Furthermore, a closed-loop feedback mechanism is designed to suppress false alarms. For example, when the camera detects "misaligned clothing," the IMU needs to verify the continuity of the action (excluding temporary adjustments). Furthermore, monthly health reports are generated, including behavioral trend charts and heatmaps of abnormal events, to help doctors adjust intervention plans.

[0025] As an optional embodiment, the specific steps for performing the daily behavior anomaly monitoring function using this method are as follows: As part of the image acquisition and data preprocessing stage of this daily behavior anomaly monitoring, the system first integrates visual sensing, environmental assistance, and physiological parameters to collect multi-source data. This mainly includes using millimeter-wave radar (with strong penetration) and RGB-D cameras (such as Kinect) to simultaneously capture three-dimensional spatial movements, and using skeletal key point modeling to track human posture in real time. It also includes deploying a UWB / Bluetooth beacon positioning system, combined with smart home device logs (such as refrigerator opening and closing frequency) to assist in judging spatial orientation anomalies. At the same time, wearable devices (IMU sensors) are used to monitor gait stability and cross-verify fall risk with video stream data. As the intelligent analysis stage of abnormal behavior monitoring in this daily behavior anomaly monitoring, the first step is to use a multimodal algorithm model for analysis. This includes repetitive action recognition, which mainly uses an LSTM network to analyze action sequences (such as kitchen operations) and detect abnormal cyclical patterns (such as opening and closing the refrigerator ≥ 5 times within 10 minutes). An optical flow method is also designed to track movement trajectories and identify "repeated wandering" behavior (path overlap rate > 80% triggers an alert). It also includes orientation disorder determination, which uses a graph neural network (GNN) to build a "human-space" relationship model. When someone stays in a familiar area (such as the living room) for an extended period (> 15 minutes) and their movement path is chaotic, they are judged as lost. A life skills regression detection is also designed, which uses a YOLOv7 model to identify misuse of items (such as dish soap appearing in the toothbrush area) and combines action sequence analysis to confirm the misuse behavior. Furthermore, dynamic risk features are extracted, with key indicators including: frequency of action repetition, spatial positioning deviation, item misuse rate, and speech repetition. These indicators are input into a random forest classifier. Simultaneously, a Bayesian network is used to dynamically update the risk probability (e.g., if disorientation occurs for 3 consecutive days, the probability of illness increases by 20%). As the risk assessment and graded alarm stage in this daily abnormal behavior monitoring, a multi-level early warning mechanism is adopted for risk assessment, specifically divided into: when a single abnormality occurs (such as accidentally adding salt once), it is judged as low risk, and measures such as device voice reminders and playing soothing music are taken; when the average number of abnormalities per day is ≥3 (such as repeated inquiries), it is judged as medium risk, and measures such as APP push notifications to family members and telephone follow-ups by community doctors are taken; when a complex abnormality occurs (getting lost + misuse of items), it is judged as high risk, and measures such as calling emergency contacts, coordinating with the community to visit the home, and calling 120 if necessary; It should be further explained that a false alarm suppression mechanism is also designed. For example, when the camera detects "misaligned clothing", IMU data is required to verify the continuity of the action (excluding temporary clothing adjustments); normal behaviors such as getting up to drink water at night are ignored, and high-risk scenarios (such as squatting in the bathroom for more than 90 minutes) are focused on. As part of the dynamic threshold update and system optimization phase of this daily behavior abnormality monitoring, it includes the execution of personalized threshold adjustments. The initial threshold is set based on preset standard values ​​of clinical data (e.g., a walking speed of <0.4m / s is used to determine gait abnormality). Through an adaptive learning mechanism, user habit data (e.g., the elderly move slowly in daily life) is integrated into the federated learning framework to automatically increase the threshold for determining gait stagnation duration. In addition, the model parameters can be dynamically optimized according to the false alarm rate (e.g., the sensor sensitivity is reduced when the false alarm rate is >10%). Furthermore, long-term risk trend management is implemented, which mainly involves generating monthly behavior reports, marking abnormal frequency trend lines, and assisting doctors in adjusting intervention plans; at the same time, it combines urine biomarker data (such as AD7c-NTP concentration) to calibrate behavioral risk models.

[0026] As an optional second embodiment, the specific steps for performing the emotion and personality change recording function using this method are as follows: As the audio-visual information acquisition and preprocessing stage in this emotional and personality change recording, the first step is to integrate speech analysis, facial expression recognition, and physiological parameters for multi-source data acquisition. This mainly includes using a microphone array to capture dialogue content, using voiceprint recognition to identify the target user, and combining dialect recognition technology (such as incremental training of the Wuhan dialect database) to extract speech features (tone, speech rate, repetition frequency). It also includes using an infrared multispectral camera to capture facial micro-expressions (such as drooping corners of the mouth and dull eyes), converting them into an emotion vector with 17 key point coordinates at the local edge node (avoiding the storage of raw images). At the same time, wearable devices (smart bracelets / rings) are used to simultaneously monitor heart rate variability (HRV) and skin conductance response (GSR) to capture physiological stress signals during emotional fluctuations (such as a sudden increase in heart rate due to fear). As the intelligent analysis and feature extraction stage in this emotional and personality change record, the main aspects of emotional state recognition include: Voice features are input into an NLP model to identify semantic confusion (such as repeated accusations of "family members stealing") or emotional apathy (short and indifferent responses). Facial key point data are input into a lightweight CNN model to output emotion labels (anxiety / depression / apathy). Physiological data (such as abnormal fluctuations in HRV) and behavioral data (such as a sudden decrease in social activity) are cross-validated to verify the authenticity of the emotions. Furthermore, the detection of personality changes mainly includes: using graph neural networks (GNNs) to construct a "behavior-time-space" relationship graph to identify abnormal patterns (such as a mild-mannered person suddenly exhibiting aggressive behavior); and simultaneously analyzing smart home logs (such as access control records) and UWB location data to quantify social withdrawal (such as not leaving the room for three consecutive days). Furthermore, abnormal behavior patterns are matched against typical symptom features in a clinical database. For example, when the words "steal" and "persecution" appear frequently in speech, accompanied by an alert expression, it is marked as "persecution complex"; when facial expressions are lacking, speech response delay is greater than 5 seconds, and daytime activity level decreases by 40%, it is marked as "emotional blunting"; when the IMU sensor captures a sudden arm-waving motion and simultaneously detects the speech anger index, aggressive behavior is marked as "emotional blunting". As a dynamic risk assessment and alarm stage in the recording of emotional and personality changes, a multi-level early warning mechanism is also used for risk assessment, specifically divided into: when a single episode of low mood occurs (such as silence for more than 1 hour), it is judged as low risk; when social withdrawal occurs for 3 consecutive days and the anxiety index is greater than the threshold, it is judged as medium risk; when aggressive behavior or self-harm tendencies occur (such as banging one's head against a wall), it is judged as high risk. The coping measures for each level correspond to Example 1. It should be further noted that the same false alarm suppression mechanism is also implemented. For example, when the voice analysis suggests depression, IMU data is required to verify that the activity level decreases synchronously. As part of the dynamic updating and long-term management phase of this daily abnormal behavior monitoring, the system includes dynamically calibrating alarm thresholds based on baseline personality; employing a probabilistic dynamic update mechanism, utilizing a Bayesian network to detect paranoid delusions for three consecutive days, increasing the probability of developing the condition by 15%; and integrating long-term trends to generate a monthly mood fluctuation heatmap, marking personality mutation nodes (such as a 30% weekly increase in the frequency of irritability).

[0027] As an optional third embodiment, the specific steps for performing home safety monitoring using this method are as follows: As the real-time acquisition stage of multi-source data in this home safety monitoring, it mainly utilizes millimeter-wave radar for monitoring and wearable devices for verification. Specifically, it involves emitting electromagnetic waves through a 60GHz band millimeter-wave radar (such as Quectel RD6000CC), generating 3D point cloud data through echo signals, tracking the center of gravity position, movement trajectory and posture changes (such as sudden falls, abnormal stillness) of a person in real time, and combining environmental perception to detect slippery ground and obstacles (such as water accumulation in the bathroom, debris at night), and marking high-risk areas. Furthermore, the auxiliary verification method uses the accelerometer and gyroscope built into the wristband / badge to monitor the center of gravity shift and gait imbalance precursors (such as a sudden drop in walking speed and a body tilt angle >30°). Combined with physiological parameters, such as real-time monitoring of heart rate variability (HRV) and respiratory rate by smart rings, it captures the physiological stress response at the moment of fall (such as a sudden increase in heart rate >50%). As part of the collaborative analysis phase of home safety monitoring, multimodal feature fusion is mainly used for analysis. Specifically, this includes identifying "sudden fall + stationary" actions using millimeter-wave radar, while simultaneously detecting "weightlessness acceleration > 3g + abnormal heart rate" using wearable devices, triggering a secondary verification mechanism. Simultaneously, algorithms are used to eliminate false alarm scenarios (such as sitting down slowly or pet interference). Specifically, point cloud density analysis is used to determine the continuity of the action, and IMU data is combined to verify whether it is accompanied by violent shaking. As a graded response and proactive intervention phase in home safety monitoring, a multi-level early warning mechanism is also used for risk assessment, specifically divided into: when an environmental warning occurs (slippery ground + obstacles), it is judged as low risk; when abnormal posture occurs but physiological parameters are stable, it is judged as medium risk; when sudden posture changes, physiological stress, or no response (>30 seconds) occur, it is judged as high risk. Furthermore, real-time local intervention measures are adopted simultaneously. For example, when a slippery floor or obstacle is detected, the system automatically plays a warning such as "Caution: Slippery floor, please hold on to the handrail"; and at night, automatic lighting is triggered (such as gradually brightening the lights in the restrooms and corridors) to reduce the risk of falling in dark environments. As a closed-loop optimization and false alarm suppression stage in this home safety monitoring, adaptive optimization is achieved based on user habits, such as increasing the "stillness judgment time" (from 30 seconds to 60 seconds) for elderly people with slow movements; federated learning is used to integrate multi-user data to improve the recognition accuracy in complex scenarios (such as multiple people living together); and a false alarm cross-validation method is adopted at the same time. For example, after the millimeter-wave radar detects a fall, the wearable device needs to confirm "continuous no body movement" or "abnormal heart rate", otherwise it will be automatically downgraded to a false alarm.

[0028] As an optional fourth embodiment, the specific steps for performing the cognitive function self-test using this method are as follows: As the multimodal interaction and data collection stage of this cognitive function self-test, it mainly includes collecting speech through a microphone array, using voiceprint to lock the target user, combining dialect recognition model to improve the accuracy of dialect understanding, and then performing anti-interference and noise reduction processing, and using beamforming technology to filter out environmental noise (such as TV sound) to ensure clear reception of test instructions. Furthermore, by combining visual device-assisted feedback, the system monitors the user's micro-expressions (such as frowning in confusion) and body movements (such as counting on fingers) during the test using a camera, and uses a lightweight CNN model to analyze the degree of attentional distraction. As the intelligent analysis and assessment stage of this cognitive function self-test, it is mainly used to execute core test items, including a three-word recall test. The test process is as follows: the system first reads three unrelated words (such as "kite, purple, hospital") at a soothing speed and asks the elderly to repeat them immediately. The accuracy of the repetition and the response delay are analyzed through an NLP model. Then, the elderly are guided to perform other activities for 5 minutes (such as simple calculation problems). During this period, millimeter-wave radar monitors whether their attention is distracted. After 5 minutes, the elderly are asked to repeat the words. The system records the number of omissions and incorrect substitutions (such as saying "clinic" instead of "hospital"). The degree of memory decline is assessed by combining voice tremor detection and the completeness of the first repetition is analyzed by combining LSTM network (full marks are awarded for all 3 words correctly). Furthermore, the test also includes a continuous subtraction of 7 calculation. The test process is as follows: the system subtracts 7 from 100 one by one (100→93→86→...). The elderly person answers verbally, and the speech recognition engine converts the answers to text in real time and verifies them. During the test, abnormal behaviors are captured simultaneously. For example, anxiety is identified and calculated by analyzing facial micro-expressions (such as frowning and wandering eyes) using a camera. Heart rate variability (HRV) is monitored through wearable devices to determine physiological stress caused by cognitive load. If the difference between two results is greater than 14 (such as 100→79), a voice prompt "Please recalculate" is triggered. Furthermore, a multimodal feature fusion assessment is performed, integrating features such as three-word recall error rate, number of computational interruptions, and response delay. These features are then input into a random forest classifier to output a cognitive risk level (normal / mild / high risk). A Bayesian network is applied to update the risk probability: for example, if the computational error rate is >30% in three consecutive tests, the probability of developing the disease increases by 25%. Through monthly cognitive ability trajectory reports, changes in key indicators are marked (such as a 20% monthly average decrease in delayed recall score triggering an alert). As the risk assessment and graded alarm stage in this cognitive function self-test, it specifically includes: when there is ≤1 missing word recall but no calculation error, it is judged as low risk, and the device will provide voice encouragement and push brain training games; when there are ≥2 missing words or ≥2 calculation errors, it is judged as medium risk, and the APP will push measures such as family members; when it is impossible to complete any test item, it is judged as high risk, and emergency contact person calls and medical advice will be pushed. As an adaptive optimization and long-term management phase in the cognitive function self-test, it includes adaptive threshold adjustment. The system dynamically adjusts the test difficulty based on the baseline ability. For example, for elderly people with slower daily reactions, the calculation test response time limit is extended from 15 seconds to 25 seconds. At the same time, a dynamic fault tolerance mechanism is set up. If the user makes two consecutive calculation errors, the test is automatically downgraded to "minus 3" and re-evaluated. Furthermore, a cognitive ability curve is generated monthly, marking abnormal fluctuation points (such as a sudden drop in memory ability) to help family members adjust care strategies.

[0029] like Figure 2 As shown, the home monitoring system for Alzheimer's disease risk based on multimodal behavior analysis adopts an "edge-cloud" collaborative architecture, which is divided into four levels: perception layer, edge computing layer, platform layer, and application layer. The perception layer is used for data acquisition and includes: a millimeter-wave radar array deployed on the ceiling / corner for real-time monitoring of human posture and fall risk (e.g., in a slippery bathroom scenario); a multispectral camera, combined with infrared illumination, to capture facial micro-expressions (drooping corners of the mouth) and behavioral trajectories (repeated pacing); wearable devices, including a wristband with a built-in IMU sensor for monitoring gait stability and heart rate variability (HRV), and a smart ring for continuously measuring blood oxygen and skin conductance response (GSR) and recognizing emotional fluctuations; and environmental sensors that use UWB positioning beacons to track spatial location and identify orientation obstacles (e.g., getting lost in the living room), and temperature / humidity / gas sensors to detect risks such as slippery floors and gas leaks. The edge computing layer is used for real-time data processing and is equipped with a lightweight AI model. It includes a built-in skeletal joint extraction model (video → coordinates) and a speech feature extraction model (audio → semantic vector). It also deploys a multimodal data fusion engine to perform cross-validation mechanisms (such as camera detection of falls + IMU data confirmation of action continuity). The platform layer is used for data analysis and storage. It is deployed with multi-source databases, including: storing relevant content such as action frequency and path trajectory according to the data type of behavior time sequence, which is used to identify repetitive actions / orientation disorders; storing relevant content such as HRV, GSR, and sleep quality according to the data type of physiological parameters, which is used to identify emotional apathy and anxiety analysis; and storing relevant content such as water and electricity usage and location records according to the data type of environmental logs, which is used to assist in the analysis of abnormal behaviors (such as prolonged stillness). Furthermore, an intelligent analysis engine is deployed to analyze abnormal behavioral sequences (such as adding salt three times in a row) using an LSTM+Transformer model, construct "human-object-space" relationships using a GNN graph model to detect misuse of items (using dish soap as toothpaste), and update the probability of illness based on historical data using a Bayesian dynamic network (such as getting lost for 3 days → risk +20%). The application layer provides functional services and has four core functional modules: First, a daily behavior monitoring module, which identifies repetitive actions / life skill decline through a behavior time sequence analyzer; second, an emotion and personality analysis module, which detects paranoia (voice and semantic analysis) and emotional apathy (facial micro-expressions) through a multimodal emotion computing engine; third, a home safety monitoring module, which uses a fall prediction model and an active intervention unit, combined with millimeter-wave radar for real-time early warning, triggering voice reminders / automatic lighting; and fourth, a cognitive self-testing interaction module, which uses an integrated NLP dialogue robot to perform three-word recall tests and continuous subtraction of 7 operations to assess memory decline. Furthermore, a tiered response system has been developed. For low-risk levels, measures such as device voice reminders (e.g., "The ground is slippery, please walk slowly") are taken; for medium-risk levels, measures such as sending push notifications to family members and community doctors via the APP are taken; and for high-risk levels, measures such as automatically calling emergency contacts and coordinating with the community to provide door-to-door services are taken. Furthermore, an adaptive threshold management system has been developed to perform personalized baseline calibration, specifically by dynamically adjusting alarm thresholds based on user habits (e.g., lower thresholds for social frequency in introverted elderly people); and a built-in false alarm suppression algorithm, which is mainly used to ignore normal behaviors such as drinking water at night and focus on high-risk scenarios (squatting in the bathroom for more than 90 minutes).

[0030] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A home-based monitoring method for Alzheimer's disease risk based on multimodal behavioral analysis, characterized in that, The specific steps are as follows: It can monitor human posture in real time, analyze repetitive movements, orientation disorders and decline in daily living skills, and simultaneously issue early warnings for abnormal movements, monitor gait stability and heart rate variability. It also continuously collects blood oxygen saturation and skin conductance response, and can also perform voice interaction operations through a built-in AI dialogue terminal. After initial screening at the edge, the data undergoes multimodal fusion analysis in the cloud, and behavioral, emotional, and physiological correlation models are constructed. In addition, a cognitive decline dynamic assessment is performed through cognitive function self-testing, and the risk level is determined based on the analysis and assessment results. Based on risk level information, a multi-level early warning mechanism is implemented to automatically take corresponding emergency measures. In addition, an active intervention mechanism is implemented to assist in cognitive function self-testing and home safety monitoring. By integrating user habit data to dynamically calibrate thresholds and designing a closed-loop feedback mechanism to suppress false alarms, monthly health reports are generated simultaneously to assist doctors in adjusting intervention plans.

2. The method for home monitoring of Alzheimer's disease risk based on multimodal behavioral analysis according to claim 1, characterized in that, When using the aforementioned monitoring method to perform daily behavioral anomaly monitoring, the specific steps are as follows: It integrates visual sensing, environmental assistance, and physiological parameters to collect multi-source data, including capturing three-dimensional spatial movements, tracking human posture in real time, combining smart home device logs to help judge spatial orientation anomalies, and cross-validating with wearable devices and video stream data. Analyze action sequences to detect abnormal loop patterns, track movement trajectories to identify "repeated wandering" behavior, construct a "human-space" relationship model to determine orientation impairment, analyze object misuse behavior to detect life skills decline; extract dynamic risk features and dynamically update risk probabilities; A multi-level early warning mechanism is adopted to conduct risk assessments, determine low, medium, and high risk levels, and take emergency measures accordingly. Integrate user habit data to perform personalized threshold adjustments; Generate monthly behavior reports and mark trend lines for abnormal frequency.

3. The method for home monitoring of Alzheimer's disease risk based on multimodal behavioral analysis according to claim 1, characterized in that, When using the aforementioned monitoring method to perform daily behavioral anomaly monitoring, the specific steps are as follows: First, multi-source data collection is carried out by integrating voice analysis, facial expression recognition and physiological parameters. Specifically, this includes locking the target user to capture the dialogue content, collecting facial micro-expressions, and simultaneously monitoring heart rate variability and skin conductance response through wearable devices to capture physiological stress signals during emotional fluctuations. Voice features are input into an NLP model to identify emotional states, and physiological and behavioral data are cross-validated to verify the authenticity of emotions. Abnormal patterns are identified by constructing a "behavior-time-space" relationship graph. Smart home logs and UWB positioning data are analyzed simultaneously to quantify social withdrawal. Abnormal behaviors are matched by comparing them with typical symptom features.

4. The home-based monitoring method for Alzheimer's disease risk based on multimodal behavioral analysis according to claim 3, characterized in that: Risk assessment is conducted based on the aforementioned abnormal behaviors, including low risk when there is a single instance of low mood, medium risk when there is social withdrawal and anxiety index > threshold for 3 consecutive days, and high risk when there is aggressive behavior and self-harm tendencies. The alarm threshold is dynamically calibrated based on baseline personality, and the probability is dynamically updated to generate a monthly mood fluctuation heatmap, marking personality change nodes.

5. The method for home monitoring of Alzheimer's disease risk based on multimodal behavioral analysis according to claim 1, characterized in that, When using the aforementioned monitoring method for home safety monitoring, the specific steps are as follows: It tracks the position of the human body's center of gravity, movement trajectory and posture changes in real time, combines environmental perception to detect slippery ground and obstacles, and marks high-risk areas. Wearable devices are used to monitor the center of gravity shift and gait imbalance precursors, and physiological parameters are combined to capture the physiological stress response at the moment of fall. It identifies "sudden fall and stillness" actions, and simultaneously uses wearable devices to detect weightlessness acceleration and abnormal heart rate to trigger a secondary verification mechanism, while simultaneously using algorithms to eliminate false alarm scenarios.

6. The method for home monitoring of Alzheimer's disease risk based on multimodal behavioral analysis according to claim 5, characterized in that: The home safety monitoring system indicates low risk when an environmental warning is issued, medium risk when abnormal posture occurs but physiological parameters remain stable, and high risk when sudden posture changes, physiological stress, or no response occurs. Simultaneously execute local intervention operations such as voice prompts and emergency lighting; Adaptive optimization is achieved based on user habits, and multi-user data is integrated to improve the recognition accuracy in complex scenes.

7. The method for home monitoring of Alzheimer's disease risk based on multimodal behavioral analysis according to claim 1, characterized in that, The specific steps for using the aforementioned monitoring method to perform a self-test of cognitive function are as follows: The system identifies target users and collects their voice information. It also combines visual devices to provide feedback on users' micro-expressions and body movements, and simultaneously analyzes the degree of attention distraction. Execute core test projects, integrate three-word recall error rate, calculate number of interruptions, response delay characteristics, output cognitive risk level, update risk probability, and mark key indicator changes based on historical monthly cognitive ability trajectory reports; Low risk is indicated by missing ≤1 word recall but no calculation errors; medium risk is indicated by missing ≥2 words or making ≥2 calculation errors; high risk is indicated by being unable to complete any test item; local intervention operations such as simultaneous voice encouragement and push brain training games are performed. The test difficulty is dynamically adjusted based on baseline ability, and a dynamic error tolerance mechanism is set up; a cognitive ability curve is generated every month, and abnormal fluctuation nodes are marked.

8. The method for home monitoring of Alzheimer's disease risk based on multimodal behavioral analysis according to claim 7, characterized in that, The core test items include: The three-word recall test is conducted as follows: The system first reads three unrelated words at a slow pace and asks the elderly to repeat them immediately. The accuracy of the repetition and the response delay are analyzed. The elderly are then guided to engage in other activities for 5 minutes. After 5 minutes, the elderly are asked to repeat the words. The system records the number of omissions and incorrect substitutions, detects and assesses the degree of memory decline, and analyzes the completeness of the first repetition. The test involved subtracting 7 consecutively from 100. The elderly person answered verbally, and the speech recognition engine transcribed the answers into text in real time and verified the answers. Abnormal behavior was captured simultaneously.

9. A home-based monitoring system for Alzheimer's disease risk based on multimodal behavioral analysis, executing the monitoring method as described in any one of claims 1-8, characterized in that, The monitoring system comprises four layers: a perception layer, an edge computing layer, a platform layer, and an application layer. The sensing layer is used for data acquisition and includes a millimeter-wave radar array deployed on the ceiling / corner of the wall, a multispectral camera with infrared illumination, a wearable device with a built-in IMU sensor, a smart ring that measures blood oxygen and skin conductance, and environmental sensors. The edge computing layer is used for real-time data processing and is equipped with a lightweight AI model and a multimodal data fusion engine. The platform layer is used for data analysis and storage, and it is deployed with multi-source databases and intelligent analysis engines. The application layer is used to provide functional services and has four core functional modules developed internally. The first is the daily behavior monitoring module, which identifies repetitive actions / life skills degradation through a behavior time sequence analyzer. Secondly, the emotion and personality analysis module detects paranoia and emotional apathy through a multimodal emotion computing engine. Thirdly: the home safety monitoring module, which uses a fall prediction model and active intervention unit, combined with millimeter-wave radar for real-time early warning, triggering voice reminders / automatic lighting; Fourthly: the cognitive self-testing interaction module, which uses an onboard NLP dialogue robot to perform three-word recall tests and continuous subtraction of 7 operations to assess memory decline; Furthermore, a hierarchical response system and an adaptive threshold management system have also been developed.

10. The home-based monitoring system for Alzheimer's disease risk based on multimodal behavioral analysis according to claim 9, characterized in that: The multi-source database stores corresponding action frequencies and path trajectories according to the data type of the behavior time sequence, which is used to identify repetitive actions / orientational obstacles; Based on the data type of physiological parameters, store corresponding HRV, GSR, and sleep quality content for identifying apathy and anxiety analysis; based on the data type of environmental logs, store corresponding water and electricity usage and location records for assisting in the analysis of abnormal behavior. The intelligent analysis engine analyzes abnormal behavior sequences using an LSTM+Transformer model, constructs "human-object-space" relationships using a GNN graph model, detects misuse of items, and updates the probability of disease based on historical data using a Bayesian dynamic network.

Images