A cognitive impairment identification method based on natural behavior analysis

Abstract

The application discloses a cognitive impairment recognition method based on natural behavior analysis, comprising the following steps: S1, video acquisition and individual separation, real-time acquisition of activity scene video stream, detection of the boundary box of all human bodies, and cross-frame tracking of the detected human body target; S2, natural behavior feature extraction, posture estimation, expression recognition, social interaction analysis and task participation analysis are carried out, and after all the features are normalized, the fixed-length feature vectors are packaged and uploaded to the cloud; S3, mapping of behavior features to cognitive dimensions, the cloud receives the time sequence feature flow of each human body target, calls the pre-trained multi-level mapping model to perform time sequence coding, attention fusion, cognitive dimension decoding, risk identification and suggestion generation; S4, result storage and display, the risk identification result and the suggestion of each time are stored into a time sequence database, and a personal cognitive health atlas is constructed. The application can realize high ecological validity, low cost, high acceptance and fine early screening.

Description

Technical Field

[0001] This invention relates to a method for identifying cognitive impairment, and more particularly to a method for identifying cognitive impairment based on natural behavior analysis. Background Technology

[0002] With the increasing aging of the population, cognitive impairment (including mild cognitive impairment (MCI) and dementia) has become a serious public health problem affecting the health and quality of life of older adults. Early identification and intervention are key to slowing disease progression. Currently, the main clinical assessment tools are neuropsychological scales, such as the Brief Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA). These scales test subjects' memory, attention, language, visuospatial, and other functions through a series of standardized questions. However, these scales have inherent limitations that prevent them from meeting the needs of universal community screening.

[0003] 1. The assessment scenario is out of touch with reality: Traditional neuropsychological scales (such as MMSE and MoCA) must be conducted in artificial environments such as clinics. The elderly are in a state of stress, and their performance is difficult to reflect their real daily abilities, resulting in low ecological validity of the assessment results and inability to accurately identify early cognitive decline.

[0004] 2. Reliance on professional personnel: The scale assessment requires one-on-one testing by professionally trained doctors or assessors, which is costly and makes it difficult to promote and apply in large-scale community screening.

[0005] 3. Poor cooperation and strong resistance among the elderly: The assessment process is similar to an exam. Many elderly people refuse to cooperate due to the stigma of their illness, anxiety, or resistance to the medical environment, resulting in low screening coverage and a large number of potential patients being missed.

[0006] 4. Limited assessment results and lack of guidance: Traditional scales only output a total score or a few rough scores in a few dimensions, which cannot provide a multi-dimensional cognitive function profile or automatically generate personalized care suggestions, thus having limited guiding value for subsequent interventions.

[0007] In recent years, with the development of sensor and artificial intelligence technologies, several alternative solutions have emerged: Wearable device monitoring: Using devices such as smart bracelets and watches to monitor physiological indicators of the elderly, such as steps, sleep, and heart rate, in an attempt to indirectly infer cognitive status from changes in activity levels and sleep patterns. For example, some studies have analyzed the association between gait abnormalities and cognitive decline using accelerometer data.

[0008] Video surveillance and abnormal behavior detection: Deploying surveillance cameras in elderly care facilities and using computer vision technology to detect abnormal behaviors in the elderly, such as wandering, falling, and nighttime loitering, are common behaviors in patients with mid-to-late stage dementia. For example, some commercial elderly care monitoring systems can identify elderly people who remain still for extended periods or leave designated areas.

[0009] Human-computer interaction game assessment: Cognitive training games using touchscreens or voice interaction record the elderly person's reaction time, accuracy, etc., as indicators of cognitive function. For example, some cognitive training apps have built-in simple assessment modules.

[0010] Speech analysis technology: By analyzing the speech of elderly people in their daily conversations or during specific tasks, features such as speech rate, pauses, and pitch changes are extracted to detect language function decline. For example, companies like Winterlight Labs use speech analysis to screen for cognitive impairment.

[0011] The aforementioned wearable devices and simple video monitoring can only monitor coarse-grained activity levels or obvious abnormal behaviors (such as wandering), and cannot capture micro-behavioral characteristics related to cognitive decline (such as delayed response, decreased motor fluency, social withdrawal, etc.), which may cause them to miss the golden window for early intervention.

[0012] As can be seen from the above, the main drawbacks of existing technologies (including scales, wearable devices, simple video surveillance, etc.) are: 1. Low ecological validity: The scale assessment is conducted in an artificial environment, and the results cannot truly reflect daily functioning.

[0013] 2. Reliance on professional personnel and difficulty in scaling: The scale requires one-on-one administration, which is costly and inefficient.

[0014] 3. Low acceptance among the elderly: The assessment process can feel intrusive and easily trigger resistance, leading to insufficient screening coverage.

[0015] 4. Poor sensitivity: It can only detect obvious symptoms and cannot identify early subtle behavioral changes, thus missing the intervention window.

[0016] 5. Limited assessment dimensions: Only outputs total score or coarse-grained indicators, failing to provide a multi-dimensional cognitive function profile and lacking intervention guidance.

[0017] 6. Non-natural scenarios: Some technologies (such as wearable devices and game assessments) still require the elderly to actively cooperate or wear the devices, which presents compliance issues.

[0018] To address the aforementioned issues, it is necessary to provide a method and system that can automatically and imperceptibly identify cognitive impairments in natural, everyday settings, aiming to achieve high ecological validity, low cost, high acceptability, and precise early screening. Summary of the Invention

[0019] The technical problem to be solved by the present invention is to provide a cognitive impairment identification method based on natural behavior analysis, which can automatically identify early signs of cognitive impairment in completely natural and unconscious daily activity scenarios through refined multimodal behavior analysis.

[0020] To address the aforementioned technical problems, this invention provides a cognitive impairment identification method based on natural behavior analysis, comprising the following steps: S1, video acquisition and individual separation: real-time acquisition of video streams of activity scenes, inference on each frame, detection of bounding boxes for all human bodies, and cross-frame tracking of detected human targets; S2, natural behavior feature extraction: posture estimation, facial expression recognition, social interaction analysis, and task participation analysis are performed. All features are normalized and packaged into fixed-length feature vectors, which are then uploaded to the cloud; S3, mapping of behavioral features to cognitive dimensions: the cloud receives the temporal feature stream of each human target, and calls a pre-trained multi-level mapping model for temporal encoding, attention fusion, cognitive dimension decoding, risk identification, and suggestion generation; S4, result storage and display: each risk identification result and suggestion is stored in a temporal database to construct a personal cognitive health atlas; an institutional management platform is used to display the cognitive status, trend chart, and early warning information of elderly people currently participating in activities in real time; and a family member mini-program is provided to view weekly reports and daily interaction prompts for loved ones.

[0021] Furthermore, in step S1, a high-definition camera is used to capture the video stream of the activity scene in real time and transmit it to the edge computing unit. The edge computing unit uses the NVIDIA Jetson AGX Orin development kit and is pre-installed with the Ubuntu operating system. The edge computing unit completes individual separation and natural behavioral feature extraction and uploads it to the cloud.

[0022] Furthermore, the edge computing unit uses a TensorRT-optimized YOLOv8 as the object detection model to perform object detection inference for each frame. Specifically, this includes: using TensorRT to fuse network layer operators, merging the convolutional layers, batch normalization layers, and activation function layers in the model into a single operator; configuring a dynamic batch processing dimension to adapt to the dynamically changing number of elderly people in the activity scene; and employing an entropy-based calibration algorithm to perform INT8 quantization calibration on the model using a pre-collected dataset of natural activity scenes of the elderly, thereby compressing the model size while maintaining the detection accuracy for edge views and non-standard sitting postures.

[0023] Furthermore, the edge computing unit employs DeepSORT for multi-target tracking and assigns a unique ID to each detected human target; simultaneously, it modifies the DeepSORT correlation matrix to fuse appearance and motion features. When a target leaves the frame and returns, it is re-identified based on the feature library to retrieve the original ID; specifically, this includes: constructing a dynamic adaptive cost matrix C. ij =α·D m (i,j)+ β·D α (i,j), where D m For the Mahalanobis distance based on Kalman filtering, D αThe cosine distance of appearance features is based on a lightweight ReID network. The weight coefficients α and β are dynamically adjusted according to the intersection-union ratio of the detection boxes between targets: when the IoU of the detection boxes is greater than the preset occlusion threshold, occlusion is determined to have occurred, the weight factor α of Mahalanobis distance is automatically reduced, and the weight factor β of appearance feature cosine distance is simultaneously increased to overcome trajectory drift and ID jump caused by occlusion in complex social interaction scenarios.

[0024] Furthermore, the edge computing unit uses the TensorRT-optimized MediaPipe Pose as the pose estimation model. Specifically, the pose estimation is as follows: the MediaPipe pose inference subnetwork is converted into an FP16 half-precision TensorRT engine for hardware-level acceleration; the image affine transformation preprocessing required for pose estimation, and the post-processing operation of mapping the skeletal keypoint heatmap to absolute coordinates are all encapsulated as CUDA custom operators and directly embedded into the TensorRT execution pipeline; the optimized model is used to extract the coordinates and confidence scores of multiple skeletal keypoints; the instantaneous velocity, acceleration, and joint angle of each skeletal keypoint are calculated; and motion features are statistically analyzed every 2 seconds to form a time series.

[0025] Furthermore, in step S2, the expression recognition is as follows: the face region is cropped, and the probability of joy, sadness, confusion, indifference, anxiety, surprise, and neutral expressions is output based on MobileNet training in the edge computing unit; the average probability is calculated every 2 seconds.

[0026] Furthermore, the social interaction analysis in step S2 is as follows: using the center point of the detection box of all people, calculate the Euclidean distance between the ID and other IDs, and record the closest distance, average distance, and distance change rate; detect whether the person is facing others by estimating head posture, and count the number of active interactions by moving towards others and communicating through gestures.

[0027] Furthermore, the task participation analysis in step S2 is as follows: based on the activity type, a task template is preset, and by detecting whether the elderly are looking at the task center and whether they are performing task-related actions, the participation time percentage and action synchronization rate are calculated, and a task participation score is output every 2 seconds.

[0028] Further, the multi-level mapping model in step S3 is processed as follows: Temporal encoding: The feature sequence is input into a bidirectional LSTM or Transformer to encode the contextual information of each time window and output the behavioral pattern embedding vector; Attention fusion: A multi-head attention mechanism is introduced to calculate the importance weights of different feature dimensions and generate scene-aware behavioral representations; Cognitive dimension decoding: The scene-aware behavioral representations are input into a multilayer perceptron and output quantitative scores for the following six cognitive dimensions: memory, executive function, attention, language function, social cognition, and visuospatial ability; Risk identification: The six-dimensional scores are input into a classifier and output the probabilities of three risk levels: normal, high risk of mild cognitive impairment, and suspected dementia; Suggestion generation: Based on a rule base predefined by clinical experts, personalized care suggestion text is generated according to the scores of each dimension and the risk level.

[0029] Furthermore, the edge computing unit converts the trained model into TensorRT format and quantizes it into FP16 or INT8. It then deploys the model in the cloud using ONNX Runtime and encapsulates the mapped model into ONNX format.

[0030] Compared with existing technologies, this invention offers the following advantages: The cognitive impairment identification method based on natural behavior analysis provided by this invention can achieve high ecological validity, low cost, high acceptability, and refined early screening. Specific advantages include: 1. Achieving high ecological validity in assessment, with results accurately reflecting daily abilities. 2. Requiring no professional intervention, allowing for large-scale deployment in communities and homes. 3. Completely undetectable by the elderly, requiring no device wearing or active cooperation, with near 100% acceptance. 4. Capable of capturing early micro-behavioral changes, providing early warning of cognitive decline risks. 5. Outputting multi-dimensional cognitive function scores and impairment pattern profiles, providing a basis for personalized intervention. 6. Achieving real-time analysis while protecting privacy through an edge-cloud collaborative architecture. Attached Figure Description

[0031] Figure 1 This is a flowchart of the cognitive impairment identification process based on natural behavior analysis according to the present invention. Detailed Implementation

[0032] The present invention will now be further described with reference to the accompanying drawings and embodiments.

[0033] I. Device Components (System Composition) The system of this invention mainly consists of the following components: Video capture unit: High-definition cameras deployed in natural settings (such as community activity rooms or family living rooms) are used to capture real-time video streams containing multiple targets. This unit features wide-angle and low-light adaptability, ensuring clear imaging under various lighting conditions.

[0034] Edge computing unit: An edge device (such as the NVIDIA Jetson series) integrated or connected to the video acquisition unit. It has built-in dedicated algorithms responsible for real-time processing of the video stream, extracting anonymized features, and uploading them to the cloud. The edge computing unit includes the following modules: object detection and tracking module, pose estimation module, facial expression recognition module, social interaction analysis module, task participation analysis module, and feature encryption transmission module. Social interaction analysis can incorporate graph neural networks (GNNs) to model the group as a dynamic graph and learn the interaction patterns between individuals.

[0035] The cloud computing unit, deployed on a cloud server, is responsible for receiving feature data uploaded from the edge, running the core cognitive assessment model, and generating assessment results and recommendations. The cloud computing unit includes the following modules: a temporal encoding module, an attention fusion module, a cognitive dimension decoding module, a risk classification module, a recommendation generation module, and a database (storing the temporal health map). The temporal encoding module can be replaced with a Transformer encoder only, or a Temporal Convolutional Network (TCN) can be used; the attention fusion module can be replaced with a simple feature concatenation followed by a fully connected layer, but the performance may decrease; the cognitive dimension decoding module can be designed as a multi-task learning architecture, sharing underlying features and outputting scores for each dimension separately.

[0036] Results output units include the organization management platform (web version), family mini-program, partner API, etc., used to present assessment results and recommendations to different users.

[0037] II. Apparatus Structure (Detailed Preparation / Configuration Process) Video capture unit: Use commercially available high-definition network cameras (such as Hikvision, Dahua, etc.), with a resolution of no less than 1080p and a frame rate of no less than 25fps. The lens focal length is selected according to the size of the scene (usually 2.8-4mm) to ensure coverage of the entire activity area. The camera is powered via PoE and connected to the edge computing unit.

[0038] Edge computing unit: Utilizes the NVIDIA Jetson AGX Orin development kit, pre-installed with the Ubuntu operating system, and deploys the following algorithm environments: Install acceleration libraries such as CUDA (Compute Unified Device Architecture), cuDNN, and TensorRT.

[0039] Deploy the object detection model (YOLOv8, optimized with TensorRT); the TensorRT optimization specifically includes: using TensorRT to fuse network layer operators, merging the convolutional layers, batch normalization layers, and activation function layers in the model into a single operator to reduce memory access overhead; configuring dynamic batch dimensions (Dynamic Shapes) to adapt to the dynamically changing number of elderly people in the activity scene; and adopting an entropy-based calibration algorithm (Entropy Calibration), using a pre-collected dataset of natural activity scenes of the elderly to perform INT8 quantization calibration on the model, compressing the model size while maintaining the detection accuracy for edge views and non-standard sitting postures.

[0040] Deploy a multi-target tracking algorithm: DeepSORT, modifying the correlation matrix to fuse appearance and motion features; specifically: Kalman filter Mahalanobis distance, D α The cosine distance of appearance features is based on a lightweight ReID network. The weight coefficients α and β are dynamically adjusted according to the intersection-over-union (IoU) of the detection boxes between targets: when the IoU of the detection boxes is greater than the preset occlusion threshold, occlusion is determined, the weight factor α of Mahalanobis distance is automatically reduced, and the weight factor β of appearance feature cosine distance is simultaneously increased to overcome trajectory drift and ID jump caused by occlusion in complex social interaction scenarios.

[0041] Deploy a pose estimation model (MediaPipe Pose or OpenPose, optimized with TensorRT; or use HRNet, ViTPose, etc., for higher accuracy, but this may increase computational requirements). Specific pose estimation is as follows: The MediaPipe pose inference subnetwork is converted into an FP16 half-precision TensorRT engine for hardware-level acceleration; and the image affine transformation preprocessing (image cropping and scaling) required for pose estimation, as well as the post-processing operation of mapping skeletal keypoint heatmaps to absolute coordinates, are all encapsulated as CUDA custom operators and directly embedded into the TensorRT execution pipeline, eliminating the frequent data transfer latency between the CPU and GPU.

[0042] Deploy a lightweight facial expression recognition model (trained on MobileNet, outputting probabilities for 7 types of facial expressions; facial expression feature extraction can be replaced by using a 3D convolutional neural network to directly identify expressions from video clips, instead of the frame-by-frame model plus statistical method).

[0043] Develop a feature extraction pipeline using a mix of Python and C++ programming to ensure real-time performance.

[0044] Cloud computing unit: Rent a cloud server (such as Alibaba Cloud ECS, configured with a GPU such as NVIDIA T4) and deploy the following environment: Operating System: Ubuntu 20.04 Deep learning framework: PyTorch 1.10+ Database: PostgreSQL + TimescaleDB (for time-series data) API Framework: FastAPI Model: A pre-trained multi-level mapping model (Transformer+LSTM+MLP), deployed via ONNX Runtime or TorchServe.

[0045] Output Units: Institutional Management Platform: Developed using Vue3 + Element Plus and deployed on an Nginx server. Family Member Mini Program: Utilizes WeChat Mini Program development tools to retrieve data via cloud APIs.

[0046] III. Working principle of the device (the process of coordinated operation of each component) Please see Figure 1 The overall workflow of this invention is as follows: Step A: Video Acquisition and Individual Separation The video acquisition unit collects video streams of the event scene in real time and transmits them to the edge computing unit.

[0047] The object detection module (YOLOv8) of the edge computing unit performs inference for each frame and detects the bounding boxes of all human figures.

[0048] The target tracking module (Improved DeepSORT, Deep Simple Online and Realtime Tracking) assigns a unique ID to each detected person and tracks them across frames. When occlusion occurs, the algorithm combines appearance features (color histogram) and motion prediction (Kalman filtering) to maintain ID stability. When the target briefly leaves the frame and then returns, the original ID is retrieved by the re-identification module (based on a feature library).

[0049] Step B: Extraction of Natural Behavioral Features For each ID, perform the following parallel processing in each frame: Pose estimation: The coordinates and confidence scores of 17 skeletal keypoints were extracted using MediaPipe Pose. Instantaneous velocities, accelerations, and joint angles (elbow, knee, etc.) of the keypoints were calculated. Motion features were analyzed every 2 seconds to form a time-series sequence.

[0050] Facial Expression Recognition: The face region is cropped, input into the facial expression recognition model, and the model outputs the probabilities of seven types of expressions: joy, sadness, confusion, indifference, anxiety, surprise, and neutral. The average probability is calculated every 2 seconds.

[0051] Social interaction analysis: Using the center points of all bounding boxes, calculate the Euclidean distance between this ID and other IDs, and record the closest distance, average distance, and rate of change of distance. Detect whether the user is facing others (based on head pose estimation). Count the number of proactive interactions (such as moving towards others, gestural communication).

[0052] Task participation analysis: Based on the activity type (e.g., choir, crafts), a pre-set task template is used. The analysis detects whether the elderly person is focusing on the task center (e.g., the lead singer, the craft table) and whether they are performing task-related actions (e.g., opening their mouth, moving their hands), calculating the percentage of participation time and the synchronization rate of actions. A task participation score is output every 2 seconds.

[0053] After all features are normalized, they are packaged into a fixed-length feature vector (approximately 200 dimensions every 2 seconds) and uploaded to the cloud computing unit via an encrypted transmission module (TLS protocol).

[0054] Step C: Mapping behavioral characteristics to cognitive dimensions (cloud computing unit) The cloud receives the temporal feature stream of each elderly person (one vector every 2 seconds) and calls a pre-trained multi-level mapping model for processing: Temporal encoding: Input the feature sequence into a bidirectional LSTM (or Transformer), encode the context information of each time window, and output the behavior pattern embedding vector (128 dimensions).

[0055] Attention fusion: A multi-head attention mechanism is introduced to calculate the importance weights of different feature dimensions (such as posture, expression, and social interaction) to generate scene-aware behavioral representations (256 dimensions). Attention weights can be interpreted as which behavioral features contribute the most to the current evaluation.

[0056] Cognitive Dimension Decoding: Input the scene perception behavior representation into the multilayer perceptron and output quantitative scores (0-100) for 6 cognitive dimensions: memory, executive function, attention, language function, social cognition, and visuospatial ability.

[0057] Risk identification: Input the 6-dimensional score into the classifier (softmax) and output the probability of 3 risk levels: normal, high risk of mild cognitive impairment, and suspected dementia requiring further diagnosis.

[0058] Recommendation generation: Based on a rule base (predefined by clinical experts), personalized care recommendation texts are generated according to scores and risk levels across various dimensions.

[0059] Step D: Result Storage and Display Each assessment result (score, risk level, recommendation) is stored in a time-series database to construct a personal cognitive health profile.

[0060] The organization's management platform displays the cognitive status, trend charts, and early warning information of elderly people currently participating in activities in real time.

[0061] The family member mini-program allows users to view their loved ones' weekly reports and daily interaction reminders.

[0062] IV. Manufacturing Method of the Device The core of this invention lies in the algorithm and software, while the hardware uses commercially available mature components. Therefore, "manufacturing" mainly refers to the construction and deployment of the software system, with the following steps: 1. Dataset preparation: Collect videos of community activities, each video featuring multiple elderly people participating in different activities.

[0063] Clinical experts were invited to conduct cognitive assessments (such as MMSE scores and diagnostic labels) on each elderly person in the video as a monitoring signal.

[0064] The video is manually annotated: each frame contains a human bounding box, an identity ID, and behavioral events (such as smiling or clapping), which are used to train the intermediate model.

[0065] 2. Model Training: The object detection model (YOLOv8) and pose estimation model (MediaPipe, which can be finely tuned) were trained using labeled data.

[0066] Use facial expression datasets (such as AffectNet) to train facial expression recognition models.

[0067] A multi-level mapping model is constructed: using behavioral features of video clips as input and clinical labels as output, an end-to-end mapping model is trained. Cross-entropy loss (classification) and mean squared error loss (regression) are jointly used for optimization.

[0068] 3. Edge optimization: The trained model is converted to TensorRT format and quantized (FP16 or INT8) to adapt it to the Jetson platform.

[0069] Write a C++ / Python hybrid inference pipeline to ensure real-time processing (25fps).

[0070] 4. Cloud deployment: The mapping model is encapsulated in ONNX format and deployed using ONNX Runtime.

[0071] Set up a FastAPI service to receive feature data, call the model, and return the results.

[0072] Configure the database to enable long-term data storage and retrieval.

[0073] 5. System integration testing: End-to-end testing was conducted in a simulated laboratory setting to adjust algorithm parameters.

[0074] Conduct small-scale pilot programs in partner communities, collect feedback, and iterate and optimize accordingly.

[0075] Deployed in the activity room of a community senior day care center, this device automatically completed a cognitive assessment of eight elderly participants during a 45-minute choral activity. The system output a six-dimensional cognitive score and risk level for each participant, and generated the following care recommendations: "Grandpa Wang, your attention score is low, and your social participation is declining. We recommend you participate in more interactive activities, such as a chess or card group, and arrange for volunteers to pair you with a senior twice a week." Community staff adjusted the activity schedule based on the recommendations and regularly tracked the effects. The time-series graphs accumulated by the system showed that after three months of intervention, Grandpa Wang's social cognitive score improved, indicating that the intervention was effective.

[0076] The cognitive impairment identification method based on natural behavior analysis provided by this invention can extract multi-dimensional behavioral features from natural behavior videos in a refined manner, construct a multi-level mapping model, and identify three levels—normal, high-risk MCI, and suspected dementia—based on multi-dimensional cognitive scores. It also generates personalized care suggestions by combining a rule base. Specific advantages are as follows: 1. High ecological validity: The assessment is conducted in completely natural daily activities, and the results truly reflect the daily functional status of the elderly, avoiding the stress bias of clinic assessments.

[0077] 2. No professional personnel required: Fully automated analysis, no doctors or assessors needed, can be deployed on a large scale in communities, greatly reducing labor costs.

[0078] 3. High acceptance among the elderly: Completely contactless and without any physical contact, the elderly do not need to wear equipment or actively cooperate, and their resistance is almost zero, resulting in a significant increase in screening coverage.

[0079] 4. High sensitivity: By extracting multimodal micro-behavioral features, it can capture subtle signals of early cognitive decline, provide early warnings, and buy time for intervention.

[0080] 5. Detailed assessment results: Outputs cognitive scores and impairment pattern profiles across 6 dimensions, providing a precise basis for personalized intervention, rather than a single total score.

[0081] 6. Strong privacy protection: The original video is not uploaded, only the anonymized version is transmitted, which complies with the requirements of the Personal Information Protection Law and reduces ethical risks.

[0082] 7. System scalability: Through the "Lingmou" module, it can empower third-party devices such as robots and smart home devices to form an ecosystem.

[0083] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be defined by the claims.

Claims

1. A method for identifying cognitive impairment based on natural behavior analysis, characterized in that, Includes the following steps: S1. Video acquisition and individual separation: real-time acquisition of video streams of activity scenes, inference on each frame, detection of bounding boxes of all human bodies, and cross-frame tracking of detected human targets. S2. Natural behavior feature extraction: pose estimation, facial expression recognition, social interaction analysis, and task participation analysis are performed. All features are normalized and packaged into a fixed-length feature vector, which is then uploaded to the cloud. S3. Mapping from behavioral features to cognitive dimensions: The cloud receives the temporal feature stream of each human target and calls the pre-trained multi-level mapping model to perform temporal encoding, attention fusion, cognitive dimension decoding, risk identification, and suggestion generation. S4. Results storage and display: Store each risk identification result and suggestion into a time-series database to build a personal cognitive health map; The institution's management platform displays the cognitive status, trend charts, and early warning information of elderly participants in real time. Family members can use a mini-program to view their loved ones' weekly reports and daily interaction reminders.

2. The cognitive impairment identification method based on natural behavior analysis as described in claim 1, characterized in that, Step S1 involves using a high-definition camera to capture real-time video streams of the activity scene and transmitting them to an edge computing unit. The edge computing unit uses the NVIDIA Jetson AGX Orin development kit and is pre-installed with the Ubuntu operating system. The edge computing unit performs individual separation and natural behavioral feature extraction and uploads the data to the cloud.

3. The cognitive impairment identification method based on natural behavior analysis as described in claim 2, characterized in that, The edge computing unit uses a TensorRT-optimized YOLOv8 as the object detection model to perform object detection inference for each frame. Specifically, it includes: using TensorRT to fuse network layer operators, merging the convolutional layers, batch normalization layers, and activation function layers in the model into a single operator; configuring a dynamic batch processing dimension to adapt to the dynamically changing number of elderly people in the activity scene; and using an entropy-based calibration algorithm to perform INT8 quantization calibration on the model using a pre-collected dataset of natural activity scenes of the elderly, so as to maintain the detection accuracy for edge views and non-standard sitting postures while compressing the model size.

4. The cognitive impairment identification method based on natural behavior analysis as described in claim 2, characterized in that, The edge computing unit uses DeepSORT for multi-target tracking and assigns a unique ID to each detected human target. At the same time, it modifies the correlation matrix of DeepSORT to fuse appearance and motion features. When the target leaves the frame and returns, it is re-identified based on the feature library to retrieve the original ID. Specifically, it includes: Constructing a dynamic adaptive cost matrix C ij =α·D m (i,j)+ β·D α (i,j), where D m For the Mahalanobis distance based on Kalman filtering, D α The cosine distance of appearance features is based on a lightweight ReID network. The weight coefficients α and β are dynamically adjusted according to the intersection-union ratio of the detection boxes between targets: when the IoU of the detection boxes is greater than the preset occlusion threshold, occlusion is determined to have occurred, the weight factor α of Mahalanobis distance is automatically reduced, and the weight factor β of appearance feature cosine distance is simultaneously increased to overcome trajectory drift and ID jump caused by occlusion in complex social interaction scenarios.

5. The cognitive impairment identification method based on natural behavior analysis as described in claim 2, characterized in that, The edge computing unit uses MediaPipe Pose optimized by TensorRT as the pose estimation model. The specific pose estimation is as follows: the MediaPipe pose inference subnetwork is converted into an FP16 half-precision TensorRT engine for hardware-level acceleration; and the image affine transformation preprocessing required for pose estimation, as well as the post-processing operation of mapping the skeletal keypoint heatmap to absolute coordinates, are all encapsulated as CUDA custom operators and directly embedded into the TensorRT execution pipeline. The optimized model is used to extract the coordinates and confidence scores of multiple skeletal key points; the instantaneous velocity, acceleration, and joint angle of each skeletal key point are calculated; and the motion features are statistically analyzed every 2 seconds to form a time series.

6. The cognitive impairment identification method based on natural behavior analysis as described in claim 2, characterized in that, In step S2, the facial expression recognition is performed as follows: the face region is cropped, and the probability of joy, sadness, confusion, indifference, anxiety, surprise, and neutral expressions is output based on MobileNet training in the edge computing unit; the average probability is calculated every 2 seconds.

7. The cognitive impairment identification method based on natural behavior analysis as described in claim 1, characterized in that, The social interaction analysis in step S2 is as follows: using the center point of the detection box of all people, calculate the Euclidean distance between the ID and other IDs, and record the closest distance, average distance, and distance change rate; detect whether the person is facing others by estimating head posture, and count the number of active interactions by moving towards others and communicating through gestures.

8. The cognitive impairment identification method based on natural behavior analysis as described in claim 1, characterized in that, The task participation analysis in step S2 is as follows: Based on the activity type, a task template is preset. By detecting whether the elderly are looking at the task center and whether they are performing task-related actions, the participation time ratio and action synchronization rate are calculated, and the task participation score is output every 2 seconds.

9. The cognitive impairment identification method based on natural behavior analysis as described in claim 1, characterized in that, The multi-level mapping model in step S3 is processed as follows: Temporal encoding: Input the feature sequence into a bidirectional LSTM or Transformer, encode the context information of each time window, and output the behavior pattern embedding vector; Attention fusion: Introducing a multi-head attention mechanism to calculate the importance weights of different feature dimensions and generate scene-aware behavior representations; Cognitive Dimension Decoding: Inputting scene-aware behavioral representations into a multilayer perceptron, the output is a quantitative score of the following six cognitive dimensions: memory, executive function, attention, language function, social cognition, and visuospatial ability; Risk identification: Input the 6-dimensional score into the classifier and output the probability of 3 risk levels: normal, high risk of mild cognitive impairment, and suspected dementia; Recommendation generation: Based on a rule base predefined by clinical experts, personalized care recommendation texts are generated according to scores and risk levels across various dimensions.

10. The cognitive impairment identification method based on natural behavior analysis as described in claim 2, characterized in that, The edge computing unit converts the trained model into TensorRT format and quantizes it into FP16 or INT8. It then deploys the model in the cloud using ONNXRuntime and encapsulates the mapped model into ONNX format.

Images