Edge multi-mode grading detection-based falling early warning method for old people

Abstract

The invention provides an edge multi-mode grading detection elder falling early warning method comprising the following steps: deploying an edge computing network comprising a master control node and a data node, and loading a context model according to a position; binding the face information of the user with the RFID tag through an identity confirmation camera; the data node triggers a millimeter wave radar to collect point cloud data through a human body infrared sensor; performing contour change rate primary detection, context model deep learning secondary detection and voice module state confirmation tertiary response; and after the tumble is confirmed, the master control node gives an early warning to the emergency center. The invention relates to the technical field of elderly monitoring.

Description

Technical Field

[0001] This invention relates to the field of elderly monitoring technology, and in particular to a fall warning method for the elderly based on edge multimodal hierarchical detection. Background Technology

[0002] With the development of an aging society, fall detection technology for elderly people living alone is receiving increasing attention. Current technologies mainly rely on wearable devices (such as accelerometers) or environmentally deployed cameras for fall detection. These solutions have significant limitations: wearable devices require the elderly to wear them actively and have limited battery life, while camera-based solutions are prone to privacy violations and are limited by lighting conditions. In recent years, millimeter-wave radar has been introduced into this field due to its advantages of non-contact sensing and privacy protection. Existing millimeter-wave radar solutions typically use a single sensor combined with a fixed recognition model, but they still face challenges in practical applications—the posture characteristics of different indoor scenes (such as bedrooms and bathrooms) vary significantly, making it difficult for a single model to adapt to the recognition needs of multiple scenarios; at the same time, the problem of identity confusion when multiple users coexist has not been effectively solved, and detection relying solely on point cloud data is prone to false alarms triggered by everyday actions (such as sitting down quickly).

[0003] Existing millimeter-wave radar fall detection systems have three main shortcomings: First, their models do not consider the semantic differences in the scene at which nodes are deployed (such as the impact of a smooth bathroom floor versus a soft bedroom bed on fall posture), leading to a decrease in cross-scene recognition accuracy. Second, in multi-user scenarios, they lack a reliable identity binding mechanism, making it impossible to distinguish the posture data of different users. Finally, the detection process lacks a hierarchical verification mechanism. When the millimeter-wave radar misjudges a daily action as a fall (such as bending over to pick up an object causing a sudden change in the outline), the system directly triggers a false alarm, which not only increases the burden on emergency resources but may also reduce users' trust in the system.

[0004] Therefore, the inventors urgently need a fall warning method for the elderly based on edge multimodal hierarchical detection to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention provides an edge multimodal hierarchical detection method for elderly fall warning, aiming to solve the problems of low cross-scene recognition accuracy and high false alarm rate in existing millimeter-wave radar fall detection systems, which suffer from a lack of consideration for scene semantic differences, multi-user identity binding mechanisms, and hierarchical verification processes.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a method for early warning of elderly falls based on edge multimodal hierarchical detection, comprising the following steps: S1. Deploy an edge computing network in the monitoring environment. The edge computing network includes a master control node and several data nodes. Load the corresponding context model according to the deployment location of each node. S2. The facial information of the user entering the monitoring environment is obtained through the identity verification camera, bound to the user identity and RFID tag stored on the server, and the binding information is distributed to several data nodes through the main control node; S3. The data node detects human activity through a human infrared sensor and activates millimeter-wave radar to collect millimeter-wave radar point cloud data in real time when a human body is detected. S4. Conduct graded fall detection: Primary detection: Based on the millimeter-wave radar point cloud data, analyze the rate of change of the three-dimensional contour of the monitored object. If it exceeds a set threshold, secondary detection is triggered. Secondary detection: Based on the loaded context model, a deep learning algorithm is used to identify the fall posture on the millimeter-wave radar point cloud data; Level 3 response: When the Level 2 detection identifies a fall, the voice module is activated to confirm the status of the monitored object; S5. If the fall is confirmed by voice or there is no response, a warning message is sent to the emergency center through the main control node.

[0007] Based on the above, the beneficial effect of the edge multimodal hierarchical detection method for elderly fall warning is that it solves the problems of low cross-scene recognition accuracy and high false alarm rate in existing millimeter-wave radar fall detection systems, which are caused by the lack of consideration for scene semantic differences, the absence of multi-user identity binding mechanisms, and hierarchical verification processes. This is mainly reflected in: 1. This invention solves the problem of low cross-scene recognition accuracy caused by millimeter-wave radar fall detection system due to failure to consider scene semantic differences by deploying an edge computing network containing master control nodes and data nodes in the monitoring environment and loading a context model according to the deployment location of each node. Data nodes in different locations of the edge computing network (such as bedroom and bathroom) perform scene-adaptive posture recognition. 2. This invention acquires user facial information through an identity verification camera and binds it to RFID tags stored on a server. The master control node distributes the binding information to the data nodes. This step establishes a multimodal identity association between face, RFID, and millimeter-wave radar data, solving the data confusion problem caused by the lack of an identity binding mechanism in a multi-user environment. 3. This invention solves the problem of high false alarm rate for daily actions caused by the lack of a hierarchical fall detection process (primary detection of contour change rate → secondary detection of context model deep learning → tertiary response of voice module status confirmation). This step is based on millimeter-wave radar point cloud data for step-by-step verification and combined with offline interaction of the voice module to eliminate false judgments.

[0008] Furthermore, the loading of the corresponding context model in step S1 specifically includes: the master control node collecting the deployment location semantic information of several data nodes, applying to the server for a point cloud pose recognition model that matches the location semantics, and distributing the point cloud pose recognition model to the data nodes at the corresponding locations.

[0009] Based on the above, the beneficial effects of the master control node collecting the semantic information of the deployment location of data nodes are: establishing a mapping relationship between location and recognition model, realizing the basic conditions for on-demand allocation of scenario-based models; the beneficial effects of requesting the point cloud attitude recognition model with location semantic matching from the server are: calling pre-trained differentiated models (such as the slippery bathroom floor fall model and the soft landing floor fall model in the bedroom) from the server based on location semantics, realizing accurate parsing of millimeter-wave radar point cloud data in different scenarios; the beneficial effects of distributing the point cloud attitude recognition model to the corresponding location data nodes are: enabling data nodes to load dedicated models that match their deployment scenario (such as loading the bathroom fall recognition model on the data node deployed in the bathroom), realizing localized execution of scenario-adaptive attitude recognition at the edge, and eliminating cross-scenario misjudgment.

[0010] Furthermore, the primary detection in step S4 specifically involves: caching the millimeter-wave radar point cloud data for 5 consecutive seconds, extracting the rate of change of the three-dimensional contour of the monitored object, and triggering the secondary detection when the rate of change of the three-dimensional contour exceeds a set threshold.

[0011] Based on the above, the beneficial effect of caching 5 consecutive seconds of millimeter-wave radar point cloud data is that by temporarily storing 5 consecutive seconds of raw point cloud data, a complete time window is provided for contour change analysis, thus realizing the continuous feature capture of the fall action.

[0012] Furthermore, the binding of user identity and RFID tag stored on the server in step S2 specifically includes: the identity verification camera captures facial information at the entrance of the monitoring area and sends it to the main control node; the main control node obtains the RFID code associated with the face from the server and distributes the RFID code to each of the data nodes for real-time identity verification.

[0013] Based on the above, the beneficial effects of the identity verification camera capturing facial information at the entrance of the monitoring area are as follows: by fixing the camera at the entrance of the environment, facial data is obtained in real time when the user enters, realizing the initial accurate binding of the user's identity and avoiding identity confusion during the monitoring process; the beneficial effect of the main control node obtaining the RFID code associated with the face from the server is that it calls the face-RFID mapping relationship pre-stored on the server, dynamically associating biometrics with electronic tags, and realizing unified management of multimodal identity identification; the beneficial effect of the main control node distributing the RFID code to each data node is that it enables all data nodes to synchronize the RFID tag of the target user, and combined with the RFID receiver of the data node to verify the identity in real time, realizing accurate matching of millimeter-wave radar data with specific objects in multi-user scenarios.

[0014] Furthermore, the three-level response in step S4 includes: the voice module playing a preset prompt, parsing user feedback through offline voice recognition, and determining a fall state if it is recognized that the user has confirmed falling or has not responded within a time limit.

[0015] Based on the above, the beneficial effects of the voice module playing preset prompts are: by broadcasting fixed inquiry content locally without generating voice data online, it achieves real-time status confirmation with privacy and security, avoiding the risk of leakage of online voice data transmission; the beneficial effects of parsing user feedback through offline voice recognition are: by utilizing the offline voice recognition capabilities built into the edge device, it can parse user responses in real time, achieving rapid feedback verification without network dependence and ensuring the real-time response of the system; the beneficial effects of determining a fall status by recognizing a confirmed fall or failure to respond within a timeout are: by combining voice active confirmation with timeout logic, it eliminates accidental touches from daily actions (such as suddenly lying down), and achieves final manual verification of the millimeter-wave radar detection results.

[0016] Furthermore, the method also includes a model optimization process: after user authorization, the millimeter-wave radar point cloud data is uploaded to the server, the server classifies the millimeter-wave radar point cloud data according to node location semantics and automatically labels new data through clustering algorithm, and retrains the context model in combination with the new dataset.

[0017] Based on the above, the beneficial effects of server-side semantic classification of millimeter-wave radar point cloud data by node location are: automatically dividing the dataset according to the deployment scenario of the data source node (such as bedroom / bathroom), realizing isolated management of multi-scenario data, and avoiding cross-scenario feature interference with model training; the beneficial effect of automatically labeling new data through clustering algorithms is that posture patterns can be identified without manual labeling, realizing efficient automated labeling of massive point cloud data and reducing model iteration costs; the beneficial effect of retraining the context model with new datasets is that the ResNet+UNet network is continuously optimized based on incremental data, enabling the bedroom / bathroom scenario-specific model to dynamically adapt to changes in user behavior, and achieving autonomous evolution of fall recognition accuracy.

[0018] Furthermore, the edge computing network initialization includes: several data nodes reporting location configuration parameters to the master control node; the master control node obtaining a corresponding context model from the server based on the location configuration parameters; and sending the context model and user RFID binding information to the data nodes.

[0019] Based on the above, the beneficial effect of the master control node sending the context model and user RFID binding information to the data node is to synchronously transmit the scene-specific model and user identity tag, enabling the data node to simultaneously load gesture recognition capability and identity verification basis, thus realizing the integrated readiness of the monitoring task of "scene adaptation + identity binding".

[0020] To make the above features of the present invention and the objectives to be achieved clearer, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0021] Figure 1 : This is a flowchart of the present invention; Figure 2 : This is a flowchart illustrating the specific implementation of the edge computing network initialization method of the present invention; Figure 3 : This is a flowchart illustrating the specific implementation of the fall warning method for the elderly according to the present invention; Figure 4 : This is a schematic diagram of the system hardware architecture of the present invention. Detailed Implementation

[0022] See Figures 1-4 As shown, This invention provides a fall warning method for the elderly based on edge multimodal hierarchical detection, characterized by comprising the following steps: S1. Deploy an edge computing network in the monitoring environment. The edge computing network includes a master control node and several data nodes. Load the corresponding context model according to the deployment location of each node. S2. The facial information of the user entering the monitoring environment is obtained through the identity verification camera, bound to the user identity and RFID tag stored on the server, and the binding information is distributed to several data nodes through the main control node; S3. The data node detects human activity through a human infrared sensor and activates millimeter-wave radar to collect millimeter-wave radar point cloud data in real time when a human body is detected. S4. Conduct graded fall detection: Primary detection: Based on the millimeter-wave radar point cloud data, analyze the rate of change of the three-dimensional contour of the monitored object. If it exceeds a set threshold, secondary detection is triggered. Secondary detection: Based on the loaded context model, a deep learning algorithm is used to identify the fall posture on the millimeter-wave radar point cloud data; Level 3 response: When the Level 2 detection identifies a fall, the voice module is activated to confirm the status of the monitored object; S5. If the fall is confirmed by voice or there is no response, a warning message is sent to the emergency center through the main control node.

[0023] In this embodiment, step S1, loading the corresponding context model, specifically includes: the master control node collecting the deployment location semantic information of several data nodes, requesting a point cloud attitude recognition model that matches the location semantics from the server, and distributing the point cloud attitude recognition model to the data nodes at the corresponding locations; the millimeter-wave radar uses a Texas Instruments IWR6843AOP module, operating in the 60-64GHz frequency band, and transmits data containing spatial location, contour, and attitude point cloud to the Hi3861 master control system via a serial port; the point cloud attitude recognition model is a bedroom / bathroom scene-specific model trained based on a ResNet+UNet network, which is pre-stored by the server and updated weekly.

[0024] The primary detection in step S4 of this embodiment specifically involves: caching the millimeter-wave radar point cloud data for 5 consecutive seconds, extracting the rate of change of the three-dimensional contour of the monitored object, and triggering the secondary detection when the rate of change of the three-dimensional contour exceeds a set threshold; the rate of change of the three-dimensional contour is achieved by calculating the aspect ratio change acceleration of the point cloud data within 5 seconds, and the set threshold is 0.8 m / s²; when an elderly person is detected to suddenly bend over or slip, the rate of change of the contour exceeds the threshold and triggers the secondary detection, while sitting down slowly is filtered out.

[0025] In this embodiment, step S2, binding the user identity and RFID tag stored on the server, specifically includes: the identity verification camera captures facial information at the entrance of the monitoring area and sends it to the main control node; the main control node obtains the RFID code associated with the face from the server and distributes the RFID code to each of the data nodes for real-time identity verification; the identity verification camera is fixed at the entrance of the monitoring area and uses a facial recognition algorithm to compare the information pre-stored on the server; the RFID tag is worn by the user, and the data nodes verify the identity in real time through a UHF band receiver, ensuring that millimeter-wave radar data is bound to the target in a multi-user environment (such as distinguishing between the elderly and visitors).

[0026] The three-level response in step S4 of this embodiment includes: the voice module plays a preset prompt, and the user feedback is analyzed through offline voice recognition. If a fall is confirmed or no response is received within 10 seconds, the user is determined to be in a fall state. The voice module uses the Hailink HLK-V20 offline voice module with preset prompts. The offline recognition supports 10 wake-word-free commands, and the timeout is set to determine a fall if there is no response within 10 seconds. According to experiments, the actual recognition rate within 5 meters is 95%.

[0027] In this embodiment, the method further includes a model optimization process: after user authorization, the millimeter-wave radar point cloud data is uploaded to the server; the server semantically classifies the millimeter-wave radar point cloud data according to node location and automatically labels the new data using a clustering algorithm; the context model is then retrained using the new dataset; the clustering algorithm uses DBSCAN density clustering to automatically label the newly uploaded point cloud data with tags such as "slipping in the bathroom" and "falling off the bed"; the retraining uses incremental learning to update the ResNet+UNet network parameters, and the accuracy of the optimized model will gradually improve.

[0028] In this embodiment, the edge computing network initialization includes: several data nodes reporting location configuration parameters to the master control node; the master control node obtaining a corresponding context model from the server based on the location configuration parameters; and sending the context model and user RFID binding information to the data nodes.

[0029] In this embodiment, the specific implementation process for edge computing network initialization is as follows: After installation, deployment, and parameter configuration, the edge computing network powers on and initializes. First, the master control node, data nodes, and identity verification cameras complete hardware self-tests and basic system initialization. Then, the master control node requests control permissions from the identity verification camera, which returns a device-ready status to the master control node. Each data node reports pre-configured location semantic parameters to the master control node. Based on the received location semantic parameters, the master control node requests the corresponding point cloud attitude recognition model from the server. The server responds to the request and transmits the model data to the master control node. Finally, the master control node distributes the point cloud attitude recognition model to each data node, and after model loading is complete, the edge computing network enters standby mode.

[0030] In this embodiment, the specific implementation process of the elderly fall warning method is as follows: When a monitored individual arrives at the entrance of the monitoring area, the identity verification camera captures a facial image and uploads it to the main control node. The main control node queries the server for the user's identity and RFID code associated with that face. After the server returns the corresponding RFID code, the main control node distributes it to all data nodes. The data nodes verify the monitored individual's identity and establish a binding relationship through RFID receivers. After successful verification, the data nodes drive millimeter-wave radar to collect the monitored individual's posture point cloud data. Based on the millimeter-wave radar point cloud data, the data nodes first perform a preliminary fall screening through 3D contour change rate analysis. If the threshold is exceeded, a secondary detection is initiated, using a point cloud posture recognition model for deep learning posture recognition. When a fall posture is detected, the voice module plays a preset prompt requesting status confirmation and parses the user's feedback through offline voice recognition. If the voice confirms the fall or there is no response after a timeout, the data node sends a fall confirmation signal to the main control node. After receiving the signal, the main control node updates the status record to the server and automatically sends a warning request containing location information to the emergency center. It should be noted that the information exchange between the server, the emergency center, the main control node, the data nodes, and the identity verification camera is completed through a public network egress router.

[0031] The above description is merely the optimal embodiment of the present invention and is not intended to limit the present invention. Any modifications or substitutions made by those skilled in the art without departing from the essence and scope of protection of the present invention should also be within the scope of protection of the present invention.

Claims

1. A method for early warning of elderly falls based on edge multimodal hierarchical detection, characterized in that, Includes the following steps: S1. Deploy an edge computing network in the monitoring environment. The edge computing network includes a master control node and several data nodes. Load the corresponding context model according to the deployment location of each node. S2. The facial information of the user entering the monitoring environment is obtained through the identity verification camera, bound to the user identity and RFID tag stored on the server, and the binding information is distributed to several data nodes through the main control node; S3. The data node detects human activity through a human infrared sensor and activates millimeter-wave radar to collect millimeter-wave radar point cloud data in real time when a human body is detected. S4. Conduct graded fall detection: Primary detection: Based on the millimeter-wave radar point cloud data, analyze the rate of change of the three-dimensional contour of the monitored object. If it exceeds a set threshold, secondary detection is triggered. Secondary detection: Based on the loaded context model, a deep learning algorithm is used to identify the fall posture on the millimeter-wave radar point cloud data; Level 3 response: When the Level 2 detection identifies a fall, the voice module is activated to confirm the status of the monitored object; S5. If the fall is confirmed by voice or there is no response, a warning message is sent to the emergency center through the main control node.

2. The method for early warning of elderly falls using edge multimodal hierarchical detection according to claim 1, characterized in that, The loading of the corresponding context model in step S1 specifically includes: the master node collecting the deployment location semantic information of several data nodes, applying to the server for a point cloud pose recognition model that matches the location semantics, and distributing the point cloud pose recognition model to the data nodes at the corresponding locations.

3. The method for early warning of elderly falls based on edge multimodal hierarchical detection according to claim 1, characterized in that, The primary detection in step S4 specifically involves: caching the millimeter-wave radar point cloud data for 5 consecutive seconds, extracting the rate of change of the three-dimensional contour of the monitored object, and triggering the secondary detection when the rate of change of the three-dimensional contour exceeds a set threshold.

4. The method for early warning of elderly falls based on edge multimodal hierarchical detection according to claim 1, characterized in that, The binding of user identity and RFID tag stored on the server in step S2 specifically includes: the identity verification camera captures facial information at the entrance of the monitoring area and sends it to the main control node; the main control node obtains the RFID code associated with the face from the server and distributes the RFID code to each of the data nodes for real-time identity verification.

5. The method for early warning of elderly falls based on edge multimodal hierarchical detection according to claim 1, characterized in that, The three-level response in step S4 includes: the voice module playing a preset prompt, parsing user feedback through offline voice recognition, and determining a fall status if it is recognized that the user has fallen or has not responded within a time limit.

6. The method for early warning of elderly falls based on edge multimodal hierarchical detection according to claim 1, characterized in that, The method also includes a model optimization process: after user authorization, the millimeter-wave radar point cloud data is uploaded to the server, the server classifies the millimeter-wave radar point cloud data according to node location semantics and automatically labels new data through clustering algorithm, and retrains the context model in combination with the new dataset.

7. The method for early warning of elderly falls based on edge multimodal hierarchical detection according to claim 1, characterized in that, The edge computing network initialization includes: several data nodes reporting location configuration parameters to the master control node; the master control node obtaining a corresponding context model from the server based on the location configuration parameters; and sending the context model and user RFID binding information to the data nodes.

Images