Indoor old-age care safety monitoring and positioning system based on multi-sensor fusion

Abstract

The invention discloses an old-age care indoor safety monitoring and positioning system based on multi-sensor fusion, which belongs to the technical field of intelligent old-age care, and comprises a data acquisition module, a data processing and analysis module, a positioning module, an alarm and notification module, a storage module and a power supply module, the indoor safety monitoring and positioning system based on multi-sensor fusion is designed by comprehensively using technologies of indoor safety monitoring, FMCW radar positioning, YOLOv8 dangerous scene detection, multi-sensor fusion and the like. The system is composed of a data acquisition module, a data processing and analysis module, a positioning module, an alarm and notification module, a storage module and a power supply module, a series of system optimization schemes are designed, the false alarm rate of the system is reduced by adopting modes of optimizing a detection threshold value, analyzing historical data by means of machine learning and the like, and the data transmission efficiency is improved. Secondly, fall detection and fire detection are optimized by using an AI algorithm, so that the system can identify potential dangers more accurately and timely and take corresponding measures timely.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent elderly care, and particularly to an indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion. Background Art

[0002] Existing elderly care safety monitoring systems have comprehensively applied a number of technologies related to computers and big data, and integrated communication technologies and data analysis tools to achieve real-time monitoring of the health status and living environment of the elderly. Existing elderly care safety detection systems have been applied in various places such as homes, communities, shopping malls, etc., and certain results have been achieved.

[0003] However, there are still many deficiencies in existing elderly care safety monitoring systems. In indoor scenarios, especially in home environments, the related defects of existing technologies are particularly significant. Due to the particularity of the health status and physical fitness of the elderly, sudden dangerous scenarios such as falls and fires may have a fatal impact on the personal safety of the elderly. Therefore, it is particularly important to accurately monitor these dangerous scenarios and locate the position and movement trajectory of the elderly. However, so far, existing elderly care safety detection systems still cannot achieve the effect of accurate monitoring and precise positioning.

[0004] The application of existing technologies in the field of intelligent elderly care is subject to a series of restrictions, mainly including data accuracy and reliability issues, privacy protection issues, system integration and compatibility issues, technical limitations issues, user-friendliness and ease of use issues, insufficient personalized needs issues, imperfect emergency response mechanisms issues, and cost and penetration issues.

[0005] Therefore, it is necessary to develop an indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion to solve the above problems. Summary of the Invention

[0006] The purpose of the present invention is to provide an indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion.

[0007] To achieve the above purpose, the present invention provides the following technical solutions:

[0008] An indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion includes a data acquisition module, a data processing and analysis module, a positioning module, an alarm and notification module, a storage module, and a power supply module, wherein,

[0009] The data acquisition module: is used for accurately acquiring data;

[0010] The data processing and analysis module: converts the format of sensor data, and performs denoising and preprocessing of abnormal data, effectively avoiding false alarm situations;

[0011] Location module: Adopts a frequency-modulated continuous wave (FMCW) radar system, combines radar data with camera data to obtain the location trajectory of the elderly; simultaneously uses filtering algorithms for optimization;

[0012] Alarm and notification module: Sets up alarm trigger mechanisms for falls, fires, excessive carbon dioxide concentration, etc., and sets different alarm levels; after the alarm is triggered, the system immediately sends alarm notifications via text messages, APPs, and phone reminders;

[0013] Storage module: The system stores environmental data, health data, and historical alarm records;

[0014] Power supply module: Adopts joint management of mains power and backup batteries to improve the power supply capacity in emergency situations.

[0015] Specifically, the data acquisition module includes environmental sensors, smoke sensors, and fire sensors, and uses a camera based on the YOLOv8 algorithm, as well as an FMCW radar system; specifically,

[0016] Temperature and humidity sensor: Installed in the corners or on the walls of the room, avoiding direct sunlight and proximity to heat sources to ensure the accuracy of measurement results. The temperature and humidity sensor is used to monitor the indoor temperature and humidity in real time;

[0017] Carbon dioxide sensor: Installed in areas with poor ventilation to more accurately monitor changes in indoor carbon dioxide concentration. The carbon dioxide sensor is used to monitor the indoor carbon dioxide concentration in real time;

[0018] Smoke sensor: Installed on the ceiling of the kitchen, in a position closer to the heat source, so as to quickly detect smoke;

[0019] FMCW radar: Installed in a position that can cover the main activity area. Based on frequency-modulated continuous wave technology, it monitors the position information and movement trajectory of the elderly indoors in real time to achieve precise positioning and behavior analysis;

[0020] Camera: Installed in areas prone to falls and fire hazard areas, avoiding facing the door or window directly to prevent interference from strong light and affecting the monitoring effect; Based on an improved indoor dangerous scene detection method using the YOLOv8 algorithm, it conducts real-time video monitoring of the indoor environment to identify dangerous scenes such as fires, smoke, and people falling.

[0021] Specifically, the data processing and analysis module uses the YOLOv8 algorithm and AI algorithms to optimize the recognition of dangerous situations such as fires and falls; at the same time, it conducts joint analysis of environmental data, visual data, and radar data.

[0022] Specifically, the storage module adopts a combination of local storage and cloud storage to increase the storage capacity.

[0023] Specifically, the system adopts a data transmission method that combines wired and wireless means.

[0024] Specifically, for the FMCW radar and camera, a wired transmission method is adopted, and the sensors are connected to the data processing module through network cables or dedicated data transmission cables; for the temperature and humidity sensors, carbon dioxide sensors, and smoke sensors, a wireless transmission method is adopted, and the data collected by the sensors is transmitted to the data processing and analysis module by using wireless communication technologies such as WIFI, Bluetooth, or ZigBee.

[0025] The beneficial effects of the present invention are as follows:

[0026] The present invention comprehensively applies technologies such as indoor safety monitoring, FMCW radar positioning, YOLOv8 dangerous scenario detection, and multi-sensor fusion to design an indoor safety monitoring and positioning system based on multi-sensor fusion. The system consists of six modules: data acquisition, data processing and analysis, positioning, alarm and notification, storage, and power supply. A series of system optimization schemes are designed, and the false alarm rate of the system is reduced and the data transmission efficiency is improved by optimizing the detection threshold and analyzing historical data with the help of machine learning. Secondly, the present invention uses AI algorithms to optimize fall detection and fire detection, enabling the system to more accurately and timely identify potential dangers and take corresponding measures in a timely manner.

[0027] In terms of monitoring and positioning, the present invention adopts the fusion of vision and radar data, effectively improving the monitoring accuracy of the system, significantly enhancing the perception ability, reliability, and adaptability of the system, and at the same time enhancing the safety and comfort of the living environment of the elderly, providing all-round protection and support for the elderly on the premise of respecting privacy. This fusion not only enhances functions such as target detection, tracking, and obstacle avoidance, but also supports more advanced and complex indoor application scenarios.

[0028] In terms of alarm and notification, the system adopts a technology that combines intelligent alarm and remote notification, significantly enhancing the safety and response efficiency of the living environment of the elderly.

[0029] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly and implement it in accordance with the content of the specification, the following takes the preferred embodiments of the present invention and combines the drawings to describe in detail as follows. Description of the Drawings

[0030] Figure 1 It is the working flow chart of the indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion shown in the present invention;

[0031] Figure 2It is the data acquisition flow chart of the indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion shown in the present invention;

[0032] Figure 3 It is the data preprocessing flow chart of the indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion shown in the present invention;

[0033] Figure 4 It is the working flow chart of the YOLOv8 model applied to the camera shown in the present invention;

[0034] Figure 5 It is the flow chart of the DeepSORT algorithm combined with YOLOv8 used in the present invention;

[0035] Figure 6 It is the working flow chart of the positioning module shown in the present invention;

[0036] Figure 7 It is the flow chart of the dual power supply switching operation of the power supply module shown in the present invention. Detailed implementation manners

[0037] Next, the technical solutions of the present invention will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention. In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] Please refer to Figure 1, An indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion, including a data acquisition module, a data processing and analysis module, a positioning module, an alarm and notification module, a storage module, and a power module. Among them,

[0039] Data acquisition module: It includes environmental sensors, smoke sensors, and fire sensors, and adopts a camera based on the YOLOv8 algorithm, as well as a Frequency Modulated Continuous Wave (FMCW) radar; it realizes the accurate acquisition of data;

[0040] Specifically, the temperature and humidity sensor: It is installed in the corner or on the wall of the room, avoiding direct sunlight and being close to heat sources to ensure the accuracy of the measurement results. This temperature and humidity sensor is used to monitor the indoor temperature and humidity in real time;

[0041] Carbon dioxide sensor: It is installed in areas with poor ventilation, and can more accurately monitor the change of indoor carbon dioxide concentration. This carbon dioxide sensor is used to monitor the indoor carbon dioxide concentration in real time. This smoke sensor can quickly detect smoke;

[0042] Smoke sensor: It is installed on the ceiling of the kitchen, in a position close to the heat source, so as to be able to quickly detect smoke;

[0043] FMCW radar: It is installed in a position that can cover the main activity area. Based on the frequency modulated continuous wave technology, it can monitor the position information and movement trajectory of the elderly in the room in real time, and realize accurate positioning and behavior analysis;

[0044] Camera: It is installed in places where falls are likely to occur and areas with fire hazards, avoiding facing the door or window directly, so as not to be interfered by strong light and affect the monitoring effect; based on an improved indoor dangerous scene detection method of the YOLOv8 algorithm, it conducts real-time video monitoring of the indoor environment and identifies dangerous scenes such as fires, smoke, and people falling;

[0045] Data processing and analysis module: It converts the format of the sensor data, and performs denoising and preprocessing of abnormal data, effectively avoiding false alarms; for dangerous scene detection, the present invention uses the YOLOv8 algorithm and AI algorithm to optimize the recognition of dangerous situations such as fires and falls; at the same time, it conducts joint analysis of environmental data, visual data, and radar data to realize data fusion processing and efficient utilization of data;

[0046] Positioning module: It adopts a Frequency Modulated Continuous Wave (FMCW) radar, combines the radar data with the camera, and obtains the position trajectory of the elderly; at the same time, it uses a filtering algorithm for optimization, effectively reducing the positioning error and improving the positioning accuracy;

[0047] Alarm and Notification Module: An alarm trigger mechanism for falls, fires, excessive carbon dioxide concentration, etc. is set, and different alarm levels are defined. At the same time, after the alarm is triggered, the system immediately sends alarm notifications to relevant personnel via text messages, APPs, phone reminders, etc., so as to take emergency measures in a timely manner;

[0048] Storage Module; The system stores environmental data, health data, and historical alarm records. While diversifying the types of stored data, the present invention optimizes the existing storage method, adopts a combination of local storage and cloud storage, increases the storage capacity, and effectively prevents the loss of system-stored data;

[0049] Power Supply Module; To ensure the stable operation of the system, the present invention jointly manages the mains power and backup batteries to improve the power supply capacity of the power supply in emergency situations. In addition, the system adopts a low-power design, effectively improving the battery life of the system and reducing the power consumption cost to a certain extent.

[0050] Specifically, the present application adopts a data transmission method that combines wired and wireless. For sensors with high requirements for data transmission stability and real-time performance, such as FMCW radars and cameras, a wired transmission method is adopted, and the sensors are connected to the data processing module through network cables or dedicated data transmission cables to ensure high-speed, stable, and delay-free data transmission. For sensors with relatively low requirements for data transmission real-time performance, such as temperature and humidity sensors and carbon dioxide sensors, a wireless transmission method is adopted, and the data collected by the sensors is transmitted to the data processing module using wireless communication technologies such as WIFI, Bluetooth, or ZigBee. This combination of wired and wireless transmission methods not only ensures the stability of data transmission but also improves the convenience of system installation and maintenance.

[0051] Data Acquisition Process

[0052] Please refer to Figure 2 , after the system is started, all sensors enter the initialization state and are ready to start data acquisition. The temperature and humidity sensors and carbon dioxide sensors start first, and respectively monitor the indoor temperature, humidity, and carbon dioxide concentration. These environmental parameters are crucial for ensuring that the indoor environment is suitable for the elderly to live in.

[0053] In terms of safety monitoring, the smoke sensor remains vigilant at all times. Once it detects smoke, it immediately emits an alarm signal. In addition, the camera, in conjunction with an improved indoor dangerous scene detection method based on the YOLOv8 algorithm, conducts real-time video monitoring of the indoor environment, and can accurately identify dangerous scenes such as fires, smoke, and people falling, providing additional protection for the safety of the elderly. All data collected by the sensors, whether environmental parameters, physiological indicators, or dangerous scene information, will be transmitted in real time to the data processing and analysis module through wired or wireless data transmission methods.

[0054] During the data transmission process, in order to ensure the integrity and accuracy of the data, the system adopts advanced data verification and encryption technologies to prevent data loss or tampering during transmission. After receiving the original data, the data processing and analysis module will immediately perform format conversion and preprocessing for subsequent analysis and fusion processing. This process not only involves the preliminary screening and sorting of data, but also includes the marking and preliminary analysis of abnormal data to ensure that subsequent processing can be based on high-quality data.

[0055] The preprocessed data then enters the fusion processing stage. The system comprehensively analyzes the data from different sensors to form a comprehensive assessment of the indoor environment for the elderly and the safety status of the elderly. For example, combining the dangerous scenes captured by the camera with the radar positioning data can quickly determine the specific location where the danger occurs, providing key information for timely rescue. Finally, the data after fusion processing will be classified and stored in the storage module to support subsequent data analysis, system optimization, and query by guardians.

[0056] The data acquisition process, through the collaborative work of multiple sensors, realizes the comprehensive monitoring of indoor environment parameters for the elderly, physiological parameters of the elderly, and potential dangerous scenes. The data collected by each sensor, after preprocessing and fusion processing, ensures the accuracy and integrity of the data, providing high-quality data support for subsequent data analysis and system optimization, thus effectively ensuring the home safety of the elderly.

[0057] Data Processing and Analysis

[0058] 1. Data Preprocessing

[0059] The preprocessing of the data includes marking, deduplicating, and splitting the data to create a dataset for training and testing, and ensuring that the dataset is balanced, containing the same number of images of dangerous scenes and non-dangerous scenes.

[0060] The sensor uses tools such as LabelImg or manually draws bounding boxes in images and videos to mark the location and scope of dangerous scenes such as fires, and assigns a class label (such as fire, smoke) to each bounding box.

[0061] Duplicate data removal: Check if there are duplicate images or videos in the dataset. If so, delete one of them to avoid data redundancy.

[0062] Data splitting: Divide the dataset into a training set and a test set, and allocate the data according to a certain ratio (such as 80% and 20%) for training and testing the performance of the model.

[0063] Preprocess the data. Perform some necessary preprocessing steps on the images and videos, such as resizing, cropping, rotating, flipping, grayscaling, normalizing, etc., to improve the quality and diversity of the data and reduce the computational load.

[0064] Data balancing: Ensure that the training set and the test set contain the same number of images of dangerous and non - dangerous scenes to prevent the model from being biased towards one type of image. If there are too many images of a certain type, some can be deleted or data augmentation techniques (such as random noise, blurring, contrast adjustment) can be used to generate more images of the other type. For the specific process of data preprocessing, please refer to Figure 3 。

[0065] 2. Detection of dangerous scenes by YOLOv8:

[0066] (1) Model training:

[0067] Model training is one of the core steps to achieve dangerous scene detection using YOLOv8. First, a high - quality dataset needs to be prepared. For the detection of scenes such as falls and fires, collecting diverse images or videos is crucial. The data should cover different lighting conditions, perspectives, backgrounds, and scene variations, and the objects in each image or video (such as a fallen person or fire smoke) need to be manually annotated, specifying the bounding boxes of the objects and assigning corresponding labels. After the dataset is prepared, use tools (such as LabelImg) for image annotation to ensure the accuracy and comprehensiveness of the annotation. These annotated data will be used to train the YOLOv8 model so that it can learn how to recognize specific dangerous scenes.

[0068] Next, when training the YOLOv8 model, the data needs to be preprocessed and augmented. YOLOv8 requires the input images to be of a fixed size (such as 416x416 or 640x640), so the images need to be resized to a unified size. In addition, data augmentation techniques (such as rotation, cropping, flipping, color perturbation, etc.) can be used to increase the diversity of the training data, improve the generalization ability of the model, and reduce the risk of overfitting. The training of the model is usually based on fine - tuning a pre - trained model, which can accelerate the training process and improve performance. During training, the model will learn how to accurately predict the location and category of the target by optimizing the loss function (including localization loss, classification loss, and confidence loss), and at the same time, through a suitable optimizer (such as Adam or SGD) and adjusting the learning rate, the model gradually converges.

[0069] During the training process, it is also necessary to monitor the model's performance and make adjustments. The validation set is used to regularly evaluate the model's precision and recall to check for overfitting. If the model performs poorly on the validation set, its performance can be improved by further adjusting hyperparameters, using more data augmentation techniques, or increasing the dataset size. Additionally, after training is completed, the model is exported in a format suitable for inference (such as.pt or ONNX) and deployed in practical applications.

[0070] (1)The model identifies dangerous scenarios (fall / fire)

[0071] After training the YOLOv8 model, for real-time identification of falls and fires, it is first necessary to load the trained model. After training is completed, the YOLOv8 model is usually saved in the.pt format, which contains the weights and structural information during the training process. In the inference stage, by loading this model and passing in the image or video stream to be detected, the model will output the bounding box positions, class labels (such as "fall" or "fire"), and confidence scores for each target. The model can be loaded through code and an image can be passed in for inference. The model will return an object containing the detection results, including the position information and class of each target.

[0072] Next, the inference process will obtain the class and confidence of each detection box. However, since there may be multiple targets or overlapping boxes in the image, post-processing is usually required to optimize the results. YOLOv8 already has the Non-Maximum Suppression (NMS) algorithm built-in, which can automatically remove redundant boxes. Additionally, a confidence threshold can be set, such as only retaining targets with a confidence higher than a certain value (such as 0.5), to improve the accuracy of detection. Each detected target is identified based on the class label (such as "fall" or "fire"). By comparing the class information output by the model, it can be determined whether there is a dangerous scenario of a fall or fire in the image.

[0073] Finally, in practical applications, the YOLOv8 model is usually used for real-time monitoring of video streams, such as continuously detecting indoor scenes through a camera. The video stream is read through OpenCV and passed frame by frame into the model for inference. The system can detect falls or fires in real-time and respond. When the model detects a fall or fire, it can trigger an alarm system or an automatic response program, such as sending an alarm notification, starting an emergency plan, etc. This real-time monitoring and automated response make YOLOv8 an efficient tool in the security monitoring system, capable of providing timely warnings at critical moments. For details, please refer to Figure 4 。

[0074] 3. Optimization solutions for YOLOv8:

[0075] Combining the DeepSORT algorithm with YOLOv8 can significantly improve the accuracy of object detection. Especially in the multi-object tracking (MOT) task, this combination provides a powerful optimization solution. YOLOv8, as an efficient object detection model, can quickly and accurately identify objects in images or videos and output the bounding boxes, class labels, and confidence scores of each object. However, in dynamic scenarios, when used alone, YOLOv8 may be affected by factors such as object occlusion, fast movement, or lighting changes, resulting in a decrease in the accuracy of object detection. At this time, the introduction of DeepSORT can greatly enhance the stability and accuracy of detection by providing continuous multi-object tracking capabilities. DeepSORT uses the appearance features of each object for continuous tracking and re-identification of the object, ensuring that even when the object is briefly occluded or appears in a complex background, the correct identification of the object can still be maintained.

[0076] After the combination, the re-identification ability of DeepSORT further enhances the accuracy of YOLOv8. YOLOv8 mainly focuses on the spatial position and category of the object, while DeepSORT focuses on the appearance information (such as color, texture, etc.) of the object through deep feature extraction. This enables DeepSORT to accurately re-identify and associate the object from different frames even when the object is slightly occluded or the background is complex. This ability significantly reduces the problems of object loss or misdetection that YOLOv8 may encounter, especially in the case of fast-moving or overlapping objects. In addition, DeepSORT can also help YOLOv8 optimize the processing of low-confidence objects, enhancing the robustness of YOLOv8 in multi-object detection. For details, please refer to Figure 5 .

[0077] 4. Data Fusion Processing

[0078] In the multi-sensor fusion-based indoor safety monitoring and positioning system for the elderly, data fusion processing is the core link, involving the joint analysis of environmental data, visual data, and radar data to achieve accurate and efficient indoor safety monitoring.

[0079] During the data fusion process, the data processing and analysis module is responsible for jointly analyzing environmental data, visual data, and radar data. For example, when the visual data detects a fall event, the system will combine the radar data to determine whether the elderly is still on the ground and refer to the environmental data to evaluate whether there are environmental risks such as low oxygen or high temperature. Only when multiple data sources jointly verify the abnormal situation will the system trigger the alarm and notification module to avoid false alarms and improve the accuracy and reliability of detection. Through this multi-modal data fusion method, the system can more comprehensively and intelligently ensure the home safety of the elderly.

[0080] Environmental data (such as temperature and humidity, carbon dioxide concentration, smoke, and fire sensor data) are used to monitor changes in the indoor environment, ensure suitable air quality and temperature, and detect potential fire hazards in a timely manner. These data mainly adopt a threshold judgment method, that is, when the sensor data exceeds the safe range, the system will trigger an alarm.

[0081] Visual data are collected through cameras and the improved YOLOv8 algorithm is used for indoor dangerous scene detection, such as fires, smoke, and elderly people falling. YOLOv8 can not only accurately identify dangerous events but also combine with the DeepSORT algorithm for multi-object tracking to ensure the accurate identification and continuous monitoring of dangerous scenes.

[0082] Radar data (FMCW radar) are used to obtain the position information of the elderly in real time. The radar system emits and receives swept-frequency signals and combines with a multi-object tracking algorithm to extract the trajectory coordinates and speed information of the elderly. These data can be used to determine whether the elderly are moving normally or have been stationary for a long time (which may indicate a fall or abnormal condition).

[0083] Positioning module

[0084] 1. Obtain the position trajectory of the elderly through radar data

[0085] Data acquisition: The FMCW radar continuously scans the environment to obtain the distance and speed information of the elderly.

[0086] Target detection: Identify the target points in the radar data through signal processing algorithms (such as CFAR detection) to distinguish the elderly from other objects. Using constant false alarm rate detection (CFAR), the relationship between the detection probability Pd and the signal-to-noise ratio (SNR) is:

[0087] ;

[0088] is the Marcum Q function, is the false alarm probability.

[0089] Trajectory generation: Combine multiple frames of radar data and use the position and speed information of the target to generate the movement trajectory of the elderly.

[0090] Multi-object tracking: If there are multiple targets in the environment, multi-object tracking algorithms (such as Kalman filtering or particle filtering) can be used to distinguish and track the position of the elderly.

[0091] Using the Joint Probability Data Association (JPDA) algorithm, the association probability is:

[0092] ;

[0093] is the Mahalanobis distance between the i-th target and the j-th measurement.

[0094] Noise and multipath effects may exist in radar measurements, leading to positioning errors. To reduce the errors, the following filtering algorithms can be used:

[0095] Kalman filter: The Kalman filter is an optimal linear recursive estimation method. By predicting and updating the state of the system, it can optimally estimate the state of the system in the presence of noise and uncertainty. In FMCW radar positioning, the position and velocity of the elderly can be used as the state variables of the system, and the distance, velocity, and angle information measured by the radar can be used as the observations. The Kalman filter algorithm can be used to estimate the position and velocity of the elderly, thereby reducing the measurement error.

[0096] Extended Kalman filter (EKF): When the state equation and observation equation of the system are nonlinear, the Kalman filter is no longer applicable. In this case, the extended Kalman filter can be used. The extended Kalman filter linearizes the nonlinear equations to transform them into approximate linear equations, and then applies the Kalman filter algorithm for estimation. In FMCW radar positioning, if the motion model of the elderly is considered to be nonlinear, such as the elderly may make accelerating or turning motions, the extended Kalman filter can be used to improve the positioning accuracy.

[0097] Particle filter: The particle filter is a nonlinear filtering algorithm based on the Monte Carlo method. It represents the state distribution of the system through a set of random samples (particles). At each sampling moment, the particles are updated and resampled according to the state transition model and observation model of the system, so as to obtain the estimated value of the system state. The particle filter is suitable for nonlinear and non-Gaussian systems and can be used to handle complex motion models and noise distributions in FMCW radar positioning.

[0098] Data fusion: Combine the data of multiple radar sensors and perform data fusion through the extended Kalman filter (EKF) or unscented Kalman filter (UKF) to further optimize the positioning accuracy.

[0099] Please refer to Figure 5, By fusing the data of RFID, WiFi, and IMU, their advantages can be comprehensively utilized to make up for each other's deficiencies. For example, in an indoor environment, RFID can provide local high-precision location information, WiFi can provide a large-scale location reference, and IMU can accurately track the movement of an object in a short period of time. Specifically, when an object is moving, the IMU can measure its acceleration and angular velocity in real time to obtain the movement trajectory of the object; WiFi can determine the general area where the object is located based on the signal strength; RFID can provide accurate location correction at specific positions. By fusing the data of these three sensors, high-precision positioning can be achieved, and the accuracy can be improved to within 10 cm.

[0100] 2. Optimization of the positioning system

[0101] (1) Combining camera + radar data to improve positioning accuracy A single radar sensor may have limitations in a complex environment. Combining camera data can significantly improve positioning accuracy:

[0102] Camera data: The camera can provide rich visual information, such as the posture and direction of the elderly and details of the surrounding environment.

[0103] Data fusion: By fusing the distance and speed information of the radar with the visual information of the camera, the position and movement trajectory of the elderly can be estimated more accurately.

[0104] Deep learning: Using deep learning algorithms (such as convolutional neural network CNN) to process camera data, extract the features of the elderly, and fuse them with radar data to further improve positioning accuracy.

[0105] (2) Obstacle avoidance processing scheme

[0106] In an indoor environment, the elderly may encounter obstacles, so an obstacle avoidance processing scheme needs to be designed:

[0107] Environmental modeling: Construct a two-dimensional or three-dimensional map of the environment through radar and camera data to identify the positions of obstacles.

[0108] Path planning: Use path planning algorithms (such as A* algorithm or Dijkstra algorithm) to plan an optimal path for the elderly to avoid obstacles.

[0109] Real-time obstacle avoidance: During the movement of the elderly, update the environmental map and path planning in real time to ensure that the elderly can safely avoid dynamic obstacles.

[0110] For the detailed flowchart of positioning, please refer to Figure 6 , which specifically includes the following steps:

[0111] Data collection: The FMCW radar and camera collect environmental data simultaneously. The radar provides distance and speed information, and the camera provides visual information.

[0112] Data processing: Perform target detection and tracking on radar data, and feature extraction and target recognition on camera data.

[0113] Data fusion: Integrate the data from the radar and camera, and use filtering algorithms (such as Kalman filtering or particle filtering) to optimize position estimation.

[0114] Environmental modeling: Based on the fused data, construct an indoor environmental map to identify obstacles and walkable areas.

[0115] Path planning: Use path planning algorithms to plan an optimal path for the elderly to avoid obstacles.

[0116] Real-time obstacle avoidance: During the movement of the elderly, update the environmental map and path planning in real time to ensure the safety of the elderly.

[0117] Alarm, remote notification and storage

[0118] This system is built based on FMCW radar positioning, YOLOv8 visual detection and multi-sensor fusion technology, and includes an environmental perception layer, a data processing layer, a decision-making layer and a user interaction layer. Among them:

[0119] Environmental perception layer: Deploy devices such as temperature and humidity sensors, CO2 sensors, smoke sensors, millimeter-wave radars, and cameras.

[0120] Data processing layer: Integrate sensor data and perform real-time analysis through edge computing.

[0121] Decision-making layer: Trigger alarm logic and generate notification strategies.

[0122] User interaction layer: Push information through APP / sms / phone, and store data collaboratively in the cloud and locally.

[0123] Alarm and notification

[0124] I. Design of alarm trigger mechanism

[0125] 1. Multi-dimensional trigger conditions

[0126] (1) Health-related alarms: Fall detection by fusing millimeter-wave radar and vision

[0127] Technical implementation process:

[0128] FMCW radar signal processing:

[0129] Transmit a frequency-modulated continuous wave with a frequency of 76 - 81 GHz, and obtain the difference frequency signal through a mixer;

[0130] Use FFT to extract the target distance (resolution 0.15 m), speed (accuracy 0.1 m / s), and angle (±60°);

[0131] Generate 3D point cloud data (XYZ coordinate update rate 30 Hz).

[0132] YOLOv8 pose recognition optimization:

[0133] Deploy the lightweight YOLOv8s model (accelerated by TensorRT, inference speed 45 ms / frame);

[0134] For key point detection, use the 17-point COCO format to calculate the torso tilt angle (θ) and the change rate of the centroid height (Δh / Δt);

[0135] Mark as a suspected fall when θ > 60° and Δh / Δt > 1.2 m / s.

[0136] Multi-modal decision logic:

[0137] The radar detects a sudden change in vertical speed (>2.5 m / s downward);

[0138] The vision system confirms that the human body is in a lying position and there is no limb movement;

[0139] The standing posture has not been restored for 30 seconds;

[0140] Trigger condition formula:

[0141] ;

[0142] (2) Environmental alarms

[0143] Fire warning: Smoke concentration > 2.5% obs / m³ and sudden temperature rise > 3℃ / minute, or YOLOv8 recognizes open fire / dense smoke.

[0144] Gas leakage: CO2 concentration > 1500 ppm or methane concentration > 5% LEL, and it has not decreased for 30 seconds.

[0145] Intrusion detection: Unregistered personnel fail the YOLOv8 face recognition + abnormal movement detected by radar positioning.

[0146] (3) Equipment alarms

[0147] Heartbeat monitoring protocol:

[0148] The sensor sends a UDP heartbeat packet every 60 seconds (including device ID, battery level, signal strength);

[0149] The server maintains a device status table and marks it as offline if it times out for 5 minutes;

[0150] Use a Kalman filter to predict the power consumption curve and give early warnings of low power.

[0151] II. Notification System

[0152] 1. Multi-channel intelligent push strategy

[0153] Red alert:

[0154] Automatically call 3 preset emergency contacts and play TTS voice warnings;

[0155] APP pop-up window + strong vibration reminder;

[0156] Synchronously push text messages containing location map links (integrated with Amap API).

[0157] Orange alert:

[0158] APP message top pinning + text message push;

[0159] The community management platform automatically generates work orders.

[0160] Yellow / Blue alert:

[0161] APP message bar notification + WeChat service account push.

[0162] 2. Five-fold false alarm filtering mechanism

[0163] (1) Multi-source Validation

[0164] The system adopts multi-sensor fusion decision-making to ensure that the data of at least two different sensors meet the alarm conditions at the same time before triggering an alarm:

[0165] Radar fall detection + video pose analysis: If the radar sensor detects a fall event but the visual analysis does not detect an abnormal pose, the alarm is not triggered; only when both detect abnormalities, an alarm is generated.

[0166] Gas sensor + air quality camera analysis: If the CO2 sensor detects an excessive concentration and at the same time the visual system finds a high indoor population density, the system confirms the abnormality; otherwise, it may be a false alarm caused by short-term sensor drift.

[0167] 3. Time window confirmation

[0168] The system introduces a time window mechanism (Temporal Sliding Window) to ensure that an alarm is triggered only after the abnormal state lasts for more than the set threshold time, so as to filter out false alarms caused by instantaneous fluctuations:

[0169] Air quality alarm: If the CO2 concentration exceeds the set threshold (e.g., 1000 ppm) but only lasts for 30 seconds, it may be caused by air flow fluctuations; if it lasts for more than 60 seconds, it is determined as a real anomaly.

[0170] Temperature and humidity anomaly: If the temperature briefly exceeds the set upper limit but only lasts for 20 seconds, it is regarded as a fluctuation; if it lasts for more than the set time (e.g., 300 seconds), an alarm is triggered.

[0171] The above formula is as follows:

[0172] ;

[0173] Among them, is the anomaly score of the current time window, is the current data point, is the smoothing factor (the typical value can be set to 0.1 - 0.3). When exceeds the set threshold T and lasts for the time window ΔT, an alarm is triggered.

[0174] 4. Behavior pattern learning

[0175] To further reduce the false alarm rate, the system uses the "Long Short-Term Memory (LSTM)" model for behavior pattern learning, trains the normal activity pattern based on historical data, and deeply identifies the abnormal state. For example:

[0176] The morning behavior pattern of the elderly: Under normal circumstances, the elderly will experience a series of behaviors such as slow movement, sitting up, and standing up in the morning. If the system detects a sudden fall without previous activities, it may be a false alarm.

[0177] Abnormal breathing detection: By analyzing the individual's daily breathing pattern, if a sudden short breathing pause is detected but does not exceed the abnormal range of the individual's historical statistics, the system will not trigger an alarm.

[0178] The LSTM model can perform long-term dependence modeling on the input data through the "Temporal Feature Extraction" method and output the predicted value :

[0179]

[0180] Among them, , , b are the parameters of the LSTM network, is the hidden state. The system calculates the anomaly score by comparing the deviation between the actual state and the predicted state. If the deviation exceeds the set threshold, an alarm is triggered.

[0181] 5. Manual Review Channel

[0182] To further optimize the false alarm management of the system, a Human-in-the-Loop AI mechanism is added. When an alarm is pushed, a 10-second video clip is attached, allowing the recipient to quickly report a false alarm with one click. The specific mechanism includes:

[0183] Alarm Confirmation Interface: The user can select the "Confirm Alarm" or "False Alarm" button and can provide remarks;

[0184] False Alarm Feedback Optimization: The system uses false alarm data to update the machine learning model to improve the accuracy of future detections;

[0185] Intelligent Review System: Combining the update of user feedback weights, the system dynamically adjusts the sensor weights to reduce the false alarm rate.

[0186] This strategy adopts the Bayesian Update method to adjust the alarm confidence based on historical feedback data:

[0187]

[0188] Among them, P(A∣B) is the probability that the system will detect the same event in the future after the user confirms the alarm, P(B∣A) is the probability predicted by the current model, and P(A) is the prior probability of historical alarms.

[0189] 6. Dynamic Threshold Adjustment

[0190] The traditional alarm mechanism with a fixed threshold is difficult to adapt to environmental changes. Therefore, the system introduces the "Adaptive Thresholding" method to dynamically optimize the alarm conditions according to environmental factors. For example:

[0191] Temperature Alarm Adjustment: In summer, the upper limit of the temperature alarm is automatically relaxed by 2°C, and in winter, it is tightened to adapt to seasonal changes.

[0192] Optimization of Nighttime Fall Detection: At night, due to insufficient light, the false alarm rate of video fall detection is relatively high. Therefore, the system can increase the weight of radar signals to reduce the impact of visual false alarms.

[0193] The mathematical model can adopt the dynamic upper and lower limit adjustment formula:

[0194]

[0195] Among them, T′ is the threshold after adaptive adjustment, T is the default threshold, ΔE is the change in environmental factors (such as temperature, humidity, etc.), and k is the environmental sensitivity coefficient.

[0196] Storage

[0197] 2. Key Technologies for Storage Optimization

[0198] Hierarchical Storage:

[0199] Hot Data: Alarm videos in the past 3 days are stored in the local NVMe SSD;

[0200] Warm Data: Data from 3 to 30 days is stored in the cloud SSD;

[0201] Cold Data: Historical data is transferred and stored in Alibaba Cloud Archive Storage.

[0202] Compression Algorithm:

[0203] Videos are encoded using H.265, and the bitrate is controlled at 2 Mbps;

[0204] Sensor data is compressed in real time using Snappy, and the volume is reduced by 65%.

[0205] Privacy Protection:

[0206] Videos are uploaded after being blurred at the edge (OpenCV GaussianBlur);

[0207] Data transmission uses the national secret SM4 encryption, and AES-256-GCM is used for storage encryption.

[0208] Power Module

[0209] 1. Please refer to Figure 7 , this system adopts a power management solution that combines mains power and backup batteries to ensure the stable operation of the system. In the normal working state, the system is powered by the mains power to ensure that the device can work continuously and stably, meeting the power requirements of daily monitoring tasks. At the same time, the system is equipped with high-performance backup batteries. In the event of a mains power outage or abnormality, the backup batteries can seamlessly switch and provide power support for the system, ensuring that the system can still operate normally in case of emergencies, avoiding monitoring interruption due to power problems, and ensuring the continuity of indoor security monitoring.

[0210] 2. The system adopts a low-power design. By optimizing the hardware circuit and software algorithm, the energy consumption of the system during operation is reduced. In terms of hardware, low-power sensors, processors, and communication modules are selected, and unnecessary energy consumption is reduced through reasonable circuit design. In terms of software, an intelligent power management strategy is adopted to dynamically adjust the operating mode of the device according to the working state of the system. For example, when it is detected that there is no one or no abnormality in the room, the sampling frequency of the sensor and the operating speed of the processor are automatically reduced to reduce energy consumption. At the same time, the data processing and transmission algorithms are optimized to reduce the amount of data transmission and processing complexity, further reducing the system power consumption, thereby improving the endurance of the system, extending the usage time of the backup battery, and ensuring that the system can still operate for a long time without mains power supply.

[0211] The usage method of this application: After the system is started, the sensors in the data acquisition module automatically start to work. The temperature and humidity sensors, carbon dioxide sensors regularly collect indoor environmental data, the smoke sensor monitors fire hazards in real time, and the camera continuously collects image or video data. The collected data is transmitted to the data processing and analysis module in real time. This module performs format conversion and preprocessing on the data, analyzes the image and video data using an improved indoor dangerous scenario detection method based on the YOLOv8 algorithm, and at the same time performs threshold judgment and trend analysis on the environmental and physiological parameter data. The FMCW radar in the positioning module continuously emits and receives signals, obtains the location information of the elderly through data processing, and fuses it with other data to achieve comprehensive monitoring. If the data processing and analysis module detects an abnormality, the alarm and notification module immediately issues a sound alarm and a flashing light, and at the same time sends a notification containing the type of abnormality and location information to the guardian's mobile phone or other terminal devices through communication networks such as WIFI, GPRS / CDMA. After receiving the notification, the guardian can view the detailed information through the mobile application or other terminal devices and take corresponding measures.

[0212] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not conflict, they should all be considered to be within the scope described in this specification.

[0213] The above-described embodiments only represent several implementation manners of the present invention. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent should be subject to the appended claims.

Claims

1. The indoor safety monitoring and positioning system for elderly care based on multi-sensor fusion is characterized by: It includes a data acquisition module, a data processing and analysis module, a positioning module, an alarm and notification module, a storage module and a power module, among which: Data collection module: used for accurate data collection; Data processing and analysis module: converts the sensor data format, performs denoising and abnormal data preprocessing, effectively avoiding false alarms; Positioning module: It uses frequency modulated continuous wave radar and combines radar data with the camera to obtain the location trajectory of the elderly. It also uses filtering algorithms for optimization. Alarm and notification module: set alarm trigger mechanisms such as falls, fires, and excessive carbon dioxide concentrations, and set different alarm levels; after the alarm is triggered, the system immediately notifies the user through SMS, APP, and phone reminders; Storage module; the system will store environmental data, health data and historical alarm records; Power module: It adopts the joint management of mains power and backup battery to improve the power supply capacity in emergency situations.

2. The indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion as claimed in claim 1, characterized in that: The data acquisition module includes an environmental sensor, a smoke sensor and a fire sensor, and uses a camera based on the YOLOv8 algorithm and a frequency modulated continuous wave radar; specifically, Temperature and humidity sensor: Installed in the corner or on the wall of the room, avoid direct sunlight and close to heat sources to ensure the accuracy of the measurement results. The temperature and humidity sensor is used to monitor the indoor temperature and humidity in real time; Carbon dioxide sensor: installed in poorly ventilated areas, it can more accurately monitor changes in indoor carbon dioxide concentration. The carbon dioxide sensor is used to monitor indoor carbon dioxide concentration in real time; Smoke sensor: Installed on the kitchen ceiling, close to the heat source so that smoke can be detected quickly; FMCW radar: Installed in a location that covers the main activity area, based on frequency modulated continuous wave technology, it monitors the location information and movement trajectory of the elderly indoors in real time to achieve precise positioning and behavior analysis; Cameras: Installed in places where falls are likely to occur and in areas with fire hazards, avoid facing doors or windows to avoid interference from strong light and affect the monitoring effect; based on the improved indoor dangerous scene detection method of the YOLOv8 algorithm, real-time video monitoring of the indoor environment is carried out to identify dangerous scenes such as fire, smoke, and people falling.

3. The indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion as claimed in claim 1, characterized in that: The data processing and analysis module uses the YOLOv8 algorithm and AI algorithm to optimize the recognition of dangerous situations such as fire and falls; at the same time, it jointly analyzes environmental data, visual data and radar data.

4. The indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion as claimed in claim 1, characterized in that: The storage module combines local storage with cloud storage to increase storage capacity.

5. The indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion as claimed in claim 1, characterized in that: The system adopts a data transmission method that combines wired and wireless.

6. The indoor safety monitoring and positioning system for the elderly based on multi-sensor fusion as claimed in claim 5, characterized in that: The FMCW radar and camera use wired transmission to connect the sensor to the data processing module through a network cable or a dedicated data transmission cable; the temperature and humidity sensor, carbon dioxide sensor and smoke sensor use wireless transmission to transmit the data collected by the sensor to the data processing and analysis module using WIFI, Bluetooth or ZigBee wireless communication technology.

Images