A multi-source sensing data collection and analysis method and monitoring system for home-based care for the elderly

Abstract

The application discloses a multi-source sensing data collection and analysis method and a monitoring system for home-based endowment, and relates to the technical field of Internet of Things and intelligent endowment monitoring. The method synchronously collects water immersion, smoke, gas sensor data, human behavior state data of a millimeter wave radar and visitor and door opening and closing event data of an intelligent doorbell, performs multi-modal fusion analysis on the environmental sensor data by taking the human behavior state of the millimeter wave radar as a verification factor and the intelligent doorbell event as auxiliary context, generates a hierarchical alarm decision and synchronously pushes the decision to a large-screen monitoring terminal, a mobile terminal and an intelligent doorbell interaction terminal. The application effectively reduces the false alarm rate of a single sensor, realizes accurate identification and cross-terminal cooperative alarm of multiple dangerous situations such as fire, gas leakage, fall, water leakage, getting lost and abnormal visitors, protects the privacy of the old by using non-visual sensing, and is suitable for various scenes such as home monitoring of the old, community endowment monitoring center and the like.

Description

Technical Field

[0001] This invention relates to the fields of Internet of Things, intelligent elderly care monitoring and security alarm technology, and in particular to a multi-source sensor data acquisition and analysis method that integrates water immersion sensors, smoke sensors, gas sensors, millimeter-wave radar and smart doorbells, as well as an alarm monitoring system for multiple terminals such as large-screen monitoring terminals and mobile terminals. Background Technology

[0002] As my country's population ages rapidly, the home safety and health issues of elderly people living alone or in empty nests are becoming increasingly prominent. The elderly face various risks in their home environment, such as falls, forgetting to turn off the stove, gas leaks, and water leaks, while traditional monitoring methods are clearly inadequate.

[0003] The existing technical solutions mainly have the following problems: (1) Single sensor and independent operation: Traditional systems only deploy single sensors such as smoke detectors, gas detectors or water immersion detectors. Each device works independently and lacks linkage analysis capabilities, making it impossible to form a comprehensive judgment on the elderly's home status.

[0004] (2) High false alarm rate: Smoke detectors are easily affected by cooking fumes, water immersion sensors cannot distinguish between water accumulation from mopping and actual leaks, and gas alarms still frequently give false alarms even in well-ventilated conditions, causing trouble for the elderly and their caregivers.

[0005] (3) Lack of human behavior verification: The existing system cannot confirm whether the elderly are present, whether they have fallen or have not moved for a long time, which makes it impossible for alarm information to distinguish between real danger and environmental interference.

[0006] (4) Single alarm terminal: Traditional solutions mostly rely on local sound and light alarms or SMS notifications, lacking a multi-terminal collaborative push mechanism such as large screen command terminal and mobile APP terminal, resulting in information fragmentation between the community, property management and family members.

[0007] (5) Privacy issues: Some solutions use camera surveillance, which infringes on the privacy of the elderly and is difficult to accept.

[0008] Existing patents, such as CN215180872U, disclose a multi-functional indoor alarm system that uses a millimeter-wave radar module in conjunction with environmental sensors such as smoke and gas detectors to achieve remote alarms. However, it does not integrate a water immersion sensor and does not involve the fusion of access control and visitor information in a smart doorbell. CN121938105A discloses a multi-modal fusion fire monitoring and emergency evacuation system that integrates smoke detectors and millimeter-wave radar for fire location, but it does not involve the comprehensive analysis of multi-source data such as water immersion, gas, and doorbell data in home-based elderly care scenarios. CN205563873U discloses a multi-functional doorbell for elderly people living alone, which integrates smoke, gas, and water immersion alarms and sends text messages through the doorbell host. However, its doorbell is only used as an alarm host and does not incorporate millimeter-wave radar for human behavior recognition, resulting in limited fusion judgment capabilities.

[0009] Therefore, there is an urgent need for a home-based elderly care monitoring system and method that can integrate multi-source sensor data, combine human behavior verification, and achieve cross-terminal collaborative alarm. Summary of the Invention

[0010] The purpose of this invention is to overcome the problems of isolated sensors, high false alarm rate, lack of human behavior verification, and insufficient terminal collaboration in the existing technology, and to provide a multi-source sensor data acquisition and analysis method and monitoring system for home-based elderly care. Through millimeter-wave radar, it can accurately perceive the presence, posture and activity of the human body, and combine multi-source data such as water immersion, smoke detection, gas detection, and smart doorbell for fusion analysis to achieve low false alarm, high reliability and cross-terminal home-based elderly care safety monitoring. Technical solution

[0011] The system described in this invention comprises four layers: a perception layer, an edge computing layer, a cloud platform layer, and an application layer.

[0012] The perception layer includes: (1) Water immersion sensor, deployed in areas prone to water leakage such as kitchen, bathroom, and balcony, to detect water accumulation on the ground; (2) Photoelectric smoke sensor, deployed in areas such as kitchens, bedrooms and corridors, to detect smoke concentration; (3) Catalytic combustion or NDIR gas sensors are deployed near the kitchen gas stove to detect the concentration of combustible gas; (4) Millimeter-wave radar module, with a working frequency of 24GHz or 77GHz, is deployed in key activity areas such as living room, bedroom, and bathroom to detect human presence, breathing, posture (standing / sitting / falling), activity trajectory and long-term static state; (5) Smart doorbell, deployed at the entrance door, has visitor identification, bell ringing event recording, audio and video intercom, door magnetic status detection functions, and can be expanded to carry a camera or infrared auxiliary sensor.

[0013] The edge computing gateway layer includes: (1) Data preprocessing module, which performs filtering, noise reduction and normalization on the data from each sensor; (2) Multimodal fusion analysis engine, which executes the data fusion and alarm decision algorithm described in this invention; (3) Local storage and network outage disaster recovery module, which caches data when the network is interrupted and retransmits it after the network is restored.

[0014] The cloud platform layer includes: (1) Time-series database, storing historical data and alarm events of each sensor; (2) Elderly behavior profiling module, which constructs a baseline of elderly people's daily routines based on long-term data; (3) Alarm rule engine and work order management module; (4) Device management and OTA upgrade module.

[0015] The application layer includes: (1) Large screen monitoring terminal, for community elderly care command center and property monitoring room, provides GIS map, alarm pop-up, heat map, trend curve and work order processing interface; (2) Mobile terminal APP / mini-program, targeting the elderly’s family members and guardians, providing tiered push notifications, one-click handling, and real-time status viewing; (3) Smart doorbell interactive terminal, for visitors and the elderly, providing audio and video intercom, ringing reminder, and abnormal door opening alarm; (4) Third-party interface, which can be connected to the 119 fire platform, community grid management system and property work order system.

[0016] The system synchronously collects data from three types of environmental sensors—water immersion, smoke, and gas—at a fixed period (e.g., adjustable from 0.1 to 1 Hz), collects millimeter-wave radar point cloud and human behavior tag data at a higher frequency (e.g., 10 to 30 Hz), and collects data from smart doorbell ringing, visitors, and door opening / closing via event triggering.

[0017] Sliding window mean filtering and outlier removal are applied to water immersion, smoke detection, and gas data; clutter suppression, static background filtering, target tracking, and trajectory extraction are performed on millimeter-wave radar data; and event timestamp alignment and visitor identification are applied to doorbell data. All data is uniformly timestamped and tagged with device IDs for use by the fusion engine.

[0018] The core of this invention lies in constructing a joint decision-making model with millimeter-wave radar human behavior as a verification factor, and introducing smart doorbell visitor and door opening / closing events as auxiliary context to achieve multi-scenario fusion judgment.

[0019] Scenario 1: Leak Authenticity Determination. When the water immersion sensor is triggered, the current human behavior status output by the millimeter-wave radar is queried. If the radar determines that no one is present or that people are far away from the water accumulation area, it is determined to be a Level 1 real leak alarm; if the radar determines that someone is active (such as mopping the floor or washing up), it is downgraded to a Level 3 suspected alarm and continued monitoring is performed.

[0020] Scenario 2: Gas Leak Verification. When the gas concentration exceeds the threshold, the presence of people is assessed in conjunction with millimeter-wave radar detection. If the kitchen area is unoccupied for more than the set threshold (e.g., 10 minutes), it is classified as a Level 1 gas leak alarm; if people are active in the kitchen, it may be normal gas usage for cooking, and the alarm is downgraded to a Level 2 alert with continuous monitoring of concentration trends.

[0021] Scenario 3: Fire and Trapped Status Alarm. When a smoke detector is triggered and millimeter-wave radar detects that a person has fallen or remained stationary for an extended period, it is classified as a Level 1 fire and trapped alarm. If a smoke detector is triggered but radar detects that a person is moving normally, it may be cooking smoke, and it is classified as a Level 2 alarm.

[0022] Scenario 4: Falls and Prolonged Inactivity. Millimeter-wave radar independently detects falls or prolonged inactivity in the bedroom / bathroom area (e.g., more than 30 minutes), combining this with smart doorbell door opening and closing records to determine recent absences. If the doorbell record shows no recent door openings or exits, it is classified as a Level 1 fall / unconsciousness alarm; if the doorbell record shows someone leaving, it is downgraded to a Level 2 alert.

[0023] Scenario 5: Abnormal Door Opening and Lost Person Alert. The smart doorbell detects a door opening event at night (e.g., 10:00 PM to 6:00 AM), and the millimeter-wave radar confirms that the elderly person is away from home and has not returned for an extended period. Based on the elderly person's historical daily routine, a Level 2 lost person alert is issued, and the family is notified.

[0024] Scenario 6: Abnormal Visitor Stay. The smart doorbell records visitor entry, and the millimeter-wave radar detects that the visitor stays in the elderly person's home for an extended period (exceeding the set time) while the elderly person remains stationary, triggering a level 2 abnormal visitor alarm and prompting family members to verify.

[0025] The system classifies alarms into three levels: Level 1 Alarm (Emergency): Fire trapped, gas leak, fall / unconsciousness, actual water leak. The push notification method is full-screen pop-up window on the large screen + voice broadcast + sound and light warning, strong reminder push on the mobile phone + SMS + automatic call to emergency contacts, and alarm sound triggered on the smart doorbell. Level 2 Alarm (Important): Lost Person Warning, Abnormal Visitor, Suspected Gas / Smoke. The push notification method is a highlighted label and prompt sound on the large screen, a notification on the mobile APP, and an event recording on the doorbell. Level 3 Alarm (Prompt): Suspected false alarm or environmental disturbance. The push notification is only recorded in the large screen log, and mobile devices can choose to subscribe.

[0026] Once the alarm is triggered, an emergency response work order is generated simultaneously and pushed to community grid workers or property duty personnel, forming a closed loop of early warning, dispatch, response, and feedback.

[0027] After alarm confirmation, the system can automatically or manually trigger the following linkages: a gas level 1 alarm automatically shuts off the smart gas solenoid valve and turns on the exhaust fan; a water immersion level 1 alarm automatically shuts off the smart water inlet valve; a fall level 1 alarm automatically turns on the emergency lighting and unlocks the smart door lock for rescue; a smoke level 1 alarm triggers the shutdown of the gas valve and prompts for power outage.

[0028] The cloud platform learns the elderly's daily routines based on at least 7 days of historical sensor data, including wake-up time, toilet frequency, nighttime activity patterns, and frequency of going out, to build a personalized baseline model. When real-time behavior deviates significantly from the baseline (such as prolonged periods away from bed at night or prolonged periods of inactivity during the day), a deviation warning is triggered. This baseline model supports weekly / monthly updates to adapt to the gradual changes in the elderly's lifestyle. Beneficial effects

[0029] (1) Significantly reduce false alarm rate: By using the human behavior state of millimeter-wave radar as a verification factor, the authenticity of water immersion, smoke and gas alarms is verified, effectively distinguishing between real danger and environmental interference. Actual measurements show that the false alarm rate can be reduced by more than 40%.

[0030] (2) Multi-source data complementarity: Introducing visitor and door opening / closing events from smart doorbells as auxiliary contexts to compensate for the inability of pure environmental sensors to determine the entry and exit of personnel and visitor status, thereby achieving more accurate missing person warnings and abnormal visitor identification.

[0031] (3) Privacy protection: Millimeter-wave radar and doorbell replace traditional cameras to achieve non-visual human body monitoring and visitor management. They do not collect sensitive images other than faces, which meets the privacy needs of the elderly.

[0032] (4) Multi-terminal collaboration: The large screen terminal realizes community-level macro-monitoring and command and dispatch, the mobile phone terminal realizes precise push and one-click handling for family members, and the doorbell terminal realizes visitor interaction and local alerts, forming a three-dimensional monitoring network.

[0033] (5) Adaptive learning: Based on the long-term behavioral data of the elderly, a personalized baseline is constructed to upgrade from rule alarm to intelligent deviation warning, thereby improving the system's adaptability to individual differences.

[0034] (6) Fast response speed: The edge computing gateway realizes local millisecond-level decision-making without relying on cloud latency; the network outage disaster recovery module ensures that no network anomalies are missed. Attached Figure Description

[0035] Figure 1 This is a diagram of the overall system architecture of the present invention.

[0036] Figure 2 This is a flowchart of the multimodal data fusion and alarm decision-making process of the present invention.

[0037] Figure 3 This is a schematic diagram of the interface of the large-screen monitoring terminal of the present invention.

[0038] Figure 4 This is a schematic diagram of the alarm push interface of the mobile APP of the present invention.

[0039] Figure 5 This is a schematic diagram of the interactive interface of the smart doorbell of the present invention. Detailed Implementation

[0041] The following devices are deployed in the residences of elderly people living alone: ​​a water immersion sensor and a 24GHz millimeter-wave radar are installed in the bathroom; a photoelectric smoke detector, an NDIR gas sensor, and a 24GHz millimeter-wave radar are installed in the kitchen; a smart doorbell (with a door sensor and camera) is installed on the entrance door; and a 77GHz millimeter-wave radar is installed in each of the living room and bedrooms. All sensors are connected to an edge computing gateway via the Zigbee protocol, and the gateway uploads data to the cloud platform via 4G / broadband.

[0042] Typical Case 1: An elderly person forgot to turn off the stove while cooking in the kitchen, causing the smoke concentration to slowly increase. The smoke detector triggered a level two alarm, and the millimeter-wave radar detected the elderly person moving normally in the kitchen. The system determined this to be cooking smoke and sent a level two alert to the mobile phone. Three minutes later, the smoke concentration continued to rise, and the radar detected that the elderly person had fallen in the living room after leaving the kitchen. The system upgraded to a level one fire and entrapment alarm, with a pop-up window on the large screen, a strong alert on the mobile phone, and a warning sound on the doorbell. At the same time, the gas valve was automatically shut off, and a community grid worker was notified to come to the door.

[0043] Typical Case 2: An elderly person fell in the bathroom while getting up at night. The millimeter-wave radar detected the fall and the person remained still for an extended period. The water immersion sensor was triggered when the elderly person knocked over a water glass. The fusion engine determined that this was a Level 1 fall alarm, triggering a red pop-up window on the large screen, a call to emergency contacts on the mobile phone, and the activation of emergency lighting and unlocking of the doorbell.

[0044] Typical Case 3: During the day, the smart doorbell detected the door being opened, and the millimeter-wave radar confirmed that the elderly person had left home. Six hours later, the radar detected that the elderly person had not returned and there was no record of the doorbell being opened again. The system triggered a level-two missing person alert, and a notification was pushed to the mobile phone, suggesting that the family contact the elderly person to confirm their whereabouts.

[0045] A community deployed this invention system for 50 elderly people living alone within its jurisdiction. The community's elderly care monitoring center deployed a large-screen monitoring terminal, accessing sensor data from all the elderly residents' homes. The large screen displays the location of each resident on a GIS map, using green / yellow / red to indicate normal / alert / alarm status. When a resident triggers a Level 1 alarm, the large screen automatically displays detailed information about that resident, including a millimeter-wave radar heat map, the most recent sensor readings, and recent visitor records from the doorbell. The duty officer dispatches a work order to the nearest grid worker with a single click. The grid worker receives the work order via a mobile app and navigates to the resident's home. After the incident is resolved, the system is marked as closed on the app, and the large screen status is updated to "resolved."

[0046] The system also records long-term behavioral data of each resident and generates a monthly report on the elderly’s home behavior for the community and their families to refer to. The report includes information such as sleep quality trends, frequency of nighttime activities, and changes in outing patterns.

[0047] A smart doorbell at an elderly person's residence recorded an opening at 2:00 AM and captured an image of an unfamiliar visitor. Millimeter-wave radar detected the elderly person was stationary in the bedroom (sleeping), but the visitor lingered in the living room area for an extended period. The fusion engine flagged this as a Level 2 abnormal visitor alarm, sending a notification to the family member's mobile phone and displaying the captured image. The family member then remotely communicated with the doorbell via voice chat on their mobile phone to verify the situation. Once confirmed safe, they manually deactivated the alarm. If the family member could not contact the elderly person, the alarm was escalated to Level 1, and the community and police were notified.

Claims

1. Claim 1 A method for multi-source sensor data acquisition and analysis for home-based elderly care, characterized in that, Includes the following steps: Simultaneously collect environmental data from water immersion sensors, smoke sensors, and gas sensors; human behavior data from millimeter-wave radar; and visitor and door opening / closing event data from the smart doorbell. The multi-source sensor data is preprocessed, including filtering, denoising, normalization, and timestamp alignment. Using human behavior data from millimeter-wave radar as a validation factor and smart doorbell visitor and door opening / closing events as auxiliary context, multimodal fusion analysis of environmental sensor data is performed to generate alarm decisions. The alarm level is determined based on the alarm decision, and the alarm information is simultaneously pushed to the large screen monitoring terminal, mobile terminal, and smart doorbell interaction terminal.

2. Claim 2 The method according to claim 1, characterized in that, The millimeter-wave radar's human behavior data includes: presence / absence of a person, standing, sitting / lying down, falling, respiratory rate, activity trajectory, and prolonged stillness; the multimodal fusion analysis includes: When the water immersion sensor is triggered and the millimeter-wave radar determines that there is no one or that people are far away from the water accumulation area, it is determined to be a level one water leakage alarm; When the gas concentration exceeds the threshold and the millimeter-wave radar determines that the kitchen area is unoccupied and this continues for more than the set time, it is considered a Level 1 gas alarm. When the smoke sensor is triggered and the millimeter-wave radar detects that a person has fallen or has been stationary for an extended period of time, it is determined to be a Level 1 fire entrapment alarm.

3. Claim 3 The method according to claim 1, characterized in that it further comprises: By using smart doorbell records of door opening events and millimeter-wave radar to confirm the status of people leaving home, it can determine whether an elderly person has not returned home for an extended period of time and trigger a missing person alert. The system uses a smart doorbell to record visitor entry events and millimeter-wave radar to detect when an elderly person is stationary and a visitor lingers for an extended period, triggering an abnormal visitor alarm.

4. Claim 4 The method according to claim 1, characterized in that it further comprises: Based on at least 7 days of historical multi-source sensor data, learn the daily routines of the elderly and build a personalized behavioral baseline model; When real-time sensor data deviates significantly from the behavioral baseline model, a deviation warning is triggered. The behavioral baseline model is updated weekly or monthly.

5. Claim 5 The method according to claim 1, characterized in that, The alarm levels are divided into three levels: Level 1 alarms include fire entrapment, gas leak, fall / unconsciousness, and actual water leak, which trigger a full-screen pop-up on the large screen, voice broadcast, strong mobile phone reminder, SMS, and automatic call to emergency contacts. Level 2 alarms include missing person alerts, abnormal visitor alerts, and suspected gas or smoke alerts, which trigger a highlight on the large screen and a notification on the mobile app. Level 3 alarms include suspected false alarms and environmental disturbances, which are only recorded in the logs on the large screen.

6. Claim 6 The method according to claim 1, characterized in that, After alarm confirmation, the system automatically or manually triggers equipment linkage control, including: The gas alarm at level one automatically shuts off the intelligent gas solenoid valve and turns on the exhaust fan. The intelligent water inlet valve automatically shuts off upon a level one water immersion alarm. A fall alarm will automatically activate emergency lighting and unlock the smart door lock. The smoke detector triggers a level one alarm and shuts off the gas valve.

7. Claim 7 A multi-source sensor monitoring system for home-based elderly care, employing the method described in any one of claims 1 to 6, characterized in that, The system includes: The sensing layer includes water immersion sensors, smoke sensors, gas sensors, millimeter-wave radar modules, and smart doorbells; The edge computing gateway layer includes a data preprocessing module and a multimodal fusion analysis engine; The cloud platform layer includes a time-series database, an elderly behavior profiling module, an alarm rule engine, and a device management platform; The application layer includes large-screen monitoring terminals, mobile terminal apps / mini-programs, smart doorbell interaction terminals, and third-party interfaces.

8. Claim 8 The system according to claim 7 is characterized in that, The large-screen monitoring terminal interface includes GIS map alarm point markings, millimeter-wave radar heat map, sensor numerical trend curves, alarm level color block markings, and a work order processing interface; the mobile terminal APP supports differentiated reminders according to alarm level and displays the area heat map generated by millimeter-wave radar and doorbell snapshot images; the smart doorbell interactive terminal supports audio and video intercom, ringing reminders, abnormal door opening alarms, and remote voice verification.

9. Claim 9 The system according to claim 7 is characterized in that, The edge computing gateway also includes a local storage and network outage disaster recovery module, which encrypts and stores the collected data and alarm events locally when the network is interrupted, and automatically synchronizes them to the cloud platform after the network is restored.

Images