Real-time dynamic sensing method based on artificial intelligence

Abstract

The invention relates to the technical field of Internet of Things and artificial intelligence, in particular to a real-time dynamic sensing method based on artificial intelligence, which comprises the following steps: S1, a multi-source data acquisition step: synchronously acquiring multi-dimensional real-time dynamic data of a user through a plurality of heterogeneous sensors deployed in a monitoring space, the sensor at least comprises a falling monitoring millimeter wave radar used for monitoring human body existence, a sign monitoring millimeter wave radar used for monitoring heart rate and breath, a sleep monitoring millimeter wave radar or an AI intelligent mattress used for monitoring on-bed / off-bed states and body positions, and an AI intelligent camera used for monitoring behaviors. According to the real-time dynamic sensing method based on artificial intelligence, false alarms possibly generated by a single sensor are effectively eliminated through multi-modal data cross validation. For example, a millimeter wave radar and an AI camera perform dual confirmation on a fall event, and the accuracy is far higher than that of a single technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things and artificial intelligence, in particular to a real-time dynamic sensing method based on artificial intelligence. BACKGROUND

[0002] With the acceleration of the aging process of society, the number of the elderly living alone and empty-nest elderly is increasing, and their home safety and health problems have become the focus of social attention. Traditional pension models, such as regular phone greetings, family visits or emergency call buttons, have the disadvantages of not timely response and limited coverage scenarios when dealing with unexpected events (such as falling down at night, sudden illness losing consciousness, etc.). The emergency call device relies on the user to actively seek help under the premise of being conscious and having the ability to operate, and is helpless for many unexpected conditions.

[0003] In order to solve this problem, a variety of monitoring devices based on single sensor have appeared in the market. For example, an infrared detector can detect human movement, but cannot distinguish specific personnel and cannot judge their vital signs; a surveillance camera can provide intuitive video images, but has serious privacy invasion problems and is not effective in insufficient light or with obstructions; a wearable device (such as a smart bracelet) can monitor heart rate, step count and other information, but faces many challenges such as poor user wearing compliance, forgetting to charge, taking off during bathing or sleeping, and resulting in monitoring interruption.

[0004] These single-technology solutions often cannot accurately and comprehensively understand the user's real state and the complex situation of the environment due to the single dimension of information, resulting in high false alarm rate and high risk of missing reports. For example, the system cannot effectively distinguish whether the user is normally lying down to rest or accidentally falling down, and it is also difficult to judge whether the user's long time in the bathroom is normal behavior or a syncope has occurred.

[0005] In order to solve the above problems, we have made improvements and proposed a real-time dynamic sensing method based on artificial intelligence. SUMMARY

[0006] In order to solve the above technical problems, the present application provides the following technical solutions: The present application provides a real-time dynamic sensing method based on artificial intelligence, comprising the following steps: S101. Multi-source data acquisition step: synchronously acquiring multi-dimensional real-time dynamic data of the user through a plurality of heterogeneous sensors deployed in the monitoring space, the sensors at least including a fall monitoring millimeter wave radar for monitoring human presence, a vital sign monitoring millimeter wave radar for monitoring heart rate and respiration, a sleep monitoring millimeter wave radar or an AI intelligent mattress for monitoring in-bed / off-bed state and body position, and an AI intelligent camera for monitoring behavior; S102. Data fusion and preprocessing step: receiving real-time dynamic data streams collected by the plurality of heterogeneous sensors, performing timestamp alignment, data cleaning and format standardization, and forming a unified multi-modal data structure; S103. Scene and state recognition step: calling a preset AI analysis engine to comprehensively analyze the unified multi-modal data structure, and recognizing the current state of the user in real time, the state at least including a spatial existence state (someone / no one), a position state (in bed / out of bed), a vital sign state (heart rate / respiration), and a posture and behavior state (body position / activity / stillness); S104. Abnormal event detection step: based on the recognized current state, using an event detection algorithm library to determine possible unexpected events in real time, the unexpected events at least including a fall, a long-time stillness, an abnormal out-of-bed at night, and an abnormal vital sign; S105. Warning and response step: once the unexpected event is detected, the system immediately generates a warning information, and sends an alarm to the preset guardian or management center through a specified notification mode, the warning information including the event type, the occurrence time, the position and the related sensor data snapshot.

[0007] As a preferred technical solution of the present application, the determination of the fall event is confirmed by the instantaneous sharp height change of the human body detected by the fall monitoring millimeter wave radar and the sudden body posture falling detected by the AI intelligent camera.

[0008] As a preferred technical solution of the present application, the determination of the long-time stillness is that the vital sign monitoring millimeter wave radar and the AI intelligent camera simultaneously detect that the user has no activity in the non-bed area for more than a preset time in the non-sleep period and has a vital sign signal.

[0009] As a preferred technical solution of the present application, the determination of the abnormal out-of-bed at night is that the user gets out of bed and does not return to bed within a preset time in the preset sleep period detected by the sleep monitoring millimeter wave radar or the AI intelligent mattress.

[0010] As a preferred technical solution of the present application, the AI analysis engine adopts a hierarchical structure, the bottom layer is a single sensor data analysis model, the middle layer is a multi-modal data space-time fusion model, and the top layer is a scene understanding and event reasoning model based on fusion features.

[0011] As a preferred technical solution of the present application, the system can dynamically adjust the parameter threshold of the determination of the unexpected event according to the personal habits and health status of the user, and realize personalized monitoring.

[0012] The present application has the following advantages: This AI-based real-time dynamic perception method effectively eliminates false alarms that may arise from a single sensor through multimodal data cross-validation. For example, the dual confirmation of a fall event by millimeter-wave radar and an AI camera is far more accurate than that of a single technology.

[0013] This AI-based real-time dynamic perception method covers multiple dimensions, from spatial presence and vital signs to specific behaviors. It can detect various potential risks, including falls, immobility, abnormal sleep, and nighttime wandering, forming a three-dimensional safety protection network.

[0014] This AI-based real-time dynamic perception method primarily uses non-visual sensors such as millimeter-wave radar and smart mattresses as its information sources, only calling upon AI cameras for auxiliary analysis when necessary. Furthermore, it can de-identify videos, maximizing the protection of users' personal privacy.

[0015] This AI-based real-time dynamic sensing method uses all non-contact, environmentally deployable sensors. Users do not need to wear any devices or change their lifestyles to achieve continuous 24 / 7 protection, solving the fundamental problem of poor compliance with wearable devices.

[0016] This AI-based real-time dynamic sensing method, through AI engine and user profiling, enables the system to not only issue simple threshold alarms, but also understand scenarios, identify complex events, and make adaptive adjustments based on individual differences, making the monitoring service more intelligent and humanized. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a real-time dynamic perception method based on artificial intelligence according to the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0019] Example: Figure 1 As shown, a real-time dynamic perception method based on artificial intelligence includes the following steps: S101. Multi-source data acquisition steps: Through multiple heterogeneous sensors deployed in the monitoring space, multi-dimensional real-time dynamic data of users are collected simultaneously. The sensors include at least a fall monitoring millimeter-wave radar for monitoring the presence of the human body, a vital sign monitoring millimeter-wave radar for monitoring heart rate and respiration, a sleep monitoring millimeter-wave radar or AI smart mattress for monitoring in-bed / out-of-bed status and body position, and an AI smart camera for monitoring behavior. S102. Data fusion and preprocessing steps: Receive real-time dynamic data streams collected by multiple heterogeneous sensors, perform timestamp alignment, data cleaning and format standardization to form a unified multimodal data structure; S103. Scene and State Recognition Steps: Call the pre-built AI analysis engine to perform comprehensive analysis on the unified multimodal data structure and identify the user's current state in real time. The state includes at least the spatial presence state (occupied / unoccupied), location state (in bed / out of bed), vital signs state (heart rate / respiration), and posture and behavior state (position / active / still). S104. Abnormal event detection steps: Based on the identified current state, use the event detection algorithm library to make real-time judgments on possible unexpected events. Unexpected events include at least falls, prolonged immobility, abnormal nighttime departure from bed, and abnormal vital signs. S105. Warning and Response Steps: Once an unexpected event is detected, the system immediately generates a warning message and sends an alarm to the preset monitoring personnel or management center through the specified notification method. The warning message includes the event type, time of occurrence, location, and relevant sensor data snapshots.

[0020] The determination of a fall incident is based on a combination of the sudden and drastic change in height of the human body detected by fall monitoring millimeter-wave radar and the sudden fall captured by AI smart camera.

[0021] The determination of prolonged immobility is made during non-sleep periods by simultaneously detecting, through millimeter-wave radar and AI smart camera, that the user has been inactive and has vital signs for more than a preset time in a non-bed area.

[0022] The determination of abnormal nighttime bed leaving is based on the detection of the user leaving the bed by sleep monitoring millimeter-wave radar or AI smart mattress within a preset sleep period, and the user not returning to the bed within the preset time.

[0023] The AI ​​analysis engine adopts a layered structure: the bottom layer is a single-sensor data parsing model, the middle layer is a multimodal data spatiotemporal fusion model, and the top layer is a scene understanding and event reasoning model based on fusion features.

[0024] The system can dynamically adjust the threshold parameters for judging unexpected events based on the user's personal habits and health status, thereby achieving personalized monitoring.

[0025] Specifically, this platform will be deployed in a typical senior living apartment. Fall detection millimeter-wave radar (providing wide-area personnel positioning and fall detection) will be installed on the ceiling of the living room and the hallway leading to the bathroom. Vital sign monitoring millimeter-wave radar (facing the bed, used to monitor heart rate and respiration during sleep) will be installed on the wall opposite the head of the bed in the bedroom. An AI smart mattress will be placed on the bed (monitoring bed-on / off status, body position, and number of times the user turns over). An AI smart camera (providing a global view for behavior analysis) will be installed high in the corners of both the living room and bedroom.

[0026] The edge computing gateway within the apartment collects all sensor data in real time. For example, it receives reports from fall radar ("Target 1, coordinates (x, y), velocity v, state: moving"), vital sign radar ("Heart rate: 65 bpm, respiration: 16 rpm"), AI mattress ("State: in bed, position: supine"), and AI camera ("Human skeletal points detected, behavior: standing"). The gateway timestamps these heterogeneous data packets and stores them in a cache queue.

[0027] The AI ​​analytics engine processes this multimodal data stream. At one point, the engine analyzes and concludes that: "The user is currently 'in bed' (from the mattress signal) and is in a 'sleep' state (from the steady, low-frequency heart rate and breathing from the vital signs radar, and from the camera's detection of prolonged periods without significant movement)."

[0028] At 2 AM, the AI ​​mattress first reported "Status: Away from bed." Sleep monitoring millimeter-wave radar confirmed no one was in bed. Fall detection radar then detected a moving target in the bedroom and quickly tracked it along the path to the bathroom. The AI ​​camera confirmed it was the user. This combination of statuses was identified as "getting up at night," a normal behavior.

[0029] However, after the user entered the bathroom, the fall detection radar continued to report that the target was inside, but the status changed to "still." This "still" status persisted for 15 minutes. At this point, the "prolonged stillness in the bathroom" event model was triggered. The model further queried the vital signs radar (which, despite the wall, could still weakly detect life signals) or the dedicated vital signs radar deployed in the bathroom, confirming that vital signs were still present. Based on this comprehensive assessment, the system classified this event as "suspected bathroom accident (such as slipping and being unable to get up or fainting)."

[0030] The system immediately generates an alert: "[Emergency Alert] Mr. Wang experienced an accident in the bathroom at 02:15 and has been motionless for more than 15 minutes. Please check immediately!" This alert is sent simultaneously to his son and the contracted community elderly care service center via app push notification and telephone voice message. Service center staff, with authorization, can view the anonymized images of the scene accompanying the alert (e.g., showing only human silhouettes) to assess the situation and take appropriate action.

[0031] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time dynamic perception method based on artificial intelligence, characterized in that, Includes the following steps: S101. Multi-source data acquisition steps: Multi-dimensional real-time dynamic data of users are collected synchronously through multiple heterogeneous sensors deployed in the monitoring space. The sensors include at least a fall monitoring millimeter-wave radar for monitoring the presence of the human body, a vital sign monitoring millimeter-wave radar for monitoring heart rate and respiration, a sleep monitoring millimeter-wave radar or AI smart mattress for monitoring in-bed / out-of-bed status and body position, and an AI smart camera for monitoring behavior. S102. Data fusion and preprocessing steps: Receive the real-time dynamic data streams collected by the multiple heterogeneous sensors, perform timestamp alignment, data cleaning and format standardization to form a unified multimodal data structure; S103. Scene and State Recognition Steps: Call the pre-set AI analysis engine to perform comprehensive analysis on the unified multimodal data structure and identify the user's current state in real time. The state includes at least the spatial presence state (occupied / unoccupied), location state (in bed / out of bed), vital signs state (heart rate / respiration), and posture and behavior state (position / activity / stillness). S104. Abnormal event detection step: Based on the identified current state, use the event detection algorithm library to make real-time judgments on possible unexpected events, including at least falls, prolonged immobility, abnormal nighttime departure from bed, and abnormal vital signs. S105. Warning and Response Steps: Once an unexpected event is detected, the system immediately generates a warning message and sends an alarm to the preset monitoring personnel or management center through a specified notification method. The warning message includes the event type, time of occurrence, location, and related sensor data snapshots.

2. The real-time dynamic perception method based on artificial intelligence according to claim 1, characterized in that, The determination of the fall event is based on a combination of the instantaneous and drastic change in height of the human body detected by the fall monitoring millimeter-wave radar and the sudden fall captured by the AI ​​smart camera.

3. The real-time dynamic perception method based on artificial intelligence according to claim 1, characterized in that, The determination of prolonged immobility is made when, during non-sleep periods, the user is detected simultaneously by a millimeter-wave radar for vital signs monitoring and an AI smart camera to be inactive and showing vital signs for more than a preset duration in a non-bed area.

4. The real-time dynamic perception method based on artificial intelligence according to claim 1, characterized in that, The determination of abnormal nighttime bed leaving is based on the detection of the user leaving the bed by sleep monitoring millimeter-wave radar or AI smart mattress within a preset sleep period, and the user not returning to the bed within the preset time.

5. The real-time dynamic perception method based on artificial intelligence according to claim 1, characterized in that, The AI ​​analysis engine adopts a layered structure, with the bottom layer being a single-sensor data parsing model, the middle layer being a multimodal data spatiotemporal fusion model, and the top layer being a scene understanding and event reasoning model based on fusion features.

6. The real-time dynamic perception method based on artificial intelligence according to claim 1, characterized in that, The system can dynamically adjust the parameter thresholds for judging the unexpected events based on the user's personal habits and health status, thereby achieving personalized monitoring.

Images