Fall detection and physiological sign and sleep monitoring and early warning system and method

Abstract

The invention discloses a fall detection, physiological sign and sleep monitoring and early warning system and method, and relates to the field of health monitoring, the system comprises a millimeter wave radar, a function module and a data real-time monitoring and early warning module; the signal output end of the millimeter wave radar is connected with the signal input end of the function module. The signal output end of the function module is connected with the signal input end of the data real-time monitoring and early warning module. The function module obtains physiological sign data through the phase change original signal and obtains the sleep condition based on the physiological sign data; generating point cloud data based on the original signal, and further obtaining human body posture data; the data real-time monitoring and early warning module is used for processing, storing and visually displaying the physiological sign data and the human body posture data, and carrying out real-time detection and fall early warning on abnormal physiological sign data, sleep and human body postures. According to the invention, the safety of medical monitoring can be improved, and more efficient and reliable technical support is provided for intelligent medical treatment and health management.

Description

Technical Field

[0001] This application relates to the field of health monitoring, and particularly to a fall detection, physiological sign, and sleep monitoring and warning system and method. Background Art

[0002] In recent years, with the booming development of artificial intelligence, cloud computing, Internet of Things, and big data technologies, intelligent health monitoring technologies have continuously made breakthroughs and have been widely applied in fields such as healthcare, smart elderly care, and safety monitoring. Especially in the scenarios of elderly people's home care and medical care, fall accidents and sudden physiological abnormalities (such as cardiovascular diseases) have become one of the major health risks. How to efficiently and accurately detect fall events, simultaneously achieve non-contact real-time monitoring of physiological signs, and provide timely warnings has become an important research direction in health monitoring technologies.

[0003] Currently, there are various fall detection, physiological sign, and sleep monitoring solutions on the market, which can be roughly divided into two categories: contact type and non-contact type. Contact type methods mainly rely on wearable devices, such as smart bracelets, heart rate monitors, sleep monitors, etc. Although these devices can achieve basic physiological parameter measurements, there are limitations in wearing comfort, user compliance, and long-term use convenience. In addition, the elderly may, due to reasons such as forgetting, rejection, or mobility difficulties, cause the devices to be unable to continuously and effectively function. Non-contact type methods mainly rely on technologies such as visible light cameras, infrared sensors, and lidar. Although they can monitor the human body's activity state to a certain extent, they have poor stability in complex environments (such as light changes, occlusion interference, etc.), and there are also concerns about privacy protection.

[0004] Millimeter-wave radar, as an emerging non-contact sensing technology, has shown broad application prospects in fields such as smart home, smart healthcare, and security monitoring in recent years. Millimeter-wave radar has strong penetration, all-weather working ability, and can achieve real-time monitoring of respiration, heart rate, and sleep by accurately capturing the micro-motion signals of the human body. In addition, combined with advanced signal processing algorithms and deep learning models, millimeter-wave radar can efficiently identify human body posture changes, accurately detect fall events, and perform abnormal warnings in combination with the changing trends of physiological parameters. Compared with traditional detection technologies, millimeter-wave radar has higher environmental adaptability and privacy protection advantages, providing a more intelligent, unobtrusive, and non-invasive solution for health monitoring. However, currently, only the methods of processing the signals of millimeter-wave radar to achieve fall detection or physiological sign and sleep monitoring have been studied, lacking the research on an integrated system structure for fall detection, physiological sign, and sleep monitoring using millimeter-wave radar. Summary of the Invention

[0005] The objective of this application is to provide a fall detection, physiological sign, and sleep monitoring and warning system and method to enhance the safety of medical monitoring, and further provide more efficient and reliable technical support for intelligent healthcare and health management.

[0006] To achieve the above objective, this application provides the following solutions:

[0007] In the first aspect, this application provides a fall detection, physiological sign, and sleep monitoring and warning system, including: a millimeter-wave radar, a functional module, and a data real-time monitoring and warning module;

[0008] The signal output end of the millimeter-wave radar is connected to the signal input end of the functional module; the signal output end of the functional module is connected to the signal input end of the data real-time monitoring and warning module;

[0009] The functional module is configured to execute: based on the original signal output by the millimeter-wave radar, obtain physiological sign data through phase change, and obtain the sleep situation based on the physiological sign data; generate point cloud data based on the original signal output by the millimeter-wave radar, and obtain human body posture data based on the point cloud data; the physiological sign data includes: respiration data and heart rate data; the human body posture data includes the human body movement trajectory and the human body posture recognition result;

[0010] The data real-time monitoring and warning module is configured to execute: process and store the physiological sign data and the human body posture data, and visually display the processed physiological sign data, sleep situation, and human body posture data. At the same time, perform real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture and fall warning.

[0011] Optionally, the functional module includes: a target locking unit, a physiological sign monitoring unit, and a human body posture detection unit;

[0012] The signal input end of the target locking unit serves as the signal input end of the functional module; the signal input end of the target locking unit is respectively connected to the signal input ends of the physiological sign monitoring unit and the human body posture detection unit; the signal output ends of the physiological sign monitoring unit and the human body posture detection unit are integrated as the signal output end of the functional module;

[0013] The target locking unit is configured to execute: perform fast Fourier transform on the original signal output by the millimeter-wave radar based on the distance dimension to obtain the distance information of the target, and lock the target position through adaptive threshold screening;

[0014] The physiological sign monitoring unit is configured to perform: after locking the target position, extracting a respiration signal and a heartbeat signal from the signal corresponding to the target position; performing spectral analysis on the respiration signal and the heartbeat signal by using continuous wavelet transform, and obtaining the corresponding waveforms of respiration and the corresponding waveform of heartbeat frequency in real time to obtain the physiological sign data;

[0015] The human body posture detection unit is configured to perform: obtaining the azimuth angle and elevation angle information of the target according to the original signal output by the millimeter wave radar by using a spatial spectrum estimation algorithm based on sparse reconstruction; enhancing the accuracy of the distance information of the target by using a peak detection method, and determining the velocity information of the target by using a Doppler fast Fourier transform method to generate point cloud data of the target; inputting the point cloud data into a deep learning model to perform feature extraction and classification recognition on the human body posture to obtain the human body posture data.

[0016] Optionally, the data real-time monitoring and early warning module includes: a data receiving unit, a data display unit, a data storage unit, and an abnormal early warning unit;

[0017] The signal input end of the data receiving unit serves as the signal input end of the data real-time monitoring and early warning module; the signal input end of the data storage unit is connected to the signal output end of the data receiving unit; the signal output end of the data storage unit is respectively connected to the signal input end of the data display unit and the signal input end of the abnormal early warning unit; the signal output ends of the data storage unit, the data display unit, and the abnormal early warning unit are integrated into the signal output end of the data real-time monitoring and early warning module;

[0018] The data receiving unit establishes a connection with the data storage unit, performs format conversion and processing on the data, so that the converted data meets the storage requirements of the data storage unit;

[0019] The data storage unit processes and stores the physiological sign data and the human body posture data; the data display unit visually displays the processed physiological sign data, sleep condition, and human body posture data; the abnormal early warning unit performs real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture and fall early warning.

[0020] Optionally, both the data display unit and the abnormal early warning unit include data interaction interfaces; the data interaction interfaces perform data and early warning interactions with peripheral intelligent terminals.

[0021] Optionally, the peripheral intelligent terminals include a Web end and a mini program end;

[0022] Warnings are given on the Web side and the mini-program side to remind the user to check the status of the monitored person and the device status; and when there are abnormalities in the status of the monitored person and the device status, warning notifications are sent via wireless communication; the recipients of the warning notifications are the contacts set on the Web side and the mini-program side.

[0023] Optionally, the data storage unit includes an InfluxDB time series database and a MySQL structured database;

[0024] The InfluxDB time series database stores the physiological sign data, sleep condition data, and human body posture data received from the functional module; the MySQL structured database is used to store the user's device data, personal information data, sleep report data, and historical statistical data.

[0025] Optionally, the display unit reads static structured data from the MySQL structured database to support data maintenance and query.

[0026] Optionally, the Web side and the mini-program side provide a front-end UI interface for data interaction; the front-end UI interface is built using the Vue framework;

[0027] The front-end UI interface includes a home page, a physiological signs and sleep module, a human body posture module, and a device management module; the home page is used to display the working status of the device; the physiological signs and sleep module is used to display the real-time physiological sign data of all monitored persons, the change of physiological sign data in different time periods within a set date, and the sleep condition; the human body posture module is used to display the real-time point cloud data, posture recognition results, and movement trajectory information of all monitored persons; the device management module is used to provide a data interaction interface.

[0028] In a second aspect, the present application provides a fall detection, physiological signs and sleep monitoring and warning method, including:

[0029] Based on the original signal output by the millimeter-wave radar, physiological sign data is obtained through phase change, and the sleep condition is obtained based on the physiological sign data; the physiological sign data includes: respiration data and heart rate data;

[0030] Point cloud data is generated based on the original signal output by the millimeter-wave radar, and human body posture data is obtained based on the point cloud data; the human body posture data includes the human movement trajectory and the human body posture recognition result;

[0031] Process and store the physiological sign data and the human body posture data, and visually display the processed physiological sign data, sleep conditions, and human body posture data. At the same time, perform real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture, as well as fall warnings.

[0032] Optionally, the process of performing real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture, as well as fall warnings, includes:

[0033] When the respiratory rate in the respiratory data is not within the normal threshold range of the respiratory rate or the heart rate in the heart rate data is not within the normal threshold range of the heart rate, determine that the physiological sign data is abnormal data;

[0034] When the difference between the relevant data of sleep quality and the normal range threshold reaches a set value, determine that there is an abnormal sleep situation;

[0035] Determine whether the human body posture has changed within a set time. If there is no change, determine that the human body posture is abnormal;

[0036] Use a deep learning algorithm to identify the transmitted point cloud data to obtain the human body posture prediction result;

[0037] When it is determined that the human body posture prediction result belongs to the fall state, generate a fall warning.

[0038] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0039] The present application provides a fall detection, physiological sign and sleep monitoring and warning system and method, which can use millimeter-wave radars with different functions as data sources, and realize real-time processing and detection of physiological sign data, sleep conditions, and human body posture data through functional modules. The data real-time monitoring and warning module performs real-time anomaly detection and fall warnings based on the data obtained from the functional modules, enabling users to view the status and historical data fluctuations of the monitored personnel in real time, and manage the operation of the device. Moreover, when an abnormal situation is detected, a reminder can be sent to the user in a timely manner through warning means for timely handling, realizing all-weather and all-round monitoring of the monitored personnel, thereby improving the safety of medical monitoring and providing more efficient and reliable technical support for intelligent healthcare and health management. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 It is a schematic structural diagram of a fall detection, physiological sign and sleep monitoring and warning system provided by an embodiment of the present application;

[0042] Figure 2 It is a schematic structural diagram of a functional module provided by an embodiment of the present application;

[0043] Figure 3 It is a schematic structural diagram of a data real-time monitoring and warning module provided by an embodiment of the present application;

[0044] Figure 4 It is another schematic structural composition diagram of a fall detection, physiological sign and sleep monitoring and warning system provided by an embodiment of the present application;

[0045] Figure 5 It is a basic working flowchart of a fall detection, physiological sign and sleep monitoring and warning system provided by an embodiment of the present application;

[0046] Figure 6 It is a schematic framework structure diagram of a fall detection, physiological sign and sleep monitoring and warning system provided by an embodiment of the present application. Detailed implementation manners

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0048] To make the above objects, features and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the drawings and specific implementation manners.

[0049] In an exemplary embodiment, the present application provides a fall detection, physiological sign and sleep monitoring and warning system, as Figure 1 shown, the system includes: a millimeter-wave radar 100, a functional module 200, and a data real-time monitoring and warning module 300.

[0050] The signal output end of the millimeter-wave radar 100 is connected to the signal input end of the functional module 200. The signal output end of the functional module 200 is connected to the signal input end of the data real-time monitoring and early warning module 300.

[0051] The functional module 200 is configured to perform: Based on the original signal output by the millimeter-wave radar 100, obtain physiological sign data through phase change, and obtain sleep conditions based on the physiological sign data. Generate point cloud data based on the original signal output by the millimeter-wave radar 100, and obtain human body posture data based on the point cloud data. The physiological sign data includes: respiration data and heart rate data. The human body posture data includes the human body movement trajectory and the human body posture recognition result.

[0052] The data real-time monitoring and early warning module 300 is configured to perform: Process and store the physiological sign data and the human body posture data, and visually display the processed physiological sign data, sleep conditions, and human body posture data. At the same time, perform real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture and fall early warning.

[0053] In another exemplary embodiment of the present application, on the basis of including the target locking unit, as Figure 2 shown, it may further include a physiological sign monitoring unit 201 and a human body posture detection unit 202.

[0054] The signal input end of the target locking unit serves as the signal input end of the functional module 200. The signal input end of the target locking unit is respectively connected to the signal input end of the physiological sign monitoring unit 201 and the signal input end of the human body posture detection unit 202. The signal output ends of the physiological sign monitoring unit 201 and the human body posture detection unit 202 are integrated into the signal output end of the functional module 200.

[0055] The target locking unit is configured to perform: Perform fast Fourier transform on the original signal output by the millimeter-wave radar 100 based on the distance dimension to obtain the distance information of the target, and lock the target position through adaptive threshold screening. Among them, the target locking unit is essentially to calculate the distance information of the target based on distance dimension FFT.

[0056] The physiological sign monitoring unit 201 is configured to perform: After locking the target position, extract the respiration signal and the heartbeat signal from the signal corresponding to the target position. Perform spectrum analysis on the respiration signal and the heartbeat signal by using continuous wavelet transform, and obtain the corresponding waveforms of respiration and the corresponding waveforms of heart rate in real time to obtain physiological sign data.

[0057] In practical applications, the process of the physiological sign monitoring unit 201 acquiring physiological sign data can be described as follows: extracting the phase change curve of the target position, performing phase unwrapping based on the region growing method, and simultaneously using phase difference to calculate the variation relationship of the true phase at the target position over time. Subsequently, low-pass filtering is used to smooth the signal to suppress noise interference, and a band-pass filter and empirical mode decomposition (EMD) are used to separate the mixed physiological signals to extract the respiration and heartbeat signals. The purpose of using continuous wavelet transform (CWT) for spectral analysis is to calculate the real-time respiration and heartbeat frequencies and their corresponding waveforms, and transmit the obtained data to the data real-time monitoring and early warning module to achieve continuous monitoring of vital signs and abnormal early warning, and then estimate the sleep quality of the monitored person through a deep learning algorithm.

[0058] The human body posture detection unit 202 is configured to perform: obtaining the azimuth and elevation angle information of the target according to the original signal output by the millimeter-wave radar 100 by using a spatial spectrum estimation algorithm based on sparse reconstruction. Using a peak detection method to enhance the accuracy of the target's distance information, and determining the target's speed information through the Doppler fast Fourier transform method to generate the point cloud data of the target. Inputting the point cloud data into a deep learning model based on point cloud feature extraction to perform feature extraction and classification recognition of the human body posture to obtain human body posture data. Transmitting the recognized human body posture data to the data real-time monitoring and early warning module can achieve real-time perception and abnormal detection of the human body posture.

[0059] In another exemplary embodiment of the present application, as Figure 3 shown, the data real-time monitoring and early warning module 300 includes: a data receiving unit 301, a data display unit 303, a data storage unit 302, and an abnormal early warning unit 304.

[0060] The signal input end of the data receiving unit 301 serves as the signal input end of the data real-time monitoring and early warning module 300. The signal input end of the data storage unit 302 is connected to the signal output end of the data receiving unit 301. The signal output end of the data storage unit 302 is respectively connected to the signal input end of the data display unit 303 and the signal input end of the abnormal early warning unit 304. The signal output ends of the data storage unit 302, the data display unit 303, and the abnormal early warning unit 304 are integrated into the signal output end of the data real-time monitoring and early warning module 300.

[0061] Among them, the data receiving unit 301 includes a plurality of real-time data receiving interfaces. The plurality of real-time data receiving interfaces are respectively connected to each physiological sign monitoring unit 201 and the human body posture detection unit 202.

[0062] The data storage unit 302 uses the InfluxDB time series database to save all the real-time physiological sign data and human body posture data received from the functional module 200, and uses the MySQL structured database to save static structured data.

[0063] The data display unit 303 is used to display the physiological sign data, human body posture data, and calculated sleep quality data received from the functional module 200 in the form of charts. The data display unit 303 is connected to a data interaction interface, and this data interaction interface is connected to a peripheral intelligent terminal, such as a Web terminal and a mini-program terminal.

[0064] The abnormal warning unit 304 judges the received physiological sign data, sleep quality data, and human body posture. In the case of abnormal data, it gives a warning on the Web terminal and the mini-program terminal to remind the user to check the status of the monitored person and the device. If there is a long-term abnormality, it notifies the contacts or emergency contacts set in the Web terminal and the mini-program terminal by means of text messages or phone calls.

[0065] In some embodiments, the real-time data receiving interface provides independent interface addresses for the physiological sign monitoring unit 201 and the human body posture detection unit 202 to receive the corresponding data streams. The human body posture data, physiological sign data, and sleep quality data are written through the real-time data receiving interface, and a HyperText Transfer Protocol (HTTP) request is sent to the backend in the form of a Post request. After receiving the HTTP request, the data receiving unit 301 parses the Json format data therein, establishes connections with the MySQL structured database and the InfluxDB time series database, performs format conversion and processing on the data to make it meet the storage requirements of the target table, and writes it into the corresponding table to achieve the persistent storage of the data.

[0066] In some embodiments, the InfluxDB time series database includes user physiological sign data tables, user movement trajectory data tables, point cloud data tables, etc. The MySQL structured database includes user information data tables, device information data tables, sleep quality data tables, alarm record data tables, and user access log data tables, etc.

[0067] In some embodiments, the data display unit 303 reads real-time data from the InfluxDB time series database and provides the data to the Web terminal and the mini-program terminal through the interface address for display. At the same time, the data display unit 303 also reads static structured data from the MySQL structured database to support the maintenance and query of the data.

[0068] In some embodiments, the data display unit 303 provides a data interaction interface (such as a front-end UI interface) for the Web side and the mini-program side for the user, so that the user can view the real-time data of each module in real time and manage the information of the device and the detected personnel. In addition, the front-end UI interface is built using the Vue framework (the Vue framework based on JavaScript and some plugins) to implement the display function of the real-time data monitoring and warning module 300. The front-end UI interface can also implement the state management of the front end through Vuex to share and manage the data, states, functions, etc. shared by multiple components. The data is better displayed through third-party libraries and some utility classes, plugins, etc. For example, Echarts displays the data in the form of charts, and Canvas draws a canvas to implement some functions. Various requests are sent to the back-end server through alova, including Post requests, Get requests, etc.

[0069] For example, the front-end UI interface may include a home page, a physiological signs and sleep module, a human body posture module, and a device management module. Among them, the home page provides the display of the working status of all devices. The physiological signs module displays the real-time physiological signs data of all monitored personnel, the change of physiological signs data in different time periods within a set date, and the daily sleep situation, etc. The human body posture module displays information such as the point cloud data of the real-time human body postures of all monitored personnel, the posture recognition results, and the movement trajectories. The device management module provides an interaction interface for the user to manage the millimeter-wave radar 100, the functional module 200, and the information of the monitored personnel.

[0070] In some embodiments, the abnormal warning unit 304 is configured to:

[0071] Set the normal range threshold of the breathing frequency and the normal range threshold of the heart rate. When the breathing frequency in the breathing data and the heart rate in the heart rate data are not within the normal range threshold of the breathing frequency or the normal range threshold of the heart rate, it is determined that the physiological signs data is abnormal data.

[0072] Use a deep learning algorithm to calculate the data related to sleep quality, and set the normal range of sleep time and sleep quality. When the data related to sleep quality differs greatly from the normal range threshold, it is determined that the sleep quality data is abnormal.

[0073] Use a deep learning algorithm to identify the transmitted point cloud data. When the posture is in a falling state, it is determined that the human body posture data is abnormal.

[0074] In another exemplary embodiment of the present application, the structural composition and working principle of the fall detection, physiological signs and sleep monitoring and warning system provided by the present application are described in combination with a specific application case, where:

[0075] Such as Figure 4As shown in the figure, the real-time data monitoring and warning module 300 provided by this application may include a backend, a database, a mini-program terminal, and a Web terminal. Among them, the backend is used for data reception, processing, and transmission, and signal interaction is carried out between the backend and the function module 200, the database, the mini-program terminal, and the Web terminal respectively.

[0076] Multiple radars are connected to the function module 200, and each radar selects the existing millimeter-wave radar 100. Based on this, the function module 200 is a module that realizes different functions based on the hardware of the millimeter-wave radar 100. As the lower layer of the real-time data monitoring and warning system, it provides real-time data for the system. After obtaining the signal of the millimeter-wave radar 100 (i.e., the original signal), the phase change of the signal is extracted, and the respiration and heartbeat signals are separated and extracted through a filter or a modal decomposition algorithm, and then the respiration and heart rate are obtained. Subsequently, the sleep quality is estimated using a deep learning algorithm. After obtaining the multi-channel millimeter-wave radar 100 data, the obtained multi-channel millimeter-wave radar 100 data is converted into point cloud data, and the human body posture is recognized and detected through a deep learning algorithm for the point cloud data.

[0077] The real-time data monitoring and warning module 300 provides a real-time data reception interface for the physiological sign monitoring, sleep quality monitoring, and human body posture detection module to return data, realizes that multiple radars do not interfere with each other, and passes the real-time data into the real-time data monitoring and warning module 300 through the interface. After processing, it is stored in the database, and the data is displayed in a graphical form, which is convenient for users to view the physiological signs, sleep quality, human body posture data, and historical records of the monitored person, and realizes the information management of the real-time state of the device and the information of the monitored person. At the same time, when the real-time data monitoring and warning module 300 receives abnormal data, it can be processed according to different abnormal information, and warnings are sent on the mini-program terminal and the Web terminal to remind users to go to check the status of the monitored person and the device in time.

[0078] Based on the above description, the overall process of the fall detection, physiological sign, and sleep monitoring and warning system provided in this embodiment is as follows:

[0079] The function module 200 receives the original signal of the millimeter-wave radar 100 based on the hardware of different millimeter-wave radars 100, extracts the phase change in the original signal, and then separates and extracts the respiration and heartbeat signals through a filter or a modal decomposition algorithm, and then estimates its sleep quality using a deep learning algorithm. For the multi-channel millimeter-wave radar 100 data (i.e., the original signal), it is first converted into point cloud data, and then the human body posture is recognized and detected through deep learning.

[0080] Then, the functional module 200 transmits the data back to the server in the backend through the data receiving interface provided by the data real-time monitoring and warning module 300. After the backend obtains the real-time data, it processes the data, changes the data format, connects to the database, and stores the data in the corresponding tables according to the different data.

[0081] The backend reads the real-time data from the database and provides an API interface to the front-end interface of the system (such as the front-end UI interface) in the form of an interface address, waiting for the front-end to call. For example, the front-end refers to the Web side and the mini-program side.

[0082] When the front-end opens the interface, it obtains the real-time data through the API interface provided by the backend, visualizes the data, and when the user performs other operations, it sends a corresponding request to the backend, and after obtaining the data, it displays the operation result in a chart.

[0083] When the system detects an abnormal situation in the received data, it writes the abnormal situation into the corresponding table in the database. The front-end regularly queries the abnormal situation interface, and when an abnormality is found, it gives a timely warning at the front-end to remind the user to go and handle it.

[0084] Based on the above description, the workflow of the real-time monitoring and warning of human body postures and physiological signs based on the millimeter-wave radar 100 provided by this embodiment is as Figure 5 shown.

[0085] Furthermore, the data real-time monitoring and warning module 300 provided above in this application can provide an interface for the user to query the real-time physiological signs and human body posture data of the detected person, the real-time status and associated information data of the device in the database, and obtain the physiological signs and human body posture data transmitted back by each radar. The backend sets the response status code and response information, and the front-end judges the interface status according to the returned corresponding information. The backend processes and stores various received data, stores them in different database tables according to the data type, and reads the real-time data from the database to provide various data acquisition API interfaces to the front-end.

[0086] The backend uses the SpringBoot framework to build the server, which is responsible for communicating with the functional module 200, the database, and the front-end, obtaining and processing data and providing an interface to the front-end. When the front-end calls the relevant interface, it returns the data in the JSON format.

[0087] Furthermore, the data storage unit 302 provided in the present application uses the InfluxDB time series database and the MySQL structured database as data storage tools, and sets storage modes such as user vital sign data tables, user movement trajectory data tables, point cloud data tables, user information data tables, device information data tables, sleep quality data tables, alarm record data tables, and user access log data tables, etc., to be used for persistently recording and storing relevant information of the monitored person and device information. The backend writes the real-time physiological sign data, sleep quality data, human body posture data, and device information of the monitored person through data interaction with the database, and records the abnormal information in the database when it is found that the received data is abnormal.

[0088] Furthermore, as Figure 6 shown in the framework structure of the fall detection, physiological sign and sleep monitoring and warning system, in the actual application process, intelligent devices providing Web terminals and mini-program terminals can be used as the access layer. Based on this, in the SpringBoot framework of the backend, the entire business process can be divided into an entity layer, a persistence layer, a business layer, a control layer, and some tool or static resource layers. Among them, the entity layer defines the attributes of the class and keeps them consistent with the fields in the database, and one table corresponds to one entity class. The persistence layer is responsible for interacting with the database (i.e., data interaction), and completes the basic tasks of adding, deleting, modifying, and querying the database through interfaces and configuration files. The JDBC (Java Database Connectivity) library can be used to connect and operate the database, which is equivalent to implementing the function of the persistence layer by itself. The business layer encapsulates one or more interfaces of the persistence layer into a service and provides the interfaces to the control layer. This function can be implemented through SQL statements based on the JDBC library. The control layer is responsible for the interaction between the front end and the back end. After receiving the front-end request, it calls the specific interfaces of the business layer and returns specific data to the access layer. The functions of the persistence layer and the business layer can be integrally implemented by using JDBC and SQL statements, and interfaces are provided for the front-end UI (i.e., the front-end UI interface) through the @RequestMapping and @GetMapping annotations and the data is returned in JSON format.

[0089] In summary, the fall detection, physiological sign, and sleep monitoring and warning system provided by this application uses millimeter-wave radars with different functions as data sources, and provides a data receiving interface at the backend of the data real-time monitoring and warning module to receive real-time physiological sign data, human body posture point cloud data, radar device information, and other data transmitted back by the radar. The received data is processed and calculated by algorithms, and then stored in the corresponding data tables of the database. At the same time, real-time data is read from the database, and a data interface is provided to the front end to achieve visual display of the data. The data real-time monitoring and warning module provides an interaction interface for users, enabling users to view the status of the monitored person and the fluctuations of historical data in real time, and manage the operation of the device. When the data real-time monitoring and warning module detects an abnormal situation, it can promptly send a reminder to the user to prompt the user to handle it in a timely manner, realizing all-weather and all-round monitoring of the monitored person.

[0090] Based on the same inventive concept, an embodiment of this application also provides a monitoring and warning method applied to the above-mentioned fall detection, physiological sign, and sleep monitoring and warning system. The implementation solutions provided by this method to solve problems are similar to those recorded in the above system. Therefore, the specific limitations in one or more embodiments of the fall detection, physiological sign, and sleep monitoring and warning method provided below can refer to the limitations on the fall detection, physiological sign, and sleep monitoring and warning system in the above text, and will not be repeated here. Based on this, the fall detection, physiological sign, and sleep monitoring and warning method provided by this application includes:

[0091] Step 1: Based on the original signal output by the millimeter-wave radar, physiological sign data is obtained through phase change, and the sleep situation is obtained based on the physiological sign data. The physiological sign data includes: respiratory data and heart rate data.

[0092] Step 2: Point cloud data is generated based on the original signal output by the millimeter-wave radar, and human body posture data is obtained based on the point cloud data. The human body posture data includes the human body movement trajectory and the human body posture recognition result.

[0093] Step 3: Process and store the physiological sign data and the human body posture data, and visually display the processed physiological sign data, sleep situation, and human body posture data. At the same time, real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture, and fall warning are carried out.

[0094] As an implementation manner of this application, in Step 3, the process of real-time detection of abnormal physiological sign data, abnormal sleep, and abnormal human body posture, and fall warning includes:

[0095] When the respiratory rate in the respiratory data is not within the normal threshold range of the respiratory rate or the heart rate in the heart rate data is not within the normal threshold range of the heart rate, it is determined that the physiological sign data is abnormal data.

[0096] When the difference between the relevant data of sleep quality and the normal range threshold reaches the set value, it is determined that there is an abnormal sleep situation.

[0097] Determine whether the human body posture changes within the set time. If there is no change, it is determined that the human body posture is abnormal.

[0098] Use a deep learning algorithm to identify the transmitted point cloud data to obtain the human body posture prediction result.

[0099] When it is determined that the human body posture prediction result belongs to the fall state, a fall warning is generated.

[0100] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0101] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0102] In each of the embodiments provided in the present application, the database involved may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., and is not limited thereto. In each of the embodiments provided in the present application, the processor may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., and is not limited thereto.

[0103] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0104] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A fall detection, physiological sign and sleep monitoring warning system, characterized in that: The fall detection, physiological sign and sleep monitoring warning system includes: a millimeter wave radar, a functional module and a data real-time monitoring and warning module; The signal output end of the millimeter wave radar is connected to the signal input end of the functional module; the signal output end of the functional module is connected to the signal input end of the data real-time monitoring and early warning module; The functional module is configured to execute: based on the original signal output by the millimeter wave radar, obtain physiological sign data through phase change, and obtain sleep status based on the physiological sign data; based on the original signal output by the millimeter wave radar, generate point cloud data, and obtain human posture data based on the point cloud data; the physiological sign data includes: breathing data and heart rate data; the human posture data includes human movement trajectory and human posture recognition result; The data real-time monitoring and early warning module is configured to execute: processing and storing the physiological sign data and the human body posture data, and visually displaying the processed physiological sign data, sleep conditions and human body posture data, and at the same time, performing real-time detection of abnormal physiological sign data, abnormal sleep conditions and abnormal human body posture and fall warning.

2. The fall detection, physiological sign and sleep monitoring warning system according to claim 1, characterized in that: The functional modules include: a target locking unit, a physiological sign monitoring unit and a human posture detection unit; The signal input end of the target locking unit serves as the signal input end of the functional module; the signal input end of the target locking unit is respectively connected to the signal input end of the physiological sign monitoring unit and the signal input end of the human posture detection unit; the signal output end of the physiological sign monitoring unit and the signal output end of the human posture detection unit are integrated as the signal output end of the functional module; The target locking unit is configured to perform: performing a fast Fourier transform on the original signal output by the millimeter wave radar based on the distance dimension to obtain the distance information of the target, and locking the target position through adaptive threshold screening; The physiological sign monitoring unit is configured to perform: after locking the target position, extracting a breathing signal and a heartbeat signal from the signal corresponding to the target position; performing spectrum analysis on the breathing signal and the heartbeat signal using continuous wavelet transform, acquiring the corresponding waveform of breathing and the corresponding waveform of heartbeat frequency in real time, and obtaining the physiological sign data; The human posture detection unit is configured to perform: using a spatial spectrum estimation algorithm based on sparse reconstruction to obtain the azimuth and pitch angle information of the target according to the original signal output by the millimeter wave radar; using a peak detection method to enhance the accuracy of the distance information of the target, and determining the speed information of the target through a Doppler fast Fourier transform method to generate point cloud data of the target; inputting the point cloud data into a deep learning model to perform feature extraction and classification recognition on the human posture to obtain the human posture data.

3. The fall detection, physiological sign and sleep monitoring warning system according to claim 1, characterized in that: The data real-time monitoring and early warning module includes: a data receiving unit, a data display unit, a data storage unit and an abnormal early warning unit; The signal input end of the data receiving unit serves as the signal input end of the data real-time monitoring and early warning module; the signal input end of the data storage unit is connected to the signal output end of the data receiving unit; the signal output end of the data storage unit is respectively connected to the signal input end of the data display unit and the signal input end of the abnormal early warning unit; the signal output end of the data storage unit, the signal output end of the data display unit and the signal output end of the abnormal early warning unit are integrated as the signal output end of the data real-time monitoring and early warning module; The data receiving unit establishes a connection with the data storage unit, and performs format conversion and processing on the data so that the converted data meets the storage requirements of the data storage unit; The data storage unit processes and stores the physiological sign data and the human body posture data; the data display unit visualizes the processed physiological sign data, sleep conditions and human body posture data; the abnormal warning unit performs real-time detection of abnormal physiological sign data, abnormal sleep and abnormal human body posture and provides fall warning.

4. The fall detection, physiological sign and sleep monitoring warning system according to claim 3, characterized in that: The data display unit and the abnormal warning unit both include a data interaction interface; the data interaction interface interacts with the external intelligent terminal for data and warning.

5. The fall detection, physiological sign and sleep monitoring warning system according to claim 4, characterized in that: The peripheral intelligent terminal includes a Web terminal and a mini-program terminal; An early warning is issued in the Web end and the mini-program end to remind the user to check the status of the monitored person and the equipment; and when there are abnormalities in the status of the monitored person and the equipment, an early warning notification is issued through wireless communication; the objects of the early warning notification are the contacts set in the Web end and the mini-program end.

6. The fall detection, physiological sign and sleep monitoring warning system according to claim 3, characterized in that: The data storage unit includes an InfluxDB time series database and a MySQL structured database; The InfluxDB time series database stores the physiological sign data, sleep condition data and human posture data received from the functional module; the MySQL structured database is used to store the user's device data, personal information data, sleep report data and historical statistical data.

7. The fall detection, physiological sign and sleep monitoring warning system according to claim 6, characterized in that: The display unit reads static structured data from the MySQL structured database to support data maintenance and query.

8. The fall detection, physiological sign and sleep monitoring warning system according to claim 5, characterized in that: The Web end and the mini program end provide a front-end UI interface for data interaction; the front-end UI interface is built using the Vue framework; The front-end UI interface includes a home page, a physiological sign and sleep module, a human posture module, and a device management module; The home page is used to display the working status of the device; the physiological signs and sleep module is used to display the real-time physiological signs data of all monitored persons, the changes in physiological signs data in different time periods within a set date, and the sleep status; the human posture module is used to display the real-time point cloud data, posture recognition results and movement trajectory information of all monitored persons; the device management module is used to provide a data interaction interface.

9. A fall detection, physiological sign and sleep monitoring early warning method, characterized in that: The fall detection, physiological sign and sleep monitoring early warning method comprises: Based on the original signal output by the millimeter wave radar, physiological sign data is obtained through phase change, and the sleep condition is obtained based on the physiological sign data; the physiological sign data includes: breathing data and heart rate data; Generate point cloud data based on the original signal output by the millimeter wave radar, and obtain human posture data based on the point cloud data; the human posture data includes a human movement trajectory and a human posture recognition result; The physiological sign data and the human body posture data are processed and stored, and the processed physiological sign data, sleep conditions and human body posture data are visualized. At the same time, real-time detection of abnormal physiological sign data, abnormal sleep and abnormal human body posture and fall warning are performed.

10. The fall detection, physiological sign and sleep monitoring and early warning method according to claim 9, characterized in that: The process of real-time detection of abnormal physiological sign data, abnormal sleep, abnormal human posture, and fall warning includes: When the respiratory frequency in the respiratory data is not within the normal respiratory frequency threshold range or the heart rate in the heart rate data is not within the normal heart rate threshold range, determining that the physiological sign data is abnormal data; When the difference between the sleep quality related data and the normal range threshold reaches a set value, it is determined that there is a sleep abnormality; Determine whether the human body posture changes within a set time, and if no change occurs, determine that the human body posture is abnormal; Use deep learning algorithms to identify the returned point cloud data and obtain human posture prediction results; When it is determined that the human body posture prediction result belongs to a fall state, a fall warning is generated.

Images