Multifunctional health monitoring method based on millimeter wave radar

Abstract

The present application discloses a multifunctional health monitoring method and system based on a millimeter wave radar, and a computer readable storage medium. The method comprises: receiving an echo signal in a space, and on the basis of the echo signal, calculating point cloud data of a user and phase change data caused by the respiration and heartbeat of the user; calculating behaviors of the user on the basis of the point cloud data; calculating vital signs of the user on the basis of the phase change data, wherein the vital signs comprise respiration rate and heart rate; and on the basis of the behaviors and / or the vital signs, determining that the user triggers a set alarm event, and on the basis of the type of the triggered alarm event, performing a corresponding alarm operation.

Description

Multifunctional health monitoring method based on millimeter-wave radar Priority Declaration

[0001] This patent claims priority to the invention patent with application number 202311580515.9 submitted to the State Intellectual Property Office of China on November 23, 2023, and application name "Multifunctional health monitoring method based on millimeter wave radar". Technical Field

[0002] The present application relates to the field of radar monitoring technology, and in particular to a multifunctional health monitoring method, system and computer-readable storage medium based on millimeter-wave radar. Background Art

[0003] With the trend of population aging, the health and daily care of the elderly are becoming more and more important. Health monitoring, accidental falls of the elderly, and elderly going out alone require a lot of manpower and material resources to support. Now there are many wearable and non-wearable devices on the market to monitor the health, falls, and going out of the elderly.

[0004] Wearable devices are usually wristbands or bracelets. These wearable devices often have problems such as uncomfortable wearing experience and inability to provide all-day monitoring.

[0005] Non-wearable devices typically rely on cameras and radar. Cameras are a major concern because they require real-time video of user activity, potentially infringing privacy. Radars are also limited in their ability to perform only one of several monitoring functions, such as fall detection or headcount detection, leading to significant limitations in daily care. Summary of the Invention

[0006] The embodiments of the present application provide a multifunctional health monitoring method based on millimeter-wave radar, aiming to achieve all-weather, contactless and privacy-free multifunctional monitoring.

[0007] To achieve the above objectives, the present invention provides a multifunctional health monitoring method based on millimeter wave radar, comprising:

[0008] Receive echo signals in the space, and calculate the user's point cloud data and phase change data caused by the user's breathing and heartbeat based on the echo signals;

[0009] Calculating user behavior based on the point cloud data;

[0010] Calculating the user's vital signs based on the phase change data, where the vital signs include respiratory rate and heart rate;

[0011] It is determined based on the behavior and / or the vital signs that the user has triggered a set alarm event, and a corresponding alarm operation is performed based on the type of the triggered alarm event.

[0012] It can be understood that the multifunctional health monitoring method based on millimeter-wave radar in the technical solution of this application can obtain the point cloud data and phase change data of the monitored user through the echo signal in space, and then calculate the behavior and vital signs of the monitored user based on the point cloud data and phase change data. In this way, multiple monitoring functions can be realized simultaneously based on the behavior and vital signs, such as number detection, fall detection, heart rate detection, breathing detection, and going out detection. These monitoring functions are all-weather and non-sensing. Therefore, compared with existing monitoring solutions, this solution has the following advantages:

[0013] 1. Multi-function monitoring: The technical solution of this application is based on millimeter-wave radar technology and can simultaneously realize multiple detection functions such as number detection, fall detection, heart rate detection, breathing detection, and going out detection. Users can independently select one or more of these functions according to their needs;

[0014] 2. Low cost: The technical solution of this application only requires one radar to achieve multiple monitoring functions. Compared with systems that combine multiple non-wearable devices to achieve multiple monitoring functions, it is lower in cost and more cost-effective.

[0015] 3. Non-contact all-weather monitoring: The monitoring method of the technical solution of this application can achieve 24-hour all-weather monitoring without any contact, achieving true non-sensing protection;

[0016] 4. Privacy protection: Unlike cameras, millimeter-wave radar does not generate images during monitoring, but only captures simple trajectory and point cloud data. This means that user privacy is effectively protected and personal privacy issues are not affected.

[0017] 5. Two-way communication: When the monitored user triggers an alarm event, the system can automatically call the bound terminal through the cloud server platform so that the monitored user and the terminal user can get in touch as soon as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0019] FIG1 is a module structure diagram of an embodiment of a multifunctional health monitoring system based on millimeter-wave radar of the present application;

[0020] FIG2 is a flow chart of an embodiment of a multifunctional health monitoring method based on millimeter-wave radar of the present application.

[0021] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0023] To better understand the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0024] It should be noted that in the claims, any reference signs placed between brackets should not be construed as limiting the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order, and these words may be interpreted as names.

[0025] As shown in FIG1 , FIG1 is a schematic structural diagram of a multifunctional health monitoring system based on millimeter-wave radar according to an embodiment of the present application.

[0026] As shown in FIG1 , the server 1 includes: a millimeter wave radar 15 , a memory 11 , a processor 12 and a network interface 13 .

[0027] In this embodiment, the radar of the present application adopts TD-MIMO (Time Division Multiple Access Multiple Input Multiple Output) signal transmission mode, which sends a linear frequency modulated continuous wave signal (chirp signal). The radar signal is transmitted by the transmitting antenna. The electromagnetic wave signal emitted encounters an obstacle and is reflected back. After a period of time, After that, the receiving antenna receives the echo signal, passes through the low noise amplifier to filter out the noise, and then mixes it with one of the transmitting signals. After passing through the low pass filter, the intermediate frequency signal (IF signal) is obtained. The ADC signal can be obtained by digitally sampling the intermediate frequency signal. The frequency of the intermediate frequency signal is for: , where k is the signal frequency modulation slope, is the target delay, and the delay The relationship between the target distance d and the target distance d is as follows: , where d is the target distance and c is the speed of light. Therefore, the distance between the target and the radar can be obtained from the frequency of the intermediate frequency signal: .

[0028] Furthermore, the millimeter wave radar of this application adopts a 3-transmit 4-receive antenna design. The 3 transmitting antennas transmit signals in a time-division multiplexing manner, and the 4 receiving antennas receive signals simultaneously. Each transmitting antenna sends 32 chirps in one frame, and the number of ADC sampling points for each chirp is 512. I / Q complex sampling is used, and the amount of ADC data received in one frame is .

[0029] Optionally, the radar of this application has two installation methods, namely ceiling-mounted and side-mounted. The ceiling-mounted radar is installed on the ceiling or suspended ceiling directly above the bed, and the side-mounted radar is installed above the middle position of the bed head, 1.5m to 1.8m from the ground.

[0030] Furthermore, the memory 11 may include both an internal storage unit of the server 1 and an external storage device. The memory 11 can be used not only to store application software and various data installed on the server 1, such as the code of the multifunctional health monitoring program 10 based on millimeter-wave radar, but also to temporarily store data that has been output or is about to be output.

[0031] In some embodiments, the processor 12 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program codes or process data stored in the memory 11, such as executing a multi-functional health monitoring program 10 based on millimeter-wave radar.

[0032] The network interface 13 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the server 1 and other electronic devices.

[0033] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth communication protocol, or a combination thereof.

[0034] Optionally, the server may further include a user interface, which may include a display and an input unit such as a keyboard. Optional user interfaces may also include standard wired and wireless interfaces. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display, which may also be referred to as a display screen or display unit, is used to display information processed by server 1 and to display a visual user interface.

[0035] Figure 1 only shows a server 1 having components 11-13 and a multifunctional health monitoring program 10 based on millimeter-wave radar. Those skilled in the art will understand that the structure shown in Figure 1 does not constitute a limitation on the server 1, and may include fewer or more components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0036] Based on the hardware architecture of the millimeter-wave radar-based multifunctional health monitoring system, an embodiment of the millimeter-wave radar-based multifunctional health monitoring method of the present application is proposed. The millimeter-wave radar-based multifunctional health monitoring method of the present application aims to achieve all-weather, contactless, and privacy-free multifunctional monitoring.

[0037] 2 , which illustrates an embodiment of a multifunctional health monitoring method based on millimeter-wave radar of the present application, the multifunctional health monitoring method based on millimeter-wave radar includes the following steps:

[0038] S10 , receiving echo signals in the space, and calculating the user's point cloud data and the phase change data caused by the user's breathing and heartbeat according to the echo signals.

[0039] Here, the user refers to a subject that needs to be monitored, cared for, and protected. For example, the user can be one of the following objects: a patient, an elderly person, or an infant.

[0040] Point cloud data is a collection of three-dimensional coordinate points that describe the shape and position of an object or scene in three-dimensional space. Furthermore, because a person's breathing and heartbeat cause minute movements of the chest and thoracic cavity, these movements cause periodic phase shifts in the echo signal. Based on these periodic phase shifts, the radar system can set specific filtering conditions to distinguish between the user in space and the static background, thereby accurately monitoring the phase shift data caused by the user's breathing and heartbeat.

[0041] Specifically, the millimeter-wave radar system first sends millimeter-wave signals, which interact with objects in space and reflect back. The receiver receives these echo signals and uses their time delay and intensity information to calculate the user's point cloud data and phase change data.

[0042] S20: Calculate the user's behavior based on the point cloud data.

[0043] Specifically, behavior refers to information about user behavior inferred by analyzing point cloud data acquired from millimeter-wave radar. This data includes a detailed description of the user's activities and movements during a presumed time period. For example, behavior can include posture data (such as standing, sitting, walking, running, stretching, etc.), motion trajectory data (such as direction, speed, and path of movement), and posture stability data (such as monitoring whether the user maintains balance, shakes, or sways).

[0044] Specifically, by analyzing the shape and position in the point cloud, the system can infer the user's current posture, such as standing, sitting, lying down, etc. In this way, the user's posture data can be obtained.

[0045] At the same time, by comparing the point cloud data between consecutive frames, the system can analyze the user's movement trajectory, speed and direction, thereby obtaining the user's trajectory data.

[0046] It is also worth adding that the number of people in the space can be detected through point cloud data to obtain the number of people in the current space.

[0047] S30. Calculate the user's vital signs based on the phase change data, where the vital signs include respiratory rate and heart rate.

[0048] Specifically, when a person breathes or their heart beats, the movement of the chest and heart causes the phase of the millimeter wave signal to change. This change typically manifests as a phase shift or rate of change in the signal. The phase change caused by breathing typically has a different frequency and characteristics than the phase change caused by a heartbeat. Breathing typically has a lower frequency (usually between 0.1 Hz and 0.8 Hz), while heartbeats typically have a higher frequency (usually above 0.8 Hz to 2.5 Hz). Based on the above principles, by monitoring the phase changes in the echo signal caused by the user's breathing and heartbeat, each user's breathing frequency and / or heart rate can be calculated using methods such as fast Fourier transform, thereby achieving non-contact vital sign monitoring.

[0049] It is worth noting that, in some embodiments, the system can display the detected behaviors and vital signs in real time on a terminal, which can be a mini-program, an APP on a mobile device, a web page, etc.

[0050] S40: Determine, based on the behavior and / or the vital sign, that the user has triggered a set alarm event, and then execute a corresponding alarm operation based on the type of the triggered alarm event.

[0051] Specifically, a set of alarm events refers to a series of specific situations or conditions pre-defined and configured in the radar system. When the user's behavior or vital signs meet these conditions, the system will trigger corresponding alarm notifications or emergency measures. It is worth noting that these alarm events can be customized based on the user's health status, medical standards, personal needs, or special circumstances.

[0052] For example, the set alarm events may include the following events: falling, being still for a long time, abnormal breathing, abnormal heart rate, going out alone, etc.

[0053] Specifically, after obtaining behavior and vital signs, the radar system can monitor these point cloud data and vital signs in real time and compare them with the set alarm thresholds to monitor whether the user triggers the set alarm event.

[0054] In some embodiments, the system may monitor whether the user triggers an alarm event simply by comparing the user's behavior with a set alarm threshold, or comparing the user's vital signs with a set alarm threshold.

[0055] In other embodiments, the system may combine behavior and vital signs to more accurately monitor the user's health. For example, if a user has been inactive for an extended period (behavior) and has an abnormally low heart rate (vital sign), the system may detect that the user is potentially in danger.

[0056] Furthermore, after determining that the user has triggered a set alarm event, the system can perform corresponding alarm operations according to the type of alarm event triggered. Depending on the degree of urgency and situation, the system may perform different operations to ensure the user's safety and provide appropriate assistance.

[0057] For example, depending on the type of alarm event triggered by the user, the system can send different types of alarm information to pre-bound terminals. For example, when a monitored user triggers a fall event, the system can send a fall alarm message to the pre-bound terminals via phone call, SMS, app push, email, instant message, etc., and remind the terminal user to take appropriate action in a timely manner.

[0058] Among them, the end user can be the relative, guardian, friend, etc. of the user being tested, or a family doctor, medical institution, nursing home, etc. who has a care obligation with the monitored user.

[0059] In some embodiments, based on this alarm operation, the radar system of the present application's technical solution can also be pre-installed with a microphone and speaker, thus enabling the system to have a two-way voice communication function. Thus, after the monitored user triggers an alarm event and the system performs an alarm operation, the system can proactively initiate a communication request with the bound terminal and, once communication is established, enable a two-way voice call. This allows the user to receive timely communication and assistance, facilitating rapid rescue or support in emergency situations.

[0060] It can be understood that the multifunctional health monitoring method based on millimeter-wave radar in the technical solution of this application can obtain the point cloud data and phase change data of the monitored user through the echo signal in space, and then calculate the behavior and vital signs of the monitored user based on the point cloud data and phase change data. In this way, multiple monitoring functions can be realized simultaneously based on the behavior and vital signs, such as number detection, fall detection, heart rate detection, breathing detection, and going out detection. These monitoring functions are all-weather and non-sensing. Therefore, compared with existing monitoring solutions, this solution has the following advantages:

[0061] 1. Multi-function monitoring: The technical solution of this application is based on millimeter-wave radar technology and can simultaneously realize multiple detection functions such as number detection, fall detection, heart rate detection, breathing detection, and going out detection. Users can independently select one or more of these functions according to their needs;

[0062] 2. Low cost: The technical solution of this application only requires one radar to achieve multiple monitoring functions. Compared with systems that combine multiple non-wearable devices to achieve multiple monitoring functions, it is lower in cost and more cost-effective.

[0063] 3. Non-contact all-weather monitoring: The monitoring method of the technical solution of this application can achieve 24-hour all-weather monitoring without any contact, achieving true non-sensing protection;

[0064] 4. Privacy protection: Unlike cameras, millimeter-wave radar does not generate images during monitoring, but only captures simple trajectory and point cloud data. This means that user privacy is effectively protected and personal privacy issues are not affected.

[0065] 5. Two-way communication: When the monitored user triggers an alarm event, the system can automatically call the bound terminal through the cloud server platform so that the monitored user and the terminal user can get in touch as soon as possible.

[0066] In some embodiments, calculating the user's point cloud data based on the echo signal includes:

[0067] S11. Convert the echo signal into a digital signal.

[0068] Specifically, the echo signal returned by an object in space is an analog signal, which needs to be converted into a digital signal through an analog-to-digital conversion module (ADC) before subsequent processing.

[0069] S12 , performing a range-dimensional fast Fourier transform (RangeFFT) and a Doppler-dimensional Fourier transform (DopplerFFT) on the digital signal to obtain a range-Doppler map (RD-map) of all target units in the space.

[0070] Among them, in radar signal processing, the range dimension is used to measure the distance between the target object and the radar, and the Doppler dimension is used to measure the speed of the target object.

[0071] Furthermore, RangeFFT is a method that uses the relationship between the frequency change of the chirp signal and the target distance to measure the target distance. Doppler FFT is a method that uses the relationship between the frequency change of the echo signal caused by the target movement and the target speed to measure the target speed.

[0072] In this embodiment, the system first performs a RangeFFT on each radar antenna to obtain distance information for each target unit in space. The system then performs a DopplerFFT on the RangeFFT results to obtain Doppler-dimensional data for each target unit in space. Combining these RangeFFT and DopplerFFT results yields an RD-map.

[0073] S13, performing incoherent accumulation on the range-Doppler spectrum, and performing a two-dimensional constant false alarm rate (2D CFAR) detection on the accumulation result using a preset threshold parameter to extract the target unit corresponding to the user on the range-Doppler spectrum.

[0074] After obtaining the RD-map for each antenna, the RD-maps for all antennas can be non-coherently accumulated to produce the accumulated RD-map. Non-coherent accumulation is a method that adds the RD-maps from different antennas or frames based on amplitude without considering phase differences. This non-coherent accumulation improves the signal-to-noise ratio of the target element, making it easier to detect the target element during subsequent 2D constant false alarm detection.

[0075] Furthermore, 2D CFAR detection is a method that performs constant false alarm detection in two dimensions: distance and Doppler. It can determine whether a target unit exists based on the average value of each unit and the surrounding units in the RD-map.

[0076] Specifically, the preset threshold parameter is the false alarm rate threshold required for 2D CFAR detection. A two-dimensional threshold decision is performed on the accumulated range-Doppler data, and data points exceeding the threshold are considered to be present targets. Therefore, after completing the 2D CFAR detection, target units in space can be screened out.

[0077] S14: Perform azimuth-elevation joint angle measurement on the target unit corresponding to the user to obtain the azimuth and elevation angles of the target unit.

[0078] Azimuth and elevation are parameters used to describe the position of a target unit within the radar's detection volume. These are angular measurements in a polar coordinate system, used to determine the target's direction and elevation relative to the radar. The azimuth describes the horizontal orientation of the target unit relative to the radar's position. The elevation describes the vertical orientation of the target unit relative to the radar's position.

[0079] The azimuth-elevation joint angle measurement algorithm is a method used to calculate the azimuth and elevation angles of a target object within the radar detection space. The core principle of the azimuth-elevation joint angle measurement algorithm is to use multiple antennas or sensors to measure the target object's signals and combine the phase and amplitude information of these signals to calculate the azimuth and elevation angles.

[0080] Specifically, after the target unit is screened, all antenna data of the target unit can be extracted from the RD-map. Then, the azimuth and elevation angles of the target unit are calculated using the azimuth-elevation joint angle measurement algorithm.

[0081] S15. Obtain point data corresponding to the target unit in space according to the azimuth angle and the elevation angle and the corresponding range unit and Doppler unit on the range-Doppler map.

[0082] Specifically, after calculating the azimuth and elevation angles of the target unit, combined with the range unit and Doppler unit recorded in the RD-map, the three-dimensional position of the target unit in the radar detection space can be determined, and then the point data of the target unit can be obtained.

[0083] S16. Generate point cloud data of the user in space based on all point data of the user.

[0084] Specifically, the user's point cloud data can be obtained through the point data of all monitoring points of the user.

[0085] In this way, through the above steps S11 to S16, the user's point cloud data can be extracted from the echo signal to achieve monitoring of the user's movement and position.

[0086] In some embodiments, calculating the user's behavior based on the point cloud data includes:

[0087] S21. Calculate the distribution status data of all monitoring points in the point cloud data.

[0088] Specifically, the system analyzes the user's point cloud data, including the location and density of monitoring points in space. This distribution data helps the system understand the areas of space where monitoring points are located and their distribution density. By analyzing these features in the point cloud data, the system can determine the user's current posture, such as standing or lying down.

[0089] S22: performing clustering and fusion on the user's point cloud data to obtain a fitting center of the point cloud data.

[0090] Specifically, the user's point cloud data can be clustered and fused, and then the least squares method can be used for spherical fitting to obtain the sphere center coordinates of the clustering results, and then the sphere center coordinates are used as the fitting center of the user in space.

[0091] Among them, spherical fitting is a mathematical method used to find the parameters of a sphere that best fits a set of data points, including the coordinates of the sphere's center and radius.

[0092] S23 , performing Kalman filter tracking on the fitting center to obtain motion data of the fitting center, where the motion data includes motion trajectory data and motion speed data.

[0093] Kalman filtering is an algorithm that uses the linear system state equation and observation data from the system's input and output to optimally estimate the system state. By tracking the coordinates of the fitting center through Kalman filtering, we can determine how the coordinates of the fitting center change over time.

[0094] Specifically, by performing Kalman filter tracking on the coordinates of the fitting center, the coordinates of the fitting center at each sampling moment can be obtained, and then the change of the coordinates of the fitting center over time can be obtained, and the user's movement trajectory data and movement speed data can be generated.

[0095] S24: Generate the user's behavior according to the distribution state data and the motion data.

[0096] Specifically, in this step, the system combines distribution data and motion data to generate user behaviors. These behaviors can include posture, trajectory, speed, and dwell time. By analyzing this data, the system can identify user behavior patterns, such as whether the user is walking, lying in bed, standing, or sitting.

[0097] It's understandable that using algorithms like Kalman filtering to track the fitting center yields highly accurate motion data. Meanwhile, point cloud data typically contains a certain degree of noise. Clustering and calculating the fitting center can reduce the impact of this noise on motion data calculations. Furthermore, combining the distribution of cloud data with the motion data of the fitting center to simultaneously generate user behavior increases the data dimension, further improving the accuracy of user behavior assessments.

[0098] In some embodiments, the set alarm event includes a fall event;

[0099] Monitor whether the user has triggered a set alarm event based on the behavior, including:

[0100] S411. Calculate the ratio of monitoring points in the point cloud data that are lower than or equal to a set height to all monitoring points based on the distribution status data.

[0101] Specifically, based on the user's point cloud data, the system analyzes the number of monitoring points in the point cloud data that are lower than or equal to the set height and calculates the ratio of these monitoring points to all monitoring points. This ratio reflects the degree of contact between the user's body and the bed, floor, and other furniture.

[0102] For example, if the set height is lower than the height of the bed, then the higher the proportion of monitoring points in the point cloud data that are lower than or equal to the set height is, the closer the user is to the ground.

[0103] S412: Calculate a speed change value of the user according to the motion speed data.

[0104] Specifically, based on the motion data, the system can calculate the user's speed change between adjacent frames. This speed value represents the degree of change in the user's motion speed within a set time. This speed change value can be used to monitor the user's motion status, such as whether there are sudden rapid movements or stops.

[0105] S413: When it is confirmed based on the motion trajectory data that the user's action state is in bed, and the speed change value is greater than or equal to the set value, and the ratio is greater than or equal to the set ratio, it is determined that the user has triggered a fall-out event.

[0106] Specifically, the system can determine the user's movement status based on the user's movement trajectory data. For example, if the user's movement trajectory always remains within the range of the bed, it can be determined that the user is in bed.

[0107] Specifically, if the system detects that the user is in bed, the speed change value is greater than or equal to the set threshold, and the proportion of monitoring points below or equal to the set height is greater than or equal to the set ratio, the system will determine that the user has triggered a fall from bed event.

[0108] In some embodiments, the set alarm event includes a falling-out-of-bed event.

[0109] Monitor whether the user has triggered a set alarm event based on the behavior, including:

[0110] S421. Calculate the ratio of monitoring points in the point cloud data that are lower than or equal to a set height to all monitoring points based on the distribution status data.

[0111] Specifically, based on the user's point cloud data, the system analyzes the number of monitoring points in the point cloud data that are lower than or equal to the set height and calculates the ratio of these monitoring points to all monitoring points. This ratio reflects the degree of contact between the user's body and the bed, floor, and other furniture.

[0112] For example, if the set height is lower than the height of the bed, then the higher the proportion of monitoring points in the point cloud data that are lower than or equal to the set height is, the closer the user is to the ground.

[0113] S422: Calculate a speed change value of the user according to the motion speed data.

[0114] Specifically, based on the motion data, the system can calculate the user's speed change between adjacent frames. This speed value represents the degree of change in the user's motion speed within a set time. This speed change value can be used to monitor the user's motion status, such as whether there are sudden rapid movements or stops.

[0115] S423: When it is confirmed according to the motion trajectory data that the user's action state is in bed, and the speed change value is greater than or equal to the set value, and the ratio is greater than or equal to the set ratio, it is determined that the user has triggered a fall-out event.

[0116] Specifically, the system can determine the user's action status based on the user's motion trajectory data. For example, if the user's motion trajectory remains outside the range of the bed, it can be determined that the user is in the out-of-bed state.

[0117] Specifically, if the system detects that the user is out of bed, the speed change value is greater than or equal to the set threshold, and the proportion of monitoring points below or equal to the set height is greater than or equal to the set ratio, the system will determine that the user has triggered a fall event.

[0118] Through the above steps, we can combine the height information, movement speed, movement trajectory and other multi-dimensional data in the point cloud data to accurately monitor whether the user has triggered a fall or a fall from bed, providing a reliable basis for the discovery and alarm of emergency situations.

[0119] In some embodiments, the set alarm event includes a lone out event.

[0120] Specifically, a solo outing event refers to a monitored user leaving their home, medical facility, or other safe space without anyone else by their side. Such an event can pose risks or safety concerns in certain situations, particularly for people who require special care or supervision, such as the elderly, children, and those with cognitive impairments or other medical conditions (such as autism and epilepsy).

[0121] In some embodiments, monitoring whether the user has triggered a set alarm event based on the behavior includes:

[0122] When it is monitored according to the motion trajectory data that the user leaves the space and does not return to the space after a first set time period, it is determined that the user has triggered a solo going-out event.

[0123] Specifically, when faced with the need to monitor users going out alone, regional boundaries of the monitoring space can be set in the system. In this way, when the user's movement trajectory is detected to have crossed these boundaries, the system can determine that the user has left the space.

[0124] Furthermore, when a user leaves a room, the system records the time they left and monitors whether they return within a set timeframe. If they do not return within the set timeframe, the system determines that they have triggered a Solo Out event.

[0125] Once the system determines that a user has triggered an "out-of-home" event, it immediately triggers the appropriate alert action. This can include sending an alert notification to a guardian, family member, or relevant agency so they can take appropriate action, such as contacting the user, locating the user, or notifying the police.

[0126] Through the above steps, the system can detect and alarm in time when the user goes out alone to ensure the safety of the monitored user.

[0127] In some embodiments, the set alarm events also include no-person activity events.

[0128] Specifically, an unattended activity event refers to a monitored user remaining alone in a home, medical facility, or other safe space for longer than a set time period without anyone else by their side. Such events can pose risks or safety concerns in specific situations, particularly for individuals requiring special attention or supervision, such as the elderly, children, and those with cognitive impairments or other medical conditions (such as autism and epilepsy).

[0129] In some embodiments, monitoring whether the user has triggered a set alarm event based on the behavior further includes:

[0130] When it is monitored according to the motion trajectory data that the duration of a user's single stay in the space exceeds a second set duration, it is determined that the user has triggered an unmanned activity event.

[0131] Specifically, when monitoring for extended periods of time, the system can set boundaries for the monitored space. This way, if the user's movement trajectory indicates that they haven't crossed these boundaries for an extended period (meaning greater than a second set duration, such as 24 or 36 hours), the system can determine that the user has stayed in the space for too long. Furthermore, the system will determine that the monitored user has triggered an unattended activity event.

[0132] Once the system determines that a monitored user has triggered an unattended activity event, it will immediately trigger the appropriate alarm action. This can include sending an alarm notification to a guardian, family member, or relevant agency so that they can take appropriate action, such as contacting the user, locating the user, or notifying the police.

[0133] Through the above steps, the system can detect and alarm in time when the monitored user stays in the space for a long time, so as to ensure the safety of the monitored user.

[0134] In some embodiments, calculating the phase change data caused by the user's breathing and heartbeat based on the echo signal includes:

[0135] S110 , after obtaining the range-Doppler spectra of all target units in the space, extracting phase data where Doppler is 0 from the range-Doppler spectra of any antenna.

[0136] Specifically, the range-Doppler maps of all target units in the space can be obtained during the step of acquiring the user's point cloud data. This process can reduce the system's computational process and save computing power and energy.

[0137] Furthermore, in a multi-channel millimeter-wave radar system, the range-Doppler spectrum of any antenna can be selected for data processing. In the selected range-Doppler spectrum, the point with a Doppler frequency of 0 corresponds to a static or slowly moving target (person or object). The phase data of this target can change with the target's slight movement.

[0138] Therefore, the extracted Doppler frequency 0 phase data can be used to screen out users who are static (usually in a sleeping state) in the space.

[0139] S120 , accumulating multiple frames of phase data to obtain slow Doppler phase data with a Doppler of 0.

[0140] Among them, slow Doppler refers to the change in echo signal frequency caused by slight movement of the target.

[0141] Specifically, the system processes multiple frames (e.g., 10 consecutive seconds) of phase data and calculates the differences between the multiple frames to obtain the slow Doppler phase data of the target object with a Doppler of 0. This allows the temporal variation of this data to be determined.

[0142] It can be understood that by selecting data with a Doppler of 0 and performing multi-frame data accumulation processing on them, and by superimposing the multi-frame data, the observation time of the target object can be increased, thereby improving the reliability of detection.

[0143] S130: Perform static filtering on the slow Doppler phase data to filter out slow Doppler phase data of users that are static in space.

[0144] Specifically, static filtering of the slow Doppler data (subtracting the average of multiple frames from each frame) leverages the stability of background signals to remove completely static background signals, retaining only the signals that vary. The objects corresponding to these varying signals can be considered static users in space. Accordingly, these varying signals serve as the slow Doppler phase data for these static users required for subsequent calculations.

[0145] S140 , filtering peak values ​​from the statically filtered slow Doppler phase data based on a preset peak threshold, and determining a target range unit where the static user is located according to the peak values.

[0146] Specifically, after performing static filtering, the system selects peaks in the slow Doppler phase data based on a preset peak threshold. These peaks represent the user's breathing and heartbeat signals. By identifying these peaks, the system can determine the target range cell where the static user is located.

[0147] S150 , extracting the phase at the target range unit of the slow Doppler phase data to obtain the phase change data.

[0148] Finally, by extracting the phase data at the determined target distance unit, the phase change data caused by the user's breathing and heartbeat can be obtained.

[0149] In some embodiments, calculating the user's vital signs based on the phase change data includes:

[0150] S311, using the 6th order Butterworth filter designed according to the frequency band of the respiratory frequency to filter the phase change data and extract the respiratory waveform data.

[0151] The Butterworth filter is a digital filter commonly used in signal processing. It enhances signals within a specific frequency range while suppressing unwanted frequency components. A sixth-order Butterworth filter designed for the respiratory frequency range can emphasize respiratory waveform signals while reducing noise.

[0152] Specifically, after filtering the phase change data, the obtained signal can be considered as respiratory waveform data, which reflects the periodic changes in the respiration of the target object.

[0153] S312: Perform fast Fourier transform on the respiratory waveform data to extract the respiratory cycle and calculate the user's respiratory frequency.

[0154] Specifically, by performing a fast Fourier transform (FFT) on the extracted respiratory waveform data, the frequency components of the target object's respiratory cycle can be extracted from the waveform. Based on the periodic changes in the frequency components of the respiratory cycle, the target object's respiratory frequency can be calculated.

[0155] In addition, by analyzing the FFT results, the amplitude and phase information corresponding to the respiratory frequency in the spectrum can be extracted. Among them, the amplitude information of the respiratory frequency can be used to indicate the intensity of breathing.

[0156] In some embodiments, calculating the user's vital signs based on the phase change data includes:

[0157] S321 , using a 6th-order Butterworth filter designed according to the frequency band of the heart rate to filter the phase change data and extract the heart rate waveform data.

[0158] The Butterworth filter is a digital filter commonly used in signal processing. It enhances signals within a specific frequency range while suppressing unwanted frequency components. A sixth-order Butterworth filter designed for the heart rate frequency range can emphasize the heartbeat waveform signal and reduce noise.

[0159] Specifically, after filtering the phase change data, the obtained signal can be considered as heart rate waveform data, which reflects the periodic changes in the heart rate of the target object.

[0160] S322: Perform wavelet transform on the heart rate waveform data and then perform fast Fourier transform to extract the heart rate cycle and calculate the user's heart rate.

[0161] Among them, wavelet transform is a transform that can decompose the signal into basis functions of different scales and frequencies, which is used to extract the detailed features of the signal or remove noise.

[0162] Specifically, by performing wavelet transform and fast Fourier transform (FFT) on the extracted respiratory waveform data, the frequency components of the target object's heart rate cycle can be extracted from the waveform. Based on the periodic changes in the frequency components of the heart rate cycle, the target object's heart rate can be calculated.

[0163] In addition, by analyzing the FFT results, we can also extract the amplitude and phase information corresponding to the heart rate in the spectrum. Among them, the amplitude information of the heart rate can be used to indicate the intensity of the heartbeat.

[0164] In some embodiments, the set alarm events also include abnormal breathing events and abnormal heart rate events.

[0165] Specifically, in addition to falls, falling out of bed, and going out alone, the system can also set abnormal breathing and abnormal heart rate events as alarm events. Abnormal breathing events can include abnormal breathing rate and apnea, while abnormal heart rate events can include abnormal heart rate frequency, irregular heartbeat, or arrhythmia.

[0166] In some embodiments, monitoring whether the user has triggered a set alarm event based on the vital signs further includes:

[0167] S210: If the user's heart rate is detected to be outside the set heart rate range, it is determined that the user has triggered an abnormal heart rate event;

[0168] S220: If the user's breathing rate is monitored to be outside the set breathing range, it is determined that the user has triggered an abnormal breathing event.

[0169] Specifically, the system monitors the user's vital signs (heart rate and respiratory rate) in real time and sets specific heart rate and respiratory rate ranges. If the user's heart rate is detected to be outside the set heart rate range, the system determines that the user has triggered an abnormal heart rate event. Similarly, if the user's respiratory rate is detected to be outside the set respiratory rate range, the system determines that the user has triggered an abnormal respiratory event.

[0170] For example, if the user's heart rate exceeds the set upper limit, the system will determine it as an abnormal heart rate event. An excessively fast heart rate may indicate that the user is in a state of cardiovascular stress and needs to be alert.

[0171] For example, if the user's heart rate is lower than a set lower limit, it is also determined to be an abnormal heart rate event. A heart rate that is too slow may indicate that the user may have arrhythmia or other heart problems.

[0172] For example, if the user's breathing rate exceeds a set upper limit, the system will determine it as an abnormal breathing event. Excessively fast breathing may indicate that the user may be in anxiety, respiratory disease or other emergency conditions.

[0173] For example, if the user's breathing rate is lower than the set lower limit, it is also determined to be an abnormal breathing event. Slow breathing may indicate that the user may have respiratory problems or other potential health risks.

[0174] It is worth adding that when the system detects that the user's heart rate or respiratory rate exceeds the set range, it will immediately trigger the corresponding alarm operation so that medical measures can be taken in time or medical personnel, guardians, etc. can be notified to ensure that the user can receive timely help and treatment when facing health risks.

[0175] In addition, embodiments of the present application further provide a computer-readable storage medium. The computer-readable storage medium may be any one of, or any combination of, a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), or a USB memory device. The computer-readable storage medium includes a multifunctional health monitoring program 10 based on millimeter-wave radar. The specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the multifunctional health monitoring method based on millimeter-wave radar and the server 1 described above, and will not be further described herein.

[0176] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram.

[0178] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0180] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0181] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A multifunctional health monitoring method based on millimeter wave radar, wherein: include: Receiving echo signals in the space, and calculating point cloud data of the user and phase change data caused by the user's breathing and heartbeat according to the echo signals; Calculating the user's behavior based on the point cloud data; Calculating the user's vital signs based on the phase change data, where the vital signs include breathing rate and heart rate; It is determined based on the behavior and / or the vital sign that the user has triggered a set alarm event, and a corresponding alarm operation is performed based on the type of the triggered alarm event.

2. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: Calculating the user's point cloud data according to the echo signal includes: Converting the echo signal into a digital signal; Performing a range-dimensional fast Fourier transform and a Doppler-dimensional Fourier transform on the digital signal to obtain a range-Doppler spectrum of all target units in the space; Performing incoherent accumulation on the range-Doppler spectrum, and performing two-dimensional constant false alarm detection on the accumulation result using preset threshold parameters, and extracting the target unit corresponding to the user on the range-Doppler spectrum; Performing azimuth-elevation joint angle measurement on the target unit corresponding to the user to obtain the azimuth and elevation angles of the target unit; Obtaining point data corresponding to the target unit in space according to the azimuth angle and the pitch angle and the corresponding range unit on the range-Doppler spectrum; and The point cloud data of the user is generated according to all the point data of the user.

3. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 2, wherein: Calculating the user's behavior according to the point cloud data includes: Calculating the distribution status data of all monitoring points in the point cloud data; Performing clustering and fusion on the user's point cloud data to obtain a fitting center of the point cloud data; Performing Kalman filter tracking on the fitting center to obtain motion data of the fitting center, wherein the motion data includes motion trajectory data and motion speed data; and The user's behavior is generated according to the distribution state data and the motion data.

4. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 3, wherein: Clustering and fusing the user's point cloud data to obtain a fitting center of the point cloud data includes: After clustering and fusing the user's point cloud data, the least square method is used to perform spherical fitting to obtain the sphere center coordinates of the clustering result, and the sphere center coordinates are used as the fitting center of the point cloud data.

5. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 3, wherein: The set alarm events include fall events; Determining that the user has triggered a set alarm event based on the behavior includes: Calculate the ratio of monitoring points in the point cloud data that are lower than or equal to a set height to all monitoring points according to the distribution state data; Calculate the speed change value of the user according to the motion speed data; If it is confirmed according to the motion trajectory data that the user's action state is in bed, and the speed change value is greater than or equal to the set value, and the ratio is greater than or equal to the set ratio, it is determined that the user has triggered a fall from bed event.

6. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 3, wherein: The set alarm events include falling out of bed; Determining that the user has triggered a set alarm event based on the behavior includes: Calculate the ratio of monitoring points in the point cloud data that are lower than or equal to a set height to all monitoring points according to the distribution state data; Calculate the speed change value of the user according to the motion speed data; If it is confirmed according to the motion trajectory data that the user's action state is out of bed, and the speed change value is greater than or equal to the set value, and the ratio is greater than or equal to the set ratio, it is determined that the user has triggered a fall event.

7. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 3, wherein: The set alarm events include the event of going out alone; Determining that the user has triggered a set alarm event based on the behavior includes: If it is monitored according to the motion trajectory data that the user leaves the space and does not return to the space after a first set time period, it is determined that the user has triggered a solo going-out event.

8. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 7, wherein: Monitoring the user leaving the space according to the motion trajectory data includes: Set the area boundaries of the space; If it is detected through the motion trajectory data that the user has crossed the area boundary, it is determined that the user has left the monitoring space.

9. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 3, wherein: The set alarm events include no-person activity events; Determining that the user has triggered a set alarm event according to the behavior includes: if it is monitored according to the motion trajectory data that the single stay duration of the user in the space exceeds a second set duration, then determining that the user has triggered an unmanned activity event.

10. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 9, wherein: The time duration of a single stay of a user in a space monitored according to the motion trajectory data exceeds a second set time duration, including: Set the area boundaries of the space; If it is monitored through the motion trajectory data that the user has not crossed the area boundary for a period of time exceeding the second set time, it is determined that the single stay time of the user in the space exceeds the second set time.

11. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 2, wherein: The millimeter wave radar has multiple antennas; Calculating phase change data caused by the user's breathing and heartbeat according to the echo signal, including: After obtaining the range-Doppler spectrum of all target units in the space, extract the phase data of Doppler 0 from the range-Doppler spectrum of any antenna; Accumulating multiple frames of phase data to obtain slow Doppler phase data with a Doppler of 0; Static filtering is performed on the slow Doppler phase data to filter out the slow Doppler phase data of users that are static in space; Filtering a peak value from the slow Doppler phase data after static filtering based on a preset peak value threshold, and determining a target distance unit where a static user is located according to the peak value; and The phase is extracted at the target distance unit of the slow Doppler phase data to obtain the phase change data.

12. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: Calculating the user's vital signs according to the phase change data includes: The phase change data is filtered using a 6th-order Butterworth filter designed according to the frequency band of the respiratory frequency to extract the respiratory waveform data; Perform fast Fourier transform on the respiratory waveform data to extract the respiratory cycle and calculate the user's respiratory frequency.

13. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: Calculating the user's vital signs according to the phase change data includes: The phase change data is filtered using a 6th-order Butterworth filter designed according to the frequency band of the heart rate to extract the heart rate waveform data; The heart rate waveform data is processed by wavelet transform and then fast Fourier transform to extract the heart rate cycle and calculate the user's heart rate.

14. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: The set alarm events include abnormal heart rate events; Determining, according to the vital signs, that the user has triggered a set alarm event, includes: If the user's heart rate is detected to be outside the set heart rate range, it is determined that the user has triggered an abnormal heart rate event.

15. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: The set alarm events include abnormal breathing events; Determining, according to the vital signs, that the user has triggered a set alarm event, includes: It is monitored that the user's breathing rate exceeds the set breathing range, and it is determined that the user has triggered an abnormal breathing event.

16. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: According to the type of alarm event triggered, the corresponding alarm operation is performed, including: When a user falls, an alarm message is sent to the pre-bound terminal in at least one of the following ways: Phone calls, text messages, app push, emails, instant messaging.

17. The multifunctional health monitoring method based on millimeter wave radar as claimed in claim 1, wherein: The method further comprises: After the monitored user triggers an alarm event and performs an alarm operation, a communication request is initiated with the bound terminal, and after the communication is established, a two-way voice call is realized.

18. A multifunctional health monitoring system based on millimeter wave radar, wherein: It includes a millimeter-wave radar, a memory, a processor, and a multifunctional health monitoring program based on the millimeter-wave radar stored in the memory and executable on the processor. When the processor executes the multifunctional health monitoring program based on the millimeter-wave radar, the multifunctional health monitoring method based on the millimeter-wave radar as described in any one of claims 1 to 17 is implemented.

19. A computer-readable storage medium, wherein: The computer-readable storage medium stores a multifunctional health monitoring program based on millimeter-wave radar, and when the multifunctional health monitoring program based on millimeter-wave radar is executed by the processor, the multifunctional health monitoring method based on millimeter-wave radar as described in any one of claims 1-17 is implemented.

Images