Multifunctional health monitoring and early warning system based on radar and vision fusion

Abstract

The invention provides a multifunctional health monitoring and early warning system based on radar and vision fusion, and relates to the technical field of intelligent early warning, and the system comprises a feedback receiving module which receives a radar feedback signal and an image feedback signal; the monitoring identification module traverses the target monitoring area to determine monitoring data, and carries out monitoring identification; the abnormity judgment module is used for carrying out fusion analysis on the first monitoring data set and the second monitoring data set, carrying out bidirectional abnormity judgment and generating an early warning instruction; and the risk analysis module uploads the abnormal judgment result to a cloud platform for risk analysis, and sends early warning information to a remote terminal according to the risk level. According to the invention, the technical problem that the accuracy of early warning information is affected due to high false alarm rate caused by environmental infection due to dependence on a single sensor in health monitoring in the prior art can be solved, risk assessment is carried out by combining radar and visual fusion, the accuracy of anomaly recognition is improved, and the accuracy of early warning information is improved. And the early warning real-time performance and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent warning technology, and particularly to a multi-functional health monitoring and warning system based on the fusion of radar and vision. Background Art

[0002] With the advent of an aging society, the health and safety issues of groups such as the elderly living alone have become increasingly prominent. Traditional health monitoring methods mostly rely on a single sensor for data collection and on wearable devices. Single-sensor technologies (such as vision or radar) are effective in some specific application scenarios. However, in complex environmental conditions, there are problems such as inconvenient use and limited monitoring range. Contact devices may be affected by human movement, sweat, or other factors, resulting in inaccurate data, and they cannot provide continuous monitoring without contact, making it difficult to identify abnormal behaviors in a timely manner, leading to a high false alarm rate, which in turn affects the accuracy of warning information.

[0003] In summary, there is a technical problem in the prior art that because health monitoring relies on a single sensor, it is easily affected by the environment, resulting in a high false alarm rate, which in turn affects the accuracy of warning information. Summary of the Invention

[0004] The purpose of this application is to provide a multi-functional health monitoring and warning system based on the fusion of radar and vision, so as to solve the technical problem in the prior art that because health monitoring relies on a single sensor, it is easily affected by the environment, resulting in a high false alarm rate, which in turn affects the accuracy of warning information.

[0005] In view of the above problems, this application provides a multi-functional health monitoring and warning system based on the fusion of radar and vision. The multi-functional health monitoring and warning system based on the fusion of radar and vision includes: a feedback receiving module, which is used to transmit radar signals through a radar module, receive radar feedback signals, record the target monitoring area in real time through a camera module, and receive image feedback signals; a monitoring and recognition module, which is used to traverse the target monitoring area to determine the target monitoring human body data, and respectively monitor and recognize the target monitoring human body data based on the radar feedback signal and the image feedback signal to generate a first monitoring data set and a second monitoring data set; an abnormal determination module, which is used to perform fusion analysis on the first monitoring data set and the second monitoring data set, generate a health monitoring result for two-way abnormal determination, and generate a warning instruction according to the abnormal determination result. The warning instruction includes warning information; a risk analysis module, which is used to trigger a warning mechanism through the warning instruction to upload the abnormal determination result to the cloud platform for risk analysis, and send warning information to a remote terminal according to the abnormal risk level.

[0006] Optionally, a radar signal receiving unit is configured to transmit a frequency-modulated continuous wave radar signal to a target monitoring area through a radar module in the host, receive the reflected signal in the target area through a receiving antenna, and obtain an initial radar echo signal; a frequency shift screening unit is configured to perform frequency shift screening based on the initial radar echo signal to obtain the radar feedback signal; a multi-view acquisition unit is configured to adjust the acquisition angle of the camera module in the host at a preset interval period, determine the data acquisition angle, and perform multi-view acquisition through the camera module according to the data acquisition angle to obtain the image feedback signal.

[0007] Optionally, a grid determination unit is configured to divide the target monitoring area into N×M grid cells, determine a plurality of grid cells, where N and M are positive integers greater than 1, and N and M can be equal; a scan result determination unit is configured to control the radar module and the camera module to traverse the plurality of grid cells for human body recognition scanning according to a preset path, and obtain target monitoring human body data, where the target monitoring human body data includes a radar scan result and an image scan result; a feature extraction unit is configured to perform feature extraction on the radar scan result based on the radar feedback signal, determine human body micro-motion feature data, and perform change monitoring according to the human body micro-motion feature data to generate the first monitoring data set; a trajectory monitoring unit is configured to perform feature extraction on the image scan result based on the image feedback signal, determine human body contour feature data, and perform trajectory monitoring according to the human body contour feature data to generate the second monitoring data set.

[0008] Optionally, a multi-feature extraction sub-unit is configured to perform short-time Fourier transform on the radar scan result based on the radar feedback signal to extract human body micro-motion feature data, where the human body micro-motion feature data includes a respiration fundamental frequency feature, a heartbeat harmonic feature, and a body motion acceleration feature; a state monitoring sub-unit is configured to perform change state monitoring based on the respiration fundamental frequency feature, the heartbeat harmonic feature, and the body motion acceleration feature, obtain a plurality of body motion change events, and add the plurality of body motion change events to the first monitoring data set.

[0009] Optionally, a frame difference processing sub-unit is configured to perform frame difference processing on the image scan result based on the image feedback signal to construct human body bounding box parameters; an identifier assignment sub-unit is configured to assign an identity identifier to the human body bounding box parameters to determine the human body contour feature data; a vector calculation sub-unit is configured to perform motion vector calculation based on the human body contour feature data to obtain human body motion trajectory data, and add the human body motion trajectory data to the second monitoring data set.

[0010] Optionally, a matrix construction unit is configured to perform wavelet denoising on the first monitoring data set to construct a radar feature matrix, extract key frames from the second monitoring data set to construct a visual feature matrix; a regularization fusion unit is configured to perform dynamic time warping fusion on the radar feature matrix and the visual feature matrix to generate a joint feature vector; a feature division unit is configured to divide the joint feature vector into a physiological feature subset and a behavior feature subset; a correlation analysis unit is configured to perform temporal correlation analysis based on the physiological feature subset to obtain a physiological health monitoring result; a motion analysis unit is configured to perform skeletal point motion analysis based on the behavior feature subset to obtain a behavior health monitoring result; a result generation unit is configured to associate and integrate the physiological health monitoring result and the behavior health monitoring result to generate the health monitoring result.

[0011] Optionally, a physiological abnormality determination unit is configured to perform physiological abnormality determination based on the physiological health monitoring result to obtain a physiological abnormality determination result; a behavior abnormality determination unit is configured to perform behavior abnormality determination based on the behavior health monitoring result to obtain a behavior abnormality determination result; a physiological warning unit is configured to generate a physiological warning message when only the physiological abnormality determination result exists, perform confidence analysis according to the physiological warning message, and trigger a first warning instruction according to a first confidence level; a behavior warning unit is configured to generate a behavior warning message when only the behavior abnormality determination result exists, perform confidence analysis according to the behavior warning message, and trigger a second warning instruction according to a second confidence level; a confidence analysis unit is configured to generate a two-way warning message for confidence analysis when both the physiological abnormality determination result and the behavior abnormality determination result exist, and trigger a third warning instruction according to a third confidence level.

[0012] Optionally, a first risk analysis unit is configured to upload the physiological abnormality determination result to a cloud platform for risk analysis to generate a first abnormal risk level when the first warning instruction triggers the warning mechanism; a first warning unit is configured to send the physiological warning message to a remote terminal according to the first abnormal risk level; a second risk analysis unit is configured to upload the behavior abnormality determination result to a cloud platform for risk analysis to generate a second abnormal risk level when the second warning instruction triggers the warning mechanism; a second warning unit is configured to send the behavior warning message to a remote terminal according to the second abnormal risk level; a third risk analysis unit is configured to upload the physiological abnormality determination result and the behavior abnormality determination result to a cloud platform for risk analysis to generate a third abnormal risk level when the third warning instruction triggers the warning mechanism; a third warning unit is configured to send the two-way warning message to a remote terminal according to the third abnormal risk level.

[0013] Optionally, a level determination unit is configured to retrieve a historical abnormal event set within a target monitoring area, perform causal analysis based on the historical abnormal event set, label risk level tags for the historical abnormal event set according to the analysis results, and determine an abnormal risk level list; a weight optimization unit is configured to perform weight optimization according to the abnormal risk level list to construct a gradient boosting decision tree; a decision tree embedding unit is configured to embed the gradient boosting decision tree into a cloud platform for risk analysis.

[0014] Optionally, a response recording unit is configured to record the response processing of the remote terminal receiving the warning information to generate a response record report; a processing efficiency determination unit is configured to traverse the response record report to analyze the processing results of the remote terminal to generate the terminal processing efficiency; a mapping relationship determination unit is configured to perform mapping analysis on the abnormal risk level and the warning information to determine a risk-warning mapping relationship; a warning mode determination unit is configured to dynamically adjust the risk-warning mapping relationship according to the terminal processing efficiency to generate a risk level score, and replace the warning mode for the remote terminal according to the risk level score.

[0015] The technical solution provided in this application has at least the following beneficial effects: Through a feedback receiving module, which is configured to transmit a radar signal through a radar module, receive a radar feedback signal, record the target monitoring area in real time through a camera module, and receive an image feedback signal; a monitoring and recognition module is configured to traverse the target monitoring area to determine target monitoring human body data, and respectively monitor and recognize the target monitoring human body data based on the radar feedback signal and the image feedback signal to generate a first monitoring data set and a second monitoring data set; an abnormality determination module is configured to perform fusion analysis on the first monitoring data set and the second monitoring data set to generate a health monitoring result for two-way abnormality determination, and generate a warning instruction according to the abnormality determination result, where the warning instruction includes warning information; a risk analysis module is configured to trigger a warning mechanism through the warning instruction to upload the abnormality determination result to a cloud platform for risk analysis, and send warning information to a remote terminal according to the abnormal risk level. That is to say, through the monitoring of the radar module and visual fusion, two-way abnormality determination is performed on the monitoring results, abnormal risks are quickly identified and uploaded to the cloud platform for risk assessment, warning information is sent, false alarms and missed alarms are reduced, and the accuracy and timeliness of abnormal situation identification are improved, thereby improving the real-time performance and accuracy of early warning.

[0016] The above description is only an overview of the technical solution of this application. In order to understand the technical means of this application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the following specific embodiments of this application are specifically given. It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of this application, nor is it used to limit the scope of this application. Other features of this application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in this application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0018] Figure 1 FIG. is a schematic structural diagram of a multi-functional health monitoring and early warning system based on radar and vision fusion of this application.

[0019] Figure 2 FIG. is a schematic structural diagram of a monitoring and recognition module in a multi-functional health monitoring and early warning system based on radar and vision fusion of this application.

[0020] Description of reference numerals: feedback receiving module 11, monitoring and recognition module 12, abnormal determination module 13, risk analysis module 14, grid determination unit 21, scan result determination unit 22, feature extraction unit 23, trajectory monitoring unit 24. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] By providing a multi-functional health monitoring and early warning system based on radar and vision fusion, this application solves the technical problem in the prior art that due to the dependence of health monitoring on a single sensor, it is easily affected by the environment, resulting in a high false alarm rate, thus affecting the accuracy of early warning information. Through the monitoring by the radar module and vision fusion, a two-way abnormal determination is carried out on the monitoring results, the abnormal risks are quickly identified and uploaded to the cloud platform for risk assessment, and early warning information is sent, reducing false alarms and missed alarms, improving the accuracy and timeliness of abnormal situation recognition, and thus improving the real-time and accuracy of early warning.

[0022] Next, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the example embodiments described herein. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application. Additionally, it should be noted that for the sake of description, only the parts related to the present application are shown in the accompanying drawings rather than all of them.

[0023] For the embodiments, please refer to the attached Figure 1 , the present application provides a multi-functional health monitoring and warning system based on radar and vision fusion. Among them, the multi-functional health monitoring and warning system based on radar and vision fusion includes: A feedback receiving module 11, configured to transmit a radar signal through a radar module, receive a radar feedback signal, record the target monitoring area in real time through a camera module, and receive an image feedback signal.

[0024] Furthermore, the feedback receiving module 11 in the multi-functional health monitoring and warning system based on radar and vision fusion is further configured to: A radar signal receiving unit, configured to transmit a continuous wave frequency modulated radar signal to the target monitoring area through the radar module in the host, receive the reflected signal in the target area through a receiving antenna, and obtain an initial radar echo signal; a frequency shift screening unit, configured to perform frequency shift screening based on the initial radar echo signal to obtain the radar feedback signal; a multi-view acquisition unit, configured to adjust the acquisition angle of the camera module in the host at a preset interval period, determine the data acquisition angle, and perform multi-view acquisition through the camera module according to the data acquisition angle to obtain the image feedback signal.

[0025] Specifically, the host and the slave are registered through the cloud platform to complete device binding and network configuration. Through the internal network communication module (such as supporting WiFi or Bluetooth), efficient communication between the host and the slave is ensured. Through the external network communication module (such as supporting 4G / 5G transmission), the real-time performance and stability of data transmission are ensured. The internal network communication module is used to transmit the radar signal collected by the slave to the host through the internal network to ensure that the data can be transmitted quickly and stably. The external network communication module is used for transmission with the cloud platform to ensure the real-time performance and stability of data transmission.

[0026] The host is installed at the core position that can cover the target monitoring area to the maximum extent to ensure that the radar and the camera can cover the entire monitoring range. The radar module and the camera module are both installed on the host. The radar module in the host continuously transmits and receives radar signals, and the camera module collects video signals in real time. The radar signals and video signals are transmitted to the calculation module for human body recognition and posture analysis through a preset AI data model.

[0027] The host includes both a radar module and a camera module. The radar module transmits and receives radar signals for monitoring human postures, breathing, and heartbeats. The camera module includes a binocular camera and a 270-degree rotating base for collecting video signals to assist the radar module in human recognition and posture analysis. When there are areas where cameras are not allowed to be installed, such as bedrooms, bathrooms, etc., only radar monitoring is carried out. After the fall monitoring slave device (focusing on monitoring human postures and identifying fall states) and the breathing and heart rate monitoring device (focusing on monitoring human breathing and heart rate states) emit millimeter-wave radar, the reflected millimeter waves are transmitted to the host through the intranet communication module. The host judges whether there is a person in the target area based on the radar signals uploaded by the slave devices. When it is judged that there is a person in the target area, it analyzes the person's posture and continuously monitors the person's breathing and heartbeats through the fall monitoring slave device and the breathing and heart rate monitoring slave device.

[0028] The millimeter-wave radar signal will have a frequency shift as the object moves. The frequency shift in the radar echo reflects the moving speed of the target object. The movement of the human body is different from that of other objects (such as furniture, pets, etc.), usually showing different speed patterns and frequency shift characteristics. The movement of the human body usually has relatively continuous and specific movement patterns (such as walking, falling, standing still, etc.), while the movement of other objects is relatively simple or irregular. For example, fixed objects such as walls and tables do not move, and the frequency change of the reflection signal of static objects is very small, while the human body shows a continuously changing frequency shift signal. The movement frequency range of pets (such as dogs, cats) is relatively wide, showing fast, jumpy, and irregular movements (such as running, jumping, etc.). The volume of pets is usually small, so the radar signal reflected by them will be relatively weak, but the movement frequency characteristics are highly irregular. Based on these rules, it can be judged whether there is a person in the target area.

[0029] The intensity and arrival time of radar echo signals can reflect the position of an object, based on which the distance and approximate position of the object can be determined. If the signal intensity is high and the time delay is stable, it indicates that the target is a human body, and further posture analysis is carried out. The frequency change of radar signals reflects the movement of the human body. According to the frequency shift amount (i.e., the magnitude of frequency offset), the speed and direction of human movement are inferred. When the human body performs different movements (such as walking, standing, sitting, squatting, etc.), the frequency change of radar signals presents different patterns. For example, when the human body is walking or standing, the frequency shift is relatively stable, with a small amplitude and a certain periodicity; when the human body is running or moving quickly, the amplitude of the frequency shift is large and the period is short, reflecting high-speed movement; when the human body is sitting or squatting, the amplitude of frequency change is small, and the persistence and stability of the reflected signal are relatively high. Through the frequency change of radar echo signals, the movement states of human limbs are inferred. Different parts of the human body, such as the arms and legs, have different movement patterns during walking or movement, and the frequency change of radar signals can reflect the dynamic characteristics of these different parts.

[0030] The fall monitoring slave device and the respiration and heart rate monitoring slave device are respectively deployed at the center of the areas that need to be monitored intensively (such as bedrooms, bathrooms, etc.). The radar module of the fall monitoring slave device continuously transmits and receives radar signals, and the signals are transmitted to the host through the internal network communication module. In addition, besides the fall monitoring slave device and the respiration and heart rate monitoring slave device, there is also a user presence monitoring slave device, which focuses on monitoring whether there is a human body in the target area. Time periods and warning thresholds are set for the host and the slave devices respectively. During the set time period, monitoring is carried out through the user presence monitoring slave device. If the presence of the user cannot be detected within the preset time period, that is, no vital signs are detected during the time period when a person should have appeared in the target area (for example, an old person should get up and enter the monitoring area within a certain time period in the morning), it will be judged as abnormal, thus triggering an alarm. The presence monitoring slave device will emit radar signals and judge whether there is someone in the monitoring area based on the reflected signals. The warning threshold is set to expect to detect the signs or activities that should be present in the target area within a time period. That is to say, it is normal for there to be human activities in the target area within this time period. If this threshold is exceeded and no activity or sign is still detected, an alarm will be triggered. For example, if within a certain specific time period (such as 5 - 7 o'clock), it is expected that an old person should enter areas such as the kitchen and bathroom after getting up, and assuming the set threshold is "detect at least 1 human activity or sign signal", if no sign is detected within this time period (i.e., the threshold is not met), it is considered abnormal and an alarm is triggered.

[0031] The computing module of the host judges the human body posture according to the radar signals uploaded by the slave. The radar module of the breathing and heart rate monitoring slave continuously transmits and receives radar signals, and the radar signals are transmitted to the host through the intranet communication module. The computing module of the host judges the human body breathing and heart rate states according to the radar signals uploaded by the slave. The host and the slave are networked through the intranet communication module. One host can be connected to multiple slaves, and one slave can only be connected to one host. Non-contact monitoring is realized through radar and camera, improving user comfort. The computing module in the host is built with an AI data model for processing radar signals and video signals to realize functions such as human body recognition, posture analysis, breathing and heart rate monitoring. The main task of the computing module is to receive and process data uploaded from different sensors (such as radar, camera, etc.), conduct signal interpretation and analysis, be responsible for data processing, feature extraction, pattern recognition and final health monitoring and alarm decision-making. The computing module usually has high-performance computing capabilities, especially when processing complex signals (such as radar and video signals), often relying on high-speed computing, parallel processing and hardware acceleration. The AI data model is a model trained through methods such as machine learning and deep learning, which can analyze, classify, predict or identify the input data, be responsible for extracting useful information from the input radar signals and video signals, and realize advanced functions such as human body recognition, posture analysis, breathing and heart rate monitoring.

[0032] Collect a large amount of radar signal and video signal data, including samples of normal behaviors (such as walking, standing, sitting) and abnormal behaviors (such as falling, etc.). During the data collection process, it is necessary to label the behavior samples, including behavior types, posture information, etc. According to the task requirements, select the convolutional neural network architecture. Divide the collected labeled data into a training set and a test set. The training set is used to train the model, and the test set is used to evaluate the performance of the model. Use the training set to train the model. Through optimization algorithms such as backpropagation algorithm and gradient descent, adjust the weights and biases of the model. After multiple training cycles, gradually learn the laws in the data. Evaluate the accuracy of the model through the test set, and adjust the structure or training parameters (such as learning rate, batch size, etc.) of the model as needed, and adopt techniques such as cross-validation to improve the generalization ability of the model. Set the convergence conditions of the model, such as the validation loss changing less than 0.01 for 5 consecutive rounds or the training set accuracy reaching 95%. When the model reaches the convergence conditions, stop training to obtain the AI data model.

[0033] The host and slave machines are started to work together to monitor the human body status in real time. The radar module in the host emits a frequency-modulated continuous wave radar signal to the target monitoring area, that is, continuously changes the frequency of the transmitted signal to achieve precise detection of the target, and calculates the position and speed of the target object according to the frequency change of the reflected signal. The radar signal will pass through the target monitoring area and be reflected back by the human body or other objects. The reflected radar echo signal is received through the receiving antenna to obtain the initial radar echo signal, which contains information such as the position and speed of the target object. Frequency shift screening is performed on the initial radar echo signal. Frequency shift screening analyzes the frequency change in the radar echo signal to screen out the frequency components related to the movement of the target object, with the aim of extracting the frequency components related to the target movement from the echo signal. The radar feedback signal is obtained by the radar module transmitting a radar signal, receiving the radar feedback signal, and the camera module records the target monitoring area in real time and receives the image feedback signal, which contains information about the target object, such as position, speed, movement direction, etc.

[0034] Adjust the acquisition angle of the camera module at a preset interval period to ensure coverage of the entire monitoring area. The camera module will perform multi-view acquisition from different angles to obtain a series of image feedback signals. The preset interval period is determined according to the target area and acquisition requirements to ensure coverage of different parts of the monitoring area. Usually, the preset interval period is shorter in areas where the target activities are more frequent for more frequent acquisition. According to the adjusted acquisition angle, the camera module performs multi-view acquisition. The acquired images include images at different times and angles, and the acquired images are transmitted to the calculation module in the host to form an image feedback signal. The image feedback signal is the real-time image data of the target monitoring area captured by the camera, providing visual information about the target object, such as appearance, shape, color, actions, etc.

[0035] By jointly using the radar and the camera, the limitations of a single sensor are overcome, ensuring coverage of the entire monitoring area and reducing the influence of environmental factors. The radar signal has strong robustness in low-light and complex environments, while the visual information provided by the camera can help accurately identify abnormal behaviors (such as falling, walking, etc.), providing more comprehensive image information to help precisely analyze the target status.

[0036] The monitoring and recognition module 12 is used to traverse the target monitoring area to determine the target monitoring human body data, and respectively monitor and recognize the target monitoring human body data based on the radar feedback signal and the image feedback signal to generate a first monitoring data set and a second monitoring data set.

[0037] Further, as shown in the appendix Figure 2 The monitoring and recognition module 12 in the multifunctional health monitoring and warning system based on radar and vision fusion further is used for: A grid determination unit 21 for dividing the target monitoring area into N×M grid cells, determining a plurality of grid cells, where N and M are positive integers greater than 1, and N and M can be equal; a scan result determination unit 22 for controlling a radar module and a camera module to traverse the plurality of grid cells according to a preset path for human body recognition scanning to obtain target monitoring human body data, the target monitoring human body data including a radar scan result and an image scan result; a feature extraction unit 23 for extracting features from the radar scan result based on the radar feedback signal to determine human body micro-motion feature data, and performing change monitoring according to the human body micro-motion feature data to generate the first monitoring data set; a trajectory monitoring unit 24 for extracting features from the image scan result based on the image feedback signal to determine human body contour feature data, and performing trajectory monitoring according to the human body contour feature data to generate the second monitoring data set.

[0038] Further, the feature extraction unit 23 is further configured to: a multi-feature extraction subunit for performing short-time Fourier transform on the radar scan result based on the radar feedback signal to extract human body micro-motion feature data, the human body micro-motion feature data including a respiration fundamental frequency feature, a heartbeat harmonic feature, and a body motion acceleration feature; a state monitoring subunit for performing change state monitoring based on the respiration fundamental frequency feature, the heartbeat harmonic feature, and the body motion acceleration feature to obtain a plurality of body motion change events, and adding the plurality of body motion change events to the first monitoring data set.

[0039] Further, the trajectory monitoring unit 24 is further configured to: a frame difference processing subunit for performing frame difference processing on the image scan result based on the image feedback signal to construct human body bounding box parameters; an identifier assignment subunit for assigning an identity identifier to the human body bounding box parameters to determine the human body contour feature data; a vector calculation subunit for performing motion vector calculation based on the human body contour feature data to obtain human body motion trajectory data, and adding the human body motion trajectory data to the second monitoring data set.

[0040] Specifically, the target monitoring area is divided into several small grid cells, and each grid cell represents an independent monitoring area. N and M are the number of rows and columns of the division. N and M are greater than 1 and can be equal, ensuring a detailed division of the monitoring area. The size and number of each grid cell can be adjusted according to the scale of the actual monitoring area. For example, if the target monitoring area is a 10-meter×10-meter room, the area is divided into 5×5 grid cells, and the size of each grid cell is 2 meters×2 meters, so that the monitoring area can be refined to ensure that the activities within each grid cell can be monitored by the radar and camera modules.

[0041] According to the preset path, the radar module and camera module on the control host sequentially scan these grid units to obtain target monitoring human body data. The radar module obtains the dynamic information of an object by emitting signals and receiving reflected waves, while the camera records the shape and actions of the target by taking pictures. Each grid unit will be scanned by the radar and camera respectively to obtain the human body recognition data in that area. By traversing each grid unit, the radar and camera can comprehensively monitor every corner in the target area, thus obtaining the target monitoring human body data.

[0042] Extract the features of the radar scan results in the target monitoring human body data according to the radar feedback signal to obtain data related to minute human movements (such as breathing, heartbeat, etc.), that is, human micro-movement feature data. By analyzing the changes in the human micro-movement feature data, a first monitoring data set is generated. Specifically, the fall monitoring slave device and the breathing and heart rate monitoring slave device are devices dedicated to monitoring the human health status. The former is responsible for monitoring whether a fall occurs, and the latter is responsible for monitoring the breathing and heartbeat of the human body. The target area is scanned by the radar signal to collect the reflected radar data. The computing module on the host processes and analyzes the received radar signal, extracts key information, and generates a monitoring data set.

[0043] The fall monitoring slave device and the breathing and heart rate monitoring slave device respectively obtain the radar signals in the target area and transmit them to the computing module on the host. The computing module is responsible for processing and analyzing the received radar signals. The computing module of the host performs a short-time Fourier transform (STFT) on the received radar feedback signal, thereby extracting the frequency components of the signal. Through the time and frequency domain analysis of the radar signal by STFT, the frequency information related to human micro-movement is obtained. For example, assuming that the detected radar signal contains minute human movements (such as breathing, heartbeat, etc.), the host analyzes through STFT to obtain components of different frequencies, and extracts feature data such as the fundamental breathing frequency, heartbeat harmonics, and body movement acceleration from them.

[0044] By analyzing the frequency components in the radar signal, the host extracts the fundamental frequency features related to breathing, which reflect the normal breathing frequency of the human body. When this fundamental frequency is abnormal (such as too fast or too slow), it indicates that there may be an abnormal situation. The harmonic features of the heartbeat are obtained by frequency analysis of the radar signal and can provide the rhythm information of the heartbeat. When the heartbeat rhythm is abnormal, it is promptly identified and marked as abnormal. By extracting the acceleration features of the radar signal, the body movement of the human body is monitored. If abnormal body movements (such as rapid body movement or fall) are detected, corresponding markings are made.

[0045] Based on the extracted fundamental breathing frequency features, heartbeat harmonic features, and body movement acceleration features, change state monitoring is performed to obtain multiple body movement change events. Change state monitoring refers to dynamic monitoring based on the extracted micro-movement feature data (such as breathing, heartbeat, body movement, etc.) to identify whether an abnormal state has occurred in the human body. Multiple body movement change events are obvious change behaviors that occur in the human body during the monitoring process, such as rapid breathing, arrhythmia, abnormal body movement, etc. For example, calculate the harmonic energy ratio of the fundamental breathing frequency. When the harmonic energy ratio is greater than 1.5, it is determined as rapid breathing; analyze the phase continuity of the heartbeat harmonics. If the phase jump exceeds 30 degrees and lasts for 3 seconds, it is marked as arrhythmia; combine the variance value of the acceleration signal. When the variance suddenly increases by more than 3 times the baseline, it is determined as abnormal body movement. All monitored body movement change events will be added to the first monitoring data set as the basis for subsequent anomaly detection and early warning.

[0046] According to the image feedback signal, feature extraction is performed on the image scan results in the target monitored human body data to obtain human body shape feature data, which usually involves information such as the contour and posture of the human body and is mainly related to the subtle movements of the human body. By analyzing the trajectory of the human body contour feature data, the movement trajectory of the human body is tracked, its position change is analyzed, and whether there is abnormal behavior is detected to generate the second monitoring data set. Specifically, based on the received image feedback signal, differential processing is performed on the image scan results (i.e., consecutive video frames). Frame difference is achieved by comparing the pixel values between consecutive frames to find the changed parts, thereby extracting the moving regions in the image. The regions that change in the image usually correspond to the movement of the target object (such as the human body).

[0047] After frame difference processing, the moving target region is identified, and a bounding box is constructed within this region. The bounding box is a rectangular box drawn around the target object (such as the human body) to represent the position and size of the target, and its position is usually determined according to the boundary of the human body contour and the changed region. For example, assume that a user is detected walking in the monitoring area, a rectangular box is marked in the image to enclose the user's body contour, and its bounding box parameters, such as position (x, y coordinates), size (width, height), etc., are calculated.

[0048] Whenever a new human target is detected, a unique identity identifier will be assigned to it, which can be a number, a letter, or other unique markers, used to distinguish different target objects. In this way, no matter how the human body moves within the monitoring area, each target can be accurately tracked and the confusion between different targets can be avoided. The identity identifier refers to assigning a unique identifier to each recognized target (such as a human body) to distinguish different target objects. The human body contour feature data includes data such as the appearance, contour size, and shape of the human body, which helps to identify and track the human body. By analyzing the human body contour within the bounding box, it is judged whether the human body is standing upright, sitting, or lying down, and further analyze the motion state of the human body.

[0049] Based on the human body contour feature data, motion vector calculation is performed to obtain the human body's motion trajectory data. The motion vector can reflect the direction and speed of the human body's movement in the image. By calculating the displacement of the target in each frame, the moving trajectory of the target is speculated. The motion vector is a vector that describes the moving direction and speed of the target object (such as a human body) in the image. The motion vector is obtained by calculating the change in pixel positions in adjacent image frames. The human body motion trajectory data is the data that describes the human body's motion path obtained by tracking and calculating the human body's motion vector, which can provide the moving trajectory and direction of the target, and help to monitor and analyze human activities. For example, when a human body walks from the bedroom on the left to the living room on the right, the position change of the human body in each frame of the image is calculated, and the corresponding motion vector is generated to reflect the walking direction and speed of the human body.

[0050] Associate all the extracted human body motion trajectory data with its corresponding human body contour feature data and add them to the second monitoring dataset, which contains the dynamic information of the target in the monitoring area, including the position change, motion trajectory of the human body, and related visual feature data. By dividing the monitoring area into multiple grid cells and performing refined monitoring on each area, the possibility of missed reports is reduced. Starting from the two datasets, the human body state is monitored in real time to identify abnormal behaviors such as falls and shortness of breath.

[0051] The abnormal determination module 13 is used to perform fusion analysis on the first monitoring dataset and the second monitoring dataset, generate a health monitoring result for two-way abnormal determination, and generate a warning instruction according to the abnormal determination result. The warning instruction contains warning information.

[0052] Furthermore, the abnormal determination module 13 in the multi-functional health monitoring and warning system based on radar and vision fusion is also used for: A matrix construction unit is used to perform wavelet denoising on the first monitoring data set to construct a radar feature matrix, extract key frames from the second monitoring data set, and construct a visual feature matrix; a regularization fusion unit is used to perform dynamic time warping fusion on the radar feature matrix and the visual feature matrix to generate a joint feature vector; a feature division unit is used to divide the joint feature vector into a physiological feature subset and a behavioral feature subset; a correlation analysis unit is used to perform time series correlation analysis based on the physiological feature subset to obtain a physiological health monitoring result; a motion analysis unit is used to perform skeletal point motion analysis based on the behavioral feature subset to obtain a behavioral health monitoring result; a result generation unit is used to associate and integrate the physiological health monitoring result and the behavioral health monitoring result to generate the health monitoring result.

[0053] A physiological abnormality determination unit is used to perform physiological abnormality determination based on the physiological health monitoring result to obtain a physiological abnormality determination result; a behavioral abnormality determination unit is used to perform behavioral abnormality determination based on the behavioral health monitoring result to obtain a behavioral abnormality determination result; a physiological warning unit is used to generate a physiological warning message when only the physiological abnormality determination result exists, perform confidence analysis according to the physiological warning message, and trigger a first warning instruction according to the first confidence level; a behavioral warning unit is used to generate a behavioral warning message when only the behavioral abnormality determination result exists, perform confidence analysis according to the behavioral warning message, and trigger a second warning instruction according to the second confidence level; a confidence analysis unit is used to generate a two-way warning message for confidence analysis when both the physiological abnormality determination result and the behavioral abnormality determination result exist, and trigger a third warning instruction according to the third confidence level.

[0054] Specifically, wavelet denoising is performed on the first monitoring data set, that is, wavelet denoising processing is performed on the radar feedback signal to remove noise and retain the effective features in the signal. Wavelet denoising is a signal processing method used to reduce noise in the signal and retain useful features. Based on the multi-scale characteristics of wavelet transform, noise is removed by analyzing the distribution of the signal in different frequency bands. Feature extraction is performed on the denoised radar feedback signal. The radar feature matrix is a matrix constructed by the radar feedback signal after feature extraction and processing, including various human micro-motion features obtained from the radar feedback signal, such as breathing frequency, heartbeat rhythm, and body movement acceleration.

[0055] Key frames are extracted from the image sequence collected by the camera, and representative image frames are selected as key frames, which contain the main changes or actions in the target monitoring area, to obtain the visual feature matrix. The visual feature matrix is a matrix constructed by the image feedback signal after key frame extraction and feature extraction, which contains visual feature data about the target object, such as human contour features, motion trajectories, etc.

[0056] Since radar data and visual data are often not synchronized in time and have different acquisition frequencies, directly comparing their feature matrices may lead to errors. Therefore, the radar feature matrix and the visual feature matrix are fused, and the dynamic time warping (DTW) algorithm is used to generate a joint feature vector through the optimal alignment of two different time series, representing the fusion result of radar and visual data. DTW calculates the distance between two sets of time series and finds their optimal matching path, thus eliminating the misalignment problem on the time axis and enabling the two sets of data to be compared on the same time scale.

[0057] Specifically, the radar feature matrix and the visual feature matrix are respectively input into the DTW algorithm as two time series. DTW measures the similarity of these two time series and outputs an optimal alignment path. The data in the radar feature matrix and the visual feature matrix are aligned according to the optimal alignment path, and finally a new joint feature vector is generated, which contains the fusion information from radar and vision and is used for subsequent health monitoring analysis.

[0058] The generated joint feature vector is divided into two subsets: the physiological feature subset and the behavioral feature subset. The physiological feature subset includes user physiological data (such as respiratory rate, heart rhythm, etc.) for evaluating the physiological health status of an individual, and the behavioral feature subset includes data from the user's movement trajectory and behavior (such as posture changes, gait, etc.) for evaluating the behavioral health status of an individual.

[0059] Perform time series correlation analysis on the physiological feature subset. The physiological feature subset contains data such as heartbeats, breaths, and body movements from radar signals, which change over time. Therefore, time series analysis is required. For example, sudden changes in heart rate and respiratory rate may indicate abnormal physiological states (such as heart problems, shortness of breath, etc.). During the analysis process, it is also possible to identify whether these changes are associated with certain specific behavioral patterns (such as exercise, rest). Based on the results of physiological health monitoring, it is possible to determine whether an individual has an abnormal physiological state and issue a health warning. For example, under normal conditions, the user's heart rate is 60 - 100 beats per minute and changes slowly over a period of time. According to the monitoring results, it is found that the user's heart rate suddenly changes from 72 beats per minute to 115 beats per minute within five seconds, indicating that the user's heart rate has accelerated abnormally and there is an arrhythmia situation, and a warning needs to be issued. Under normal circumstances, the user's gait is stable and the body movement acceleration is 0.3 m / s². At a certain moment, the body movement acceleration suddenly jumps to 1.5 m / s², showing an unusual sudden change, which may indicate a fall.

[0060] Perform skeletal point motion analysis on the subset of behavioral features. By analyzing the position changes of the skeletal points of the human body (such as joints like shoulders, elbows, knees, etc.) at different time points, understand the postural changes and movement trajectories of the human body. For example, if it is detected that certain skeletal points have undergone large displacements in a short period of time, and this displacement is inconsistent with normal behavioral patterns (such as standing, walking), it may indicate abnormal events such as falls, imbalance, etc. Identify specific behavioral patterns, such as fast walking, sitting and standing, turning around, etc., through the movement trajectories and speeds of the skeletal points. Abnormal patterns (such as rapid falls, sudden stops, etc.) can be used as indicators of health risks. Based on the results of behavioral health monitoring, identify whether there are abnormal behaviors, such as sudden falls, unsteady gait, etc., so as to issue health warnings. For example, under normal circumstances, when the user walks from the sofa to the kitchen, the skeletal points of joints such as shoulders and knees move along a normal trajectory (such as a step length of about 0.5 meters and a step speed of about 0.8 steps / second). When a fall occurs, the skeletal points of the shoulders and knees undergo rapid displacements, showing a large displacement, such as the displacement of the knee point being 3 meters / second, indicating that the user has undergone strenuous exercise in a short period of time and may have serious health risks.

[0061] Correlate and integrate the results of physiological health monitoring and the results of behavioral health monitoring to generate the final health monitoring results. By combining the two, comprehensively evaluate the health status of an individual, including physiological states (such as heart rate, respiration) and behavioral states (such as movement, posture). For example, if it is detected that the user has a rapid heart rate (physiological abnormality) and has fallen (behavioral abnormality), generate an overall health monitoring result through correlation and integration, indicating that the user is in a dangerous state and may require emergency rescue. The health monitoring result is a comprehensive health assessment result generated by combining the results of physiological health monitoring and the results of behavioral health monitoring, and is used to evaluate the overall health status of an individual. By separately analyzing physiological features and behavioral features and respectively evaluating the physiological health status and behavioral health status of an individual, it helps to detect potential health problems (such as heart diseases, fall risks, etc.) earlier.

[0062] Perform physiological abnormality determination based on the results of physiological health monitoring. For example, if the heart rate suddenly rises to more than 120 beats per minute and is accompanied by shortness of breath, it may indicate that the user has acute health problems such as heart disease or difficulty in breathing. Compare the results of physiological health monitoring with the preset normal value range to determine whether there is a physiological abnormality. Perform behavioral abnormality determination based on the results of behavioral health monitoring. For example, if it is detected that the user suddenly falls, or their movement trajectory does not match normal activities (such as rapid displacement, etc.), then it is determined that there is a behavioral abnormality, such as a fall or accidental injury.

[0063] If only the situation where the physiological abnormality determination result is abnormal occurs, a physiological warning message is generated. Confidence analysis is performed based on the physiological warning message to evaluate the reliability of the warning message. Confidence refers to the degree of certainty about the occurrence of a certain abnormal state. For example, if the heart rate is too fast (such as exceeding 100 beats per minute) or too slow, then there is a heart rate abnormality; if the respiratory rate is too fast or too slow, or there is shortness of breath, etc., then there is a respiratory abnormality; if the body movement acceleration suddenly increases, which may indicate a sudden change in the body state, such as a change in body position or a sudden onset of a disease, then there is a body movement abnormality. When it is determined that there is a physiological abnormality by analyzing physiological data, a corresponding physiological warning message will be generated, including the type of abnormality (such as tachycardia, shortness of breath, etc.), the degree of abnormality, the abnormal event, the duration, etc.

[0064] Confidence analysis is a key process for evaluating the credibility of warning messages. It can be done by comparing the current physiological monitoring data with the user's historical health data to check whether the abnormality conforms to the user's health condition. If the historical data shows that the abnormality is relatively rare, the confidence may be higher. At the same time, the status of the sensor is evaluated. If the sensor is normal and has high precision, a higher confidence is given. The severity of the abnormality affects the confidence. Severe physiological abnormalities (such as tachycardia, shortness of breath) result in higher confidence. For example, if the heart rate acceleration occurs in a user with a history of heart disease and exceeds the preset threshold by 20%, the confidence may be 90%. According to the confidence analysis result, a warning instruction is triggered. If only a physiological abnormality occurs in the current user, the first warning instruction is triggered.

[0065] Similarly, if only the situation where the behavior abnormality judgment result is abnormal occurs, a behavior warning message is generated. Confidence analysis is performed based on the behavior warning message to evaluate the reliability of the warning message. For example, when a fall event is detected, the possible risk of falling or injury is warned. At the same time, confidence analysis is carried out to analyze the possibility of falling. Confidence analysis may combine the images collected by the camera and the motion data obtained by the radar, as well as the historical data. If the confidence exceeds a certain threshold (such as 90%), the second warning instruction is generated to trigger the first aid response mechanism.

[0066] Similarly, if both physiological and behavioral abnormalities are detected simultaneously (such as an accelerated heart rate and a detected fall), a two-way warning message is generated, which needs to include both physiological and behavioral abnormality information. The two-way warning message may include a more urgent warning indicating that these two abnormalities may interact with each other, such as a heart problem leading to a fall, which requires immediate treatment. After the two-way warning message is generated, confidence analysis is performed to evaluate the accuracy and reliability of the two-way warning. Considering the simultaneous occurrence of physiological and behavioral abnormalities, the confidence level needs to be determined through a joint confidence analysis of physiological and behavioral abnormalities to assess the reliability of the warning message. That is to say, based on the accuracy and degree of abnormality of physiological data, the abnormal reliability of physiological health status is evaluated; at the same time, through the analysis of behavioral data, such as the accuracy of the fall detection system, the possibility of behavioral abnormalities is evaluated. In addition, the data of radar, vision, and other sensors are combined to improve the accuracy of the overall warning.

[0067] A third confidence level is generated based on the confidence analysis result of the two-way warning message. The third confidence level is the final evaluation result after comprehensively considering physiological and behavioral abnormalities. If the third confidence level exceeds a preset threshold (such as 90%), a third warning instruction will be triggered, including emergency prompts, automatic responses, risk level assessments, etc. Since the situation is relatively serious, an emergency call can be directly made to the hospital, and at the same time, an alarm message is sent to family members. Combining the simultaneous occurrence of physiological and behavioral abnormalities, the user's health status is comprehensively evaluated to provide more accurate warning information.

[0068] The risk analysis module 14 is used to trigger the warning mechanism through the warning instruction, upload the abnormal determination result to the cloud platform for risk analysis, and send a warning message to the remote terminal according to the abnormal risk level.

[0069] Furthermore, the risk analysis module 14 in the multifunctional health monitoring and warning system based on the fusion of radar and vision is further used for: The first risk analysis unit is used to upload the physiological abnormal determination result to the cloud platform for risk analysis and generate a first abnormal risk level when the first warning instruction triggers the warning mechanism; the first warning unit is used to send the physiological warning message to the remote terminal according to the first abnormal risk level; the second risk analysis unit is used to upload the behavioral abnormal determination result to the cloud platform for risk analysis and generate a second abnormal risk level when the second warning instruction triggers the warning mechanism; the second warning unit is used to send the behavioral warning message to the remote terminal according to the second abnormal risk level; the third risk analysis unit is used to upload the physiological abnormal determination result and the behavioral abnormal determination result to the cloud platform for risk analysis and generate a third abnormal risk level when the third warning instruction triggers the warning mechanism; the third warning unit is used to send the two-way warning message to the remote terminal according to the third abnormal risk level.

[0070] Specifically, when the first warning instruction triggers the warning mechanism, that is, when there is a physiological abnormality in the user, the determination result of the physiological abnormality is uploaded to the cloud platform for risk analysis to evaluate the risk level of the current health status. For example, if it is found that the heart rate of an elderly person suddenly increases to 180 bpm, exceeding the normal value, it is identified as a heart abnormality, triggering the first warning instruction. Once the first warning instruction is triggered, the determination result of the physiological abnormality will be uploaded to the cloud platform for further analysis. The cloud platform will receive this data and run a gradient boosting decision tree, combining the user's health history data (such as heart disease, respiratory problems, etc.) for risk assessment, and generating the risk level of this abnormal event, that is, generating the first abnormal risk level.

[0071] Within the cloud platform, the risk analysis model conducts risk assessment based on the uploaded physiological abnormality data and generates the first abnormal risk level of this event. If the risk of this event is high (such as a high possibility of cardiac arrest), it will be marked as a high risk level; if the event is minor (such as a slightly accelerated heart rate without other symptoms), it may be marked as a low risk. According to the generated first abnormal risk level, the cloud platform will send physiological warning information to the remote terminal through the network, and the device of the remote terminal will receive the real-time warning information.

[0072] Exemplarily, assume the following data is obtained from a worn physiological monitoring device (such as a heart rate monitor): the user's heart rate has suddenly increased to 180 bpm within the past 15 minutes and exceeded the set threshold (120 bpm); the user has a previous history of heart disease and has not had a similar heart rate abnormality recently. Based on this data, the first warning instruction is triggered and this data is uploaded to the cloud platform. After receiving the data, the cloud platform processes it through models such as the gradient boosting decision tree and evaluates the risk level of this event as high risk. The cloud platform then sends physiological warning information to the remote terminal, showing that the user's heart rate has accelerated and exceeded the safe range, and it is recommended to take immediate measures. Through the automated warning mechanism, physiological abnormalities are promptly detected and warnings are issued, avoiding health risks caused by reaction delays.

[0073] Similarly, when the second warning instruction triggers the warning mechanism, it indicates that there is a behavioral abnormality in the user. The determination result of the behavioral abnormality is uploaded to the cloud platform for risk analysis and evaluation, generating the second abnormal risk level. For example, it is detected that someone suddenly falls to the ground during an activity or has uncoordinated movements, immediately triggering the second warning instruction to mark this behavioral abnormality. The behavioral abnormality data will be uploaded to the cloud platform in real time, including the time of the event, the type of abnormality, the user's status information (such as vital signs data), and environmental factors (such as a slippery ground, etc.). According to the second abnormal risk level, the cloud platform will send behavioral warning information to the remote terminal, including the type, time, risk level of the behavioral abnormality, and recommended handling measures, etc.

[0074] When the third warning instruction triggers the warning mechanism, it indicates that the user has both physiological abnormalities and behavioral abnormalities, or it may be behavioral abnormalities caused by physiological abnormalities. For example, through heart rate monitoring, it is found that the user's heart rate has increased sharply (for example, above 150 bpm), and behavioral abnormalities (such as falling, losing balance) are detected. This is considered a serious health threat and triggers the third warning instruction. Once the third warning instruction is triggered, both physiological abnormalities (such as a too fast heart rate) and behavioral abnormalities (such as falling) are uploaded to the cloud platform simultaneously. After receiving these data, the cloud platform will perform a risk assessment on them according to the gradient boosting decision tree and generate a comprehensive abnormal risk level. Considering the dual threat of the two abnormal events (physiological and behavioral) to the user's health, a relatively high risk level (such as high risk) will be generated. According to the generated third abnormal risk level, the cloud platform sends two-way warning information to the relevant remote terminals, including detailed abnormal descriptions, occurrence times, possible health risks, warning levels, and suggestions for emergency measures.

[0075] The corresponding warning mechanism is triggered through the warning instruction, and the warning instruction is triggered based on preset conditions. Once the warning instruction is triggered, the abnormal determination results (such as heart rate abnormalities, falls, etc.) are uploaded to the cloud platform through the external network communication module, including the abnormal type, occurrence time, abnormal parameters (such as heart rate values), etc. The external network communication module supports 4G / 5G communication and is used for data transmission with the cloud platform. After receiving the abnormal determination results, the cloud platform performs a risk analysis through the gradient boosting decision tree, determines the corresponding abnormal risk level, and sends warning information to the remote terminal, including detailed descriptions of abnormal events, risk levels, emergency suggestions, etc., to notify relevant personnel to take actions in time to prevent the accident from worsening. The warning methods include SMS, phone calls, APP push, etc., to ensure that the warning information is conveyed in a timely manner.

[0076] Furthermore, the risk analysis module 14 in the multifunctional health monitoring and warning system based on the fusion of radar and vision is further used for: The level determination unit is used to retrieve the historical abnormal event set in the target monitoring area, perform a causal analysis based on the historical abnormal event set, label the risk level tags for the historical abnormal event set according to the analysis results, and determine the abnormal risk level list; the weight optimization unit is used to perform weight optimization according to the abnormal risk level list and construct a gradient boosting decision tree; the decision tree embedding unit is used to embed the gradient boosting decision tree into the cloud platform for risk analysis.

[0077] Specifically, obtain the historical abnormal event set in the target monitoring area, that is, all abnormal event records stored in the cloud platform during the previous monitoring period, including physiological abnormalities (such as irregular heartbeat, shortness of breath, etc.) and behavioral abnormalities (such as falls, loss of consciousness, etc.). The cloud platform extracts all abnormal event data that occurred in the target monitoring area within a certain period of time in the past from the database, including the identification, occurrence time, severity assessment, and corresponding handling measures of physiological and behavioral abnormalities.

[0078] The cloud platform uses a built-in machine learning model to analyze historical abnormal events and discovers potential causal relationships through the analysis. Causal analysis can help determine whether the occurrence of certain abnormal events will lead to the occurrence of other events. Through causal analysis, it may be found that, for example, tachycardia may cause the user to fall, or the potential connection between the fall event and heart health. During the analysis process, a risk level label is assigned to each abnormal event, indicating the potential risk of the event occurring, such as low risk, medium risk, high risk, etc. The abnormal risk level list is a list obtained by sorting all historical abnormal events according to the risk level labels of the events, which can identify which events pose a greater threat to the user's health and prioritize the handling of these high-risk events.

[0079] The abnormal risk level list has assigned risk levels to each type of abnormality. To improve the accuracy of risk prediction, the gradient boosting decision tree adjusts these levels through weight optimization. For example, if certain events (such as falls) were often associated with high risks (such as serious injuries, hospitalization, etc.) in past monitoring, then the weights of these events will be increased.

[0080] During the training process, the gradient boosting decision tree gradually constructs a series of decision trees. In the training of each tree, the model focuses on how to reduce the current prediction error (usually based on the gradient descent of the loss function). The gradient boosting decision tree gradually enhances the prediction ability through ensemble learning. Each tree corrects the mispredictions of the previous tree, and finally obtains a powerful and accurate model. For example, if the model initially misjudged some low-risk events as high-risk, subsequent decision trees will correct this error by learning the error, so as to achieve the optimization goal.

[0081] Train a gradient boosting decision tree model using a historical anomaly event set. During the training process, in each round, the model will consider the residuals (i.e., prediction errors) in the previous prediction results and improve the prediction accuracy of the model by optimizing the residuals. After the training is completed, it is necessary to evaluate the performance of the model. Common evaluation metrics include accuracy, recall rate, F1-score, etc. By comparing the prediction results of the model with the actual situation, ensure that the model can effectively identify anomaly events of different risk levels. For example, for fall events, the model should be able to identify high-risk events with a relatively high recall rate (able to capture as many fall events as possible), while for low-risk events (such as small body movements, repetitive actions, etc.), false alarms should be avoided.

[0082] Deploy the trained gradient boosting decision tree model to the cloud platform and use the powerful computing power of cloud computing to perform real-time risk analysis on new data. The gradient boosting decision tree can receive real-time data from monitoring devices, combine historical data for real-time prediction, and generate anomaly risk levels. The cloud platform will issue warnings based on the analysis results to help relevant personnel take necessary intervention measures in a timely manner. Through causal analysis and weight optimization based on historical events, the gradient boosting decision tree can more accurately identify and predict anomaly events, especially for high-risk events, it can issue early warnings.

[0083] Furthermore, the risk analysis module 14 in the multifunctional health monitoring and warning system based on radar and vision fusion is also used for: A response recording unit is used to record the response processing of the remote terminal receiving the warning information and generate a response record report; a processing efficiency determination unit is used to traverse the response record report to analyze the processing results of the remote terminal and generate the terminal processing efficiency; a mapping relationship determination unit is used to perform mapping analysis on the anomaly risk level and the warning information to determine the risk-warning mapping relationship; a warning mode determination unit is used to dynamically adjust the risk-warning mapping relationship according to the terminal processing efficiency, generate a risk level score, and change the warning mode for the remote terminal according to the risk level score.

[0084] Specifically, after receiving the warning information, the remote terminal will perform response processing, automatically record the response time of the terminal, response actions (such as whether to contact rescue, whether to confirm the safety of the user, etc.), and generate a response record report. The response record report records the response situation of the remote terminal to each warning information, including the response time, action content, response personnel, etc., and is used to evaluate the response efficiency and processing quality of the remote terminal.

[0085] Traverse all response records and calculate the response efficiency of each remote terminal, including response time (such as how long it takes to process a warning) and processing quality (such as whether the response is timely and effective). The terminal processing efficiency measures the speed and quality of the remote terminal's response to warning information and is usually calculated based on factors such as response time, response frequency, and effectiveness of processing. For example, if a terminal has a response time of 10 minutes after receiving a warning, while another terminal has a response time of only 2 minutes, then the processing efficiency of the first terminal is lower.

[0086] Establish a mapping relationship between risks and warning information according to different abnormal events and corresponding risk levels. Different risk levels correspond to different warning information contents, triggering methods, and processing priorities. For example, high-risk events may trigger more urgent and frequent warnings, while low-risk events only send mild warnings. Dynamically adjust the risk-warning mapping relationship according to the response efficiency of the remote terminal. If a certain terminal frequently fails to respond effectively, then upgrade the warning mode of this terminal, such as increasing the frequency or intensity of the warning, to ensure that this terminal responds in a timely manner. At the same time, the risk level score will also be adjusted according to historical performance. For example, an event was originally rated as medium risk, but due to the slow response of the remote terminal, its risk level may be increased, and the future warning method will be changed according to this adjustment.

[0087] When a certain terminal fails to respond effectively to the same type of warning three times in a row, change the warning mode according to this situation, such as increasing the alarm frequency of this terminal, modifying the warning content (such as adding more urgent prompts), or switching this terminal to a higher-level emergency response mode. Optimize the response ability of the remote terminal in real time by dynamically adjusting the warning mode and risk level score. Improve the timeliness and processing efficiency of warnings and avoid delayed or ineffective responses.

[0088] In summary, the multifunctional health monitoring and warning system based on radar and vision fusion provided by this application has the following beneficial effects: A feedback receiving module is used to transmit radar signals through a radar module, receive radar feedback signals, record the target monitoring area in real time through a camera module, and receive image feedback signals; a monitoring and recognition module is used to traverse the target monitoring area to determine the target monitored human body data, and respectively monitor and recognize the target monitored human body data based on the radar feedback signal and the image feedback signal to generate a first monitoring data set and a second monitoring data set; an anomaly determination module is used to perform fusion analysis on the first monitoring data set and the second monitoring data set, generate a health monitoring result for two-way anomaly determination, and generate a warning instruction according to the anomaly determination result, where the warning instruction includes warning information; a risk analysis module is used to trigger a warning mechanism through the warning instruction to upload the anomaly determination result to a cloud platform for risk analysis, and send warning information to a remote terminal according to the anomaly risk level. That is to say, through the monitoring of radar module and vision fusion, two-way anomaly determination is carried out on the monitoring results, abnormal risks are quickly identified and uploaded to the cloud platform for risk assessment, warning information is sent, false alarms and missed alarms are reduced, and the accuracy and timeliness of abnormal situation identification are improved, thereby improving the real-time performance and accuracy of early warning.

[0089] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0090] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A multi-functional health monitoring and warning system based on the fusion of radar and vision, characterized in that Including: A feedback receiving module, configured to transmit a radar signal through a radar module, receive a radar feedback signal, record the target monitoring area in real time through a camera module, and receive an image feedback signal; A monitoring and recognition module, configured to traverse the target monitoring area to determine the target monitored human body data, and respectively monitor and recognize the target monitored human body data based on the radar feedback signal and the image feedback signal to generate a first monitoring data set and a second monitoring data set; An anomaly determination module, configured to perform fusion analysis on the first monitoring data set and the second monitoring data set, generate a health monitoring result for two-way anomaly determination, and generate a warning instruction according to the anomaly determination result, where the warning instruction includes warning information; A risk analysis module, configured to trigger a warning mechanism through the warning instruction, upload the anomaly determination result to a cloud platform for risk analysis, and send warning information to a remote terminal according to the anomaly risk level.

2. The multifunctional health monitoring and early warning system based on radar and vision fusion according to claim 1, wherein The feedback receiving module includes: A radar signal receiving unit, configured to transmit a frequency-modulated continuous wave radar signal to the target monitoring area through a radar module in a host, receive the reflected signal in the target area through a receiving antenna, and obtain an initial radar echo signal; A frequency shift screening unit, configured to perform frequency shift screening based on the initial radar echo signal to obtain the radar feedback signal; A multi-view acquisition unit, configured to adjust the acquisition angle of the camera module in the host at a preset interval period to determine the data acquisition angle, and perform multi-view acquisition through the camera module according to the data acquisition angle to obtain the image feedback signal.

3. The multifunctional health monitoring and early warning system based on radar and vision fusion according to claim 1, characterized in that, The monitoring and recognition module includes: A grid determination unit, configured to divide the target monitoring area into N×M grid cells, and determine a plurality of grid cells, where N and M are positive integers greater than 1, and N and M can be equal; A scan result determination unit, configured to control the radar module and the camera module to traverse the plurality of grid cells for human body recognition scanning according to a preset path to obtain the target monitored human body data, where the target monitored human body data includes a radar scan result and an image scan result; A feature extraction unit, configured to extract features from the radar scan result based on the radar feedback signal to determine human body micro-motion feature data, and perform change monitoring according to the human body micro-motion feature data to generate the first monitoring data set; A trajectory monitoring unit, configured to extract features from the image scan result based on the image feedback signal to determine human body contour feature data, and perform trajectory monitoring according to the human body contour feature data to generate the second monitoring data set.

4. The multi-functional health monitoring and early warning system based on radar and vision fusion according to claim 3, characterized in that, The feature extraction unit includes: A multi-feature extraction subunit, configured to perform short-time Fourier transform on the radar scan result based on the radar feedback signal to extract human body micro-motion feature data, where the human body micro-motion feature data includes a respiration fundamental frequency feature, a heartbeat harmonic feature, and a body motion acceleration feature; A state monitoring subunit, configured to perform change state monitoring based on the respiration fundamental frequency feature, the heartbeat harmonic feature, and the body motion acceleration feature to obtain a plurality of body motion change events, and add the plurality of body motion change events to the first monitoring data set.

5. The multifunctional health monitoring and early warning system based on radar and vision fusion according to claim 3, characterized in that The trajectory monitoring unit includes: A frame difference processing subunit, configured to perform frame difference processing on the image scanning result based on the image feedback signal, and construct human body bounding box parameters; An identifier assignment subunit, configured to assign an identity identifier to the human body bounding box parameters to determine the human body contour feature data; A vector calculation subunit, configured to perform motion vector calculation based on the human body contour feature data to obtain human body motion trajectory data, and add the human body motion trajectory data to the second monitoring data set.

6. The multi-functional health monitoring and warning system based on radar and vision fusion according to claim 1, characterized in that The anomaly determination module includes: A matrix construction unit, configured to perform wavelet denoising on the first monitoring data set to construct a radar feature matrix, and perform key frame extraction on the second monitoring data set to construct a visual feature matrix; A regularization fusion unit, configured to perform dynamic time warping fusion on the radar feature matrix and the visual feature matrix to generate a joint feature vector; A feature partitioning unit, configured to partition the joint feature vector into a physiological feature subset and a behavior feature subset; A correlation analysis unit, configured to perform time series correlation analysis based on the physiological feature subset to obtain a physiological health monitoring result; A motion analysis unit, configured to perform skeletal point motion analysis based on the behavior feature subset to obtain a behavior health monitoring result; A result generation unit, configured to perform associated integration on the physiological health monitoring result and the behavior health monitoring result to generate the health monitoring result.

7. The multifunctional health monitoring and warning system based on radar and vision fusion according to claim 6, characterized in that, The anomaly determination module includes: A physiological anomaly determination unit, configured to perform physiological anomaly determination based on the physiological health monitoring result to obtain a physiological anomaly determination result; A behavior anomaly determination unit, configured to perform behavior anomaly determination based on the behavior health monitoring result to obtain a behavior anomaly determination result; A physiological warning unit, configured to generate a physiological warning message when only the physiological anomaly determination result exists, perform confidence analysis according to the physiological warning message, and trigger a first warning instruction according to a first confidence level; A behavior warning unit, configured to generate a behavior warning message when only the behavior anomaly determination result exists, perform confidence analysis according to the behavior warning message, and trigger a second warning instruction according to a second confidence level; A confidence analysis unit, configured to generate a two-way warning message for confidence analysis when both the physiological anomaly determination result and the behavior anomaly determination result exist, and trigger a third warning instruction according to a third confidence level.

8. The multi-functional health monitoring and early warning system based on radar and vision fusion according to claim 7, characterized in that, The risk analysis module includes: A first risk analysis unit, configured to upload the physiological anomaly determination result to a cloud platform for risk analysis when the first warning instruction triggers the warning mechanism, and generate a first anomaly risk level; A first warning unit, configured to send the physiological warning message to a remote terminal according to the first anomaly risk level; A second risk analysis unit, configured to upload the behavior anomaly determination result to a cloud platform for risk analysis when the second warning instruction triggers the warning mechanism, and generate a second anomaly risk level; A second warning unit, configured to send the behavior warning message to a remote terminal according to the second anomaly risk level; A third risk analysis unit, configured to upload the physiological abnormality determination result and the behavior abnormality determination result to a cloud platform for risk analysis when the third warning instruction triggers a warning mechanism, and generate a third abnormal risk level; A third warning unit, configured to send the two-way warning information to a remote terminal according to the third abnormal risk level.

9. The multifunctional health monitoring and early warning system based on radar and vision fusion according to claim 1, wherein The risk analysis module includes: A level determination unit, configured to retrieve a historical abnormal event set within a target monitoring area, perform causal analysis based on the historical abnormal event set, label risk level tags for the historical abnormal event set according to the analysis result, and determine an abnormal risk level list; A weight optimization unit, configured to perform weight optimization according to the abnormal risk level list and construct a gradient boosting decision tree; A decision tree embedding unit, configured to embed the gradient boosting decision tree into a cloud platform for risk analysis.

10. The multifunctional health monitoring and early warning system based on radar and vision fusion according to claim 1, characterized in that, The risk analysis module further includes: A response recording unit, configured to record the response processing of the remote terminal receiving the warning information and generate a response recording report; A processing efficiency determination unit, configured to traverse the response recording report to analyze the processing result of the remote terminal and generate a terminal processing efficiency; A mapping relationship determination unit, configured to perform mapping analysis on the abnormal risk level and the warning information to determine a risk-warning mapping relationship; A warning mode determination unit, configured to dynamically adjust the risk-warning mapping relationship according to the terminal processing efficiency, generate a risk level score, and replace the warning mode for the remote terminal according to the risk level score.

Images