A monitoring method, system, electronic device and storage medium

Abstract

The application provides a monitoring method and system executed by an electronic device, an electronic device and a storage medium, comprising: acquiring human state information and environmental parameter information in a monitoring area; performing feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data; performing data fusion on the state data and the environmental data to generate comprehensive event data; and determining early warning information according to the comprehensive event data. Thus, the problem that the accuracy and reliability of a single sensor in the prior art may be affected by various factors, such as performance limitation of the sensor itself, external environmental interference and human activities, thereby causing errors or loss of monitoring data, can be effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of health monitoring, and in particular to a monitoring method, system, electronic device and storage medium. BACKGROUND

[0002] With the intensification of population aging and the increasing concern of people for health, the market demand for health monitoring devices is growing. According to the forecast of China Research Institute, the market size of intelligent medical and health devices in China will exceed 200 billion yuan in 2025, of which medical-grade wearable devices account for 45%. People expect to monitor the health status of the human body and the surrounding environmental parameters in real time and accurately through these devices, so as to better manage their own health and discover potential health problems in a timely manner.

[0003] The accuracy and reliability of a single sensor in the prior art can also be affected by various factors, such as the performance limitations of the sensor itself, external environmental interference, and human activities, thereby causing errors or missing of the monitoring data. SUMMARY

[0004] The embodiments of the present application provide a monitoring method, system, electronic device and storage medium to solve the problems in the related art, and the technical solutions are as follows:

[0005] In a first aspect, the embodiments of the present application provide a monitoring method, comprising:

[0006] obtaining human state information and environmental parameter information in a monitoring area;

[0007] performing feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data;

[0008] performing data fusion on the state data and the environmental data to generate comprehensive event data;

[0009] determining warning information according to the comprehensive event data.

[0010] In an implementation manner, the human state information includes infrared data obtained through an infrared sensor and radar data obtained through a millimeter wave radar, the environmental parameter information includes environmental data and sound data, and the obtaining of the human state information and the environmental parameter information in the monitoring area comprises:

[0011] obtaining the infrared data, the radar data, the environmental data and the sound data;

[0012] reorganizing the infrared data, the radar data, the environmental data and the sound data according to a PTP clock synchronization signal broadcast by a master node in an N-second time window to obtain the human state information and the environmental parameter information in the detection area.

[0013] In an embodiment, the acquiring the human state information and the environmental parameter information in the monitoring area further comprises:

[0014] According to the radar data, a radar coordinate system is established as a reference space;

[0015] According to the reference space, a spatial conversion matrix of the infrared data, the environmental data and the sound data is determined;

[0016] According to the spatial conversion matrix of the infrared data, the environmental data and the sound data, the infrared data, the environmental data and the sound data are three-dimensionally converted to obtain the human state information and the environmental parameter information in the monitoring area.

[0017] In an embodiment, the feature extraction of the human state information and the environmental parameter information to obtain state data and environmental data comprises:

[0018] According to the radar data, breathing information, heart rate information and motion trajectory information are determined;

[0019] According to the infrared data, presence probability information is determined;

[0020] According to the environmental data, temperature and humidity information and air information are determined;

[0021] According to the sound data, voiceprint information and sound information are determined;

[0022] According to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the temperature and humidity information, the air information, the voiceprint information and the sound information, state data and environmental data are obtained.

[0023] In an embodiment, the data fusion of the state data and the environmental data to generate comprehensive event data comprises:

[0024] According to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the voiceprint information and the sound information, a joint confidence is calculated;

[0025] According to the temperature and humidity information and the air information, the joint confidence is adjusted to generate comprehensive event data.

[0026] In an embodiment, the calculation according to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the voiceprint information and the sound information to obtain the joint confidence comprises:

[0027] acquire the infrared sensor, the millimeter wave radar and the sound sensor;

[0028] According to the historical data, determine the credibility of the infrared sensor, the credibility of the millimeter wave radar, the credibility of the sound sensor, the weight of the infrared sensor, the weight of the millimeter wave radar and the weight of the sound sensor;

[0029] According to the credibility of the infrared sensor, adjust the presence probability information to obtain the confidence of the presence probability information;

[0030] According to the credibility of the millimeter wave radar, adjust the motion trajectory information to obtain the confidence of the motion trajectory information;

[0031] According to the credibility of the millimeter wave radar, adjust the breathing information to obtain the confidence of the breathing information;

[0032] According to the credibility of the millimeter wave radar, adjust the heart rate information to obtain the confidence of the heart rate information;

[0033] According to the credibility of the sound sensor, adjust the voiceprint information to obtain the confidence of the voiceprint information;

[0034] According to the credibility of the sound sensor, adjust the sound information to obtain the confidence of the sound information;

[0035] According to the weight of the infrared sensor, the weight of the millimeter wave radar, the weight of the sound sensor, the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information and the confidence of the sound information, obtain the joint confidence.

[0036] In an implementation mode, the pre-warning information includes first-level pre-warning information, second-level pre-warning information and third-level pre-warning information, and the determination of the pre-warning information according to the comprehensive event data includes:

[0037] In the case that the comprehensive event data is greater than or equal to a first specified threshold, trigger the first-level pre-warning information;

[0038] In the case that the comprehensive event data is less than the first specified threshold and greater than or equal to a second specified threshold, trigger the second-level pre-warning information;

[0039] In the case that the comprehensive event data is less than the second specified threshold, and any one of the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the temperature and humidity information, the air information, the voiceprint information and the sound information is abnormal, trigger the third pre-warning signal.

[0040] In a second aspect, the embodiments of the present application provide a system, comprising:

[0041] an acquisition unit configured to acquire human state information and environmental parameter information in a monitoring area;

[0042] an obtaining unit configured to perform feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data;

[0043] a generating unit configured to perform data fusion on the state data and the environmental data to generate comprehensive event data;

[0044] a determining unit configured to determine early warning information according to the comprehensive event data.

[0045] In an embodiment, the human state information includes infrared data obtained by an infrared sensor and radar data obtained by a millimeter wave radar, and the environmental parameter information includes environmental data and sound data, and the acquisition of the human state information and the environmental parameter information in the monitoring area includes:

[0046] acquiring the infrared data, the radar data, the environmental data, and the sound data;

[0047] reorganizing the infrared data, the radar data, the environmental data, and the sound data according to a PTP clock synchronization signal broadcast by a master node in an N-second time window to obtain the human state information and the environmental parameter information in the detection area.

[0048] In an embodiment, the acquisition of the human state information and the environmental parameter information in the monitoring area further includes:

[0049] establishing a radar coordinate system as a reference space according to the radar data;

[0050] determining a spatial conversion matrix of the infrared data, the environmental data, and the sound data according to the reference space;

[0051] performing three-dimensional conversion on the infrared data, the environmental data, and the sound data according to the spatial conversion matrix of the infrared data, the environmental data, and the sound data to obtain the human state information and the environmental parameter information in the monitoring area.

[0052] In an embodiment, the feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data includes:

[0053] determining breathing information, heart rate information, and motion trajectory information according to the radar data;

[0054] determining presence probability information according to the infrared data;

[0055] determine temperature and humidity information and air information according to the environment data;

[0056] determine voiceprint information and sound information according to the sound data;

[0057] obtain state data and environment data according to the breathing information, the heart rate information, the motion trajectory information, the existence probability information, the temperature and humidity information, the air information, the voiceprint information and the sound information.

[0058] In an implementation, the data fusion of the state data and the environment data to generate comprehensive event data includes:

[0059] calculate a joint confidence according to the breathing information, the heart rate information, the motion trajectory information, the existence probability information, the voiceprint information and the sound information;

[0060] adjust the joint confidence according to the temperature and humidity information and the air information to generate comprehensive event data.

[0061] In an implementation, the calculation according to the breathing information, the heart rate information, the motion trajectory information, the existence probability information, the voiceprint information and the sound information to obtain a joint confidence includes:

[0062] obtain the infrared sensor, the millimeter wave radar and the sound sensor;

[0063] determine the credibility of the infrared sensor, the credibility of the millimeter wave radar, the credibility of the sound sensor, the weight of the infrared sensor, the weight of the millimeter wave radar and the weight of the sound sensor according to the historical data;

[0064] adjust the existence probability information according to the credibility of the infrared sensor to obtain the confidence of the existence probability information;

[0065] adjust the motion trajectory information according to the credibility of the millimeter wave radar to obtain the confidence of the motion trajectory information;

[0066] adjust the breathing information according to the credibility of the millimeter wave radar to obtain the confidence of the breathing information;

[0067] adjust the heart rate information according to the credibility of the millimeter wave radar to obtain the confidence of the heart rate information;

[0068] adjust the voiceprint information according to the credibility of the sound sensor to obtain the confidence of the voiceprint information;

[0069] adjust the sound information according to the credibility of the sound sensor, to obtain a confidence of the sound information;

[0070] obtain a joint confidence according to the weight of the infrared sensor, the weight of the millimeter wave radar, the weight of the sound sensor, the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information, and the confidence of the sound information.

[0071] In an implementation, the pre-warning information includes first-level pre-warning information, second-level pre-warning information, and third-level pre-warning information, and the determining the pre-warning information according to the comprehensive event data includes:

[0072] in a case where the comprehensive event data is greater than or equal to a first specified threshold, triggering the first-level pre-warning information;

[0073] in a case where the comprehensive event data is less than the first specified threshold and greater than or equal to a second specified threshold, triggering the second-level pre-warning information;

[0074] in a case where the comprehensive event data is less than the second specified threshold, and any one of the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the temperature and humidity information, the air information, the voiceprint information, and the sound information is abnormal, triggering the third-level pre-warning information.

[0075] In a third aspect, an electronic device is provided, which includes at least one processor, and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, so that the at least one processor can execute the above-mentioned monitoring method.

[0076] In a fourth aspect, a computer readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed on a computer, the method in any one of the above aspects is executed.

[0077] The above technical solutions have at least the following advantages or beneficial effects:

[0078] In the embodiment, the monitoring method comprises: acquiring human state information and environmental parameter information in a monitoring area; performing feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data; performing data fusion on the state data and the environmental data to generate comprehensive event data; and determining early warning information according to the comprehensive event data. Thus, the problem that the accuracy and reliability of a single sensor in the prior art can be affected by various factors, such as performance limitations of the sensor itself, interference of the external environment, and human activities, thereby causing errors or missing of monitoring data, can be effectively solved. The monitoring method of the embodiment has a multi-level data processing and intelligent decision mechanism, and performs comprehensive analysis based on the motion of the person and the environment, thereby reducing the misjudgment and enabling the health condition of the person to be determined more accurately, and a complete technical chain from physical signals to health early warning is constructed, all-weather and non-contact human state monitoring is realized in a family care, elderly care, and the like, and the emergency response capability of a sudden event is significantly improved.

[0079] The above summary is intended to illustrate only and is not intended to be limiting in any way. Further aspects, implementations, and features of the application will be apparent from the drawings and the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0080] In the drawings, like reference numerals refer to same or similar components throughout the several views. The drawings are not necessarily to scale. It should be understood that the drawings only depict certain embodiments in accordance with the disclosure and should not be considered limiting the scope of the disclosure.

[0081] Figure 1 is a schematic diagram of a monitoring method according to an embodiment of the application;

[0082] Figure 2 is a block diagram of an electronic device for implementing the monitoring method according to an embodiment of the application. DETAILED DESCRIPTION

[0083] In the following, only certain example embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the application. Therefore, the drawings and the description are considered to be exemplary in nature and not limiting.

[0084] In the related art, with the intensification of population aging and the increasing concern of people for health, the demand for devices capable of real-time and accurate monitoring of human health status and surrounding environmental parameters is growing. In existing human monitoring technology, a single sensor often has limitations. For example, when used for fall detection, traditional accelerometers are easily disturbed by false movements; and camera-based monitoring methods have the risk of privacy invasion. Although millimeter wave radar has certain advantages in detecting human motion state and vital signs, it is easily affected by environmental clutter and other factors in basic detection of whether a person exists, and its accuracy needs to be improved. Infrared sensors have the characteristics of simplicity and speed in detecting the presence or absence of people, but they cannot accurately obtain heart rate, respiratory rate and other vital sign data. In addition, factors such as light intensity, gas composition, temperature and humidity, and sound in the environment have an important influence on human health and life comfort, but there is a lack of effective integrated monitoring solutions. Therefore, it is of great practical significance to develop a system that can comprehensively utilize the advantages of multiple independent sensors, achieve collaborative work through wireless connection, and achieve high-precision, multi-functional human and environmental monitoring.

[0085] Figure 1 A flowchart of a monitoring method according to an embodiment of the application is shown. As shown in Figures 1-2 A monitoring method can include:

[0086] In a first aspect, the embodiments of the present application provide a monitoring method, including:

[0087] S110: obtaining human state information and environmental parameter information in a monitoring area;

[0088] S120: performing feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data;

[0089] S130: performing data fusion on the state data and the environmental data to generate comprehensive event data;

[0090] S140: determining warning information according to the comprehensive event data.

[0091] The monitoring method of the present embodiment can provide hardware support through a host device, and the method of monitoring is executed in the host device.

[0092] In the embodiment, the monitoring method comprises: acquiring human state information and environmental parameter information in a monitoring area; performing feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data; performing data fusion on the state data and the environmental data to generate comprehensive event data; and determining early warning information according to the comprehensive event data. Thus, the problem that the accuracy and reliability of a single sensor in the prior art may be affected by various factors, such as performance limitation of the sensor itself, interference of the external environment, and human activities, thereby causing errors or loss of monitoring data, can be effectively solved. The monitoring method of the embodiment has a multi-level data processing and intelligent decision mechanism, performs comprehensive analysis based on the motion of the person and the environment, reduces the misjudgment, can more accurately determine the health condition of the person, and constructs a complete technical chain from a physical signal to a health early warning, thereby realizing all-weather and non-contact human state monitoring in a family care, elderly care, or the like, and significantly improving the emergency response capability in an emergency.

[0093] In the embodiment of the present application, the monitoring method of the present application aims to provide a multifunctional human monitoring system in which multiple independent sensors work cooperatively through wireless connection. Through the division of labor and cooperation of the sensors, the accuracy and reliability of the detection of the state of the human body, such as falling, heart rate, respiratory rate, and sleep quality, are improved, the presence of a person in the monitoring area is effectively determined, and the light intensity, gas concentration, temperature and humidity, and sound information in the environment are monitored in real time, thereby providing comprehensive health and environmental monitoring services for users.

[0094] To implement the monitoring method in the embodiment, the data collected by the data acquisition device can be transmitted to the data processing device through the wireless communication unit, and the data processing device performs data processing of the monitoring method, thereby realizing

[0095] Millimeter wave radar: a radar chip capable of transmitting and receiving millimeter wave signals can be selected, and the working frequency band is [60-64GHz], which is used to acquire micro-motion information and motion trajectory information of the human body in real time. The millimeter wave radar is internally provided with a wireless communication unit, and the collected data can be wirelessly transmitted to the data processing device.

[0096] Infrared sensing device: a pyroelectric infrared sensor can be used to detect whether there is infrared radiation change in the monitoring area, so as to determine whether a person exists. Similarly, a wireless communication unit is provided to wirelessly transmit the detection signal to the data processing device.

[0097] Light sensing device: a high-precision photosensitive element is used to monitor the ambient light intensity in real time. The light intensity data is sent to the data processing device through the wireless communication unit.

[0098] Gas sensing device: equipped with sensors sensitive to specific gases (such as carbon dioxide, etc.), real-time monitoring of gas concentration in the environment. Use wireless communication function, transmit gas concentration data to data processing device.

[0099] Temperature and humidity sensing device: integrated temperature and humidity sensors, real-time acquisition of environmental temperature and humidity data, and data transmission to data processing device through wireless communication unit.

[0100] Sound sensing device: use high-sensitivity microphone to collect sound signals in the monitoring area, and convert them into electrical signals, then transmit them to the data processing device through the wireless communication unit.

[0101] Data processing device: choose high-performance microprocessor, with powerful data operation and processing capability, used to receive and process data transmitted wirelessly from various sensor devices. The monitoring method of this embodiment is realized through the data processing device.

[0102] Communication unit: can use 433M, Bluetooth, Wi-Fi or ZigBee wireless communication technology to realize data transmission between various sensor devices and data processing device, as well as data interaction between data processing device and external devices (such as mobile phones, smart bracelets or cloud servers, etc.).

[0103] Each sensor device works independently to collect data. The infrared sensing device continuously monitors whether there is a person in the area, and when it detects that a person enters, it triggers the millimeter wave radar device to start working through wireless signals. The millimeter wave radar emits millimeter wave signals, receives reflected signals and converts them into electrical signals, which are transmitted wirelessly to the data processing device. The data processing device uses specific algorithms based on time-frequency analysis to process millimeter wave radar data and extract heart rate and respiration rate information. At the same time, according to the human motion trajectory information obtained by the millimeter wave radar, combined with the pre-set fall judgment model, it judges whether the human body has fallen. During sleep monitoring, according to the human micro-motion information monitored by the millimeter wave radar, analyze the number of turning over and body movement during sleep, etc., to evaluate sleep quality.

[0104] Light sensing device: real-time monitoring of ambient light intensity, data transmission to data processing device through wireless communication. This module can combine with other sensor data, such as in sleep monitoring scenario, if the light intensity suddenly increases during sleep time and the millimeter wave radar detects an increase in human micro-motion frequency, it can be inferred that the human body may wake up from sleep due to light interference, and the sleep quality evaluation system is improved. In daily activity monitoring, light data helps to determine the scene the user is in.

[0105] The gas sensing device continuously monitors the concentration of ambient gas. When the concentration of a specific gas (such as carbon dioxide) is detected to exceed a preset threshold, the data processing device sends data to the data processing device through wireless communication. The data processing device combines the human presence information detected by the infrared device and sends a reminder to the external device through the communication module, informing the user that the ambient gas concentration is abnormal.

[0106] The temperature and humidity sensing device wirelessly transmits real-time environmental temperature and humidity data to the data processing module. The data processing module can analyze the environmental comfort level based on temperature and humidity changes and other sensor data. For example, in a sleep scenario, the data processing module can determine whether the temperature and humidity affect sleep quality based on the sleep state of the human body detected by the millimeter wave radar.

[0107] The sound sensing device collects sound signals and wirelessly transmits them to the data processing device. In the case of distress sound detection, the data processing device uses sound recognition technology to analyze the collected sound signals in real time and compares them with a preset distress sound feature template. If the matching degree exceeds a certain threshold, the data processing device confirms that a person is in an abnormal state based on the human body position and motion state information detected by the millimeter wave radar and infrared device, and sends a distress alarm to the preset emergency contact or relevant rescue platform through the communication module. In the case of sleep snoring monitoring, the data processing device uses sound recognition algorithms to identify snoring characteristics and analyzes whether the user has sleep disorders such as sleep apnea based on the human body breathing micro-motion information detected by the millimeter wave radar.

[0108] The infrared sensing device first determines whether a person is present, effectively avoiding false positives caused by environmental clutter and other factors when there is no one present, and improving the accuracy of subsequent detection of human vital signs and motion state by the millimeter wave radar. The data from each sensor complements each other, and the comprehensive multi-source data improves the accuracy of human state and environmental parameter determination.

[0109] At the same time, it can realize fall detection, heart rate monitoring, respiratory rate monitoring, sleep quality monitoring, light intensity monitoring, gas concentration monitoring, temperature and humidity monitoring, and sound monitoring, and finally form comprehensive event data to meet the user's demand for comprehensive health and environmental monitoring.

[0110] Each sensor device is independent and connected wirelessly, making it easy to install and layout flexibly according to different monitoring scenarios and needs, improving the applicability of the system.

[0111] Unlike camera-based monitoring methods, the monitoring method of the present application does not involve image acquisition, fully protecting the privacy of users. Each sensor only collects specific types of data, and encryption technology is used during data transmission to ensure user data security.

[0112] In an embodiment, the human body state information includes infrared data obtained by an infrared sensor and radar data obtained by a millimeter wave radar, the environment parameter information includes environment data and sound data, and the human body state information and the environment parameter information obtained in the monitoring area include:

[0113] The infrared data, the radar data, the environment data and the sound data are obtained.

[0114] According to the PTP clock synchronization signal broadcast by the master node, the infrared data, the radar data, the environment data and the sound data are reorganized in an N-second time window to obtain the human body state information and the environment parameter information in the detection area.

[0115] In this embodiment, the master node periodically broadcasts a synchronization message, and calculates the clock offset ;

[0116] Wherein:

[0117]

[0118] t1: master node sends synchronization message time;

[0119] t2: slave node receives synchronization message time;

[0120] t3: slave node sends delay request time;

[0121] t4: master node receives delay request time.

[0122] The master node in this embodiment can be a PTP clock in a data processing device, and the slave node is each sensor device. The master node broadcasts a PTP clock synchronization signal, which is the clock offset . N seconds can be 1 second or 1.5 seconds, etc. In 1 second, the infrared data, the radar data, the environment data and the sound data are adjusted by the clock offset , so that the adjusted infrared data, radar data, environment data and sound data are aligned in time.

[0123] The sensor data (the infrared data, the radar data, the environment data and the sound data) of the slave node are calibrated to the local clock, and the alignment accuracy is ±1ms. The data is reorganized in a 1-second time window to eliminate the timing deviation.

[0124] The sensor data (the time and frequency of different sensor data are inconsistent, such as radar sending 100 data per second and infrared sending 1 data per second) of different sampling rates can also be aligned by interpolation method.

[0125]

[0126] The original data timestamp, The reference timestamp.

[0127] Thus, the different data is adjusted to the same time axis for analysis and evaluation.

[0128] In an embodiment, the acquisition of the human state information and the environmental parameter information in the monitoring area further comprises:

[0129] According to the radar data, a radar coordinate system is established as a reference space;

[0130] According to the reference space, a spatial conversion matrix of the infrared data, the environmental data and the sound data is determined;

[0131] According to the spatial conversion matrix of the infrared data, the environmental data and the sound data, the infrared data, the environmental data and the sound data are three-dimensionally converted to obtain the human state information and the environmental parameter information in the monitoring area.

[0132] In this embodiment, a radar coordinate system is established as a reference space through radar data, and a spatial conversion matrix of other sensors is calculated:

[0133] Specifically,

[0134] Three or more reflective marker points are arranged in the monitoring area;

[0135] Millimeter wave radar records marker point coordinates ;

[0136] Other sensors trigger to record local coordinates ;

[0137] SVD solution:

[0138]

[0139] Calculation steps:

[0140] a. Calculate the centroid coordinates:

[0141]

[0142] b. Construct the covariance matrix:

[0143]

[0144] c. SVD decomposition:

[0145] d. Rotation matrix:

[0146] e. Translation vector:

[0147]

[0148] The coordinates of the infrared sensor can also be transformed according to the reference space: rotate 15° + offset 2.4m along the Z-axis;

[0149] The three-dimensional position mapping of environmental sensors is based on the reference space.

[0150] Sensor spatial alignment is achieved through three-dimensional coordinate system transformation. Taking the millimeter-wave radar coordinate system as the reference, the coordinate transformation matrices for other sensors are as follows:

[0151]

[0152] Where R is the rotation matrix and T is the translation vector, the parameters of which are obtained through calibration experiments.

[0153] This enables spatiotemporal alignment of radar data, infrared data, environmental data, and sound data, avoiding spatiotemporal errors and preventing misjudgments caused by errors between sensor data.

[0154] In one implementation, the step of extracting features from the human body state information and the environmental parameter information to obtain state data and environmental data includes:

[0155] Based on the radar data, respiratory information, heart rate information, and movement trajectory information are determined;

[0156] Based on the infrared data, the probability information is determined to exist;

[0157] Based on the environmental data, temperature, humidity, and air quality information are determined;

[0158] Based on the sound data, determine the voiceprint information and sound information;

[0159] Based on the breathing information, heart rate information, movement trajectory information, presence probability information, temperature and humidity information, air quality information, voiceprint information, and sound information, state data and environmental data are obtained.

[0160] In this embodiment, the radar data includes real-time capture of micro-motion signals of the human chest cavity by millimeter-wave radar (60GHz band), such as FMCW radar phase differential extraction of chest cavity movement, i.e. respiratory signal and heart rate signal, for example: chest cavity displacement → respiratory waveform (0.1-0.5Hz bandpass filtering) to obtain respiratory information; and heart rate information obtained by millimeter-wave radar through micro-Doppler features → heart rate estimation (0.8-3Hz spectrum analysis).

[0161] For example: Radar data is resolved by thoracic motion phase:

[0162]

[0163] For phase noise;

[0164] : millimeter wave wavelength (5 mm @ 60 GHz);

[0165] : thoracic displacement

[0166] Respiratory / heart rate separation: dual-channel bandpass filter is adopted

[0167] Respiratory signal: 0.1-0.5 Hz (6—30 times / minute)

[0168] Heart rate signal: 0.8-3 Hz (48—180 times / minute)

[0169] Radar data includes real-time capture of motion trajectory by millimeter wave radar for analysis:

[0170] Extended Kalman filter (EKF) model:

[0171]

[0172] Where F is the state transition matrix, and Q is the process noise covariance.

[0173] Infrared data obtained by infrared sensor, infrared sensor is mainly to monitor whether the human body exists in the monitoring area, human body existence state filtering (sliding window threshold method) can be adopted, for example: double threshold comparison method to judge whether the human body exists in the monitoring area.

[0174] The existence probability calculation formula can also be used to determine:

[0175]

[0176] , : empirical threshold

[0177] Thus, the existence probability information is obtained.

[0178] Through environmental data extraction:

[0179] Environmental comprehensive index (ECI):

[0180]

[0181] Temperature and humidity information (THI):

[0182]

[0183] Air information (GCI):

[0184]

[0185] The temperature and humidity information and the air information can be determined through the above formula.

[0186] Sound sensor: match abnormal voiceprint based on dynamic time warping (DTW) to obtain voiceprint information:

[0187]

[0188] Effect: convert sound into machine recognizable "fingerprint" for distinguishing between help-seeking sound, snoring sound, etc.

[0189] Symbol explanation:

[0190] E k : Energy output of Mel filter bank, simulating human ear sensitivity to different frequencies.

[0191] N: Number of Mel filters (usually take 40).

[0192] i: Serial number of MFCC coefficients (usually take the first 13).

[0193] Human ears are more sensitive to low frequencies (such as male voices), and less sensitive to high frequencies. MFCC compresses high frequency information through Mel filter bank and retains key features.

[0194] Sound information can be directly detected by sound sensor.

[0195] In one embodiment, the data fusion of the state data and the environment data to generate comprehensive event data includes:

[0196] According to the respiratory information, the heart rate information, the motion trajectory information, the existence probability information, the voiceprint information, and the sound information, a joint confidence is calculated.

[0197] According to the temperature and humidity information and the air information, the joint confidence is adjusted to generate comprehensive event data.

[0198] Through the method of the embodiment, the joint confidence can be determined first, and the joint confidence is adjusted by the weight of the temperature and humidity information and the air information, which further analyzes whether the person in the actual monitoring area has a fall or coma condition through the environmental information, and improves the accuracy of the analysis.

[0199] In an embodiment, the calculating, according to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the voiceprint information and the sound information, a joint confidence level comprises:

[0200] Obtaining the infrared sensor, the millimeter wave radar and the sound sensor;

[0201] According to the historical data, determining the credibility of the infrared sensor, the credibility of the millimeter wave radar and the credibility of the sound sensor, the weight of the infrared sensor, the weight of the millimeter wave radar and the weight of the sound sensor;

[0202] According to the credibility of the infrared sensor, adjusting the presence probability information to obtain the confidence level of the presence probability information;

[0203] According to the credibility of the millimeter wave radar, adjusting the motion trajectory information to obtain the confidence level of the motion trajectory information;

[0204] According to the credibility of the millimeter wave radar, adjusting the breathing information to obtain the confidence level of the breathing information;

[0205] According to the credibility of the millimeter wave radar, adjusting the heart rate information to obtain the confidence level of the heart rate information;

[0206] According to the credibility of the sound sensor, adjusting the voiceprint information to obtain the confidence level of the voiceprint information;

[0207] According to the credibility of the sound sensor, adjusting the sound information to obtain the confidence level of the sound information;

[0208] According to the weight of the infrared sensor, the weight of the millimeter wave radar, the weight of the sound sensor, the confidence level of the presence probability information, the confidence level of the breathing information, the confidence level of the heart rate information, the confidence level of the voiceprint information and the confidence level of the sound information, a joint confidence level is obtained.

[0209] In the embodiment, through analysis of historical data, for example, by inputting historical data into a trained neural network, the credibility of the infrared sensor, the credibility of the millimeter wave radar and the credibility of the sound sensor can be obtained. In order to make the joint confidence level more reliable, historical information can be input into another trained neural network to determine the weight of the infrared sensor, the weight of the millimeter wave radar and the weight of the sound sensor respectively under the condition of judging the fall or injury and the like, and the output accuracy is higher.

[0210] The confidence of the corresponding information is adjusted based on the reliability of each sensor obtained above, so that the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information, and the confidence of the sound information can correspond to the corresponding sensor, so that the joint confidence of the current event can be more accurate.

[0211] The joint confidence can be obtained by the following formula:

[0212]

[0213] The confidence of multiple sensors is integrated to improve the reliability of decision-making.

[0214] is the weight of the i-th sensor among the weight of the infrared sensor, the weight of the millimeter wave radar, and the weight of the sound sensor;

[0215] is the confidence of the i-th among the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information, and the confidence of the sound information;

[0216] is the probability of the i-th among the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information, and the confidence of the sound information that the event Aj occurs, for example, a fall.

[0217] N is the number of sensors.

[0218]

[0219] m(A) is the joint confidence of event A;

[0220] is the set of propositions supported by each of the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information, and the confidence of the sound information, for example: sensor 1 supports "fall", sensor 2 supports "normal";

[0221] is the intersection of the propositions supported by the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information, and the confidence of the sound information, that is, only when all confidence evidences support A, the numerator is counted.

[0222] K is a conflict factor, representing the degree of conflict between the evidences of each sensor (K=1 when completely conflicting);

[0223] a confidence level of the presence probability information, a confidence level of the breathing information, a confidence level of the heart rate information, a confidence level of the voiceprint information, and a confidence level of the sound information.

[0224] The independent evidences of the plurality of sensors are fused into a unified conclusion to obtain a final joint confidence level.

[0225] The above method can change the problem of easy misjudgment of a single sensor.

[0226] For example: single radar: 75% chance of falling;

[0227] Add infrared data: increase to 82%;

[0228] Add sound evidence: finally 89% confidence.

[0229] For the weight part,

[0230] Based on the historical accuracy rate, the sensor credibility is dynamically adjusted:

[0231] Millimeter wave radar weight 0.6 (high-precision vital signs)

[0232] Infrared sensor weight 0.3 (reliable presence detection)

[0233] Environmental sensor / sound sensor each 0.05 (auxiliary verification).

[0234] In an embodiment, the pre-warning information includes first-level pre-warning information, second-level pre-warning information, and third-level pre-warning information, and the determining pre-warning information according to the comprehensive event data includes:

[0235] In the case that the comprehensive event data is greater than or equal to a first specified threshold, triggering the first-level pre-warning information;

[0236] In the case that the comprehensive event data is less than the first specified threshold and greater than or equal to a second specified threshold, triggering the second-level pre-warning information;

[0237] In the case that the comprehensive event data is less than the second specified threshold, and any one of the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the temperature and humidity information, the air information, the voiceprint information, and the sound information is abnormal, triggering the third pre-warning signal.

[0238] In the present embodiment, by classifying the first-level pre-warning information, the second-level pre-warning information, and the third-level pre-warning information, different levels can be responded to in order to enable the caregiver to handle in time.

[0239] The vital sign data is acquired in advance, the emergency efficiency is improved by 30%, and the processing can be performed in time.

[0240] In a second aspect, the embodiments of the present application provide a system, comprising:

[0241] An acquisition unit is configured to acquire human state information and environmental parameter information in a monitoring area.

[0242] A derivation unit is configured to perform feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data.

[0243] A generation unit is configured to perform data fusion on the state data and the environmental data to generate comprehensive event data.

[0244] A determination unit is configured to determine early warning information according to the comprehensive event data.

[0245] In the embodiments, the monitoring method comprises: acquiring human state information and environmental parameter information in a monitoring area; performing feature extraction on the human state information and the environmental parameter information to obtain state data and environmental data; performing data fusion on the state data and the environmental data to generate comprehensive event data; and determining early warning information according to the comprehensive event data. Thus, the problem that the accuracy and reliability of a single sensor in the prior art can be affected by multiple factors, such as performance limitations of the sensor itself, external environmental interference, and human activities, thereby causing errors or missing of monitoring data, can be effectively solved. The monitoring method of the embodiments has a multi-level data processing and intelligent decision mechanism, performs comprehensive analysis based on the motion of a person and the environment, reduces the misjudgment, can more accurately determine the health condition of the person, and constructs a complete technical chain from physical signals to health early warning, thereby realizing all-weather and non-contact human state monitoring in scenarios such as family care and elderly care, and significantly improving the emergency response capability of a sudden event.

[0246] In an implementation, the human state information comprises infrared data obtained by an infrared sensor and radar data obtained by a millimeter wave radar, the environmental parameter information comprises environmental data and sound data, and the acquisition of the human state information and the environmental parameter information in the monitoring area comprises:

[0247] The infrared data, the radar data, the environmental data, and the sound data are acquired.

[0248] The infrared data, the radar data, the environmental data, and the sound data are reorganized according to a PTP clock synchronization signal broadcast by a master node in an N-second time window to obtain the human state information and the environmental parameter information in the detection area.

[0249] In an embodiment, the acquiring the human state information and the environmental parameter information in the monitoring area further comprises:

[0250] According to the radar data, a radar coordinate system is established as a reference space;

[0251] According to the reference space, a spatial conversion matrix of the infrared data, the environmental data and the sound data is determined;

[0252] According to the spatial conversion matrix of the infrared data, the environmental data and the sound data, the infrared data, the environmental data and the sound data are three-dimensionally converted to obtain the human state information and the environmental parameter information in the monitoring area.

[0253] In an embodiment, the feature extraction of the human state information and the environmental parameter information to obtain state data and environmental data comprises:

[0254] According to the radar data, breathing information, heart rate information and motion trajectory information are determined;

[0255] According to the infrared data, presence probability information is determined;

[0256] According to the environmental data, temperature and humidity information and air information are determined;

[0257] According to the sound data, voiceprint information and sound information are determined;

[0258] According to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the temperature and humidity information, the air information, the voiceprint information and the sound information, state data and environmental data are obtained.

[0259] In an embodiment, the data fusion of the state data and the environmental data to generate comprehensive event data comprises:

[0260] According to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the voiceprint information and the sound information, a joint confidence is calculated;

[0261] According to the temperature and humidity information and the air information, the joint confidence is adjusted to generate comprehensive event data.

[0262] In an embodiment, the calculation according to the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the voiceprint information and the sound information to obtain the joint confidence comprises:

[0263] The infrared sensor, the millimeter wave radar and the sound sensor are acquired;

[0264] determine a credibility of the infrared sensor, a credibility of the millimeter wave radar, a credibility of the sound sensor, a weight of the infrared sensor, a weight of the millimeter wave radar and a weight of the sound sensor according to the historical data;

[0265] adjust the presence probability information according to the credibility of the infrared sensor, to obtain a confidence of the presence probability information;

[0266] adjust the motion trajectory information according to the credibility of the millimeter wave radar, to obtain a confidence of the motion trajectory information;

[0267] adjust the breathing information according to the credibility of the millimeter wave radar, to obtain a confidence of the breathing information;

[0268] adjust the heart rate information according to the credibility of the millimeter wave radar, to obtain a confidence of the heart rate information;

[0269] adjust the voiceprint information according to the credibility of the sound sensor, to obtain a confidence of the voiceprint information;

[0270] adjust the sound information according to the credibility of the sound sensor, to obtain a confidence of the sound information;

[0271] obtain a joint confidence according to the weight of the infrared sensor, the weight of the millimeter wave radar, the weight of the sound sensor, the confidence of the presence probability information, the confidence of the breathing information, the confidence of the heart rate information, the confidence of the voiceprint information and the confidence of the sound information.

[0272] In an implementation, the pre-warning information includes first-level pre-warning information, second-level pre-warning information and third-level pre-warning information, and the determining the pre-warning information according to the comprehensive event data includes:

[0273] trigger the first-level pre-warning information when the comprehensive event data is greater than or equal to a first specified threshold;

[0274] trigger the second-level pre-warning information when the comprehensive event data is less than the first specified threshold and greater than or equal to a second specified threshold;

[0275] trigger the third-level pre-warning information when the comprehensive event data is less than the second specified threshold, and any one of the breathing information, the heart rate information, the motion trajectory information, the presence probability information, the temperature and humidity information, the air information, the voiceprint information and the sound information is abnormal.

[0276] The functions of each module in each device in the embodiments of the present application can be referred to the corresponding description in the above method, which will not be repeated here.

[0277] Figure 2 A structural block diagram of an electronic device according to an embodiment of this application is shown. Figure 2 As shown, the electronic device includes a memory 410 and a processor 420. The memory 410 stores instructions that can be executed on the processor 420. When the processor 420 executes the instructions, it implements the monitoring method in the above embodiments. The number of memories 410 and processors 420 can be one or more. This electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0278] The electronic device may also include a communication interface 430 for communicating with external devices and exchanging data. The devices are interconnected using different buses and can be mounted on a common motherboard or otherwise as needed. The processor 420 can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0279] Optionally, in a specific implementation, if the memory 410, processor 420 and communication interface 430 are integrated on a single chip, the memory 410, processor 420 and communication interface 430 can communicate with each other through an internal interface.

[0280] It should be appreciated that referenced processors above can be Central Processing Units (CPUs), but can also be general purpose processors, Digital Signal Processors (DSPs), Application-Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic components, discrete hardware components, or the like. The general purpose processor can be a microprocessor or any conventional processor, and the like. It is worthy to note that the processor can be an Advanced RISC Machines (ARM) architecture processor.

[0281] The computer readable storage medium (such as the memory 410 described above) stores computer instructions, and the program is executed by the processor to implement the method provided in the embodiments of the present application.

[0282] Optionally, the memory 410 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function; the data storage area can store data created according to the use of the electronic device, and the like. In addition, the memory 410 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 410 can optionally include a memory disposed remotely with respect to the processor 420, and these remote memories can be connected to the electronic device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0283] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0284] In addition, the terms "first", "second", etc. are used herein only to describe different instances, and cannot be construed as indicating or implying relative importance or an indicated number of the technical features. Thus, the features defined with "first", "second", etc. can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0285] Any process or method descriptions or descriptions of the flow diagrams described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for performing specific logic functions or steps in the process. And the various embodiments of the application can include additional or fewer steps or processes in comparison to those shown or discussed.

[0286] The logic and / or steps represented in the flow diagrams described herein, for example, can be embodied in computer-readable instructions, which can be used to cause one or more processors to perform the actions indicated in the logic flow diagrams. The computer-readable instructions can be stored on one or more storage media or memories, such as a computer-readable medium, which can be non-transitory.

[0287] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above-described embodiment methods can be instructed by a program to the relevant hardware, and the program can be stored in a computer-readable storage medium, and when executed, includes one or a combination of the steps of the method embodiments.

[0288] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software function module. The above-mentioned integrated module, if realized in the form of a software function module and sold or used as an independent product, can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0289] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various changes or replacements within the technical range disclosed by the present application, and these should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A monitoring method performed by an electronic device, characterized in that, include: Acquire information on human status and environmental parameters within the monitoring area; Feature extraction is performed on the human body state information and the environmental parameter information to obtain state feature data and environmental feature data; The state feature data and the environmental feature data are fused to generate comprehensive event data; Based on the comprehensive event data, early warning information is determined; The human body status information includes infrared data obtained through infrared sensors and radar data obtained through millimeter-wave radar; the environmental parameter information includes environmental data and sound data. Acquiring information on human status and environmental parameters within the monitoring area includes: Based on the radar data, establish a radar coordinate system as the reference space; Based on the reference space, determine the spatial transformation matrix of the infrared data, the environmental data, and the sound data; Based on the spatial transformation matrix of the infrared data, the environmental data, and the sound data, a three-dimensional transformation is performed on the infrared data, the environmental data, and the sound data to obtain human body status information and environmental parameter information within the monitoring area; The steps for determining the spatial transformation matrix are as follows: Arrange at least three reflective markers in the monitoring area; Millimeter-wave radar records the coordinates of marker points ; Record local coordinates when other sensors are triggered. ; SVD solution: ; The calculation steps for solving the SVD problem are as follows: a. Calculate the centroid coordinates: ； b. Construct the covariance matrix: ； c. SVD decomposition: ; d. Rotation matrix: ; e. Translation vector: 。 2. The method according to claim 1, characterized in that, The acquisition of human status information and environmental parameter information within the monitoring area includes: Acquire the infrared data, the radar data, the environmental data, and the sound data; Based on the PTP clock synchronization signal broadcast by the master node, the infrared data, radar data, environmental data, and sound data are reassembled in N-second time windows to obtain human status information and environmental parameter information within the monitoring area.

3. The method according to claim 2, characterized in that, The step of extracting features from the human body state information and the environmental parameter information to obtain state feature data and environmental feature data includes: Based on the radar data, respiratory information, heart rate information, and movement trajectory information are determined; Based on the infrared data, the probability information is determined to exist; Based on the environmental data, temperature, humidity, and air quality information are determined; Based on the sound data, determine the voiceprint information and sound information; Based on the breathing information, heart rate information, movement trajectory information, presence probability information, temperature and humidity information, air quality information, voiceprint information, and sound information, state feature data and environmental feature data are obtained.

4. The method according to claim 3, characterized in that, The process of fusing the state feature data and the environmental feature data to generate comprehensive event data includes: The joint confidence level is calculated based on the respiratory information, heart rate information, movement trajectory information, presence probability information, voiceprint information, and sound information. The joint confidence level is adjusted based on the temperature and humidity information and the air information to generate comprehensive event data.

5. The method according to claim 4, characterized in that, The calculation based on the respiratory information, heart rate information, movement trajectory information, presence probability information, voiceprint information, and sound information yields the joint confidence level, which includes: Based on historical data, the reliability of the infrared sensor, the reliability of the millimeter-wave radar, the reliability of the sound sensor, the weight of the infrared sensor, the weight of the millimeter-wave radar, and the weight of the sound sensor are determined. The presence probability information is adjusted based on the confidence level of the infrared sensor to obtain the confidence level of the presence probability information; The motion trajectory information is adjusted based on the reliability of the millimeter-wave radar to obtain the confidence level of the motion trajectory information; The breathing information is adjusted based on the reliability of the millimeter-wave radar to obtain the confidence level of the breathing information; The heart rate information is adjusted based on the reliability of the millimeter-wave radar to obtain the confidence level of the heart rate information; The voiceprint information is adjusted based on the confidence level of the sound sensor to obtain the confidence level of the voiceprint information; The sound information is adjusted based on the confidence level of the sound sensor to obtain the confidence level of the sound information; A joint confidence score is obtained based on the weights of the infrared sensor, the millimeter-wave radar, the sound sensor, the confidence score of the existence probability information, the confidence score of the motion trajectory information, the confidence score of the breathing information, the confidence score of the heart rate information, the confidence score of the voiceprint information, and the confidence score of the sound information.

6. The method according to claim 5, characterized in that, The early warning information includes Level 1, Level 2, and Level 3 early warning information. Determining the early warning information based on the comprehensive event data includes: If the comprehensive event data is greater than or equal to a first specified threshold, a Level 1 warning message is triggered; If the comprehensive event data is less than the first specified threshold and greater than or equal to the second specified threshold, a level two warning message is triggered; If the comprehensive event data is less than the second specified threshold, and any of the following is abnormal: respiratory information, heart rate information, movement trajectory information, presence probability information, temperature and humidity information, air quality information, voiceprint information, and sound information, a third warning signal is triggered.

7. A monitoring system, characterized in that, include: The acquisition unit is used to acquire information on the human body status and environmental parameters within the monitoring area. The obtaining unit is used to extract features from the human body state information and the environmental parameter information to obtain state feature data and environmental feature data; A generation unit is used to perform data fusion on the state feature data and the environmental feature data to generate comprehensive event data; The determining unit is used to determine the early warning information based on the comprehensive event data; The human body status information includes infrared data obtained through infrared sensors and radar data obtained through millimeter-wave radar; the environmental parameter information includes environmental data and sound data. Acquiring information on human status and environmental parameters within the monitoring area includes: Based on the radar data, establish a radar coordinate system as the reference space; Based on the reference space, determine the spatial transformation matrix of the infrared data, the environmental data, and the sound data; Based on the spatial transformation matrix of the infrared data, the environmental data, and the sound data, a three-dimensional transformation is performed on the infrared data, the environmental data, and the sound data to obtain human body status information and environmental parameter information within the monitoring area; The steps for determining the spatial transformation matrix are as follows: Arrange at least three reflective markers in the monitoring area; Millimeter-wave radar records the coordinates of marker points ; Record local coordinates when other sensors are triggered. ; SVD solution: ; The calculation steps for solving the SVD problem are as follows: a. Calculate the centroid coordinates: ; b. Construct the covariance matrix: ; c. SVD decomposition: ; d. Rotation matrix: ; e. Translation vector: .

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

Images