Multi-unit radar collaborative motion state and vital sign monitoring system and method

Abstract

The invention discloses a multi-unit radar collaborative motion state and vital sign monitoring system and a multi-unit radar collaborative motion state and vital sign monitoring method. The system comprises an edge sensing node and an upper computer which are in communication connection, the edge sensing node has complete vital sign wireless sensing, signal resolving and communication capabilities; the upper computer is used for receiving information such as a target ID, a target position, a target vital sign and a target abnormal state sent by the edge sensing node; the edge sensing node is composed of a microcontroller module, a radar sensor module, a DSP module, an FPGA neural network reasoning module, a communication module and a power supply module. The microcontroller module is in communication connection with the radar sensor module, the DSP module, the FPGA neural network reasoning module and the communication module. The radar sensor module is in communication connection with the DSP module; the DSP module is in communication connection with the FPGA neural network reasoning module; the power supply module supplies power to the edge sensing nodes. According to the method, accurate tracking of multi-target vital signs and positions is carried out, and accurate tracking of multi-target health states is realized.

Description

Technical Field

[0001] The present invention relates to the field of wireless sensing technology, and in particular, to a multi-unit radar collaborative motion state and vital sign monitoring system and method. Background Art

[0002] With the acceleration of the social aging process, the demand for real-time health status monitoring and evaluation technology in centralized elderly care places such as nursing homes is becoming increasingly prominent. The vital signs of the elderly (such as heartbeat and breathing) and behavioral states (such as falls, etc.) are important indicators for evaluating health status and preventing accidents. Long-term and accurate monitoring of them is of great significance for timely detecting health hazards and improving nursing efficiency. Traditional contact monitoring technologies (such as electrocardiogram and photoplethysmograph) rely on patches, sensors or cable connections, which are likely to restrict the daily activities of the human body, resulting in poor wearing comfort and inability to meet the all-weather and multi-target health monitoring requirements. In terms of non-contact technologies, although vision-based measurement methods avoid wearing restrictions, they require a light source and are easily affected by the field of view and ambient light. At the same time, there is a certain risk of privacy leakage, making it difficult to be widely promoted in scenarios such as nursing homes and homes.

[0003] The radar-based vital sign monitoring technology has become an important research direction in the field of health monitoring due to its non-invasive, non-contact, non-line-of-sight, multi-target simultaneous measurement and privacy protection characteristics. However, the existing single-node radar monitoring system is vulnerable to the influence of multipath effects and occlusion interference in multi-target scenarios, and it is difficult to achieve accurate coverage of multiple areas and simultaneous monitoring of multiple monitoring objects. To sum up, due to the rich information obtained by radar high-frequency signals resulting in a large amount of data, the current system still faces challenges in data processing efficiency when dealing with the monitoring of the health status of multiple targets in a large area; electromagnetic wave signals are easily blocked and there are multipath interferences, and the stability of continuous measurement of human health parameters in complex environments is insufficient. In addition, system functions such as visual display and analysis of human health status parameters and abnormal recognition and alarm still need to be improved.

[0004] The literature "Ding Yi. Development and implementation of an indoor and outdoor multi-person vital sign monitoring system based on multi-radar fusion Internet of Things [D]. Beijing University of Posts and Telecommunications, 2022." built a multi-radar fusion Internet of Things system with three ultra-wideband radars and a continuous wave radar, realizing the monitoring of the motion postures and vital signs of multiple moving targets. This system uses multiple nodes for measurement in a single room and does not expand the system to the linkage of multiple nodes in multiple rooms to achieve a more wide-area monitoring. Therefore, there is an urgent need to design a monitoring system with complete functions and a corresponding monitoring method. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the technical problem to be solved by the present invention is to provide a multi-unit radar collaborative motion state and vital sign monitoring system and method.

[0006] The technical solution of the present invention to solve the technical problem of the system is to provide a multi-unit radar collaborative motion state and vital sign monitoring system, characterized in that the system includes an edge sensing node and a host computer connected by communication;

[0007] The edge sensing node has complete wireless sensing, signal resolution and communication capabilities for vital signs; the host computer receives information such as the target ID, target position, target vital signs and target abnormal states sent by the edge sensing node;

[0008] The edge sensing node is composed of a microcontroller module, a radar sensor module, a DSP module, an FPGA neural network inference module, a communication module and a power supply module; the microcontroller module is respectively connected to the radar sensor module, the DSP module, the FPGA neural network inference module and the communication module by communication; the radar sensor module is connected to the DSP module by communication; the DSP module and the FPGA neural network inference module are connected by communication; the power supply module supplies power to the edge sensing node.

[0009] The technical solution of the present invention to solve the technical problem of the method is to provide a multi-unit radar collaborative motion state and vital sign monitoring method, characterized in that the method includes the following steps:

[0010] Step 1, build a monitoring system;

[0011] Step 2, first, the radar sensor module dynamically adjusts the measurement parameters according to the instructions issued by the microcontroller module, transmits signals and receives echo signals; then the radar sensor module completes target information acquisition, and obtains a three-dimensional data matrix composed of original ADC data; subsequently, a three-dimensional feature data set including target distance, angle and speed is output in the DSP module;

[0012] Step 3, input the three-dimensional feature data set generated in Step 2 into the DSP module to perform multi-level preprocessing, and output a primary target motion trajectory; then, based on the angle information in the primary target motion trajectory, beamforming is performed based on the minimum variance distortionless response algorithm to enhance the target direction signal, and a primary four-dimensional feature data set including target chest surface displacement information is formed;

[0013] Step 4, the microcontroller module performs multi-dimensional analysis on the primary four-dimensional feature data set obtained in Step 3, and establishes a target occlusion processing and monitoring relay mechanism; then, according to the target occlusion processing and monitoring relay mechanism, the primary target motion trajectory is processed, and a stable and continuous four-dimensional feature data set is output;

[0014] Step 5: Heartbeat and respiration feature extraction and fall detection feature extraction;

[0015] In the heartbeat and respiration feature extraction, first, based on the target thoracic micro-motion feature in the four-dimensional feature dataset output in Step 4, perform variational mode decomposition in the DSP module to obtain a number of intrinsic mode functions; in the decomposition result, if there is a mode function with energy concentrated in the respiration frequency band of 0.1 - 0.8 Hz or the heartbeat frequency band of 1 - 3 Hz, and the frequency domain sidelobe suppression ratio ≥ 20 dB, it is determined as a valid vital sign signal; otherwise, it is determined as a low signal-to-noise ratio signal that does not meet the conditions, which is input into the FPGA neural network inference module for in-depth feature mining to obtain vital sign parameters, and then perform target action influence correction on it to obtain a stable vital sign signal;

[0016] In the fall detection feature extraction, the four-dimensional feature dataset output in Step 4 is used by the DSP module to perform differential operation to extract the vertical velocity component of the human body centroid, and a two-layer detection mechanism based on kinematic criteria is constructed to determine whether a fall event occurs;

[0017] Step 6: Transmit the vital sign signal and whether a fall event occurs obtained in Step 5 to the host computer for display and statistics.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0019] 1. Construct a multi-node edge sensing system to achieve wide-area vital sign monitoring;

[0020] The present invention constructs a multi-node edge sensing system based on millimeter-wave radar, realizing all-weather wide-area monitoring and analysis of the human health status and abnormal status in places such as nursing homes. Its technical solution can efficiently solve the problems of multi-target monitoring signal loss and false detection caused by factors such as occlusion and personnel activities in complex environments. At the same time, through edge computing on the node side, the resource deployment is optimized, the problem of insufficient computing resources is solved, the remote transmission pressure of a large amount of data is reduced, and the real-time performance and reliability of monitoring are significantly improved.

[0021] 2. Propose a motion interference suppression and occlusion recovery recognition compensation algorithm to accurately track multi-target vital signs and positions. The proposed interference suppression and compensation algorithms include:

[0022] (1) Bind the target ID, target location, and vital sign information based on the short-term invariance of vital signs. Specifically, bind the target number, historical location, predicted location based on the extended Kalman filter, and vital sign similarity. In the case of target occlusion recovery, evaluate the similarity between the new target location and the predicted location group based on the Kalman filter, and between the new target vital signs and the vital signs group before occlusion to achieve target re-association, significantly improving the target recognition accuracy in the case of occlusion recovery and avoiding target loss caused by interference;

[0023] (2) Construct a vital sign regression algorithm based on historical vital sign information. If the current detection value is an outlier of the historical vital signs, it is determined as a detection false value caused by interference and regressed to the baseline value.

[0024] (3) In the case where vital signs cannot be detected at all, use the Kalman filter for short-term prediction, and iteratively optimize the prediction method by combining the true detection values after subsequent interference disappears to ensure data continuity and accuracy.

[0025] 3. Propose an adaptive network communication structure and node linkage function based on WiFi Mesh;

[0026] The present invention constructs an edge sensing node network based on WiFi Mesh, realizing efficient communication between nodes. The network is self-adaptive and the system can still work normally when some nodes are unavailable, making the system have the advantages of flexibility, scalable scale, and node linkage.

[0027] 4. Propose a task switching algorithm based on multi-node and map model to achieve wide-area uninterrupted monitoring;

[0028] By deploying multiple radar nodes in multiple rooms, when the target leaves the field of view of the current node, the current node can automatically transfer the monitoring task and target information to the new node in the room where the target enters through the WiFi network according to the moving trend of the target and the pre-input map model, so as to achieve uninterrupted and wide-area monitoring in scenarios such as nursing homes.

[0029] 5. Propose a feature extraction method based on multi-modal technology to reduce the computational burden and improve the detection robustness at the same time;

[0030] In terms of vital sign signal feature extraction, after the data undergoes multi-level FFT, noise reduction filtering, and CFAR detection, the present invention proposes a dual-modal branch structure according to different signal-to-noise ratio conditions:

[0031] (1) When the signal-to-noise ratio of the target thoracic cavity signal is high, use the variational mode decomposition method to significantly reduce the computational burden;

[0032] (2) When the signal-to-noise ratio of the target thoracic cavity signal is low, the LSTM neural network method based on FPGA is adopted to mine hidden information, effectively improving the robustness and accuracy of vital sign signal detection, and is applicable to complex scenarios with multiple targets and dynamic changes.

[0033] 6. Achieve accurate tracking of the health status of multiple targets;

[0034] The host computer parses the information packet in real time, and realizes the visual display of multiple-target vital signs on the software graphic interface. If the target vital signs exceed the abnormal threshold or there are abnormal action states such as falling, an alarm reminder will be sent to the manager. Through the data management, analysis and visualization functions implemented by the host computer, it provides intuitive health status display and abnormal alarm for nursing staff, improving the health maintenance efficiency. At the same time, the long-term data recorded by the system provides data support for subsequent health management and disease early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic structural diagram of the monitoring system of the present invention;

[0036] Figure 2 is a schematic block diagram of the edge sensing node of the present invention;

[0037] Figure 3 is a schematic structural diagram of the radar sensor module of the present invention;

[0038] Figure 4 is a schematic structural diagram of the FPGA neural network inference module of the present invention;

[0039] Figure 5 is a flow chart of the overall monitoring method of the present invention;

[0040] Figure 6 is a flow chart of the node task switching logic when the target of the present invention is blocked;

[0041] Figure 7 is a flow chart of the filtering algorithm when the vital signs of the present invention are interfered;

[0042] Figure 8 is a real-time display interface of the visualization software of the host computer of the present invention;

[0043] Figure 9 is an abnormal alarm record display interface of the visualization software of the host computer of the present invention.

[0044] In the figure, edge sensing node 1, host computer 2, microcontroller module 11, radar sensor module 12, DSP module 13, FPGA neural network inference module 14, communication module 15, power supply module 16. DETAILED DESCRIPTION OF THE INVENTION

[0045] Specific embodiments of the present invention are given below. The specific embodiments are only used to further illustrate the present invention and do not limit the protection scope of the present invention.

[0046] The present invention provides a multi-unit radar collaborative motion state and vital sign monitoring system (hereinafter referred to as the system), characterized in that the system includes an edge sensing node 1 and a host computer 2 which are communicatively connected;

[0047] The edge sensing node 1 has complete wireless sensing, signal resolution and communication capabilities for vital signs, and is arranged at a height level with the human chest cavity (in this embodiment, at a height of 1.3 to 1.5 meters in the room to be monitored) to obtain the best radar signal strength; the host computer 2 is a personal computer, which receives information such as target ID, target position, target vital signs and target abnormal states sent by the edge sensing node 1.

[0048] The edge sensing node 1 is composed of a microcontroller module 11, a radar sensor module 12, a DSP (Digital Signal Processing) module 13, an FPGA neural network inference module 14, a communication module 15 and a power supply module 16; the microcontroller module 11 is communicatively connected to the radar sensor module 12, the DSP module 13, the FPGA neural network inference module 14 and the communication module 15 respectively; the radar sensor module 12 is communicatively connected to the DSP module 13; the DSP module 13 and the FPGA neural network inference module 14 are communicatively connected; the power supply module 16 supplies power to the edge sensing node 1.

[0049] Preferably, the radar sensor module 12 continuously tracks and measures the minute displacement on the surface of the target chest cavity, emits a frequency-modulated continuous wave and acquires the original data of the reflected echo in the scene. The DSP module 13 calculates information such as the target position and the target chest cavity displacement for extracting the heart rate and breathing rate. The measurement results will be input into the FPGA neural network inference module 14 for further feature extraction; the measurement results of all targets in the current scene obtained by the edge sensing node 1 are synchronized among the nodes through the communication module 15, and the target ID, target position, target vital signs, target abnormal states and other information are sent to the host computer 2 through the communication module 15 in a wired or wireless manner through the root node for further statistics and display.

[0050] Preferably, the microcontroller module 11 is communicatively connected to the DSP module 13 through an SPI serial bus, communicatively connected to the FPGA neural network inference module 14 through an SPI protocol, and communicatively connected to the communication module 15 through an SDIO protocol; the DSP module 13 is communicatively connected to the radar sensor module 12 through a parallel interface.

[0051] Preferably, the microcontroller module 11 is composed of a high-performance microcontroller chip STM32H750XBH6 from STMicroelectronics and its peripheral circuits. The microcontroller module 11 is the control core of the edge sensing node 1, realizing functions such as system initialization, data collection, data processing, and communication control. System initialization mainly completes the establishment of the communication link, sends downlink frequency parameters and initialization instructions to the radar sensor module 12, and activates the DSP module 13 and the FPGA neural network inference module 14. Data collection is used to control the feeding of the original data stream collected by the radar sensor module 12 into the DSP module 13 and the FPGA neural network inference module 14 for processing, and then reads the calculated personnel ID, vital signs, and abnormal status into the Flash. Data processing is used to format the data in the Flash and send it to the communication module 15.

[0052] Preferably, in step 1, the radar sensor module 12 includes a transmitting antenna, a power amplifier, a voltage-controlled oscillator, a phase shifter, a mixer, a linear amplifier, a receiving antenna, an A / D converter, and a microcontroller;

[0053] The voltage-controlled oscillator is respectively connected to the power amplifier and the mixer; the transmitting antenna forms a transmitting link with the voltage-controlled oscillator through the power amplifier; the receiving antenna forms a receiving link with the mixer through the linear amplifier (a low-noise amplifier is used in this embodiment); the input ends of the two mixers are respectively connected to the voltage-controlled oscillator and the phase shifter connected to the voltage-controlled oscillator, and the output ends of the mixers are simultaneously connected to the quadrature demodulation circuit; the input end of the A / D converter is connected to the quadrature demodulation circuit, and the output end is connected to the parallel input interface of the DSP module 13. The radar radio frequency mode is frequency-modulated continuous wave, with a working frequency of 60 - 64 GHz. It obtains spatial information through the MIMO antenna array and measures the surface displacement of the multi-target chest through beamforming technology. The voltage-controlled oscillator emits electromagnetic waves of a specified waveform according to the initialization instructions and radio frequency parameters sent by the microcontroller module 11. After being reflected by the target chest surface, the electromagnetic waves are captured by the receiving antenna, down-converted by the mixer to obtain the difference frequency component, and the I / Q signal is demodulated by arctangent as the intermediate frequency signal, and then sent to the parallel input interface of the DSP module 13 after AD conversion.

[0054] Preferably, the DSP module 13 is composed of a low-power DSP TI TMS320C6748 and its peripheral circuits; the DSP module 13 executes corresponding algorithms according to the instructions sent by the controller module 11 and sends the processing results to the DMA channel of the controller module 11. The DSP module 13 is used to accelerate the radar raw data signal processing algorithms, including algorithms such as distance dimension FFT, Doppler dimension FFT, angle dimension FFT, beamforming, point cloud generation, target point cloud clustering, Kalman filtering, and variational mode decomposition.

[0055] Preferably, in step 1, the FPGA neural network inference module 14 consists of a Xilinx Zynq UltraScale+ field programmable gate array and its peripheral circuits. The FPGA neural network inference module 14 is mainly used to deploy a feature extraction neural network model based on the LSTM model, find the features of the input vital sign signals and return the respiratory rate, heart rate, and fall detection, and send the output vital sign information, abnormal conditions, and target ID back to the microcontroller module 11.

[0056] Preferably, the communication module 15 consists of the Espressif WiFi module ESP32-S3-WROOM-1U-N16R8 and its peripheral circuits. After receiving the initialization instruction from the microcontroller module 11, the WiFi communication function of the communication module 15 is activated, and Mesh self-organizing networking is performed to automatically scan all available edge sensing nodes 1 within the signal range and form a network. After successful networking, all edge sensing nodes 1 can communicate with each other, exchange information, and achieve multi-node collaborative monitoring. When the target enters an occlusion causing a non-line-of-sight situation, the communication module 15 sends the relay status quantity sent by the microcontroller module 11 to the edge sensing node 1 with the specified IP to achieve continuous monitoring. In addition, the node network will find the optimal path to transmit the monitoring data to the host computer 2.

[0057] Preferably, the power supply module 16 includes a lithium battery, a DC power supply circuit, and a voltage stabilization circuit; it is powered by a power adapter and powered by the lithium battery when the power grid is unstable or powered off to achieve all-weather monitoring of vital signs.

[0058] The host computer 2 is installed with software; the software is a data visualization display and management software written based on C# and WPF, which parses and statistics the information of the edge sensing node 1 to implement the data display and statistics functions. After the host computer 2 runs, it connects to the gateway node or the remote server node, establishes a TCP connection through the Socket interface, receives the JSON format data sent by all edge sensing nodes 1, and parses it into data such as target ID, target vital signs, target abnormal status, and target location. The host computer 2 includes a data transmission and parsing module, a node monitoring module, a data storage and reading module, a data statistics and display module, and an abnormal alarm module. Each module continuously works after the host computer 2 runs to implement the data display and statistics functions. Among them, the data reception and parsing module is responsible for establishing a TCP connection with the root node through the Socket, for sending control instructions and receiving the JSON data sent by the node, and broadcasting the parsed data to the interface; the node monitoring module is used to monitor the working status of each node; the data storage and reading module is used to save the received monitoring data according to the target ID, and can read the saved file to implement the historical record playback; the data statistics and display module is used to display the target ID, location, vital signs, and abnormal status in the three-dimensional interface, and statistically analyze the current and historical status of the target in an intuitive way for the user to analyze; the abnormal alarm module sends an alarm to the interface after receiving the abnormal value and abnormal status returned by a certain target, or sends a reminder to the user by text message.

[0059] The present invention also provides a method for monitoring the motion state and vital signs by multi-unit radar cooperation (hereinafter referred to as the method), which is characterized in that the method includes the following steps:

[0060] Step 1, build the monitoring system;

[0061] Step 2, first, the radar sensor module 12 dynamically adjusts the measurement parameters according to the instructions issued by the microcontroller module 11, emits signals and receives echo signals in the way of frequency-modulated continuous wave; then the radar sensor module 12 completes the target information acquisition through mixing and sampling processing, and obtains a three-dimensional data matrix composed of the original ADC data (in this embodiment, it is the RadarCube three-dimensional data matrix); subsequently, in the DSP module 13, the three-dimensional data matrix is sequentially subjected to distance dimension FFT (realize distance calculation), Doppler dimension FFT (analyze velocity information), angle dimension FFT combined with beamforming algorithm (complete spatial azimuth estimation), and a three-dimensional feature data set including target distance, angle and velocity is output;

[0062] Step 3: Input the 3D feature dataset generated in Step 2 into the DSP module 13 for multi-level preprocessing, and output the primary target motion trajectory. Then, based on the angle information in the primary target motion trajectory, perform beamforming using the minimum variance distortionless response algorithm to enhance the target direction signal. On the basis of the original 3D feature dataset, add the fourth-dimensional target thoracic micro-motion feature to form a primary four-dimensional feature dataset containing the target thoracic surface displacement information.

[0063] Preferably, in Step 3, the multi-level preprocessing is specifically as follows: First, eliminate environmental interference through adaptive noise reduction filtering, and use the CFAR constant false alarm rate detection algorithm to identify the candidate target coordinates. Then, convert the triple of the distance, angle, and speed of the candidate target into 3D space coordinates to generate the initial point cloud. Next, separate the independent target point cloud clusters through the DBSCAN density clustering algorithm and establish a multi-target tracking queue. Finally, connect to the Kalman prediction filter to perform temporal prediction on the target space coordinates, effectively suppressing the measurement noise jitter and compensating for the trajectory interruption caused by short-term occlusion, and output the primary target motion trajectory.

[0064] Step 4: In order to construct a multi-unit collaborative monitoring system to achieve all-weather wide-area monitoring and analysis of the human health status and abnormal status, the microcontroller module 11 performs multi-dimensional analysis on the primary four-dimensional feature dataset obtained in Step 3 to establish a target occlusion processing and monitoring relay mechanism. Then, process the primary target motion trajectory according to the target occlusion processing and monitoring relay mechanism, and output a stable and continuous four-dimensional feature dataset.

[0065] Preferably, in Step 4, the target occlusion processing and monitoring relay mechanism is specifically as follows:

[0066] When the target does not update its space coordinates for 30 consecutive frames, the system calculates the motion vector of the target according to the position sequence of the last 30 frames before the target disappears, and judges its position relationship with the field of view boundary by the following conditions:

[0067] When the target disappears within the 5% area of the field of view boundary, and a new target with similar motion characteristics to the disappearing target is detected within the 5% area of its field of view boundary by the adjacent edge sensing node 1 within 500 milliseconds, the microcontroller module 11 determines that the target triggers the out-of-line-of-sight room switching behavior, and then triggers the monitoring relay mechanism. Through the communication between the edge sensing nodes 1, synchronize the target position information and vital sign information, so as to achieve seamless handover of the target tracking task, and the next edge sensing node 1 is responsible for monitoring the target.

[0068] When the target disappears within the internal area of the field of view, or disappears within the 5% area of the field of view boundary but does not appear within the field of view of the adjacent edge sensing node 1 within 500 milliseconds, the system marks it as a temporary occlusion state and activates the hybrid prediction mechanism: In terms of target position prediction, the DSP module 13 calls the Kalman filter to perform kinematic extrapolation based on the target's historical speed and acceleration. The prediction time window is 1 second to construct motion prediction data; in terms of vital sign prediction, the FPGA neural network inference module 14 outputs the predicted values of the target's vital signs by analyzing the target's vital sign pattern and based on the target's historical vital sign data to construct vital sign prediction data; the motion prediction data and the vital sign prediction data together form the target prediction data group for future target ID re-binding;

[0069] When the target reappears in the field of view, the system triggers the target ID re-binding mechanism: The target prediction data group is input into the microcontroller module 11 for ID re-binding recognition. The microcontroller module 11 calculates the difference between the ratio of the distance d between the position of the reappearing target and the position predicted by the Kalman filter and the adaptive threshold d t and 1, where d t is the threshold distance calculated based on the target's historical maximum speed and the tolerance given by the user to verify the spatial prediction consistency of the target; at the same time, the microcontroller module 11 constructs a vital sign parameter vector through the historical vital sign data and the prediction data. This vector consists of heart rate, respiratory rate, heart rate variability, heart beat signal amplitude, and respiratory signal amplitude, and calculates the Euclidean distance similarity with the vital sign parameter vector of the reappearing target to verify the vital sign similarity; then the microcontroller module 11 performs a decision fusion mechanism based on the above two similarity parameters. By adding the value obtained by multiplying the spatial prediction consistency parameter by the user weight α and the value obtained by multiplying the vital sign similarity parameter by the user weight β, a weighted scoring model is obtained; when the score is greater than 0.85, it is confirmed that the new target is the original target and the vital sign data is continuously updated; otherwise, it is marked as a new target and a new ID file is created.

[0070] Step 5, Heartbeat and respiration feature extraction and fall detection feature extraction;

[0071] In the heartbeat and respiration feature extraction, first, based on the target chest micro-motion feature in the four-dimensional feature dataset output in Step 4, variational mode decomposition is performed in the DSP module 13 to obtain several intrinsic mode functions; in the decomposition result, if there is a mode function with energy concentrated in the respiration frequency band of 0.1 - 0.8 Hz or the heartbeat frequency band of 1 - 3 Hz and a frequency domain sidelobe suppression ratio ≥ 20 dB, it is determined as a valid vital sign signal; otherwise, it is determined as a low signal-to-noise ratio signal that does not meet the conditions and is input into the FPGA neural network inference module 14 for deep feature mining to obtain vital sign parameters, and then target motion influence correction is performed on them to obtain a stable vital sign signal;

[0072] In the fall detection feature extraction, the four-dimensional feature dataset output in step 4 is subjected to a differential operation through the DSP module 13 to extract the vertical velocity component of the human body centroid, and a two-layer detection mechanism based on kinematic criteria is constructed to determine whether a fall event occurs;

[0073] Preferably, in step 5, the variational mode decomposition is specifically: by setting the modal number constraint condition (3 to 4 intrinsic mode functions) and the adaptive noise tolerance parameter (the signal-to-noise ratio threshold is set to 15 dB), the vital sign signal components with clear physical meanings are extracted.

[0074] Preferably, in step 5, an LSTM neural network with a hierarchical feature extraction architecture is installed in the FPGA neural network inference module 14; the LSTM neural network consists of a shallow spatio-temporal perception module, a deep feature coupling module, and a global feature fusion module;

[0075] The LSTM neural network model is jointly trained based on 500 groups of synchronously collected millimeter-wave radar chest displacement data and the measured data of a medical-grade chest belt sensor. The network establishes a non-linear mapping relationship between the signal time-frequency domain features and physiological parameters, and effectively extracts the heartbeat modulation features in the respiratory harmonic components.

[0076] The shallow spatio-temporal perception module is composed of a Bi-GRU (bidirectional gated recurrent unit) and an ST-Conv (spatio-temporal convolutional layer). Among them, the Bi-GRU adopts a 64-unit structure, captures the short-term correlation of the signal through the tanh activation function and the SiLU gating mechanism, and the ST-Conv layer is configured with 5 groups of parallel time-domain and frequency-domain convolutional kernel groups, and uses cross-channel max pooling to generate spatio-temporal joint features;

[0077] The deep feature coupling module consists of a two-layer LSTM and a phase synchronization unit. Among them, the first layer of LSTM uses a 128-unit unidirectional LSTM to extract the long-range temporal dependence features of the radar signal, and the second layer of LSTM captures the bidirectional context information through a 64-unit bidirectional LSTM. The features are passed between the two layers through skip connections; the phase synchronization unit directly analyzes the phase information of the radar signal, calculates the instantaneous phase difference between adjacent time series, dynamically fuses it with the hidden state of the second-layer bidirectional LSTM, and uses the phase difference to weight-adjust the LSTM hidden state. When the phase of the radar signal mutates, the heartbeat feature response at the corresponding moment of the LSTM is amplified by multiplication, and the noise interference in the phase-stable interval is suppressed, thereby improving the detection accuracy of weak heartbeat signals in radar perception;

[0078] The global feature fusion module uses the standard self-attention mechanism to dynamically weight the feature sequence after phase synchronization processing. By calculating the three-way attention weights of the time-frequency domain features and the hidden state, a non-linear correlation mapping between the features and the heart rate and respiratory rate is established in the time-frequency dimension. The module focuses on the phase mutation points in the heartbeat harmonic frequency band, the low-frequency oscillation mode of the respiratory envelope, and the fundamental frequency stability characteristics through the multi-head attention mechanism. Finally, the weighted and aggregated features are compressed into a 128-dimensional pulse-coded feature vector. The first 64 dimensions encode the time-varying characteristics of the heartbeat rhythm, and the last 64 dimensions map the modulation mode of the respiratory waveform. In the vital sign information output stage, the 128-dimensional pulse-coded feature vector generated by the global feature fusion module is input into the dual-branch fully connected regression layer. The respiratory rate branch outputs the respiratory rate prediction value through a fully connected network with a 64-32-1 structure, and the heart rate branch uses a bottleneck structure designed as 32-16-1 to output the heart rate prediction value. Subsequently, the global feature fusion module synchronously receives the neural network prediction value, the Kalman filter prediction value, and the variational mode decomposition fundamental frequency parameter, and dynamically adjusts the fusion weights of the three based on the real-time signal quality index to generate the vital sign parameters. After the output result is smoothed by median filtering with a 1-second window length, it is uploaded to the host computer 2 at a frame rate of 30 Hz.

[0079] Preferably, in step 5, the specific steps of deep feature mining are as follows:

[0080] S51. Multi-modal signal preprocessing: The target chest surface displacement information in the primary four-dimensional feature dataset obtained in step 3 is used to generate 256-point signal segments through a sliding window. Synchronous time-frequency analysis is performed on each signal segment, and a time-frequency diagram (frequency resolution 0.1 Hz, time window length 1 second) is generated using the complex Morlet wavelet transform and spliced with the first-order difference waveform of the original signal to form a two-dimensional feature matrix. After normalization, it forms a neural network input tensor of 128×128×2. Then, the tensor is input into the shallow spatio-temporal perception module to generate spatio-temporal joint features;

[0081] S52. LSTM feature extraction: The spatio-temporal joint features are input into the deep feature coupling module. First, the long-range temporal dependence features of the radar signal are extracted through the first layer of LSTM, and the bidirectional context is captured in the second layer of LSTM and fused with the first layer through skip connections. Then, the phase mutation intervals caused by the heartbeat signal within the respiratory cycle are distinguished, and the heartbeat feature response is amplified by multiplication in the phase mutation intervals, and the noise is suppressed in the non-phase mutation intervals to obtain the phase-modulated feature sequence;

[0082] S53. Cross-modal feature fusion and calibration: A global feature fusion module is used to dynamically weight the phase-modulated feature sequence to generate a 128-dimensional pulse-coded feature vector; then a dual-branch regression output is adopted. The pulse-coded feature vector is input into a fully-connected regression layer to output the heart rate and respiratory rate parameter values; then the neural network prediction value, the Kalman filter prediction value, and the variational mode decomposition fundamental frequency parameters are dynamically fused, and the weights of the three are adjusted according to the real-time signal quality index to generate vital sign parameters.

[0083] Preferably, in step 5, the correction method for the influence of the target action is as follows:

[0084] For the measurement distortion caused by the target action (such as limb blocking the chest, rapid turning, etc.), the correction method realizes the suppression of the influence of the target action through multi-dimensional feature joint analysis: First, point cloud deformation analysis is used to identify abnormal motion frames and trigger an abnormal motion frame compensation mechanism. When the action interference ends, the system executes an iterative calibration method to finally obtain a stable vital sign signal;

[0085] The method for identifying abnormal motion frames is: When the acceleration value of the target point at the chest in a certain frame exceeds 6 m / s 2 it is determined as an action interference event and marked as an abnormal motion frame;

[0086] The abnormal motion frame compensation mechanism is: During the abnormal motion frame, the DSP module 13 constructs a Kalman filter state space model (the state variables include heart rate, respiratory rate, and their first derivatives) based on the effective vital sign data of the previous 30 seconds, dynamically adjusts the prediction weight through the time-varying process noise covariance matrix, and maintains the vital sign parameters according to the autoregressive prediction result of the historical data; and during the abnormal motion frame, the physiological parameter fluctuation threshold for this period is automatically relaxed to avoid false alarms;

[0087] The iterative calibration method after the abnormal motion frame ends is: First, calculate the residual sequence between the Kalman prediction value and the actual measurement value during the interference period, update the observation noise matrix of the Kalman filter through backpropagation, and then perform Gaussian weighted smoothing on the current vital sign parameters with a 15-second sliding window baseline, forcing the outliers generated by the action interference to regress to the baseline median.

[0088] Preferably, in step 5, the double-layer detection mechanism is specifically: First, calculate the average vertical velocity of 5 consecutive frames. When it exceeds 65% of the theoretical free-fall acceleration of 9.8 m / s 2 the initial screening is triggered; then the trajectory is verified in combination with the height change rate (window length 0.5 seconds). If it simultaneously satisfies that the height drops by more than 40% of the height within 3 seconds and the velocity curve presents a parabola-like characteristic, it is determined as a fall event.

[0089] Step 6: Transmit the vital sign signals and whether a fall event has occurred obtained in Step 5 to the host computer 2 for display and statistics.

[0090] Where the present invention is not described applies to the prior art.

Claims

1. A multi-unit radar coordinated motion state and vital signs monitoring system, characterized in that: The system comprises an edge sensor node (1) and a host computer (2) which are communicatively connected; The edge sensor node (1) has complete wireless sensing, signal analysis and communication capabilities for vital signs; the host computer (2) receives information such as target ID, target location, target vital signs and target abnormal status sent by the edge sensor node (1); The edge sensor node (1) is composed of a microcontroller module (11), a radar sensor module (12), a DSP module (13), an FPGA neural network reasoning module (14), a communication module (15) and a power module (16); the microcontroller module (11) is respectively connected to the radar sensor module (12), the DSP module (13), the FPGA neural network reasoning module (14) and the communication module (15); the radar sensor module (12) is connected to the DSP module (13); the DSP module (13) is connected to the FPGA neural network reasoning module (14); and the power module (16) supplies power to the edge sensor node (1).

2. A multi-unit radar coordinated motion state and vital sign monitoring method, characterized in that: The method comprises the following steps: Step 1: construct the monitoring system described in claim 1; Step 2: First, the radar sensor module (12) dynamically adjusts the measurement parameters according to the instructions issued by the microcontroller module (11), transmits signals and receives echo signals; then, the radar sensor module (12) completes the target information acquisition and obtains a three-dimensional data matrix composed of raw ADC data; then, the DSP module (13) outputs a three-dimensional feature data set including the target distance, angle and speed; Step 3: input the three-dimensional feature data set generated in step 2 into the DSP module (13) to perform multi-level preprocessing and output a primary target motion trajectory; then, according to the angle information in the primary target motion trajectory, beam forming is performed based on the minimum variance distortion-free response algorithm to enhance the target direction signal and form a primary four-dimensional feature data set containing the target chest surface displacement information; Step 4, the microcontroller module (11) performs multi-dimensional analysis on the primary four-dimensional feature data set obtained in step 3 to establish a target occlusion processing and monitoring relay mechanism; then processes the primary target motion trajectory according to the target occlusion processing and monitoring relay mechanism to output a stable and continuous four-dimensional feature data set; Step 5: Extract heartbeat and breathing features and fall detection features; In the extraction of heartbeat and breathing features, firstly, based on the target chest micro-motion features in the four-dimensional feature data set outputted in step 4, variational mode decomposition is performed in the DSP module (13) to obtain a number of intrinsic mode functions; in the decomposition results, if there is a mode function whose energy is concentrated in the respiratory frequency band of 0.1 to 0.8 Hz or the heartbeat frequency band of 1 to 3 Hz, and the frequency domain sidelobe suppression ratio is ≥ 20 dB, it is determined to be a valid vital sign signal; otherwise, it is determined to be a low signal-to-noise ratio signal that does not meet the conditions, and is input into the FPGA neural network inference module (14) for deep feature mining to obtain vital sign parameters, which are then corrected for the influence of target movements to obtain a stable vital sign signal; In the fall detection feature extraction, the four-dimensional feature data set output in step 4 is subjected to differential operation by the DSP module (13) to extract the vertical velocity component of the center of mass of the human body, and a double-layer detection mechanism based on kinematic criteria is constructed to determine whether a fall event occurs; Step 6: The vital sign signal obtained in step 5 and whether a fall event has occurred are transmitted to the host computer (2) for display and statistics.

3. The multi-unit radar coordinated motion state and vital sign monitoring method according to claim 2 is characterized in that: In step 2, the distance dimension FFT, the Doppler dimension FFT, and the angle dimension FFT combined with the beamforming algorithm are sequentially performed on the three-dimensional data matrix in the DSP module (13).

4. The multi-unit radar coordinated motion state and vital sign monitoring method according to claim 1, characterized in that: In step 3, the multi-level preprocessing is as follows: first, environmental interference is eliminated through adaptive noise reduction filtering, and the CFAR constant false alarm rate detection algorithm is used to identify the coordinates of the candidate targets; then the triplet of the distance, angle, and speed of the candidate targets is converted into three-dimensional spatial coordinates to generate an initial point cloud; then the DBSCAN density clustering algorithm is used to separate independent target point cloud clusters and establish a multi-target tracking queue; finally, the Kalman prediction filter is connected to perform time series prediction on the target space coordinates, effectively suppressing the measurement noise jitter and compensating for the trajectory interruption caused by short-term occlusion, and outputting the primary target motion trajectory.

5. The multi-unit radar coordinated motion state and vital sign monitoring method according to claim 1, characterized in that: In step 4, the target occlusion processing and monitoring relay mechanism is specifically: When the target does not update its spatial coordinates for 30 consecutive frames, the system calculates the target's motion vector based on the position sequence of the last 30 frames before the target disappears, and determines its positional relationship with the field of view boundary based on the following conditions: When the target disappears within the 5% area of ​​the field of view boundary, and the adjacent edge sensor node (1) detects a new target with similar motion characteristics to the disappeared target appearing within the 5% area of ​​its field of view boundary within 500 milliseconds, the microcontroller module (11) determines that the target triggers the out-of-line-of-sight room switching behavior, and then triggers the monitoring relay mechanism. Through the communication between the edge sensor nodes (1), the target position information and vital sign information are synchronized, thereby realizing seamless target tracking task handover, and the next edge sensor node (1) is responsible for the monitoring of the target; When the target disappears in the inner area of ​​the field of view, or disappears in the 5% area of ​​the boundary of the field of view but does not appear in the field of view of the adjacent edge sensor node (1) within 500 milliseconds, the system marks it as a temporary occlusion state and starts a hybrid prediction mechanism: in terms of target position prediction, the DSP module (13) calls the Kalman filter to perform kinematic extrapolation based on the target's historical velocity and acceleration, with a prediction time window of 1 second, to construct motion prediction data; in terms of vital sign prediction, the FPGA neural network reasoning module (14) analyzes the target's vital sign pattern, outputs the predicted value of the target's vital sign based on the target's historical vital sign data, and constructs vital sign prediction data; the motion prediction data and the vital sign prediction data together constitute a target prediction data group, which is used for future target ID rebinding; When the target reappears in the field of view, the system triggers the target ID rebinding mechanism: the target prediction data set is input into the microcontroller module (11) for ID rebinding identification, and the microcontroller module (11) calculates the distance d between the reappearing target position and the Kalman filter predicted position and the adaptive threshold d t The difference between the ratio and 1, d t The spatial prediction consistency of the target is verified by a threshold distance calculated based on the target's historical maximum speed and a user-given tolerance; at the same time, the microcontroller module (11) constructs a vital sign parameter vector through historical vital sign data and predicted data, the vector consisting of heart rate, respiratory rate, heart rate variability, heartbeat signal amplitude and respiratory signal amplitude, and performs Euclidean distance similarity calculation with the vital sign parameter vector of the reproduced target to verify the vital sign similarity; then the microcontroller module (11) performs a decision fusion mechanism based on the above two similarity parameters, and obtains a weighted scoring model by multiplying the spatial prediction consistency parameter by the value of the user weight α and multiplying the vital sign similarity parameter by the value of the user weight β; when the score is greater than 0.85, the new target is confirmed to be the original target and the vital sign data continues to be updated; otherwise, it is marked as a new target and a new ID file is created.

6. The multi-unit radar coordinated motion state and vital sign monitoring method according to claim 1, characterized in that: In step 5, the variational mode decomposition specifically includes: extracting the vital sign signal components with clear physical meanings by setting mode quantity constraints and adaptive noise tolerance parameters.

7. The multi-unit radar coordinated motion state and vital sign monitoring method according to claim 1, characterized in that: In step 5, an LSTM neural network using a hierarchical feature extraction architecture is installed in the FPGA neural network inference module (14); the LSTM neural network is composed of a shallow spatiotemporal perception module, a deep feature coupling module, and a global feature fusion module; The shallow spatiotemporal perception module is composed of Bi-GRU and ST-Conv. The Bi-GRU adopts a 64-unit structure and captures the short-term correlation of the signal through the tanh activation function and SiLU gating mechanism. The ST-Conv layer is configured with 5 groups of parallel time-frequency convolution kernels and uses cross-channel maximum pooling to generate spatiotemporal joint features. The deep feature coupling module consists of a two-layer LSTM and a phase synchronization unit. The first layer of LSTM uses a 128-unit unidirectional LSTM to extract the long-range temporal dependency features of the radar signal. The second layer of LSTM uses a 64-unit bidirectional LSTM to capture bidirectional context information. The features are transferred between the two layers through jump connections. The phase synchronization unit directly analyzes the phase information of the radar signal, calculates the instantaneous phase difference of adjacent time series, dynamically fuses it with the hidden state of the second-layer bidirectional LSTM, and uses the phase difference to weight the LSTM hidden state. When the radar signal phase changes suddenly, the heartbeat feature response of the LSTM at the corresponding moment is amplified by the product, suppressing the noise interference in the phase stable interval, thereby improving the detection accuracy of weak heartbeat signals in radar perception; The global feature fusion module uses a standard self-attention mechanism to dynamically weight the feature sequence after phase synchronization processing, and establishes a nonlinear correlation mapping between the feature and the heart rate and respiratory rate in the time-frequency dimension by calculating the three-way attention weights of the time-frequency domain features and the hidden state; the module uses a multi-head attention mechanism to focus on the phase mutation points of the heartbeat harmonic frequency band, the low-frequency oscillation mode of the respiratory envelope, and the fundamental frequency stability characteristics, and finally compresses the weighted aggregated features into a 128-dimensional pulse coded feature vector, of which the first 64 dimensions encode the time-varying characteristics of the heartbeat rhythm, and the second 64 dimensions map the modulation mode of the respiratory waveform; in the vital sign information output stage, the 128-dimensional pulse coded feature vector generated by the global feature fusion module is input into a dual-branch fully connected regression layer: the respiratory rate branch outputs the respiratory rate prediction value through a fully connected network with a 64-32-1 structure, and the heart rate branch uses a bottleneck structure designed as 32-16-1 to output the heart rate prediction value; Subsequently, the global feature fusion module synchronously receives the neural network prediction value, the Kalman filter prediction value and the variational mode decomposition fundamental frequency parameter, and dynamically adjusts the fusion weight of the three based on the real-time signal quality index to generate the vital sign parameters; the output result is smoothed by a median filter with a window length of 1 second and uploaded to the host computer at a frame rate of 30 Hz (2).

8. The method for monitoring motion status and vital signs using multi-unit radar collaboration according to claim 7, characterized in that: In step 5, the specific steps of deep feature mining are: S51, multimodal signal preprocessing: the target chest surface displacement information of the primary four-dimensional feature data set obtained in step 3 is used to generate 256-point signal segments through a sliding window; synchronous time-frequency analysis is performed on each signal segment, and a time-frequency graph is generated using a complex Morlet wavelet transform, and is spliced ​​with the first-order differential waveform of the original signal into a two-dimensional feature matrix, which is normalized to form a 128×128×2 neural network input tensor; the tensor is then input into a shallow spatiotemporal perception module to generate spatiotemporal joint features; S52, LSTM feature extraction: The spatiotemporal joint features are input into the deep feature coupling module. The long-range temporal dependency features of the radar signal are first extracted through the first layer of LSTM. The bidirectional context is captured in the second layer of LSTM and fused with the first layer through jump connections. Then, the phase mutation interval caused by the heartbeat signal in the respiratory cycle is distinguished. The heartbeat feature response is amplified by multiplication in the phase mutation interval, and the noise is suppressed in the non-phase mutation interval to obtain the feature sequence after phase modulation. S53. Cross-modal feature fusion and calibration: A global feature fusion module is used to dynamically weight the feature sequence after phase modulation to generate a 128-dimensional pulse coding feature vector. A dual-branch regression output is then used to input the pulse coding feature vector into a fully connected regression layer to output the heart rate and respiratory rate parameter values. The neural network prediction value, Kalman filter prediction value, and variational mode decomposition fundamental frequency parameter are then dynamically fused, and the weights of the three are adjusted according to the real-time signal quality index to generate vital sign parameters.

9. The method for monitoring motion status and vital signs using multi-unit radar collaboration according to claim 1, characterized in that: In step 5, the correction method for the influence of the target action is as follows: first, the abnormal motion frame is identified by using point cloud deformation analysis, and the abnormal motion frame compensation mechanism is triggered. When the motion interference ends, the system executes an iterative calibration method to finally obtain a stable vital sign signal; The method for identifying abnormal motion frames is: when the acceleration value of the target point at the chest cavity of a certain frame exceeds 6m / s 2 When , it is determined as a motion interference event and marked as an abnormal motion frame; The abnormal motion frame compensation mechanism is as follows: during the abnormal motion frame, the DSP module (13) constructs a Kalman filter state space model based on the valid vital sign data of the previous 30 seconds, dynamically adjusts the prediction weight through the time-varying process noise covariance matrix, and maintains the vital sign parameters according to the autoregressive prediction results of the historical data; and during the abnormal motion frame, the physiological parameter fluctuation threshold of the period is automatically relaxed to avoid false alarms; The iterative calibration method after the abnormal motion frame ends is as follows: first, the residual sequence between the Kalman prediction value and the actual measurement value during the interference period is calculated, the observation noise matrix of the Kalman filter is updated by back propagation, and then the current vital sign parameters are Gaussian weighted smoothed with a 15-second sliding window baseline to force the outliers generated by the motion interference to regress to the baseline median.

10. The multi-unit radar coordinated motion state and vital sign monitoring method according to claim 1, characterized in that: In step 5, the double-layer detection mechanism is as follows: first, the vertical velocity average of 5 consecutive frames is calculated, and when it exceeds 9.8m / s 2 The initial screening is triggered when the theoretical free fall acceleration reaches 65%; then the trajectory is verified in combination with the height change rate. If the height drop exceeds 40% of the body height within 3 seconds and the speed curve presents a parabolic feature, it is determined to be a fall event.

Images