A dual-mode human perception system and method based on millimeter wave radar

Abstract

The application discloses a kind of dual-mode human perception system and method based on millimeter wave radar, belong to intelligent perception field.The application adopts double-path parallel processing architecture, macro motion recognition branch is based on point cloud centroid trajectory dynamic adjustment PointNet++ neighborhood radius, realize high time resolution macro motion classification;Microscopic vital sign monitoring branch uses adaptive phase unwrapping and double-channel filtering, extracts respiratory rate and heart rate.Decision level fusion module carries out cross validation through dynamic weight distribution and bidirectional constraint rule, and outputs user state.The application is based on single radar platform, stationary state detection rate reaches more than 98%, respiratory rate error ±0.4 times / minute, heart rate error ±1.85bpm, fall detection response time is ≤450ms, system false alarm rate is ≤3%, realizes low cost, high reliability dual-mode fusion perception.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing technology, specifically relating to a dual-modal human body sensing system and method based on millimeter-wave radar. Background Technology

[0002] Millimeter-wave radar, as a non-contact sensor, has advantages such as strong penetration, good privacy protection, and immunity to light, showing great application potential in fields such as smart homes, smart elderly care, and security monitoring.

[0003] Currently, human body monitoring technology based on millimeter-wave radar mainly falls into two categories. The first focuses on the recognition and analysis of macroscopic human movements, with typical applications such as fall detection. The second focuses on the extraction of microscopic physiological signals, such as respiratory rate and heart rate monitoring. In terms of movement recognition, existing technologies disclose fall detection methods, devices, and millimeter-wave radar equipment based on millimeter-wave radar. These methods extract motion features and convert them into point cloud data, then use clustering algorithms to analyze gait information to determine fall risk. In terms of vital sign monitoring, existing technologies, such as a millimeter-wave radar device, method, and system for vital sign monitoring, utilize FMCW radar to acquire chest cavity reflection signals and extract phase information to obtain vital sign parameters.

[0004] However, the above-mentioned existing technical solutions have the following shortcomings in practical applications:

[0005] 1. The problem of failing to detect human activity in static states and being unable to accurately distinguish action types exists: Traditional constant false alarm rate (CFAR) detection algorithms rely on Doppler velocity thresholds to extract dynamic targets. In engineering practice and conventional signal processing configurations of general millimeter-wave radars, the basic velocity threshold of the system is usually set at 0.1-0.2 m / s. Since the radial motion velocity of the chest wall caused by physiological activities such as breathing and heartbeat is much lower than this threshold, when the subject is in a static state such as sitting or lying down, the existing system cannot effectively distinguish between "no one" and "someone but stationary" states. It is very easy to misjudge a stationary human body as static clutter and filter it out. This is the fundamental bottleneck of existing technology in the perception of stationary human bodies.

[0006] 2. Single-function devices cannot meet the actual needs of comprehensive monitoring: Most existing millimeter-wave radar human body monitoring products only support a single function: either focusing on motion recognition (such as fall detection) or only monitoring vital signs (such as respiratory and heart rate). If both functions need to be implemented simultaneously, two independent devices are usually required. This not only increases hardware costs, significantly increases installation complexity, increases maintenance difficulty, and doubles power consumption, but more importantly, it presents a data fusion problem: the time synchronization of the two independent devices is difficult to match precisely, resulting in motion recognition data and vital sign data not being accurately aligned on the timeline, which brings difficulties to subsequent multimodal data analysis.

[0007] 3. Conflict in Timing Parameters Between Action Recognition and Vital Sign Monitoring: During project development, it was discovered that the signal processing parameter requirements of action recognition and vital sign monitoring tasks are fundamentally contradictory. Action recognition requires high temporal resolution (sampling rate ≥ 20 FPS) to capture rapid actions (such as falls, the entire process of which takes only 0.5-1 seconds); while vital sign extraction requires a longer time window (≥ 10 seconds) for spectral analysis to obtain sufficient frequency resolution to distinguish between respiratory and heart rate. Using the same set of signal processing parameters to process both tasks simultaneously, verification results show that: if the real-time performance of action recognition is prioritized, the frequency resolution of vital sign monitoring is insufficient, leading to large estimation errors in respiratory and heart rate; if the accuracy of vital sign monitoring is prioritized, the response delay of action recognition is too large, failing to meet the real-time requirements (< 500 ms) of fall detection.

[0008] Therefore, there is an urgent need for a dual-modal perception scheme that can be based on a single radar platform, resolve timing parameter conflicts, and effectively integrate macroscopic action and microscopic vital sign information. Summary of the Invention

[0009] To address the problems of static human detection failure, limited functionality, and conflicting timing parameters in existing technologies, this invention provides a dual-modal human perception system and method based on millimeter-wave radar. This invention proposes a dual-path parallel processing architecture, enabling the action recognition branch and the vital sign monitoring branch to operate completely independently, each employing optimal signal processing parameters. Simultaneously, it introduces a static state detection technology based on phase unwrapping and a decision-level fusion strategy to achieve dual-modal fusion perception on a single radar platform. This invention can accurately distinguish between "unmanned" and "manned stationary" states, and simultaneously achieves dual-modal fusion perception of macroscopic action recognition and microscopic vital sign monitoring on a single radar platform, thereby reducing system cost and complexity. It also resolves the timing parameter conflicts between action recognition and vital sign monitoring, ensuring the real-time performance and measurement accuracy of both. Furthermore, this invention effectively reduces the system's false alarm rate through a decision-level fusion strategy.

[0010] This invention is achieved through the following technical solution:

[0011] A dual-modal human body sensing system based on millimeter-wave radar, comprising:

[0012] The radar signal acquisition and preprocessing module is used to configure the operating parameters of the millimeter-wave radar, transmit FMCW signals and receive echoes; mix the echoes with the local transmitted signals to generate intermediate frequency signals; perform ADC sampling on the intermediate frequency signals, and sequentially perform Fourier transforms along the fast and slow time dimensions to obtain the range spectrum and Doppler spectrum, respectively; use the CA-CFAR algorithm to perform target detection on the range spectrum and Doppler spectrum, and extract point cloud data.

[0013] The dual-path parallel processing module is connected to the radar signal acquisition and preprocessing module. Internally, it contains two independent and parallel branches: a macroscopic motion recognition branch and a microscopic vital sign monitoring branch. The macroscopic motion recognition branch receives point cloud data, extracts the centroid trajectory, dynamically adjusts the neighborhood radius based on the standard deviation of the centroid trajectory within a time window, and uses a PointNet++ network for feature extraction and classification, outputting the motion recognition result and confidence level. The microscopic vital sign monitoring branch extracts the complex signal and its instantaneous phase of the human chest cavity region from the intermediate frequency signal, sequentially performing phase extraction, phase unwrapping, phase difference, bandpass filtering, and FFT spectral analysis to extract respiratory rate and heart rate.

[0014] The decision-level fusion module is connected to the macroscopic motion recognition branch and the microscopic vital sign monitoring branch, respectively. It is used to receive motion recognition results, confidence level, respiratory rate and heart rate, calculate dynamic fusion weights and execute bidirectional constraint rules, perform cross-validation and comprehensive judgment according to the preset rule base, and output user status and alarm signals.

[0015] On the other hand, the present invention also provides a monitoring method for a millimeter-wave radar dual-mode human body sensing system, which specifically includes the following steps:

[0016] S1: Collect echo signals from the target area using millimeter-wave radar, mix the echoes with the local FMCW to generate intermediate frequency signals, and extract point cloud data P after preprocessing;

[0017] S2: Initiate dual-path parallel processing:

[0018] S21: Perform macroscopic action recognition on the point cloud data P, including extracting the centroid trajectory and using an improved PointNet++ network for feature extraction and classification; wherein, during the feature extraction process, the neighborhood radius of the network is dynamically adjusted according to the standard deviation of the centroid trajectory within the time window, and the action recognition result and confidence level are output.

[0019] S22: Perform microscopic vital sign monitoring on the intermediate frequency signal, including extracting the complex signal and its instantaneous phase of the human chest cavity region, and sequentially performing phase extraction, phase unwrapping, phase difference, bandpass filtering and FFT spectrum analysis, and extracting respiratory rate and heart rate through Fourier transform spectrum analysis;

[0020] S3: The decision-level system fuses the action recognition results with the breathing frequency and heart rate, calculates the dynamic fusion weights and executes bidirectional constraint rules, performs cross-validation and comprehensive judgment according to the preset rule base, and outputs the final user status.

[0021] Furthermore, step S1 specifically includes the following:

[0022] S11. Perform ADC sampling on the received intermediate frequency signal to obtain a discrete time series. ;

[0023] S12, along the fast time dimension for each frequency modulation pulse Perform a Fast Fourier Transform on each sampling point to obtain the distance spectrum. Next, along the slow time dimension, examine all frames within a frame. Perform a Fast Fourier Transform on the data from the same distance cell of each frequency-modulated pulse to obtain the Doppler spectrum. ;

[0024] S13. The distance spectrum obtained in step S12 and Doppler spectrum Combination formation distance-Doppler two-dimensional spectrum In this context, each element in the range-Doppler two-dimensional spectrum is called a range-Doppler unit, and is represented by... express, For distance cell index, For Doppler cell indexing, For signal strength; the CA-CFAR algorithm is used to perform target detection on the range-Doppler two-dimensional spectrum; for each range-Doppler cell to be detected... A reference window is set up around it, and the average signal strength of all distance-Doppler cells within the reference window is calculated. If the signal strength of the distance-Doppler unit to be detected is... If the distance-Doppler cell is determined to be the target point, then... The CA-CFAR scaling factor is used for target detection; and the distance, velocity, angle, and signal-to-noise ratio information of the target points are recorded, ultimately outputting point cloud data. ,in, For the first The three-dimensional spatial coordinates of the points For the first Radial velocity at each point For the first Signal-to-noise ratio at each point This represents the total number of target points detected in this frame.

[0025] Furthermore, step S21 specifically includes the following:

[0026] S211, The macroscopic motion recognition branch receives the point cloud data output from step S13. As input, the centroid position of each frame point is used for cloud computing. ,in, The current frame number. They are respectively The average value in three directions, thus constructing a length of Calculate the standard deviation of the centroid trajectory sequence of the frames. ,in, For the first The centroid position of the frame; for The average value of the frame centroid position vector; The Euclidean distance between the two vectors;

[0027] The PointNet++ network extracts local point cloud features from the point cloud through ball query operations, and dynamically adjusts the neighborhood search radius based on the action amplitude and point cloud density to match the search range with the point cloud distribution.

[0028]

[0029] in, The neighborhood search radius; This is the scaling factor; is the standard deviation of the centroid trajectory within the time window; Density compensation factor; This represents the total number of target points detected in this frame, i.e., the point cloud data output in step S13. The number of points in the middle; Given the average number of point clouds within the sliding window, output the dynamically adjusted neighborhood search radius. ;

[0030] S212. Based on the neighborhood search radius output in step S211 Set the neighborhood search radius The constraints are ,in, The preset minimum search radius; The maximum search radius is preset; key points are selected using the FPS algorithm, local features are extracted using an MLP network structure, and finally max pooling is used to aggregate the results, outputting the action recognition results and confidence scores. ,in, The value ranges from 0 to 1, and the larger the value, the more reliable the action recognition result.

[0031] Furthermore, step S22 specifically includes the following:

[0032] S221, The microscopic vital signs monitoring branch extracts the distance unit corresponding to the human thoracic cavity region from the intermediate frequency signal and obtains the complex signal of the distance unit. And calculate its instantaneous phase. ,in, For discrete-time indexing, For in-phase components, For orthogonal components, The imaginary unit is used; the instantaneous phase is calculated using the arctangent function in the four quadrants. , indicating time The phase value, with a range of ;

[0033] S222. First, the instantaneous phase obtained in step S221... Perform adaptive phase unwrapping; calculate the phase difference between adjacent time points. Extracting the signal-to-noise ratio of corresponding points in the human chest cavity region from point cloud data. The average value is taken as the signal-to-noise ratio of the current frame. ,in, The number of point clouds within the thoracic region is calculated; simultaneously, a normalized signal quality index is calculated. Normalize the SNR to the 0-1 range: ,in, The minimum effective signal-to-noise ratio threshold. The maximum signal-to-noise ratio (SNR) is set as the upper limit; a dynamic threshold related to the SNR is introduced. ,when When, it is determined that an event has occurred. Jump, based on the jump direction Compensation, restoration of continuous phase curve Output the continuous phase curve after unwrapping. and signal quality ;

[0034] Dynamic threshold The selection is based on the signal-to-noise ratio (SNR) and uses a continuously smooth dynamic threshold formula:

[0035]

[0036] in, For dynamic thresholds; Signal-to-noise ratio;

[0037] Next, the baseline drift was eliminated using moving average phase difference to refine the continuous phase curve after unwrapping. Applying a sliding window moving average filter, the filtered phase is obtained. Then calculate the phase difference between adjacent time points: ,in, For discrete-time indexing, The time interval depends on the signal quality. Adaptive adjustment; output phase change and signal quality ;

[0038] S223, Phase change amount output in step S222 Two filters were used for dual-channel filtering to obtain respiratory and heartbeat signals. FFT spectral analysis was then performed on the filtered respiratory and heartbeat signals, with a time window length of 10 seconds and a frequency resolution of 0.1 Hz, to distinguish the peak values ​​of respiratory frequency (0.1-0.67 Hz) and heartbeat frequency (0.8-3.0 Hz) and avoid measurement errors caused by spectral aliasing. Within the respiratory signal frequency band... Find the frequency corresponding to the spectral peak within Hz In the heartbeat signal frequency band Find the frequency corresponding to the spectral peak within Hz Converted to respiratory rate (beats / minute), heart rate (bpm), where bpm stands for beats per minute, i.e., the number of heartbeats per minute; the final output is the respiratory rate. Heart rate And transmit the signal quality calculated in step S222. Up to the decision-level fusion module.

[0039] Furthermore, step S3 specifically includes the following:

[0040] S31. Receive the action recognition result and confidence level output from the action recognition branch. Respiratory rate output from the microscopic vital signs monitoring branch Heart rate and signal quality Calculate the dynamic fusion weights:

[0041]

[0042]

[0043] in, This represents the weight of the action recognition result in the final state determination, with a value ranging from 0 to 1. is the weight of the vital sign result in the final state determination, with a value range of 0-1; is the motion recognition confidence, with a value range of 0-1; is the vital sign signal quality, with a value range of 0-1; is the duration of the current motion, is the time decay factor; is the reliability index of vital sign signals, with a value range of 0-1, and the calculation formula is: , wherein, is the number of consecutively and stably detected frames, (corresponding to 10 seconds), is the reliability weight factor;

[0044] S32, implementing a two-way constraint mechanism: when the motion recognition result is "fall" or "strenuous exercise", adjust the heart rate abnormality determination threshold from 100bpm to 150bpm, so as to avoid misjudging physiological heart rate elevation caused by exercise as an abnormality; when the vital sign signal quality and no effective breathing or heartbeat signal is detected for 30 consecutive seconds, forcibly correct the motion recognition result to the "no person" state, so as to prevent false detection caused by noise interference; perform cross-validation and comprehensive determination according to a preset rule base, perform comprehensive determination based on the motion recognition result and vital sign parameters, and output the final user state and alarm signal.

[0045] Further, in step S222, when SNR>15dB, ; when 10dB<SNR≤15dB, ; when SNR≤10dB, ;

[0046] when SNR>15dB, ms, when 10-15dB, ms, when <10dB ms.

[0047] Further, in step S223, the two filters include a respiratory filter and a heartbeat filter, wherein the passband of the respiratory filter is 0.1-0.67Hz, the attenuation at the cut-off frequency is -3dB, and the stopband attenuation is -40dB; the passband of the heartbeat filter is 0.8-3.0Hz, the attenuation at the cut-off frequency is -3dB, and the stopband attenuation is -40dB.

[0048] Further, in step S1, the millimeter-wave radar is a 60-64GHz FMCW radar, with: starting frequency: 60GHz; bandwidth: 4GHz; Chirp period: 60μs; number of Chirps per frame: 128; frame rate: 20FPS; number of ADC sampling points: 256.

[0049] Compared with the prior art, the advantages of the present invention are as follows:

[0050] 1. The dual-modal time-decoupled parallel architecture resolves the conflict between time-frequency domain characteristics:

[0051] By employing a dual-path parallel processing architecture, the motion recognition branch and the vital signs monitoring branch each operate independently with optimal parameters, overcoming the technical bottleneck of the traditional approach where high temporal resolution and high frequency resolution cannot be simultaneously achieved. Experiments show that the motion recognition response time is ≤450ms, meeting the real-time requirements of fall detection, while the vital signs monitoring frequency resolution of 0.1Hz ensures the accuracy of respiratory and heart rate measurements.

[0052] 2. Achieving dual-modal fusion perception with a single radar:

[0053] Eliminating the need for two separate devices reduces hardware costs and installation complexity. Dual-path shared front-end data acquisition achieves synchronous data fusion through an asynchronous collaborative mechanism, avoiding the time synchronization difficulties and data alignment issues present in multi-device solutions.

[0054] 3. Dynamic fusion strategy reduces false alarm rate:

[0055] The decision-level fusion module achieves cross-validation of action recognition and vital sign data through dynamic weight allocation and bidirectional constraint rules. The 15 preset rules are derived from real data statistics. Taking the rule of "falling and heart rate >100 bpm" as an example, the confidence level reaches 94%, effectively reducing the system's false alarm rate to ≤3%. Attached Figure Description

[0056] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0057] Figure 1 This is a block diagram of the overall structure of a millimeter-wave radar dual-mode human body perception system according to the present invention;

[0058] Figure 2 This is a schematic diagram of the dual-path parallel processing module of the present invention;

[0059] Figure 3 This is a flowchart illustrating the operation of the radar signal acquisition and preprocessing module of the present invention.

[0060] Figure 4 This is a flowchart of the macroscopic action recognition branch of the present invention;

[0061] Figure 5This is a flowchart of the microscopic vital sign monitoring branch of the present invention;

[0062] Figure 6 This is a flowchart of the decision-level fusion module of the present invention;

[0063] Figure 7 The diagram shows the effect of the adaptive phase unwrapping algorithm of the present invention, wherein (a) is a schematic diagram of the phase entanglement phenomenon and (b) is a schematic diagram of the effect of the adaptive phase unwrapping algorithm.

[0064] Figure 8 The figure shows the experimental results comparing the effects of the present invention with those of the prior art. (a) compares the four core performance indicators, (b) shows the comparison of heart rate detection errors under different signal-to-noise ratio conditions, (c) shows the accuracy of the decision rule, and (d) shows the comparison of action recognition response time.

[0065] Figure 9 This is a diagram showing the MIMO antenna array configuration used in this invention. Detailed Implementation

[0066] To clearly and completely describe the technical solution and its specific working process of the present invention, the specific embodiments of the present invention are as follows, in conjunction with the accompanying drawings:

[0067] Example 1:

[0068] like Figure 1 As shown, this embodiment provides a millimeter-wave radar dual-mode human perception system, which includes: a radar signal acquisition and preprocessing module, a dual-path parallel processing module, and a decision-level fusion module;

[0069] I. Radar Signal Acquisition and Preprocessing Module: This embodiment uses the TI IWR6843AOP 60GHz FMCW millimeter-wave radar evaluation module. This module operates in the 60-64GHz frequency band, features a 3-transmit 4-receive MIMO array, and can form 12 virtual channels (e.g., Figure 9 (As shown).

[0070] The millimeter-wave radar parameters are configured as follows: starting frequency 60 GHz, bandwidth 4 GHz, chirp period 60 μs, number of chirs per frame 128, frame rate 20 FPS, and ADC sampling points 256. Under this configuration, the range resolution is 3.75 cm and the velocity resolution is 0.05 m / s.

[0071] like Figure 3As shown, the workflow of the radar signal acquisition and preprocessing module is as follows: The radar transmits an FMCW signal and receives human body reflection echoes. The received echo signal is mixed with the local transmitted signal to generate an intermediate frequency (IF) signal. The IF signal is sampled by an ADC, and an FFT is performed along the fast time dimension (the sequence of sampling points within each chirp period) to obtain the range spectrum. An FFT is also performed along the slow time dimension (the sequence across chirps) to obtain the Doppler spectrum. The CA-CFAR algorithm is used to perform target detection on the range-Doppler two-dimensional spectrum and extract point cloud data P.

[0072] The specific process of CA-CFAR detection is as follows: For each range-Doppler cell to be detected, a reference window is set around it, and a guard cell is set between the cell to be detected and the reference window (i.e., the reference window does not contain the guard cell). This guard cell is used to isolate the target signal energy diffusion, prevent the target energy from leaking into the reference window and raising the noise estimation, and prevent weak targets from being missed. The average signal strength of all cells in the reference window is calculated. If the signal strength of the cell to be detected exceeds the average value plus the scaling factor, the detection is considered successful. The product of these parameters is then identified as the target point. The distance, velocity, angle, and signal-to-noise ratio of the target point are recorded, and point cloud data is output. ,in, For the first The three-dimensional spatial coordinates of the points For the first Radial velocity at each point For the first The signal-to-noise ratio at each point (calculated by the radar during CFAR detection, in dB). This represents the total number of target points detected in this frame.

[0073] II. Dual-path parallel processing module

[0074] like Figure 2 As shown, the dual-path parallel processing module includes an independent and parallel macroscopic motion recognition branch and a microscopic vital sign monitoring branch. The macroscopic branch receives the point cloud data P output in step S13 and processes it frame-by-frame in real time to ensure a high temporal resolution of 20 FPS; the microscopic branch receives the raw intermediate frequency signal x and uses a 10-second sliding window to accumulate data to achieve a frequency resolution of 0.1 Hz. The two paths work asynchronously and collaboratively through a timing decoupling mechanism, and the processing results are input into the decision-level fusion module.

[0075] (1) Macro-action recognition branch

[0076] like Figure 4 As shown, the macroscopic action recognition branch is used to receive the point cloud data P output in step S13, extract the centroid trajectory, and use the PointNet++ network for feature extraction and classification.

[0077] Specifically, the centroid position of the point cloud in each frame is calculated, and a centroid trajectory sequence with a time window length of 40 frames (corresponding to 2 seconds) is constructed. During feature extraction, a three-layer Set Abstraction structure is used, and the neighborhood radius of the network is dynamically adjusted based on the standard deviation of the centroid trajectory within the time window and the point cloud density. Let the scaling factor be... (Value range: 1.5-2.0), density compensation factor (Typical value is 0.3), based on the standard deviation of the centroid trajectory within the time window. The neighborhood radius is dynamically adjusted based on the point cloud density, using the following formula: and set constraints .in, The scaling factor has an empirical range of 1.5-2.0; where the amplitude of motion is determined by the standard deviation of the centroid trajectory within the time window. Characterization: This indicates minute movements (such as slight finger movements or head rotations), at which point cloud distribution is relatively concentrated. The time indicates a large movement (such as walking or falling), at which point the point cloud distribution is relatively dispersed; The time indicates a moderate amplitude movement (such as waving or bending over). "Point cloud density" is determined by the number of point clouds in this frame. Average number of point clouds within the sliding window The ratio represents the density of the point cloud in the current frame.

[0078] This is the density compensation factor, with a typical value of 0.3; This represents the total number of target points detected in this frame, i.e., the point cloud data output in step S13. The number of points in the middle; Within the sliding window (length) The average number of point clouds per frame. =0.3m: Minimum search radius, ensuring that sufficient local point cloud features can still be captured even during minute movements (such as slight finger movements or slight head turns); =1.5m: Maximum search radius, to prevent the search range from being too large during large movements (such as falling or walking), and to avoid a surge in computational load and environmental noise interference.

[0079] Finally, the probability distribution of 12 action categories (including standing, walking, sitting, standing up, lying down, falling, waving, jumping, squatting, bending over, running, and getting up after falling) is output through fully connected layers and Softmax layers.

[0080] (2) Microscopic vital signs monitoring branch

[0081] like Figure 5As shown, the microscopic vital signs monitoring branch is used to extract the complex signal and its instantaneous phase of the human chest cavity region from the intermediate frequency signal. The phase extraction, phase unwrapping, phase difference, bandpass filtering and FFT spectrum analysis are performed in sequence. The respiratory rate and heart rate are extracted by Fourier transform spectrum analysis.

[0082] Among them, phase unwrapping adopts an adaptive unwrapping algorithm based on the phase change rate (the effect is as follows). Figure 7 As shown), specifically, it includes: calculating the phase difference between adjacent time points. And introduce a dynamic threshold related to the signal-to-noise ratio (SNR). ;when When, it is determined that an event has occurred. Jump, based on the jump direction Compensation, restoration of continuous phase curve ;

[0083] Dynamic threshold The selection is based on the signal-to-noise ratio (SNR) and uses a continuously smooth dynamic threshold formula:

[0084]

[0085] in, For dynamic thresholds; Signal-to-noise ratio; output continuous phase curve after unwrapping. and signal quality ;

[0086] Baseline drift is eliminated by using moving average phase difference filtering. A sliding window moving average filter is applied to the unwrapped phase curve to calculate the phase difference between adjacent time points. .

[0087] right The signals were filtered using a respiratory filter (passband 0.1-0.67Hz, corresponding to 6-40 breaths / minute) and a heart rate filter (passband 0.8-3.0Hz, corresponding to 48-180 bpm). FFT spectral analysis was then performed on the filtered signals with a time window of 10 seconds and a frequency resolution of 0.1Hz. The frequencies corresponding to the spectral peaks within the respiratory and heart rate bands were identified and converted into respiratory rate (breaths / minute) and heart rate (bpm), respectively.

[0088] III. Decision-level Fusion Module

[0089] like Figure 6 As shown, the decision-level fusion module is connected to the macroscopic motion recognition branch and the microscopic vital sign monitoring branch, respectively. It is used to receive motion recognition results, respiratory rate and heart rate, calculate dynamic fusion weights and execute bidirectional constraint rules, perform cross-validation and comprehensive judgment according to the preset rule library, and output the final user status and alarm signal.

[0090] (1) Calculate the dynamic fusion weights

[0091]

[0092]

[0093] in, This represents the weight of the action recognition result in the final state determination, with a value ranging from 0 to 1. The weight of vital signs in the final status determination, with a value range of 0-1; The confidence level for action recognition ranges from 0 to 1. The value represents the quality of vital signs signals, ranging from 0 to 1. The duration of the current action. This is the time decay factor; The vital signs reliability index, with a value range of 0-1, is calculated using the following formula: ,in, The number of frames for continuous and stable detection. (corresponding to 10 seconds) As a reliability weighting factor;

[0094] (2) Two-way constraint mechanism

[0095] When the action recognition result is "fall" or "strenuous exercise," the threshold for judging abnormal heart rate is adjusted from 100 bpm to 150 bpm to avoid physiological increases in heart rate caused by exercise being misjudged as abnormal; when the quality of vital signs signals... If no valid breathing or heartbeat signal is detected for 30 consecutive seconds, the action recognition result is forcibly corrected to "unmanned" state to prevent false detections caused by noise interference; cross-validation and comprehensive judgment are performed according to the preset rule base, and the final user status and alarm signal are output based on the action recognition result and vital sign parameters.

[0096] (3) Rule base decision

[0097] The final state is determined according to the 15 preset rules in Table 1, and the final user status and alarm signal are output. The accuracy of each rule is derived from statistical analysis of a large number of real events, and the overall rule accuracy rate reaches 94.5%.

[0098] Table 1 is the preset rule table.

[0099]

[0100] To verify the system's performance, approximately 10,000 samples were collected from 50 volunteers, constructing a motion sample library covering various scenarios. The data collection process lasted three months and was conducted in a research institute's laboratory. Two typical environments were set up in the laboratory: a standard 4m×4m living room simulation scenario (furniture such as sofas and coffee tables) and a 3.5m×3m bedroom simulation scenario (furniture such as beds and wardrobes). The radar was installed at heights of 2.5m and 2.2m respectively. Both scenarios were equipped with air conditioners, electric fans, and other equipment to simulate interference factors in a real home environment.

[0101] For 12 target actions, multiple data collection strategies were employed: each participant performed each action 15-20 times (each time lasting 2-3 seconds), with the "falling" action further subdivided into three variations—forward, backward, and sideways—performed 5-7 times each; eight daily activity sequences were designed (e.g., "enter the room → sit down → stand up → leave"), with each participant performing each sequence 10 times; the same action was performed in different locations within the room (center, corner, near furniture), with at least 3 repetitions in each location; and the same action was performed at slow, normal, and fast speeds. Through these strategies, each participant contributed an average of approximately 200 valid samples.

[0102] The system of this embodiment is compared with existing technical solutions. Representative existing solutions are selected: a micro-motion detection algorithm based on phase extraction (baseline for stationary state detection), a single-function millimeter-wave radar product (action recognition or vital sign monitoring implemented separately), and a time-division multiplexing dual-modal solution. All comparative tests were conducted on the same hardware platform (TIIWR6843AOP), using the same dataset (10,000 samples) and under the same testing environment to ensure fairness. The results are shown in Tables 2-5.

[0103] Table 2 Comparison of the performance of the present invention and existing technologies

[0104]

[0105] Table 3 Comparison of heart rate detection performance under different signal-to-noise ratios

[0106]

[0107] Table 4 Validation data of decision-level fusion rules

[0108]

[0109] Table 5 Test Environment and Sample Statistics

[0110]

[0111] As shown in Tables 2-5, the test results demonstrate that this invention, through a dual-modal temporal decoupling parallel architecture, simultaneously achieves action recognition and vital sign monitoring on a single radar platform, fundamentally solving the technical bottleneck caused by the conflict between time and frequency domain characteristics. Under static conditions, the respiratory rate detection error is approximately ±0.4 breaths / minute, and the heart rate detection error is approximately ±1.85 bpm, achieving a static detection rate of over 98%, a 15% improvement compared to micro-motion detection algorithms based on phase extraction (from 85% to 98%). In dynamic scenarios, the action recognition response time remains stable within 450ms, a more than 55% reduction in latency compared to time-division multiplexing schemes (>1000ms). The overall system throughput is improved by 35%, with CPU utilization at only 45% and memory usage of approximately 280MB. Real-time operation is achieved on the NVIDIA Jetson Nano embedded platform, marking the first time that high temporal resolution (20FPS) and high frequency resolution (Δf=0.1Hz) have been simultaneously guaranteed on a single radar platform. The decision-level fusion module contains 15 preset rules. Taking "a fall with a heart rate >100 bpm as an emergency fall" as an example, this rule has a confidence level of 94%. Its accuracy is derived from statistical analysis of 120 real fall events, of which 113 falls were accompanied by an increased heart rate. Experimental results are as follows: Figure 8 As shown, this paper compares the present invention with the prior art in various aspects.

[0112] in, Figure 8 (a) compares four core performance indicators: the static detection rate of this invention reaches 98%, which is 25 percentage points higher than the 73% of the prior art, solving the problem that traditional CFAR detection cannot identify static human bodies; the respiratory rate error of this invention is only ±0.4 breaths / minute, which is far better than the ±5.2 breaths / minute of the prior art, thanks to the high-precision phase recovery of the adaptive phase unwrapping algorithm; the heart rate error of this invention is ±1.85 bpm, which is 73% lower than the ±6.8 bpm of the prior art; the action recognition accuracy of this invention reaches 96%, which is 11 percentage points higher than the 85% of the prior art, mainly due to the effective capture of complex actions by the dynamic neighborhood radius adjustment mechanism;

[0113] Figure 8(b) shows the comparison of heart rate detection errors under different signal-to-noise ratio conditions: in a high signal-to-noise ratio environment with SNR>15dB, the heart rate error of the present invention is 1.85bpm, and that of the prior art is 6.8bpm; in a medium signal-to-noise ratio environment with 10dB<SNR≤15dB, the heart rate error of the present invention is 2.0bpm, and that of the prior art is 7.5bpm; in a low signal-to-noise ratio environment with SNR≤10dB, the heart rate error of the present invention is 2.5bpm, and that of the prior art is 9.2bpm. Experiments show that the adaptive phase unwrapping algorithm of the present invention can still maintain high detection accuracy in a low signal-to-noise ratio environment, with an unwrapping accuracy of 92%, while the detection accuracy of the prior art decreases significantly when SNR≤10dB;

[0114] Figure 8 (c) shows the accuracy of decision rules: the accuracy of fall detection rules is 95%, the accuracy of respiratory abnormality detection rules is 93%, the accuracy of heart rate abnormality detection rules is 94%, and the accuracy of static monitoring rules is 96%. The 15 preset rules are obtained based on statistics of real data, and cross-validation of motion recognition and vital sign data is realized through dynamic fusion weights and a two-way constraint mechanism, which effectively reduces the system false alarm rate to ≤3%;

[0115] Figure 8 (d) is the comparison of motion recognition response time: the response time of the present invention is ≤450ms, which meets the real-time requirement of fall detection; the response time of the prior art is about 1000ms, which cannot trigger an alarm in time. Through a dual-channel parallel processing architecture and a timing decoupling mechanism, the present invention achieves low-latency response for motion recognition, ensuring that emergency events such as falls can be quickly detected and alarmed.

[0116] The breakthrough of this technical solution lies in overcoming the technical obstacle that time-frequency domain characteristics cannot be both obtained in traditional solutions, and providing a brand-new technical path for millimeter-wave radar dual-modal monitoring.

[0117] In a dim light environment (simulating night), the accuracy decreases, which is mainly caused by the reduction of point cloud quality. Further optimization can be achieved by enhancing the point cloud preprocessing algorithm or introducing multi-sensor fusion.

[0118] Example 2:

[0119] This embodiment provides a monitoring method of a non-contact human body monitoring system based on millimeter-wave radar, which specifically includes the following steps:

[0120] S1: Collect echo signals of a target area by a millimeter-wave radar, and extract point cloud data P through preprocessing; the parameter configuration of the millimeter-wave radar is shown in Table 6;

[0121] Table 6 is a parameter configuration table

[0122]

[0123] S2: Initiate dual-path parallel processing:

[0124] S21: The macroscopic action recognition branch adopts a frame-by-frame real-time processing mode. Let the scaling factor β = 1.8, and the density compensation factor... =0.3, based on the standard deviation of the centroid trajectory within the time window. (Calculated by S211) and the neighborhood radius dynamically adjusted by point cloud density ( This represents the total number of target points detected in this frame, i.e., the point cloud data output in step S13. The number of points in the middle; The average number of point clouds within the sliding window is set, and the constraint 0.3m≤r≤1.5m is set. The processing latency per frame for this branch is controlled within 50ms to ensure a high temporal resolution of 20FPS.

[0125] S22: The microscopic vital signs monitoring branch employs a sliding window accumulation mode. Complex signals corresponding to the human thoracic cavity region are extracted from the intermediate frequency signal. Calculate the instantaneous phase Phase unwrapping employs an adaptive unwrapping algorithm based on the phase change rate: calculating the phase difference between adjacent time points. The point cloud data output from step S13 Signal-to-noise ratio of points in the pleural region The average value is taken as the signal-to-noise ratio of the current frame. ( (The number of point clouds in the thoracic region), and calculate the dynamic threshold accordingly. ;when At that time, Compensation to restore continuous phase curve Simultaneously calculate signal quality indicators. The SNR is normalized to the 0-1 range. Moving average phase difference is used to eliminate baseline drift. Physiological signals are separated using a respiratory filter (passband 0.1-0.67Hz) and a heart rate filter (passband 0.8-3.0Hz). Respiratory rate and heart rate are extracted using 10-second sliding window FFT spectral analysis. This branch performs spectral analysis every 250ms, achieving 4 updates per second while maintaining a frequency resolution Δf=0.1Hz.

[0126] S3: The decision-level system fuses the action recognition results with the respiratory rate and heart rate, calculates dynamic fusion weights, executes bidirectional constraint rules, performs cross-validation and comprehensive judgment according to a preset rule base, and outputs the final user state. Specifically, based on the confidence level of the action recognition results and the quality of vital sign signals, the fusion weights of the action recognition results and vital sign results in the final state determination are dynamically calculated; the final state determination is performed according to a preset rule base of 15 rules.

[0127] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the protection scope of the present invention.

[0128] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

[0129] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed by the present invention.

Claims

1. A dual-modal human body sensing system based on millimeter-wave radar, characterized in that, include: The radar signal acquisition and preprocessing module is used to configure the operating parameters of the millimeter-wave radar, transmit FMCW signals, and receive echoes. The echo is mixed with the local transmitted signal to generate an intermediate frequency signal; the intermediate frequency signal is sampled by an ADC, and Fourier transforms are performed sequentially along the fast and slow time dimensions to obtain the range spectrum and Doppler spectrum, respectively. The CA-CFAR algorithm is used to perform target detection on the range spectrum and Doppler spectrum and extract point cloud data. The dual-path parallel processing module is connected to the radar signal acquisition and preprocessing module. Internally, it contains two independent and parallel branches: a macroscopic motion recognition branch and a microscopic vital sign monitoring branch. The macroscopic motion recognition branch receives point cloud data, extracts the centroid trajectory, dynamically adjusts the neighborhood radius based on the standard deviation of the centroid trajectory within a time window, and uses a PointNet++ network for feature extraction and classification, outputting the motion recognition result and confidence level. The microscopic vital sign monitoring branch extracts the complex signal and its instantaneous phase of the human chest cavity region from the intermediate frequency signal, sequentially performing phase extraction, phase unwrapping, phase difference, bandpass filtering, and FFT spectral analysis to extract respiratory rate and heart rate. The decision-level fusion module is connected to the macro-motion recognition branch and the micro-vital sign monitoring branch, respectively. It is used to receive motion recognition results, confidence, respiratory rate and heart rate, calculate dynamic fusion weights and execute bidirectional constraint rules, perform cross-validation and comprehensive judgment according to the preset rule library, and output user status and alarm signals. Among them, the macroscopic action recognition branch receives and outputs point cloud data. As input, the centroid position of each frame point is used for cloud computing. ,in, The current frame number. They are respectively The average value in three directions, thus constructing a length of Calculate the standard deviation of the centroid trajectory sequence of the frames. ,in, For the first The centroid position of the frame; for The average value of the frame centroid position vector; The Euclidean distance between the two vectors; The PointNet++ network extracts local point cloud features from the point cloud through ball query operations, and dynamically adjusts the neighborhood search radius based on the action amplitude and point cloud density to match the search range with the point cloud distribution. ； in, The neighborhood search radius; This is the scaling factor; is the standard deviation of the centroid trajectory within the time window; Density compensation factor; This represents the total number of target points detected in this frame, i.e., the point cloud data output in step S13. The number of points in the middle; Given the average number of point clouds within the sliding window, output the dynamically adjusted neighborhood search radius. ; Based on the output neighborhood search radius Set the neighborhood search radius The constraints are ,in, The preset minimum search radius; The maximum search radius is preset; key points are selected using the FPS algorithm, local features are extracted using an MLP network structure, and finally max pooling is used to aggregate the results, outputting the action recognition results and confidence scores. .

2. A dual-modal human body perception and monitoring method based on millimeter-wave radar, implemented using a dual-modal human body perception system based on millimeter-wave radar as described in claim 1, characterized in that... Includes the following steps: S1: Collect echo signals from the target area using millimeter-wave radar, mix the echoes with the local FMCW to generate intermediate frequency signals, and extract point cloud data P after preprocessing; S2: Initiate dual-path parallel processing: S21: Perform macroscopic action recognition on the point cloud data P, including extracting the centroid trajectory and using an improved PointNet++ network for feature extraction and classification; wherein, during the feature extraction process, the neighborhood radius of the network is dynamically adjusted according to the standard deviation of the centroid trajectory within the time window, and the action recognition result and confidence level are output. S22: Perform microscopic vital sign monitoring on the intermediate frequency signal, including extracting the complex signal and its instantaneous phase of the human chest cavity region, and sequentially performing phase extraction, phase unwrapping, phase difference, bandpass filtering and FFT spectrum analysis, and extracting respiratory rate and heart rate through Fourier transform spectrum analysis; S3: The decision-level system fuses the action recognition results with the breathing frequency and heart rate, calculates the dynamic fusion weights and executes bidirectional constraint rules, performs cross-validation and comprehensive judgment according to the preset rule base, and outputs the final user status.

3. The dual-modal human perception and monitoring method based on millimeter-wave radar as described in claim 2, characterized in that, Step S1 includes the following: S11. Perform ADC sampling on the received intermediate frequency signal to obtain a discrete time series. ; S12, along the fast time dimension for each frequency modulation pulse Perform a Fast Fourier Transform on each sampling point to obtain the distance spectrum. Next, along the slow time dimension, examine all frames within a frame. Perform a Fast Fourier Transform on the data from the same distance cell of each frequency-modulated pulse to obtain the Doppler spectrum. ; S13. The distance spectrum obtained in step S12 and Doppler spectrum Combination formation distance-Doppler two-dimensional spectrum In this context, each element in the range-Doppler two-dimensional spectrum is called a range-Doppler unit, and is represented by... express, For distance cell index, For Doppler cell indexing, For signal strength; the CA-CFAR algorithm is used to perform target detection on the range-Doppler two-dimensional spectrum; for each range-Doppler cell to be detected... A reference window is set up around it, and the average signal strength of all distance-Doppler cells within the reference window is calculated. If the signal strength of the distance-Doppler unit to be detected is... If the distance-Doppler cell is determined to be the target point, then... The CA-CFAR scaling factor is used for target detection; and the distance, velocity, angle, and signal-to-noise ratio information of the target points are recorded, ultimately outputting point cloud data. ,in, For the first The three-dimensional spatial coordinates of the points For the first Radial velocity at each point For the first Signal-to-noise ratio at each point This represents the total number of target points detected in this frame.

4. The dual-modal human perception and monitoring method based on millimeter-wave radar as described in claim 2, characterized in that, Step S22 specifically includes the following: S221, The microscopic vital signs monitoring branch extracts the distance unit corresponding to the human thoracic cavity region from the intermediate frequency signal and obtains the complex signal of the distance unit. And calculate its instantaneous phase. ,in, For discrete-time indexing, For in-phase components, For orthogonal components, The imaginary unit is used; the instantaneous phase is calculated using the arctangent function in the four quadrants. , indicating time The phase value, with a range of ; S222. First, the instantaneous phase obtained in step S221... Perform adaptive phase unwrapping; calculate the phase difference between adjacent time points. Extracting the signal-to-noise ratio of corresponding points in the human chest cavity region from point cloud data. The average value is taken as the signal-to-noise ratio of the current frame. ,in, The number of point clouds within the thoracic region is calculated; simultaneously, a normalized signal quality index is calculated. Normalize the SNR to the 0-1 range: ,in, The minimum effective signal-to-noise ratio threshold. The maximum signal-to-noise ratio (SNR) is set as the upper limit; a dynamic threshold related to the SNR is introduced. ,when When, it is determined that an event has occurred. Jump, based on the jump direction Compensation, restoration of continuous phase curve Output the continuous phase curve after unwrapping. and signal quality ; Dynamic threshold The selection is based on the signal-to-noise ratio (SNR) and uses a continuously smooth dynamic threshold formula: ； in, For dynamic thresholds; Signal-to-noise ratio; Next, the baseline drift was eliminated using moving average phase difference to refine the continuous phase curve after unwrapping. Applying a sliding window moving average filter, the filtered phase is obtained. Then calculate the phase difference between adjacent time points: ,in, For discrete-time indexing, The time interval depends on the signal quality. Adaptive adjustment; output phase change and signal quality ; S223, Phase change amount output in step S222 The respiratory and heartbeat signals were obtained by dual-channel filtering using two separate filters. FFT spectral analysis was then performed on the filtered respiratory and heartbeat signals, with a time window of 10 seconds and a frequency resolution of 0.1 Hz, to distinguish the peak values ​​of the respiratory and heartbeat frequencies and avoid measurement errors caused by spectral aliasing. Within the respiratory signal frequency band... Find the frequency corresponding to the spectral peak within Hz In the heartbeat signal frequency band Find the frequency corresponding to the spectral peak within Hz Converted to respiratory rate Heart rate Where bpm stands for beats per minute, i.e., the number of heartbeats per minute; the final output is the respiratory rate. Heart rate And transmit the signal quality calculated in step S222. Up to the decision-level fusion module.

5. The dual-modal human perception and monitoring method based on millimeter-wave radar as described in claim 2, characterized in that, Step S3 specifically includes the following: S31. Receive the action recognition result and confidence level output from the action recognition branch. Respiratory rate output from the microscopic vital signs monitoring branch Heart rate and signal quality Calculate the dynamic fusion weights: ； ； in, This represents the weight of the action recognition result in the final state determination, with a value ranging from 0 to 1. The weight of vital signs in the final status determination, with a value range of 0-1; The confidence level for action recognition ranges from 0 to 1. The value represents the quality of vital signs signals, ranging from 0 to 1. The duration of the current action. This is the time decay factor; The vital signs reliability index, with a value range of 0-1, is calculated using the following formula: ,in, The number of frames for continuous and stable detection. , As a reliability weighting factor; S32. Implement a two-way constraint mechanism: When the action recognition result is "fall" or "strenuous exercise", adjust the heart rate abnormality judgment threshold from 100 bpm to 150 bpm to avoid physiological heart rate increases caused by exercise being misjudged as abnormal; when the vital sign signal quality If no valid breathing or heartbeat signal is detected for 30 consecutive seconds, the action recognition result is forcibly corrected to "unmanned" state to prevent false detections caused by noise interference; cross-validation and comprehensive judgment are performed according to the preset rule base, and the final user status and alarm signal are output based on the action recognition result and vital sign parameters.

6. The dual-modal human perception and monitoring method based on millimeter-wave radar as described in claim 4, characterized in that, In step S222, when SNR > 15dB, When 10dB < SNR ≤ 15dB, When SNR≤10dB, ; When SNR > 15 dB ms, 10-15dB, ms, <10dB ms.

7. The dual-modal human perception and monitoring method based on millimeter-wave radar as described in claim 4, characterized in that, In step S223, the two filters include a breathing filter and a heartbeat filter. The breathing filter has a passband of 0.1-0.67Hz, a cutoff frequency attenuation of -3dB, and a stopband attenuation of -40dB. The heartbeat filter has a passband of 0.8-3.0Hz, a cutoff frequency attenuation of -3dB, and a stopband attenuation of -40dB.

8. The dual-modal human perception and monitoring method based on millimeter-wave radar as described in claim 2, characterized in that, In step S1, the millimeter-wave radar is a 60-64GHz FMCW radar with a starting frequency of 60GHz, a bandwidth of 4GHz, a chirp period of 60μs, 128 chirps per frame, a frame rate of 20FPS, and 256 ADC sampling points.

Images