Multi-sensor fused human body posture and physiological parameter non-inductive calculation method

Abstract

The invention relates to the technical field of data processing, and particularly discloses a multi-sensor fused human body posture and physiological parameter non-inductive calculation method, and the method comprises the steps: synchronously collecting millimeter wave radar point cloud data and a ToF sensor distance matrix, and carrying out the preprocessing and fusion, so as to achieve the precise tracking of posture key points of a head, a neck, shoulders and the like; dynamically delimiting a thoracic cavity ROI based on a posture result, compensating motion artifacts, and extracting a high-signal-to-noise-ratio heart rate variability parameter; and finally, calculating posture indexes such as a head forward inclination angle and a cervical vertebra load moment, and outputting a fatigue grade through a lightweight model in combination with HRV and a static sitting duration. According to the method, spatial constraint and correction are carried out on the radar point cloud through the ToF data, and the instability of a single radar under depth estimation and a static attitude is effectively overcome. According to the method, the posture tracking result is creatively used for guiding physiological signal extraction, and the success rate and accuracy of extracting fine parameters such as HRV in an uncontrolled desktop environment are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a non-intrusive calculation method for human posture and physiological parameters using multi-sensor fusion. Background Technology

[0002] Achieving accurate, contactless, and privacy-infringing monitoring in desktop health monitoring scenarios presents significant challenges. Existing technologies suffer from the following problems:

[0003] 1. Single sensor has significant limitations:

[0004] Using millimeter-wave radar alone: ​​Although it can generate point clouds, the point clouds are sparse and noisy when the user is stationary or making slight movements, making it unreliable for accurately reconstructing complex human postures.

[0005] Using only ToF sensors: can only provide one-dimensional distance or a simple two-dimensional depth matrix, and cannot provide accurate displacement information of joints in three-dimensional space.

[0006] Using only the camera: privacy issues remain unresolved, and the computational load is high, making it difficult to run in real time on embedded devices.

[0007] 2. Difficulty in extracting physiological parameters: Extracting physiological parameters such as heart rate and respiration from millimeter-wave radar signals usually requires a clean signal source. In desktop office scenarios, the slight movements of the user's hands and head can severely interfere with vital sign signals, resulting in low signal-to-noise ratios and unreliable extraction of parameters such as heart rate variability.

[0008] 3. Algorithm and hardware mismatch: Many complex 3D pose estimation algorithms are computationally intensive and cannot run in real time on resource-constrained embedded processors.

[0009] Therefore, a non-intrusive calculation method for human posture and physiological parameters using multi-sensor fusion has become an urgent problem to be solved. Summary of the Invention

[0010] The technical problem this invention aims to solve is to provide a computationally efficient and reliable multi-sensor fusion method for the non-intrusive calculation of human posture and physiological parameters. By leveraging the complementary advantages of millimeter-wave radar and ToF sensors, real-time and accurate 3D human posture key point tracking is achieved on an embedded platform. Using the posture tracking results, the radar signal is dynamically partitioned and motion artifacts are filtered out, thereby stably extracting fine physiological parameters such as heart rate variability even under interference environments. This forms a complete end-to-end conversion process from raw sensor data to high-level health indicators.

[0011] To solve the above-mentioned technical problems, the technical solution provided by the present invention is: a method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion, comprising the following steps:

[0012] S1. Simultaneously acquire the point cloud data sequence output by the millimeter-wave radar and the multi-zone range matrix output by the time-of-flight sensor;

[0013] S2. Clustering algorithm is used to separate point cloud clusters representing the user's torso, head, and shoulders from the preprocessed point cloud data sequence; the centroid of each point cloud cluster is calculated as the initial three-dimensional coordinates of the key points; the initial key points are projected onto the two-dimensional field-of-view coordinate system of the time-of-flight sensor, and the distance values ​​and contour information of the corresponding regions in the multi-region distance matrix are used to correct the preliminary estimated depth coordinates of the key points; the current frame observation results and historical trajectories are fused through Kalman filter or particle filter to output the final smooth key point coordinate sequence.

[0014] S3. Based on the key point locations of the torso tracked in step S2, dynamically delineate the ROI surrounding the chest cavity region in the raw data domain of the millimeter-wave radar; use the overall human movement speed to perform phase compensation on the signal within the ROI to separate the pure cardiopulmonary activity micro-Doppler signal; perform time-frequency analysis on the purified cardiopulmonary activity micro-Doppler signal to extract the heartbeat harmonic components; obtain a continuous heartbeat interval sequence through a peak detection algorithm, and calculate the time-domain HRV index or frequency-domain index based on the continuous heartbeat interval sequence;

[0015] S4. Calculate the posture index based on the key point coordinate sequence; establish a lightweight machine learning model with posture index, HRV index and sitting time as input features, and output the comprehensive fatigue level.

[0016] Furthermore, in step S2, the clustering algorithm adopts the Euclidean clustering algorithm, which realizes the separation of point cloud clusters by calculating the Euclidean distance between each point in the point cloud data sequence.

[0017] Furthermore, in step S2, the head keypoints satisfy K head =centroid(C head );

[0018] Neck key points satisfy K neck =interpolate(centroid(C shoulder ),K head );

[0019] Among them, K head K is a key point for the head. neck C is a key point on the neck. head For the head of the point cloud cluster, C shoulder Add cloud clusters to the shoulders.

[0020] Furthermore, in step S2, the specific method for correcting the initially estimated key point depth coordinates is as follows:

[0021] The coordinate transformation relationship between the millimeter-wave radar and the time-of-flight sensor is pre-calibrated. After mapping the radar key points to the multi-zone matrix of the time-of-flight sensor, the depth coordinates of the key points are corrected using the weighted average distance values ​​of the corresponding regions.

[0022] Furthermore, in step S3, the time-frequency analysis employs short-time Fourier transform, and the time-domain HRV index includes RMSSD and SDNN. The time-domain HRV index or frequency-domain index is used to quantify autonomic nervous system activity and user stress levels.

[0023] Further, in step S4, the posture indicators include the head tilt angle and the cervical spine load torque; the head tilt angle satisfies In the formula These are the Z-axis coordinates of key points on the head and neck, respectively. The X-axis coordinates of key points on the head and neck are respectively; the cervical spine load torque satisfies M=m·g·L·sin(θ), where m is the estimated mass of the head, g is the gravitational acceleration, L is the lever arm, and θ is the head tilt angle.

[0024] Furthermore, in step S4, the lightweight machine learning model is a decision tree model or a support vector machine model.

[0025] Furthermore, in step S1, outlier filtering and background noise removal are performed on the point cloud data sequence; and time-series filtering is performed on the multi-region distance matrix to smooth data fluctuations.

[0026] The advantages of this invention compared to the prior art are:

[0027] This invention uses ToF data to spatially constrain and correct radar point clouds, effectively overcoming the instability of a single radar in depth estimation and static attitude, and achieving more reliable three-dimensional attitude tracking.

[0028] This invention creatively utilizes posture tracking results to guide physiological signal extraction. Through two core technologies, "dynamic ROI" and "motion artifact compensation," it greatly improves the success rate and accuracy of extracting fine parameters such as HRV in uncontrolled desktop environments.

[0029] The entire method flow design of this invention fully considers the computing power limitations of embedded processors, avoids complex 3D reconstruction networks, and adopts lightweight filtering, clustering, and models to ensure the real-time performance of the algorithm.

[0030] This invention realizes end-to-end computation across the entire chain, from low-level sensor data to high-level health indicators, unlocking the maximum value of multimodal data and providing a reliable data foundation for precise health intervention. Attached Figure Description

[0031] Figure 1 This is a flowchart of a multi-sensor fusion method for non-sensory calculation of human posture and physiological parameters according to the present invention.

[0032] Figure 2 This is a flowchart of step 2 of the present invention.

[0033] Figure 3 This is a schematic diagram of step 3 of the present invention. Detailed Implementation

[0034] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0035] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0036] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0037] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0038] The following is a detailed description of the non-intrusive calculation method for human posture and physiological parameters based on multi-sensor fusion according to the present invention, with reference to the accompanying drawings.

[0039] Combined with appendix Figure 1-3 The specific implementation process of the multi-sensor fusion method for non-intrusive calculation of human posture and physiological parameters of the present invention is as follows:

[0040] A non-invasive method for calculating human posture and physiological parameters through multi-sensor fusion includes the following four core steps:

[0041] Step 1: Synchronous Acquisition and Preprocessing of Multimodal Data

[0042] Data Synchronous Acquisition: Synchronously acquire point cloud data sequences P output by millimeter-wave radar. t (t represents the time dimension, reflecting the point cloud distribution at different times) and the multi-zone distance matrix D output by the time-of-flight (ToF) sensor. t (The matrix elements represent distance measurements from different regions of the sensor), ensuring that the two types of data are aligned in the time dimension, laying the foundation for subsequent fusion calculations;

[0043] Point cloud data preprocessing: processing point cloud data sequence P t Outlier filtering and background noise removal are performed. Anomalies in the point cloud are identified and removed using a preset threshold algorithm. At the same time, background point cloud data that is not related to the human target is removed, while valid human point cloud data is retained.

[0044] ToF data preprocessing: Preprocessing the multi-region distance matrix D t Temporal filtering is performed using a sliding window averaging filtering algorithm to smooth out fluctuations in the distance matrix and reduce the impact of measurement noise on subsequent calculations.

[0045] Step 2: Real-time human pose key point tracking based on fused data

[0046] This step achieves precise tracking of key human posture points (head, neck, torso, left shoulder, right shoulder) by fusing millimeter-wave radar point cloud data and ToF data. It specifically includes four sub-steps:

[0047] Step 2.1 Target Clustering and Matching: For the preprocessed point cloud data sequence P t Using the Euclidean clustering algorithm, the Euclidean distance between points in the point cloud data is calculated to group points that are close together into the same cluster, thereby separating the main point cloud clusters representing the user's torso, head, and shoulders, denoted as C0, respectively. torso (Torso dotted cloud cluster), C head (Head point cloud cluster), C shoulder (Shoulder-shaped cloud clusters);

[0048] Step 2.2 Preliminary estimation of key points: Calculate the centroid of each cloud cluster and use it as the initial 3D coordinates of the corresponding attitude key points:

[0049] Initial coordinates of head key points: K head =centroid(C head ), that is, the centroid coordinates of the head point cloud cluster;

[0050] Initial coordinates of the key points in the neck: K neck =interpolate(centroid(C shoulder ),K head The position of the neck key points is obtained by interpolation calculation between the centroid of the shoulder point cloud cluster and the coordinates of the head key points, ensuring that the position of the neck key points conforms to the human physiological structure.

[0051] Initial coordinates of key points on the torso, left shoulder, and right shoulder: These are the centroid coordinates of the corresponding point cloud clusters, denoted as K. torso K L-shoulder K R-shoulder ;

[0052] Step 2.3 ToF Data-Assisted Correction and Constraints: The coordinate transformation relationship between the millimeter-wave radar and the ToF sensor is pre-calibrated. The initial key points obtained in Step 2.2 are projected onto the two-dimensional field-of-view coordinate system of the ToF sensor; using the multi-zone distance matrix D... t The distance values ​​and contour information of the corresponding regions are used to correct the preliminary estimated depth coordinates (Z-axis) of the key points. Specifically, after mapping the radar key points to a multi-region distance matrix, the weighted average distance values ​​of the corresponding regions are used to correct the Z-axis coordinates of the key points, thus solving the problem of inaccurate depth estimation by millimeter-wave radar.

[0053] Step 2.4 Temporal Filtering and Tracking: Using a Kalman filter or particle filter, fuse the fused observation results of the current frame (corrected keypoint coordinates) with the historical trajectory (keypoint coordinate sequence of previous frames) to eliminate random errors in single-frame data and output a final smooth and stable keypoint coordinate sequence. (Where i represents different key point types such as head, neck, torso, left shoulder, and right shoulder).

[0054] Step 3: Precise extraction of physiological parameters based on posture awareness

[0055] This step, based on the attitude tracking results from step 2, achieves accurate extraction of physiological parameters such as heart rate variability (HRV), and specifically includes three sub-steps:

[0056] Step 3.1 Dynamic Region of Interest (ROI) Delineation: Based on the torso keypoints K tracked in real time in Step 2... torso In the raw data domain of millimeter-wave radar, a region of interest (ROI) is dynamically defined around the chest cavity—the area where human heartbeat and respiratory signals are most concentrated. By dynamically adjusting the ROI range, the cardiopulmonary activity signals can be focused to the greatest extent and interference from limb micro-movements can be eliminated.

[0057] Step 3.2 Motion artifact compensation: First, calculate the overall motion velocity of the human body by using the difference in keypoint coordinates between adjacent frames. The formula is: (in Let be the coordinates of the key point in frame t. (where Δt is the coordinates of the key point in frame t-1, and Δt is the time interval between two frames); then, this motion speed is used to perform phase compensation on the radar signal within the ROI in step 3.1 to suppress the Doppler frequency shift caused by the overall body movement, and to separate the pure micro-Doppler signal s caused by cardiopulmonary activity from the mixed signal. vital (t);

[0058] Step 3.3 Heart Rate Variability (HRV) Calculation:

[0059] The purified micro-Doppler signal s vital(t) Time-frequency analysis is performed, and the time-domain signal is converted into a two-dimensional time-frequency spectrum using short-time Fourier transform (STFT) to extract the harmonic components corresponding to the heartbeat;

[0060] Peak positions of heartbeat harmonic components are identified by peak detection algorithms (such as adaptive threshold peak detection) to obtain a continuous heartbeat interval sequence (i.e., a time difference sequence between two adjacent heartbeat peaks).

[0061] Based on the heartbeat interval sequence, calculate the time-domain or frequency-domain indicators of HRV: the time-domain indicators include the root mean square standard deviation (RMSSD) and the standard deviation of all sinus RR intervals (SDNN), and the frequency-domain indicators include the low-frequency component (LF) and the high-frequency component (HF). The above indicators can serve as the core basis for quantifying the user's autonomic nervous system activity and stress level.

[0062] Step 4: Calculation of High-Level Health Indicators

[0063] This step, based on the posture keypoint sequence from step 2 and the HRV parameters from step 3, calculates clinically significant high-level health indicators, providing an intuitive basis for user health assessment. Specifically, this includes:

[0064] Step 4.1 Posture Index Calculation: Based on the tracked keypoint coordinate sequence Kti, calculate the indicators reflecting the health status of human posture:

[0065] Forward head tilt angle: Calculation formula is as follows In the formula These are the Z-axis coordinates (vertical direction) of key points on the head and neck, respectively. The X-axis coordinates of key points on the head and neck are shown in the horizontal front-back direction, respectively. This angle directly reflects the degree of forward tilt of the user's head. The larger the angle, the more unhealthy the posture.

[0066] The cervical spine load torque satisfies M=m·g·L·sin(θ), where m is the estimated head mass (preset based on the average head mass of an adult, or adjusted by user input), g is the gravitational acceleration, L is the lever arm (preset as the horizontal distance from the head's center of mass to the cervical spine), and θ is the forward tilt angle of the head calculated in step 4.1. This torque value reflects the magnitude of the pressure on the cervical spine; the larger the value, the higher the risk of cervical spine fatigue.

[0067] Step 4.2 Fatigue Assessment Fusion: Establish a lightweight machine learning model (such as a decision tree model or support vector machine model) using the head tilt angle and cervical spine load torque calculated in Step 4.1, the HRV index (such as RMSSD, SDNN) calculated in Step 3, and the user's sitting time (obtained through the system timing module) as input features. During model training, the user's actual fatigue state label is used as a supervision signal to optimize the mapping relationship between input features and fatigue level (such as "no fatigue", "mild fatigue", "moderate fatigue", "severe fatigue"). Finally, the lightweight machine learning model outputs a comprehensive fatigue level to provide a basis for subsequent health interventions.

[0068] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A non-intrusive calculation method for human posture and physiological parameters based on multi-sensor fusion, characterized in that, Includes the following steps: S1. Simultaneously acquire the point cloud data sequence output by the millimeter-wave radar and the multi-zone range matrix output by the time-of-flight sensor; S2. Clustering algorithm is used to separate point cloud clusters representing the user's torso, head, and shoulders from the preprocessed point cloud data sequence; the centroid of each point cloud cluster is calculated as the initial three-dimensional coordinates of the key points; the initial key points are projected onto the two-dimensional field-of-view coordinate system of the time-of-flight sensor, and the distance values ​​and contour information of the corresponding regions in the multi-region distance matrix are used to correct the preliminary estimated depth coordinates of the key points; the current frame observation results and historical trajectories are fused through Kalman filter or particle filter to output the final smooth key point coordinate sequence. S3. Based on the key point positions of the torso tracked in step S2, dynamically delineate the ROI surrounding the chest cavity region in the raw data domain of the millimeter-wave radar; use the overall human movement speed to perform phase compensation on the signal within the ROI, and separate the pure cardiopulmonary activity micro-Doppler signal. Time-frequency analysis was performed on the purified cardiopulmonary activity micro-Doppler signal to extract the heartbeat harmonic components; a continuous heartbeat interval sequence was obtained through a peak detection algorithm, and the time-domain HRV index or frequency-domain index was calculated based on the continuous heartbeat interval sequence. S4. Calculate the posture index based on the key point coordinate sequence; establish a lightweight machine learning model with posture index, HRV index and sitting time as input features, and output the comprehensive fatigue level.

2. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 1, characterized in that: In step S2, the clustering algorithm adopts the Euclidean clustering algorithm, which realizes the separation of point cloud clusters by calculating the Euclidean distance between each point in the point cloud data sequence.

3. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 2, characterized in that: In step S2, the head key points satisfy... ; Neck key points satisfied ; in, Key points of the head, Key points of the neck For the head of the cloud cluster, Add cloud clusters to the shoulders.

4. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 3, characterized in that: In step S2, the specific method for correcting the initially estimated key point depth coordinates is as follows: The coordinate transformation relationship between the millimeter-wave radar and the time-of-flight sensor is pre-calibrated. After mapping the radar key points to the multi-zone matrix of the time-of-flight sensor, the depth coordinates of the key points are corrected using the weighted average distance values ​​of the corresponding regions.

5. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 4, characterized in that: In step S3, the time-frequency analysis employs short-time Fourier transform, and the time-domain HRV index includes RMSSD and SDNN. The time-domain HRV index or frequency-domain index is used to quantify the activity of the autonomic nervous system and the user's stress level.

6. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 5, characterized in that: In step S4, the posture indicators include the head tilt angle and the cervical spine load torque; the head tilt angle satisfies In the formula These are the Z-axis coordinates of key points on the head and neck, respectively. The X-axis coordinates are for key points on the head and neck, respectively; the cervical spine load torque satisfies... In the formula To estimate the quality of the head, It is the acceleration due to gravity. As the lever arm, The angle of head tilt forward.

7. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 6, characterized in that: In step S4, the lightweight machine learning model is a decision tree model or a support vector machine model.

8. The method for non-intrusive calculation of human posture and physiological parameters by multi-sensor fusion according to claim 1, characterized in that: In step S1, outlier filtering and background noise removal are performed on the point cloud data sequence; time-series filtering is performed on the multi-region distance matrix to smooth data fluctuations.

Images