A multi-feature fusion-based millimeter wave radar human fall detection method

Abstract

The application discloses a kind of millimeter wave radar human body fall detection methods based on multi-feature fusion, comprising: obtaining original point cloud frame from each radar, time stamp calibration and space coordinate system conversion are carried out;Noise filtering, ground segmentation and human body point cloud extraction are carried out to each frame point cloud;After pre-processing, point cloud is divided into static cluster and dynamic cluster;Filter false dynamic cluster;Extract the remaining dynamic cluster set, adopt tracking algorithm to identify and track human body dynamic cluster in continuous frame;From the tracked human cluster, extract the corrected time sequence feature;The time sequence feature is input into the pre-trained deep time sequence network, learns the fall time sequence dependency relationship, and outputs the abnormal index;Multi-source fusion is carried out with the abnormal index, and the comprehensive index is obtained to determine whether to trigger alarm. It can adapt to complex home environment, improve detection accuracy and real-time performance, and reduce false alarm and missed alarm.

Description

Technical Field

[0001] This invention relates to a millimeter-wave radar method for detecting human falls based on multi-feature fusion, belonging to the field of intelligent recognition technology. Background Technology

[0002] In a home environment, it is difficult to distinguish between falling and lying down. When a user slowly lies down on a mattress or sofa, the change in their z-axis height may be similar to that of a real fall. Traditional detection methods often rely on the body's offset state, and if a person lies down slowly, it is easy to cause false alarms. Furthermore, the reflections generated by walls, floors, and metal furniture can partially overlap with the human body echo in space, which may cause furniture to misjudge a human figure.

[0003] Despite attempts to combine multiple features such as Doppler spectrum, point cloud data, and temporal information, multi-feature fusion methods still face the following challenges: Sparse point cloud data generated by millimeter-wave radar struggles to effectively extract spatial and temporal features in complex environments, impacting recognition performance; existing methods do not fuse extracted features deeply enough, failing to fully exploit the correlations between features and limiting model performance improvement. While existing models perform well on training datasets, their performance significantly degrades in real-world complex environments, such as with furniture occlusion or changes in lighting, and they have limited adaptability to environmental noise and interference, making them susceptible to non-target motion, leading to false positives and false negatives. To improve detection accuracy, many methods employ complex deep learning models, but this introduces the following problems: deep models have high hardware resource requirements, making real-time operation difficult on resource-constrained devices; in practical applications, systems need to respond quickly to fall events, and complex models may cause response delays, affecting practicality.

[0004] Furthermore, existing technologies perform poorly in complex and variable environments in practical applications, particularly in the following aspects: radar signals are easily interfered with in environments with dense furniture or obstructions, affecting detection accuracy; existing models have limited recognition capabilities for slow falls or atypical fall movements, making them prone to missed detections. Summary of the Invention

[0005] The purpose of this invention is to provide a millimeter-wave radar method for human fall detection based on multi-feature fusion, which can adapt to complex home environments, improve detection accuracy and real-time performance, and reduce false alarms and missed alarms, in order to address the problems mentioned above.

[0006] The technical solution adopted in this invention is as follows:

[0007] A millimeter-wave radar method for human fall detection based on multi-feature fusion, characterized by the following steps:

[0008] S1. Data Acquisition and Preprocessing: Obtain raw point cloud frames containing range, velocity, and reflection intensity from each radar, perform timestamp calibration and spatial coordinate system transformation; perform noise filtering, ground segmentation, and human point cloud extraction on each point cloud frame to obtain clean data;

[0009] S2. Dynamic Clustering: A density clustering algorithm based on adaptive neighborhood radius divides the preprocessed point cloud into static clusters and dynamic clusters;

[0010] S3. Multipath False Cluster Filtering: Utilizes temporal features and multi-radar fusion information to filter out false dynamic clusters caused by multipath reflection;

[0011] S4. Temporal Tracking: Extract the remaining dynamic cluster set, and use a tracking algorithm to identify and track the human dynamic clusters in consecutive frames to ensure the continuity of feature calculation;

[0012] S5. Temporal Feature Extraction: Extract the corrected centroid height, velocity, acceleration, and event duration as temporal features from the tracked human clusters;

[0013] S6. Deep Temporal Model Processing: Input temporal features into a pre-trained deep temporal network to learn the temporal dependencies of falls and output anomaly indicators;

[0014] S7. Pose data fusion: Combine the skeletal key points and pose angles obtained by the vision device with anomaly indicators to obtain a comprehensive indicator;

[0015] S8. Fall Alarm Judgment and Output: Determine whether an alarm is triggered based on comprehensive indicators. If triggered, send a reminder and record relevant information.

[0016] S1 Data Acquisition and Preprocessing, through timestamp calibration, coordinate system transformation, and noise filtering, provides clean and synchronized point cloud data for subsequent processing, solving the problems of poor original point cloud quality and asynchronous data from multiple radars. S2 Dynamic Clustering employs density clustering with adaptive neighborhood radius, achieving reasonable clustering in both furniture-dense and open areas, avoiding erroneous merging or splitting of static objects (furniture, walls) and human point clouds, effectively solving the problem of fixed-radius clustering's inability to adapt to environmental density differences. S3 Multipath False Cluster Filtering, through temporal continuity, multi-radar field-of-view verification, and reflection intensity fluctuation analysis, accurately eliminates false clusters generated by multipath reflections, reducing misjudgments caused by reflection interference from furniture, etc. S4 Temporal Tracking uses Kalman filtering to achieve continuous identification of human targets, ensuring the stability of feature extraction and facilitating the capture of the dynamic process of falls. It provides a continuous data foundation; S5 temporal feature extraction focuses on key temporal features such as centroid height, velocity, and acceleration, which can accurately depict the fall process of "rapid descent - landing - lying flat and still", solving the problem of insufficient recognition of slow or atypical fall movements; S6 deep temporal model adopts a pre-trained bidirectional LSTM network, which learns the temporal dependencies of falls while taking into account real-time performance, avoiding the defects of high resource requirements and response delay of complex models; S7 posture data fusion combines visual skeleton information to achieve complementary advantages of radar and vision - radar makes up for the shortcomings of vision in insufficient lighting or occlusion, and vision supplements the information of radar under angle limitations, significantly reducing the probability of everyday actions (such as bending over and squatting) being misjudged as falls; S8 alarm judgment and output ensure rapid response and recording of fall events, improving the system's practicality. This solution addresses the core issues in the background technology in the following ways: First, it solves the problem of false positives caused by furniture occlusion and multipath reflection through adaptive clustering and multipath filtering. Second, it improves the ability to identify slow or atypical falls and reduces missed detections through temporal feature extraction and deep temporal models. Third, it balances detection accuracy and real-time performance through multimodal fusion and lightweight model design, reduces the demand for hardware resources, and solves the problems of response latency and deployment difficulty of complex models. By organically combining adaptive clustering, multipath dynamic filtering, deep temporal learning, and multimodal fusion, a complete solution suitable for complex home environments is formed, breaking through the limitations of existing technologies in feature extraction depth, environmental adaptability, and modal complementarity, and significantly improving the accuracy, robustness, and practicality of fall detection.

[0017] Optionally, in step S1, timestamp calibration of the original point cloud frames achieves time synchronization of multi-radar data; spatial coordinate system transformation unifies the point clouds of each radar to the world coordinate system through rotation matrices and translation vectors. Timestamp calibration and spatial coordinate system transformation of the original point cloud frames are fundamental to the entire detection method, directly affecting the accuracy and reliability of subsequent processing. Timestamp calibration ensures that the point cloud frames acquired by multiple radars are aligned in the time dimension by unifying the time reference of the data collected by each radar, avoiding misalignment in capturing the motion state of dynamic targets (such as humans) due to differences in sampling times of different radars. This provides a time-consistent data foundation for subsequent steps such as dynamic clustering and time-series tracking, solving problems such as chaotic target motion trajectories and distorted feature extraction caused by time asynchrony in multi-source radar data. Spatial coordinate system transformation unifies the point cloud data collected by different radars to the world coordinate system through rotation matrices and translation vectors, eliminating spatial coordinate deviations caused by differences in installation position and orientation angle of each radar. This enables the fusion and analysis of point cloud data from multiple radars within the same spatial framework, ensuring the consistency of the positional information of human targets in different radar fields of view. It avoids problems such as human point cloud segmentation errors and confusion between static clusters (furniture, walls) and dynamic clusters (humans) caused by inconsistent coordinates, providing spatially consistent data support for subsequent accurate extraction of human point clouds and differentiation between static and dynamic targets.

[0018] Alternatively, in step S1, noise filtering is used to remove outlier noise points from the point cloud; ground segmentation uses a random sampling consensus algorithm to fit the ground plane equation and removes ground points whose distance from the plane is less than a threshold. In a home environment, the ground, as a large static area, has a large number of densely distributed reflective point clouds. If these are mixed with human point clouds, the human point cloud may be "submerged" or mistakenly merged with ground points during clustering, affecting the accurate extraction of human targets. By accurately segmenting ground points, the focus can be placed on the point clouds of static objects such as walls and furniture and dynamic targets such as humans, ensuring more accurate division between "static clusters" and "dynamic clusters" in subsequent dynamic clustering and avoiding interference from ground reflections on human point cloud extraction. Noise filtering, by removing outlier noise points from the point cloud, effectively eliminates invalid data generated by environmental interference (such as electromagnetic clutter and equipment noise) to the radar signal, significantly improving the signal-to-noise ratio of the point cloud data and providing a cleaner input for dynamic clustering. Ground segmentation uses the Random Sample Consensus Algorithm (RANSAC) to fit the ground plane equation and removes ground points that are less than a threshold from the plane. Its core function is to separate the ground from the non-ground point cloud.

[0019] Alternatively, in step S2, the adaptive neighborhood radius is determined by calculating the average of the distances from each point to its k-th nearest neighbor and multiplying this average by a coefficient α. Static and dynamic clusters are distinguished by comparing the average velocity of points within the cluster with a velocity threshold; clusters with an average velocity less than the threshold are considered static, and those with an average velocity greater than the threshold are considered dynamic. This adaptive neighborhood radius determination, by calculating the average of the distances from each point to its k-th nearest neighbor and multiplying it by a coefficient α, achieves dynamic adjustment of the neighborhood radius: in areas with high point cloud density, such as dense furniture or near walls, the average distance of local nearest neighbors is small, and the corresponding neighborhood radius decreases accordingly, preventing the point cloud of a human body from being incorrectly merged into the same cluster with the point clouds of surrounding static objects (such as furniture); in open areas, the average distance of local nearest neighbors is large, and the neighborhood radius increases accordingly, preventing the point cloud of the same human body from being split into multiple small clusters due to sparseness. This adaptive adjustment overcomes the limitation of fixed neighborhood radii in taking into account global density differences, ensuring the rationality of clustering results in different environments and providing more accurate cluster division for subsequent target recognition.

[0020] Alternatively, in step S3, the methods for filtering spurious clusters include: removing clusters that are close to the target cluster. Clusters appearing less than a set number within a frame are removed; clusters appearing only in a single radar are eliminated; clusters with a temporal standard deviation of reflection intensity exceeding the normal range for human clusters are also removed. False clusters often appear briefly due to unstable reflection conditions and are difficult to maintain in consecutive frames, while real human bodies, as continuously moving targets, will have corresponding dynamic clusters that appear stably in consecutive frames. (The last sentence appears to be incomplete and possibly refers to the removal of clusters.) Clusters appearing less than a set number within a frame are filtered by temporal continuity, effectively filtering out false targets caused by sudden noise and instantaneous reflection interference. When a real human body is in an overlapping area covered by multiple radars, its point cloud will be captured by multiple radars simultaneously. However, false clusters formed by multipath reflections are limited by the reflection path and can usually only be detected by a single radar. Eliminating clusters that only appear in a single radar reduces false detections caused by interference from a single radar signal. The reflection intensity of a human body to radar signals is relatively stable, while the reflection intensity of false clusters from multipath reflections fluctuates drastically due to changes in the reflection path, and their temporal standard deviation is significantly larger. Eliminating clusters whose temporal standard deviation of reflection intensity exceeds the normal range of human body clusters, and further distinguishing real and false targets at the signal feature level through quantitative signal stability filtering, effectively weakens the interference of multipath effects on detection accuracy.

[0021] Alternatively, in step S4, the tracking algorithm is a Kalman filter, and the tracker state variables include the three-dimensional coordinates and three-dimensional velocity of the human cluster centroid; target association is achieved by comparing the Euclidean distance between the cluster in the new frame and the tracker's predicted position in the previous frame with a matching threshold, continuously... If frames are unrelated, target tracking ends. The introduction of the Kalman filter enables dynamic prediction and correction of the motion state of human clusters. The tracker state variables include the three-dimensional coordinates of the centroid and three-dimensional velocity, which can capture the spatial position of the human body and reflect its motion trend, effectively smoothing noise interference in point cloud data. Even when the point cloud is sparse or occluded in some frames, the human body position can still be inferred based on historical motion features, ensuring the continuity of tracking. Based on the target association method of Euclidean distance and matching threshold, accurate matching of the same human body in continuous frames is achieved by quantifying spatial position differences, avoiding confusion in multi-target scenes and locking in stable target objects for subsequent temporal feature extraction. The design of ending tracking when there is no frame correlation can promptly clear targets that have left the monitoring area, reduce the consumption of unnecessary computing resources, and ensure the efficient operation of the system.

[0022] Alternatively, in step S5, the centroid height correction is obtained by calculating the dot product of the difference vector between the centroid coordinates and the ground origin coordinates and the ground unit normal vector; the extracted temporal features include the duration of the fall phase. During the descent phase, the speed is less than And lasting ≥ Frame determination: the velocity at the moment of contact with the ground is less than... Furthermore, the center of gravity height is below the standing height threshold for determination. In actual home environments, the ground may have slight slopes or bumps, and directly using the original z-axis coordinates will lead to distortion of height data. Correction can more accurately reflect the vertical height change of the human body relative to the ground. In the temporal characteristics, the duration of the descent phase... The determination of speed and the identification of the moment of impact accurately depict the typical dynamic process of a fall: when falling, the human body experiences a continuous phase of rapid descent and impact, while normal movements such as bending over or squatting involve a decrease in height, but the speed often does not meet the rapid descent threshold, or the duration is insufficient, and the impact height will not be lower than the standing threshold. This multi-dimensional determination based on speed, height, and duration can effectively distinguish between falls and daily movements.

[0023] Optionally, in step S6, the deep temporal network is a bidirectional LSTM network. The input is a temporal window feature matrix containing normalized centroid height, velocity, acceleration, body tilt angle, and horizontal movement direction vector. The output anomaly index is the probability of an abnormal fall. Compared to a unidirectional LSTM, which can only utilize past information, a bidirectional LSTM can simultaneously combine feature information from both historical and future moments, more comprehensively capturing continuous temporal dependencies during a fall. For example, when identifying a slow fall, the fundamental difference from normal movement can be distinguished by the trend of height changes and continuous fluctuations in velocity and acceleration between consecutive frames. The input temporal window feature matrix contains normalized centroid height, velocity, acceleration, body tilt angle, and horizontal movement direction vector. These features quantify the human motion state from different dimensions. Height reflects changes in vertical position, velocity and acceleration reflect the intensity of movement, and body tilt angle and horizontal direction vector characterize posture and movement trends. Normalization eliminates feature scale interference from different individuals, ensuring the model focuses on action patterns rather than individual differences.

[0024] Alternatively, in step S7, the visual device is an RGB camera, which acquires key points of the human 3D skeleton through MediaPipe; the posture angle includes the body tilt angle calculated based on the center vectors of the shoulders and hips, and the height difference between the two knees; the fusion method is to weightedly sum the fall tendency score obtained from the visual posture and the abnormal indicators to obtain a comprehensive index. The RGB camera acquires key points of the human 3D skeleton through MediaPipe, calculates the body tilt angle based on the center vectors of the shoulders and hips, which can directly reflect whether the human body is close to lying flat, and distinguishes between squatting and falling to the ground by the height difference between the two knees. These posture details are difficult for millimeter-wave radar to capture accurately; the weighted summation fusion method organically combines the fall tendency score obtained from vision with the abnormal indicators output by radar, making the advantages of the two modalities complementary. When the radar height feature is distorted due to furniture obstruction, the visual posture angle can provide supplementary verification; when insufficient lighting causes blurred visual skeleton recognition, the radar speed and acceleration as temporal features can ensure basic judgment; for daily actions, the fusion can filter out misjudgments through the comprehensive index.

[0025] Alternatively, during hardware deployment, 2-3 millimeter-wave radars can be arranged indoors, with a 15%-30% overlap in coverage area between the radars. The installation height should be 1.2m-1.5m, arranged in an equilateral triangle or inverted "T" shape, and each radar should be calibrated to the world coordinate system. Too few radars can easily create blind spots and struggle to handle scenarios where furniture obstructs the view; too many radars increase data processing burden and hardware costs. A configuration of 2-3 radars balances practicality and economy while ensuring coverage of key areas. The 15%-30% radar coverage overlap provides a physical basis for multi-radar data fusion and cross-validation. Human targets within the overlapping area can be simultaneously detected by multiple radars, facilitating subsequent filtering of multipath false clusters from a single radar by checking for simultaneous presence across multiple radars. Non-overlapping areas are covered collaboratively by multiple radars, reducing missed detections caused by furniture obstruction or blind spots, thus improving coverage integrity. The installation height of 1.2m-1.5m ensures complete capture of human point clouds from feet to head, while avoiding the problems of decreased near-ground target detection accuracy due to excessive height or excessively low height causing head point clouds to exceed the detection range. The equilateral triangle or inverted "T" shaped placement optimizes the geometric layout, ensuring that key areas such as the center and corners of the room are covered by at least two radars, reducing blind spots in areas prone to obstruction, such as densely furnished areas and corners. By standardizing the spatial coordinate reference of each radar, spatial deviations in point clouds caused by differences in installation location and orientation are eliminated, ensuring consistency in coordinates for the same human target captured by different radars. This provides a spatially consistent data foundation for subsequent dynamic clustering and temporal tracking.

[0026] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0027] (1) By automatically calculating the neighborhood radius based on the average local nearest neighbor distance of each frame's point cloud, reasonable clustering results can be obtained simultaneously in densely furnished and open areas. Compared to fixed-radius clustering, in high-density areas such as furniture or walls, human figures are not mistakenly merged with the background, maintaining the integrity of human figure clusters; in sparse and open areas, the same human figure is not split into multiple small clusters, ensuring the continuity of detection. This significantly reduces clustering errors caused by differences in environmental density and improves the robustness of the system in complex home scenarios.

[0028] (2) By combining the three criteria of temporal frame count, cross-radar visibility, and reflection intensity fluctuation, instantaneous false clusters generated by multipath reflections are accurately eliminated; transient noise artifacts are screened out by "stable appearance in recent frames," reducing false alarms; verification using overlapping views of multiple radars ensures that the corresponding cluster is retained only when a real human body can be observed from multiple perspectives; and the drastic fluctuations in multipath intensity are filtered based on the temporal standard deviation of reflection intensity, improving the ability to distinguish false reflections. This effectively reduces the negative impact of multipath effects and surrounding furniture on the detection results, improving detection accuracy.

[0029] (3) By constructing a deep temporal network with a bidirectional LSTM as its core, the model can simultaneously utilize information such as height, velocity, and acceleration from past and future moments to learn temporal patterns of typical fall processes such as "rapid descent—landing—lying flat and still" at the window level; it can distinguish between short-term falling actions such as normal bending, sitting, and squatting and real falls, effectively reducing false negatives and false positives; through large-scale labeled data training, the model has good generalization ability and can adapt to human movements of different heights and different falling methods. The model achieves a balance between accuracy and real-time performance and is suitable for online operation on embedded edge computing platforms.

[0030] (4) The height time-series anomaly indicators of millimeter-wave radar are fused with visual skeleton posture information using confidence weighting. When the signal of millimeter-wave radar is incomplete at certain angles or under occlusion, visual information can provide supplementary information, reducing missed detections. Millimeter-wave radar can still work stably when there is insufficient light or when the human body is partially obscured by furniture, compensating for the weakness of the camera. Different modalities verify each other, significantly reducing the probability of everyday actions such as sitting down and bending over being misjudged as falls. Multimodal fusion makes the overall system more stable and reliable in various complex environments. Attached Figure Description

[0031] Figure 1 This is a flowchart of a millimeter-wave radar method for detecting human falls based on multi-feature fusion. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings.

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0034] (1) Hardware deployment module:

[0035] Two to three millimeter-wave radars are deployed in the indoor spaces that need to be monitored (such as living rooms, bedrooms, and corridors). The positions of the radars should ensure that the coverage area overlaps by 15% to 30%. Each radar should be at the same height, but the horizontal viewing angle coverage range should be different to avoid blind spots. The three radars should be placed in an equilateral triangle or an inverted "T" shape to ensure that the central area of ​​the room is completely covered by at least two radars. The radars are generally installed at a height of about 1.2m to 1.5m above the ground to ensure that complete point cloud information from the feet to the head of the human body can be captured simultaneously.

[0036] Each radar interacts with the central processing unit (Edge server or embedded device with GPU) via Ethernet or wirelessly to achieve real-time point cloud reporting and command issuance.

[0037] During the deployment phase, it is necessary to first determine the installation locations of each radar. Orientation angle (yaw angle) Calibration is performed. Assume that radar A's coordinate system is consistent with the indoor "world coordinate system" and oriented north. Radars B and C undergo a known rigid transformation with respect to the world coordinate system. , ;

[0038] For the single-point cloud acquired by radar i , Indicates distance, Indicates pitch angle, To represent the azimuth angle, first convert it to a three-dimensional point in the radar's own Cartesian coordinate system, then obtain the transformed radar point cloud through rigid transformation. ,in, This represents the rotation matrix from the radar i-coordinate system to the world coordinate system. This represents the corresponding translation vector.

[0039] (2) Software modules:

[0040] Figure 1 This embodiment illustrates a flowchart of a millimeter-wave radar method for human fall detection based on multi-feature fusion, the specific steps of which include:

[0041] S1. Data Acquisition and Preprocessing

[0042] Obtain the original point cloud, point cloud for each frame. Contains several points After obtaining the ground plane equation through a random sampling consensus algorithm, and discarding points whose distance from the plane is less than a threshold as ground points, noise filtering and point cloud fusion are performed, denoted as... ;

[0043] S2, Dynamic Clustering

[0044] Specifically, it includes:

[0045] S21. Calculate the local density of each point using a counting method based on a fixed neighborhood radius:

[0046] ;

[0047] Where 1() represents the indicator function, This represents the initial coarse neighborhood radius (e.g., 0.2m); however, due to the higher point cloud density near indoor furniture or walls and the lower density in open areas, a fixed... It is difficult to consider the overall situation, therefore adaptive adjustment is required; the specific process is as follows:

[0048] For each point First, calculate the set of distances to its nearest neighbor at the nth digit. And take the average to get ; Define point The adaptive neighborhood radius is , >0.1, where The coefficient, set manually, can be empirically set to 1.5-2.0 to ensure that the neighborhood covers a sufficient number of intra-cluster points; thus, in dense point cloud areas such as furniture or walls, Smaller Consequently, it decreases; in open areas, Larger This increases accordingly, thus ensuring a reasonable local neighborhood can be obtained in both environments.

[0049] S22, Using Adaptive Radius Recalculate points Local density: ；

[0050] S23. Based on the adaptive density of step S22 For all points according to Size sorting:

[0051] According to the adaptive density in step S22 For all points according to Sort by size to obtain the sequence At the same time, a minimum distance is defined for each point. , representing the distance from the point to the nearest point with a higher density, is expressed as:

[0052] ;

[0053] according to and The scatter distribution of the data points, with several density peaks selected as cluster centers, is represented as follows:

[0054] Typically, the first M that satisfy the condition are selected. The largest point; for any other point It is then assigned to the cluster containing the nearest and densest cluster center, ultimately resulting in several clusters. , ,…, For each cluster Calculate the average velocity of all points in the cluster. Set a speed threshold ,like Then determine the cluster If the objects are static (fixed objects such as furniture, walls, and floors), then they are static clusters; otherwise, they are dynamic clusters. Let the set of all dynamic clusters be denoted as . .

[0055] S3, Multipath False Cluster Filtering

[0056] In indoor multipath environments, millimeter-wave signals are reflected multiple times by walls, floors, furniture, etc., causing false point cloud clusters to appear in locations that were not originally present. These false clusters often exhibit the following characteristics:

[0057] Point cloud has low density and scattered shape; its temporal features are discontinuous, and its velocity information often does not match the actual human motion features; when multiple radars are fused, it may only appear in a single radar and cannot be detected by other radars.

[0058] Therefore, it is necessary to remove false dynamic clusters before target tracking. The specific steps include:

[0059] Record each cluster in near Whether they exist consecutively within a frame The time window length is manually set. If a cluster is in continuous... If a frame appears only 1-2 times, it can be preliminarily considered a spurious cluster and removed; this is manifested as: let a certain cluster... The number of occurrences in frames t-4, t-3, ..., t is ,like If the number of attempts is less than the set limit, the cluster is considered unstable and will be removed.

[0060] For point clouds that have been jointly registered to the world coordinate system, if a cluster appears only from a single radar viewpoint, but other radars within its visual overlap area cannot detect the corresponding points, it is highly likely to be a multipath artifact, manifesting as: for clusters For each point p, check the reflection intensity measured by each radar. If there is a point cloud output by radar A in the same spatial interval, but radars B and C, which overlap with A in the field of view, have no corresponding reflection, then the cluster is determined to be a false cluster of a single radar and is removed.

[0061] Because the reflection intensity of a real human body is relatively stable at multiple angles, while the reflection intensity of multipath reflection fluctuates drastically, statistical clusters... If the standard deviation of the reflection intensity of all points within the cluster in recent frames is greater than the normal range for human clusters, then the cluster may be a multipath artifact and should be removed.

[0062] S4, Timing Tracing

[0063] Extract the remaining dynamic cluster set as And perform dynamic tracking, specifically as follows:

[0064] For each cluster Calculate its centroid coordinates and center of mass velocity Define tracker state variables Tracking via Kalman filter Where F represents the uniform velocity model; providing observations to the Kalman filter. The three-dimensional coordinates of the current frame cluster centroid are used; the Kalman gain K(t) is updated using the observation matrix, and a new state estimate is calculated. If a cluster in the new frame is located at a position predicted by a tracker in the previous frame... If the Euclidean distance between the target and the target is less than the matching threshold, the target is associated with that tracker; otherwise, it is considered a new target, and a new tracker is started. If a tracker continuously... If no one is associated with the target in the frame, the target is considered to have left the field of view, and tracking ends. The maximum number of frames that can be lost is set manually.

[0065] S5. Temporal Feature Extraction

[0066] include:

[0067] For the tracked number Human body number 1, extract its three-dimensional centroid coordinates of And perform height correction for the influence of ground tilt to obtain the corrected linear coordinates: ,in, Represented as the ground unit normal vector, Indicates a ground reference point;

[0068] Calculate the vertical velocity of the center of mass based on the corrected coordinates. Vertical acceleration of the center of mass ;

[0069] During the rapid descent phase, a speed threshold is set. ,when And the duration is greater than or equal to Frame, indicating the start of the descent phase, time point recorded as Determining the time of ground contact by analyzing the trend of altitude changes is specifically manifested as follows: When the center of gravity height is lower than the set standing height threshold, it is determined to be the moment of ground contact. Therefore, the duration of the fall / fall phase is expressed as: .

[0070] S6, Deep Temporal Model Processing

[0071] For the current time t, create a time window of length w. For time Extract the normalized feature vectors ,in This represents the normalized centroid height (normalized by subtracting the standing height and dividing by the height range). , These represent the normalized velocity and acceleration, respectively. , These represent the body tilt angle and horizontal movement direction vector calculated by fitting radar point cloud markers to shoulder and hip points, respectively; and the entire window is then concatenated into a matrix. As the input sequence for a bidirectional LSTM, training samples are constructed using labeled time-series data of historical normal behavior and fall times, constructed in a sliding window manner. ,in This indicates that a fall occurred inside the window. This represents the normal state; the final output layer is a single neuron, which outputs the abnormal probability of falling through an activation function. And use this probability as an indicator of fall anomalies. The output is fed into the anomaly index fusion model.

[0072] S7, Attitude Data Fusion

[0073] Since many common activities (such as bending over to tie shoelaces, picking up objects, squatting, etc.) also show short-term drops on the millimeter-wave point cloud height curve, but the skeletal posture tilt angle does not reach the level of a fall, an abnormal index fusion model is constructed to determine the target's activity state.

[0074] The anomaly indicator fusion model involves deploying a regular RGB camera indoors, using MediaPipe to acquire key points of the human 3D skeleton, and obtaining the coordinates of key joints. The body tilt is calculated using key points at the shoulder and hip. Specifically, if the shoulder center is ksho and the hip center is khip, the tilt vector is represented as follows: Take the dot product with the unit vector in the vertical direction of the world coordinate system to obtain Therefore, the target tilt angle is expressed as The larger the tilt angle, the closer the body is to lying flat or bending over; calculate the height of the knee center relative to the height of the hip center. The difference, let the left knee height be... and right knee height Then, the height difference between the two knees is expressed as ,like Less than the set threshold If the duration is less than the set time, it is judged as squatting.

[0075] Fall tendency score is calculated using the target tilt angle and the difference in knee height. ,in, Indicates the reference value for tilt angle, if If the logarithmic contribution of this term is 0, then the logarithmic contribution of this term is 0. If the value is 0, then the value of this item is 0. This represents the Sigmoid function. and Indicates the weight.

[0076] S8. Fall Alarm Detection and Output

[0077] Falling tendency score Fall Abnormality Indicators The fall threat value is obtained by weighted summation. If the fall threat value exceeds the set threshold, a fall alarm is triggered.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. The invention extends to any new features or combinations disclosed in this specification, and any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention. It is obvious to those skilled in the art that the invention is not limited to the details of the above exemplary embodiments, and that the detailed technical features not disclosed in this embodiment are all prior art, which can be obtained by those skilled in the art from the prior art. For those skilled in the art, the specific manner of terminology in the embodiments of the present invention can be understood according to the specific circumstances, and this disclosure does not specifically limit the embodiments in this regard.

Claims

1. A millimeter-wave radar method for human fall detection based on multi-feature fusion, characterized in that, Includes the following steps: S1. Data Acquisition and Preprocessing: Obtain raw point cloud frames containing range, velocity, and reflection intensity from each radar, perform timestamp calibration and spatial coordinate system transformation; perform noise filtering, ground segmentation, and human point cloud extraction on each point cloud frame to obtain clean data; S2. Dynamic Clustering: A density clustering algorithm based on adaptive neighborhood radius divides the preprocessed point cloud into static clusters and dynamic clusters; S3. Multipath False Cluster Filtering: Utilizes temporal features and multi-radar fusion information to filter out false dynamic clusters caused by multipath reflection; S4. Temporal Tracking: Extract the remaining dynamic cluster set, and use a tracking algorithm to identify and track the human dynamic clusters in consecutive frames to ensure the continuity of feature calculation; S5. Temporal Feature Extraction: Extract the corrected centroid height, velocity, acceleration, and event duration as temporal features from the tracked human clusters; S6. Deep Temporal Model Processing: Input temporal features into a pre-trained deep temporal network to learn the temporal dependencies of falls and output anomaly indicators; S7. Pose data fusion: Combine the skeletal key points and pose angles obtained by the vision device with anomaly indicators to obtain a comprehensive indicator; S8. Fall Alarm Judgment and Output: Determine whether an alarm is triggered based on comprehensive indicators. If triggered, send a reminder and record relevant information.

2. The method according to claim 1, characterized in that, In step S1, the timestamp calibration of the original point cloud frame realizes the time synchronization of multiple radar data; the spatial coordinate system transformation unifies the radar point clouds to the world coordinate system through rotation matrix and translation vector.

3. The method according to claim 1, characterized in that, In step S1, noise filtering is used to remove outlier noise points in the point cloud; ground segmentation uses a random sampling consensus algorithm to fit the ground plane equation and removes ground points that are less than a threshold distance from the plane.

4. The method according to claim 1, characterized in that, In step S2, the adaptive neighborhood radius is determined by calculating the average of the set of distances from each point to its k-th nearest neighbor, and multiplying the average value by the coefficient α. Static clusters and dynamic clusters are distinguished by comparing the average velocity of points within the cluster with a velocity threshold. Clusters with an average velocity less than the threshold are static clusters, while those with an average velocity greater than the threshold are dynamic clusters.

5. The method according to claim 1, characterized in that, In step S3, the methods for filtering fake clusters include: removing nearby clusters. Clusters that appear less than a set value within a frame are removed; clusters that appear only in a single radar are removed; clusters whose reflection intensity temporal standard deviation exceeds the normal range for human clusters are removed.

6. The method according to claim 1, characterized in that, In step S4, the tracking algorithm is a Kalman filter, and the tracker state variables include the three-dimensional coordinates and three-dimensional velocity of the human cluster centroid. Target association is achieved by comparing the Euclidean distance between the cluster in the new frame and the tracker's predicted position in the previous frame with a matching threshold. If the frames are not associated, the target tracking will end.

7. The method according to claim 1, characterized in that, In step S5, the centroid height correction is calculated by the dot product of the difference vector between the centroid coordinates and the ground origin coordinates and the ground unit normal vector; the extracted temporal features include the duration of the fall phase. During the descent phase, the speed is less than And lasting ≥ Frame determination: the velocity at the moment of contact with the ground is less than... Furthermore, the centroid height is determined to be lower than the standing height threshold.

8. The method according to claim 1, characterized in that, In step S6, the deep temporal network is a bidirectional LSTM network. The input is a temporal window feature matrix containing normalized centroid height, velocity, acceleration, body tilt angle, and horizontal movement direction vector. The output anomaly index is the fall anomaly probability.

9. The method according to claim 1, characterized in that, In step S7, the vision device is an RGB camera, which acquires the key points of the human body's 3D skeleton through MediaPipe; the posture angle includes the body tilt angle calculated based on the center vectors of the shoulder and hip, as well as the height difference between the two knees; the fusion method is to weight and sum the fall tendency score obtained from the visual posture with the abnormal indicators to obtain a comprehensive index.

10. The method according to claim 1, characterized in that, During hardware deployment, 2-3 millimeter-wave radars are arranged in the indoor space, with the coverage area of ​​the radars overlapping by 15%-30%. The installation height is 1.2m-1.5m, and they are placed in an equilateral triangle or inverted "T" shape. Each radar is converted to the world coordinate system through coordinate calibration.

Images