Trajectory smoothness evaluation and occlusion compensation for mmwave radar point cloud enhancement method

Abstract

The application discloses a millimeter wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation, and belongs to the technical field of radar signal processing. The method performs clustering and inter-frame association on original radar point cloud data, and constructs a target motion trajectory. A trajectory smoothness index is calculated. From three dimensions of velocity variation, acceleration variation and motion direction variation, product type aggregation evaluation is performed based on human motion physical constraint parameters, and two-dimensional verification is performed in combination with a signal-to-noise ratio. For real targets and suspicious targets, when occlusion occurs, virtual points are generated frame by frame based on Kalman filter prediction steps, and are dynamically calculated by an environmental complexity factor and adaptively adjusted according to the occlusion time and target speed. A position standard deviation is used as a reliability criterion, and reliable virtual points are inserted into the trajectory to compensate for data loss. The application can effectively eliminate ghost false targets and maintain trajectory continuity when occlusion occurs.

Description

Technical Field

[0001] This invention belongs to the field of radar signal processing technology, specifically relating to a millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation. Background Technology

[0002] Millimeter-wave radar, with its advantages of small size, low cost, good privacy protection, and less susceptibility to light and dust, is increasingly widely used in indoor personnel positioning, security monitoring, and smart homes. With the development of radar chip and antenna technology, the density and information content of point cloud data have significantly improved, providing a data foundation for accurate personnel tracking and behavior recognition. However, the complex indoor environment introduces serious data quality challenges, mainly in the following three aspects:

[0003] 1. Ghosting and Multipath Reflection: Ghosting occurs when radar signals are repeatedly reflected by metallic objects and mistaken for real targets. It manifests as a continuous trajectory but the location does not match the actual target, and the signal strength is relatively high. Multipath reflection is similar to ghosting, but usually weaker, when signals are reflected multiple times by walls, the ground, ceiling, etc. Both share the characteristic of continuous trajectories but motion characteristics that do not conform to human physical constraints, leading to hazards such as incorrect headcount, equipment control failure, and misidentification of behavior.

[0004] 2. Trajectory interruption due to obstruction: When a person is obstructed by furniture, partitions, or walls, the radar reflection signal attenuates sharply or is even completely lost. Traditional tracking algorithms usually passively discard the target or interrupt the trajectory after the signal is lost, resulting in the disruption of tracking continuity and seriously affecting the accuracy of subsequent behavior analysis.

[0005] 3. Environmental noise interference: Thermal noise inside the radar receiver and clutter interference in the environment will appear in the point cloud as isolated noise points or short-term false trajectories.

[0006] To address the above problems, the existing technologies include several main processing approaches, but all of them have significant technical limitations:

[0007] First, traditional filtering methods: This method mainly relies on DBSCAN clustering to remove isolated noise points, combined with Kalman filtering to smooth the trajectory. This method works reasonably well when dealing with simple random noise, but it cannot handle two types of data: one is "ghosting," because "ghosting" is very similar to the real target in statistical characteristics, and traditional filters cannot distinguish it at all; the other is occluded signals, when people are blocked by furniture or other objects, causing signal loss, traditional filtering methods can only passively discard data, resulting in trajectory breaks.

[0008] Second, there are anomaly detection methods based on deep learning. These methods learn motion patterns of normal trajectories by constructing neural networks, and then identify trajectories deviating from these patterns as anomalies. Their drawback is that they require massive amounts of labeled radar point cloud trajectory data for training. However, radar point cloud data is essentially a collection of scattered points, making it impossible to directly determine the specific affiliation of each point (e.g., whether it belongs to a real human body or a ghost) by visual inspection. This results in extremely high labeling costs and significant uncertainty in the labeling results.

[0009] Third, multi-sensor fusion methods. This type of method introduces cameras and infrared sensors for joint verification. Its drawbacks are: it increases additional hardware costs and system complexity, and cameras pose a privacy risk, making it unusable in many scenarios (such as bedrooms and bathrooms); in addition, data from different sensors cannot be synchronized in real time, and how to align the data is also a technical challenge.

[0010] In summary, existing technical solutions lack the two core technical features of "trajectory quality assessment based on human motion physical constraints" and "adaptive occlusion compensation based on state prediction." How to achieve high-precision false target elimination and reliable continuous trajectory tracking using a single radar without additional hardware or changes to the radar's operating mode is a pressing technical challenge in this field. Summary of the Invention

[0011] To address the technical problems of existing technologies failing to effectively distinguish between "ghosting" targets and real targets, and trajectory breaks caused by occlusion, this invention provides a millimeter-wave radar point cloud enhancement method for trajectory smoothness assessment and occlusion compensation. First, inter-frame correlation is performed on the raw radar point cloud data to construct the target's motion trajectory. Then, the trajectory smoothness index (TSI) is calculated based on human motion physical constraint parameters from three dimensions: inter-frame velocity change, acceleration change, and motion direction change. This is combined with a two-dimensional verification using the signal-to-noise ratio (SNR) to classify the trajectory as a real target, a suspicious target, or a false target, eliminating false targets. Finally, for trajectories identified as real or suspicious targets, when occlusion occurs, virtual points are generated frame-by-frame based on a Kalman filter prediction mechanism to compensate for trajectory loss. The process noise covariance matrix is ​​dynamically calculated by the environmental complexity factor and undergoes time-velocity dual adaptive adjustment based on the occlusion duration and target velocity. The reliability of the virtual points is determined by the position standard deviation and effective compensation duration. When the position standard deviation exceeds a threshold, the occlusion duration exceeds the effective compensation duration, or the virtual point generation time exceeds 10 seconds, compensation is stopped and the target is marked as lost.

[0012] This invention is achieved through the following technical solution:

[0013] A millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation includes the following steps:

[0014] S1: Physically Constrained TSI False Target Detection:

[0015] S11. Perform clustering and inter-frame target association on the raw radar point cloud data to construct the target motion trajectory;

[0016] S12. Calculate the inter-frame velocity change, acceleration change, and motion direction change for the target motion trajectory frame by frame. After normalization based on the physical constraint parameters of human motion, calculate the trajectory smoothness index TSI using the smoothness formula.

[0017] S13. Combining TSI and SNR, classify the trajectory into real targets, false targets, or suspicious targets, and delete false targets; output trajectory data with classification labels.

[0018] S2: Occlusion compensation based on state prediction:

[0019] S21. Monitor trajectory data. When the number of consecutive frames lost by the target reaches the first preset threshold and the signal-to-noise ratio before the loss is continuously lower than the second preset threshold, determine that an occlusion event has occurred and record the target state of the last frame before the occlusion.

[0020] S22. Based on the target state, virtual points and covariance matrices are generated frame by frame using Kalman filtering; wherein, the process noise covariance matrix of the virtual points is dynamically calculated through the environmental complexity factor and adaptively adjusted with occlusion time and target speed.

[0021] S23. Calculate the initial confidence of the virtual point based on the label type in step S13 and the speed of the target state in step S21, and then calculate the confidence of the current virtual point through the speed adaptive exponential decay model.

[0022] S24. Calculate the position standard deviation and effective compensation duration of the virtual point based on the confidence level. When the position standard deviation is less than or equal to the threshold and the occlusion duration is less than or equal to the effective compensation duration, insert the virtual point into the trajectory; otherwise, stop generating and mark the target as lost.

[0023] Further, in step S11, the clustering process adopts the DBSCAN algorithm, which sets the neighborhood radius and the minimum number of points to cluster reflection points within the neighborhood radius and with a number not less than the minimum number of points into one class; the inter-frame target association adopts the Hungarian algorithm, which uses the Euclidean distance between detection points in adjacent frames as the cost matrix element. When the distance between the detection point and the position of the last frame of the existing trajectory is less than a preset threshold, the detection point is associated with the corresponding motion trajectory.

[0024] Further, in step S12, the smoothness formula is:

[0025]

[0026]

[0027] in, , , These are the changes in velocity. Change in acceleration and change in direction of motion Normalized values ​​in three dimensions, Maximum acceleration; Maximum jerk; Maximum turning angular velocity; This represents the radar sampling time interval; The total number of frames contained in each trajectory; This is the sensitivity coefficient; Upper limit constraint on the value.

[0028] Further, in step S13, the verification includes:

[0029] Set a threshold for identifying false targets Threshold for determining the true target Effective signal-to-noise ratio threshold ;

[0030] when At that time, regardless of the SNR, the target is identified as a false target and deleted.

[0031] when At that time, regardless of the SNR, it is determined to be a real target and retained;

[0032] when <TSI≤ And SNR ≥ If identified as a suspicious target and retained; <TSI≤ And SNR < If the target is false, it will be deleted.

[0033] Further, in step S21, the first preset threshold for the number of consecutive frames lost by the target is 3 frames, the number of frames with a signal-to-noise ratio continuously lower than the preset threshold before loss is 2 frames, and the second preset threshold for the signal-to-noise ratio is 15dB; the target state in the last frame before occlusion includes position coordinates, velocity components, and covariance matrix.

[0034] Further, in step S22, the prediction equation of the Kalman filter is:

[0035] ;

[0036] ;

[0037] in, The predicted state of the virtual point in frame k contains four components. These are the X coordinate, Y coordinate, X-direction velocity, and Y-direction velocity, respectively. for The covariance matrix reflects the uncertainty of the predicted state; It is a 4×4 state transition matrix, describing the evolution of the uniform motion state; Let be the state vector at time (k-1); Let be the covariance matrix at time (k-1); Let be the process noise covariance matrix, where The duration of the current occlusion. The magnitude of the resultant velocity of the target before it is blocked; This represents the radar sampling time interval; The time step index for the Kalman filter is counted starting from the moment the occlusion occurs. The corresponding virtual point in the first frame after occlusion. The corresponding occlusion Frame virtual point; The number of frames for which the radar does not output target data is determined by counting the number of frames the radar does not output on a frame-by-frame basis, until the target reappears or the prediction becomes unreliable; For matrix The transpose of .

[0038] Furthermore, the process noise covariance matrix is ​​dynamically calculated using the environmental complexity factor and adaptively adjusted with occlusion time and target velocity. The adjustment formula is as follows:

[0039] ;

[0040] ;

[0041] ;

[0042] in, Based on the basic process noise matrix, through the environmental complexity factor Dynamic calculation; The last before the cover The average number of effective reflection points within the frame's preset radius. To calculate the number of frames in the window, if the number of trajectory frames is less than the upper limit of the window, then all accumulated frames are used. The reference reflection point density; The noise variance at the reference position, The reference velocity noise variance; This is the duration of the current occlusion; The time-adaptive growth coefficient represents the rate at which the noise in the control process increases with the duration of occlusion. The resultant velocity of the target in the last frame before the obstruction is determined by the radar Doppler measurement in the last frame before the obstruction and remains a fixed value throughout the entire obstruction event. The speed adaptive coefficient represents the rate at which the noise in the control process increases with the speed.

[0043] Further, in step S23, the initial confidence level is calculated using the following formula:

[0044]

[0045] in: As the initial confidence level, The factor for reducing the authenticity of the target. The velocity sensitivity coefficient; For reference speed; The sum of the velocity of the target in the last frame before occlusion; Covariance matrix The trace of the mid-position component reflects the uncertainty of the predicted position; This represents the maximum permissible location variance.

[0046] The velocity adaptive exponential decay model is as follows:

[0047]

[0048] in, The duration of the occlusion is The speed before the obstruction is The confidence level of the target; This is the initial confidence level; The basic attenuation coefficient; The velocity sensitivity coefficient; For reference speed; The sum of the velocity of the target in the last frame before occlusion; The duration of occlusion is determined by starting the timer from the moment the occlusion is confirmed. ,in, For the occluded frame index, This represents the radar sampling time interval.

[0049] Further, in step S24, the threshold The position standard deviation Calculated using the following formula:

[0050]

[0051] When the following conditions are met simultaneously Duration of occlusion , At a specified time, insert the virtual point into the trajectory; otherwise, stop generating virtual points and mark the target as lost; where, To effectively compensate for the duration, To meet The duration of occlusion; if the target is detected again after the trajectory is marked as lost, the false target detection and occlusion compensation process will be re-executed as a new target.

[0052] Compared with the prior art, the advantages of the present invention are as follows:

[0053] 1. The TSI method proposed in this invention aggregates the TSI values ​​of real trajectories and ghost trajectories by a product of three dimensions: velocity change, acceleration change, and motion direction change. This allows for a TSI value difference of several orders of magnitude between the real trajectory and the ghost trajectory, achieving high separation classification. After combining TSI and SNR dual-dimensional verification, the real target recognition rate reaches 98.3%, and the accuracy reaches 98.1%. This is significantly better than traditional DBSCAN clustering (accuracy 83.8%) and the single SNR threshold method (accuracy 78.8%).

[0054] 2. The occlusion compensation mechanism based on state prediction in this invention can effectively fill the trajectory gaps during occlusion. Virtual points are generated through Kalman filtering prediction equations, and a time-velocity dual adaptive process noise adjustment mechanism adapts to different scene complexities. The position standard deviation reliability criterion is used to reasonably control the accumulation of prediction errors and ensure the reliability of virtual points. Experimental results show that the average deviation reduction rate of the five sets of experiments is 78.5%, and the occlusion duration does not exceed [a certain value]. The time deviation reduction rate exceeded 80%;

[0055] 3. Compared with deep learning-based solutions, this invention does not require sample labeling and can improve the performance of the radar system through algorithms alone; compared with multi-sensor fusion solutions, it only requires a single radar sensor, avoiding the additional costs and privacy leakage risks of multi-sensor calibration and synchronization.

[0056] 4. The TSI false target detection and occlusion compensation modules are executed in sequence: The TSI module first removes false trajectories such as ghosting and multipath reflections to avoid meaningless occlusion compensation for false trajectories; the occlusion compensation module provides continuity assurance for real targets detected by TSI, so that the trajectory data output by the system has both high purity and high continuity. Attached Figure Description

[0057] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0058] Figure 1 This is a schematic diagram of the TSI false target detection process based on physical constraints in this invention;

[0059] Figure 2 This is a schematic diagram of the occlusion compensation and virtual point generation process in this invention;

[0060] Figure 3 This is a schematic diagram of the state prediction occlusion compensation mechanism in this invention. Detailed Implementation

[0061] To clearly and completely describe the technical solution and its specific working process of the present invention, the specific embodiments of the present invention are as follows, in conjunction with the accompanying drawings:

[0062] Example 1

[0063] like Figure 1 As shown, this embodiment provides a millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation, including the following steps:

[0064] Step 1: Data Collection

[0065] Raw point cloud data was acquired using a TI IWR6843AOP EVM millimeter-wave radar at a frame rate of 10Hz. The acquired data included angle, distance, timestamp, signal-to-noise ratio (SNR), and Doppler velocity. Multiple testers performed various motion modes, including normal walking, fast walking, running, and standing still, with each data acquisition session lasting approximately 15 seconds. DBSCAN clustering was performed on each frame of raw point cloud data, with a defined neighborhood radius. The minimum number of points is 3. Radar point cloud reflection points with a spatial distance within 0.3m and a number of at least 3 are clustered into one class to form a target cluster. Isolated points with fewer than 3 points are identified as noise points and removed. Through clustering, the original point cloud data is transformed into a discrete set of target points, laying the foundation for subsequent trajectory association.

[0066] Step 2: Trajectory Association

[0067] The Hungarian algorithm is used to associate target points in adjacent frames, forming complete trajectory data. Specifically, the Euclidean distance between detection points in adjacent frames is used as an element of the cost matrix. For each target point in the frame, calculate its relationship with the first... Construct the Euclidean distance between all target points in the frame. The cost matrix ( For the first Frame target points For the first The number of target points in a frame is used to solve for the optimal binary matching using the Hungarian algorithm, minimizing the total matching cost and achieving a one-to-one correspondence between target points in frames. When the distance between a detected point and the last known position of an existing trajectory is less than a preset threshold ( When the distance is 0.5m, the target points in adjacent frames are associated to form complete trajectory data. Fifteen representative trajectories are selected from the collected data for analysis, including 10 real / suspicious trajectories and 5 ghost trajectories.

[0068] Step 3: TSI Calculation

[0069] Calculate the inter-frame velocity change for each trajectory frame by frame. Inter-frame acceleration change and the change in motion direction between frames Substituting these values ​​into the TSI formula yields the smoothness index of the trajectory, as follows:

[0070] Calculate inter-frame changes: ;

[0071] Divide by the product of the corresponding human motion physical constraint parameter and the sampling time interval: Upper limit constraint when , or When the value exceeds 3, the upper limit of 3 is taken to prevent single-dimensional outliers from dominating the TSI calculation.

[0072] Taking a ghost trajectory G1 as an example (N=4 frames), refer to the motion parameters of each frame shown in Table 1. The TSI calculation process is as follows:

[0073] Table 1: Motion parameters of G1 trajectory in each frame

[0074] Frame number speed (m / s) acceleration (m / s²) Direction angle (rad) 1 1.0 0.5 0.3 2 5.0 5.0 1.5 3 12.0 8.0 2.5 4 20.0 10.0 3.5

[0075] Example of calculating the inter-frame variation and normalized ratio for frame 2 (t=2):

[0076] Inter-frame variation: m / s, m / s², rad

[0077] Normalized ratio: , , Subsequent frames ( , The results calculated using the above formula are shown in Table 2:

[0078] Table 2: Inter-frame variation of G1 trajectory and normalization ratio

[0079] Frame number (m / s) (m / s²) (rad) 2 4.0 4.5 1.2 8.0 (maximum value 3) 9.0 (maximum value 3) 3.822 (take the upper limit of 3) 3 7.0 3.0 1.0 14.0 (maximum value 3) 6.0 (maximum value 3) 3.183 (take the upper limit of 3) 4 8.0 2.0 1.0 16.0 (maximum value 3) 4.0 (maximum value 3) 3.183 (take the upper limit of 3)

[0080] Example of TSI calculation in frame 2 (t=2):

[0081] ;

[0082] Penalties:

[0083] Product term: 10 × 10 × 10 = 1000

[0084] Contribution value: 1000 - 1 = 999

[0085] Subsequent frames (t=3, t=4) are calculated using the above formula, and the results are shown in Table 3:

[0086] Table 3: Frame-by-frame calculation of G1 trajectory TSI

[0087] Frame number Product term Contribution value (product - 1) 2 10 10 10 1000 999 3 10 10 10 1000 999 4 10 10 10 1000 999

[0088]

[0089] The anomalies in ghost trails are usually far greater than in this example. Typical ghost trails exhibit abrupt changes in velocity between ±10 and ±30 m / s, frequent reversals in direction of motion, and r-values ​​in all three dimensions of each frame far exceeding the upper limit of 3, with each frame contributing 999. Taking a typical ghost trail of 5 frames as an example (N=5, each frame contributing 999): If the value is greater than the threshold of 8, it is determined to be a ghost image.

[0090] For comparison, the trajectory N1 (N=10 frames) of a normal person walking at a constant speed in the same scene, and the motion parameters and TSI calculations for each frame are shown in Tables 4-6:

[0091] Table 4: Motion parameters of N1 trajectory in each frame (excerpt)

[0092] Frame number speed (m / s) acceleration (m / s²) Direction angle (rad) 1 1.2 0.2 0.5 2 1.25 0.22 0.51 3 1.3 0.25 0.52 4 1.32 0.23 0.53 5 1.35 0.26 0.54 6 1.38 0.24 0.55 7 1.4 0.27 0.56 8 1.42 0.25 0.57 9 1.45 0.28 0.58 10 1.48 0.26 0.59

[0093] Table 5: Inter-frame variation of N1 trajectory and normalization ratio

[0094] Frame number (m / s) (m / s²) (rad) 2 0.05 0.02 0.01 0.10 0.04 0.0318 3 0.05 0.03 0.01 0.10 0.06 0.0318 4 0.02 0.02 0.01 0.04 0.04 0.0318 5 0.03 0.03 0.01 0.06 0.06 0.0318 6 0.03 0.02 0.01 0.06 0.04 0.0318 7 0.02 0.03 0.01 0.04 0.06 0.0318 8 0.02 0.02 0.01 0.04 0.04 0.0318 9 0.03 0.03 0.01 0.06 0.06 0.0318 10 0.03 0.02 0.01 0.06 0.04 0.0318

[0095] Table 6: Frame-by-frame calculation of N1 trajectory TSI

[0096] Frame number product Contribution value 2 1.01 1.0016 1.0010 1.0126 0.0126 3 1.01 1.0036 1.0010 1.0147 0.0147 4 1.0016 1.0016 1.0010 1.0042 0.0042 5 1.0036 1.0036 1.0010 1.0082 0.0082 6 1.0036 1.0016 1.0010 1.0062 0.0062 7 1.0016 1.0036 1.0010 1.0062 0.0062 8 1.0016 1.0016 1.0010 1.0042 0.0042 9 1.0036 1.0036 1.0010 1.0082 0.0082 10 1.0036 1.0016 1.0010 1.0062 0.0062

[0097] Total contribution value = 0.0126 + 0.0147 + 0.0042 + 0.0082 + 0.0062 + 0.0062 + 0.0042 + 0.0082 + 0.0062 = 0.0707

[0098] (N=10 frames, a total of N-1=9 inter-frame changes)

[0099] TSI ≈ 0.0000786, which is less than the threshold of 0.1, so it is determined to be a real target.

[0100] Sensitivity coefficient Selection criteria: Based on theoretical analysis of the TSI formula, this value ensures that the TSI of a real person walking normally is much less than 0.1, the TSI range of rapid irregular movement is 0.1–8, the TSI of ghost trails is greater than 8, the TSI ranges of the three types of targets do not overlap, and the classification boundaries are clear.

[0101] Step 4: Two-dimensional verification:

[0102] The target type was determined by combining the TSI value and the signal-to-noise ratio (SNR). The classification results of the 15 trajectories in this embodiment are shown in Table 7:

[0103] Table 7: Trajectory Classification Results

[0104] trajectory type Frames N TSI SNR (dB) Judgment result N1 Normal walking 10 0.0000786 22 Real target N2 Normal walking 8 0.00012 20 Real target N3 Normal walking 12 0.00015 18 Real target N4 Normal walking 6 0.00008 24 Real target N5 Normal walking 10 0.00020 19 Real target F1 walking fast 8 0.15 21 Suspicious Target F2 walking fast 6 0.30 23 Suspicious Target F3 walking fast 10 0.45 17 Suspicious Target F4 walking fast 8 0.80 20 Suspicious Target F5 walking fast 6 1.20 22 Suspicious Target G1 Ghost 4 9.99 28 False target G2 Ghost 4 9.99 25 False target G3 Ghost 4 9.99 30 False target G4 Ghost 4 9.99 26 False target G5 Ghost 4 9.99 24 False target

[0105] In this embodiment, the TSI threshold is taken. =8, =0.1: TSI < 0.1 for all true trajectories (N1–N5) All suspicious trajectories (F1–F5) TSI in Between these points, the TSI of all ghost trails (G1–G5) is greater than [a certain value]. (The r values ​​of all three dimensions in each frame exceed the upper limit of 3, and the contribution value is 999, which can be reproduced by the TSI formula).

[0106] Step 5: Output the results:

[0107] According to the judgment results: the TSI of the ghost trajectories of G1–G5 are all greater than 1. The target was identified as a false target and deleted; the TSI of the true trajectories N1–N5 all did not exceed [a certain value]. Identified as a real target and retained; TSI of suspicious F1–F5 trajectories in – If the target's SNR is between 15 dB and above, it is considered a suspicious target and retained. TSI is the core indicator for determining the authenticity of a target; SNR is only used as an auxiliary criterion.

[0108] Step Six: Occlusion Detection

[0109] When a target is not detected for three consecutive frames, and the SNR in the last two frames before the target is not detected is consistently less than 15dB, occlusion is determined to have occurred. In this embodiment, a real target is not detected for four consecutive frames (0.4 seconds), and the SNR in the last two frames before the target disappears is 14dB and 12dB respectively (both less than 15dB), which is determined to be occlusion.

[0110] Step 7: Virtual point generation:

[0111] Virtual points are generated frame by frame based on the Kalman filter uniform motion model (CV model).

[0112] Specific method: Utilize the last known target state before occlusion. (Including position and velocity), through the state transition equation Predict the virtual position of the current frame, and simultaneously predict the equation using covariance. Calculate the uncertainty of the predicted location. Process noise. Employing a dual adaptive adjustment mechanism of time and speed Depending on the duration of occlusion Increase and blockage speed Increased and accelerated growth. This includes a cubic increase in the time dimension. The physical law that velocity uncertainty in Kalman filtering is coupled to position uncertainty through the state transition matrix leads to a second growth in the velocity dimension. This stems from the accelerated expansion of errors caused by mismatch in the high-speed target motion model. Simultaneously, through environmental complexity factors... Dynamic calculation of basic process noise This allows the system to adapt to different environments without requiring manual switching of scene modes.

[0113] in, Through environmental complexity factor Dynamic calculation: Analyze the effective reflection points within a 2-meter radius around the target in the last 5 frames before occlusion. In this example, the average number of reflection points... Baseline density Therefore (Typical indoor environment). Basic process noise. .

[0114] Step 8: Confidence Assessment and Velocity Adaptive Process Noise:

[0115] Known parameters and initial conditions:

[0116] 1) Environmental complexity factor: ( , =10, typical indoor environment)

[0117] 2) Basic process noise matrix:

[0118] 3) Covariance matrix of the last frame before occlusion: (Position variance 0.0001 m², velocity variance 0.001 (m / s)²)

[0119] 4) State transition matrix:

[0120] 5) Sampling time interval: s

[0121] 6) Time-adaptive growth factor: s⁻³

[0122] 7) Speed ​​adaptive coefficient: m⁻²s²

[0123] 8) Target authenticity reduction factor: Actual target Suspicious targets

[0124] 9) Maximum permissible location variance: m²

[0125] 10) Basic attenuation coefficient: =0.15 s⁻¹

[0126] 11) Maximum permissible location standard deviation: m

[0127] Step 9: Recursive calculation of the covariance matrix:

[0128] covariance matrix pass Frame by frame, where .

[0129] covariance matrix It is a 4×4 symmetric matrix, and its specific form is:

[0130]

[0131] Since the X and Y directions are decoupled, the cross-dimensional covariance is zero, and the covariance matrix is ​​symmetric, only the following elements need to be calculated:

[0132]

[0133]

[0134]

[0135]

[0136]

[0137]

[0138] After adding process noise, the result is The location covariance trace is

[0139] The recursive results of the covariance trace at three different velocities are shown in Table 8:

[0140] Table 8: Summary of Covariance Trace Recursive Results at Different Speeds

[0141] k τ(s) v=1 m / s v=2 m / s v=3 m / s 1 0.1 0.0122 0.0392 0.0842 2 0.2 0.0249 0.0804 0.1728 3 0.3 0.0396 0.1278 0.2720 4 0.4 0.0575 0.1837 0.3875 5 0.5 0.0800 0.2510 0.5256 6 0.6 0.1082 0.3331 0.6928 7 0.7 0.1436 0.4335 0.8955 8 0.8 0.1876 0.5558 1.1395

[0142] Step 10: Calculate confidence score frame by frame (considering adaptive velocity decay, taking the true target η=1 as an example):

[0143] Confidence level calculation formula:

[0144]

[0145]

[0146] Reliability criteria:

[0147] Given conditions: Assume the target state in the last frame before occlusion is... (Position (2.0, 3.0) m, velocity component (1.0, 0.5) m / s), then the resultant velocity m / s. Take m / s is used as an approximation (normal walking speed) and the maximum permissible standard deviation of position. m.

[0148] Calculated frame by frame: Frame 1 m, frame 2 m, frame 3 m, frame 4 m, frame 5 m, frame 6 m, 7th frame m. First 6 frames m, virtual point is reliable; Frame 7 m, virtual points are unreliable.

[0149] Frame-by-frame calculation example:

[0150] Virtual point in frame 1 (k=1, τ=0.1 s):

[0151] 1) Process noise:

[0152] 2) Covariance trace: (From S231 recursive calculation)

[0153] 3) Velocity factor:

[0154] 4) Initial confidence level:

[0155] 5) Velocity adaptive attenuation coefficient: s⁻¹

[0156] 6) Time decay correction:

[0157] 7) Reliability assessment: ,reliable

[0158] Subsequent frames (k=2~k=10) are calculated frame by frame using the above formula, and the results are shown in Table 9:

[0159] Table 9: Frame-by-frame calculation results of occlusion compensation (velocity before occlusion) m / s, real target )

[0160] Frame number (s) reliability 1 0.1 0.0061 0.0001 0.0330 0.0122 0.111 0.234 reliable 2 0.2 0.0125 0.0034 0.0650 0.0249 0.158 0.219 reliable 3 0.3 0.0198 0.0099 0.0970 0.0396 0.199 0.205 reliable 4 0.4 0.0288 0.0196 0.1291 0.0575 0.240 0.191 reliable 5 0.5 0.0400 0.0325 0.1611 0.0800 0.283 0.178 reliable 6 0.6 0.0541 0.0486 0.1933 0.1082 0.329 0.165 reliable 7 0.7 0.0718 0.0680 0.2255 0.1436 0.379 0.153 Unreliable

[0161] Note: Covariance matrix It is a 4×4 symmetric matrix; the table only shows elements related to the X direction. Let X be the variance of the location. Let X be the covariance of X position and X velocity. (where X is the velocity variance), the elements in the Y direction are symmetrical with those in the X direction. For position component trace ( ).

[0162] The above content shows that:

[0163] (1) The trace of the covariance matrix grows quadratically with the number of prediction steps: This is because the velocity uncertainty in Kalman filtering is coupled to the position uncertainty through the state transition matrix, which leads to the accelerated expansion of the position variance.

[0164] (2) The confidence level decreases rapidly with the duration of occlusion: the confidence level is 0.234 in the first frame, drops to 0.165 in the sixth frame, and drops to 0.153 in the seventh frame. The rate of decrease is much faster than that of the linear model.

[0165] (3) Reliability is determined by Judgment: When Virtual points are unreliable at time m. In this example, frame 7. The system stops generating virtual points in frame 7.

[0166] (4) Effective compensation duration Determined by the speed before the blockage: m / s Seconds (6 frames).

[0167] Step 11, Track Update:

[0168] As can be seen from the frame-by-frame calculation results of occlusion compensation, the reliability is determined by... The decision is based on a judgment, not a fixed confidence threshold. For the velocity before occlusion in this example... The actual target speed in m / s, frames 1-6 All ≤0.35 (Frame 6) Virtual point is reliable; Frame 7 If the virtual points are unreliable, the system stops generating them and marks them as "target lost". In this example, during the 4-frame occlusion period, the virtual points in frames 1-4 are all reliable, and all are inserted into the trajectory data.

[0169] What to do after the target is lost? First time exceeding The system will mark "target lost" and stop generating virtual points. Furthermore, if virtual point generation takes longer than 10 seconds, regardless of... Whether the threshold is exceeded, target generation will stop and the target will be marked as lost. A 10-second interval is the system-level safety limit. If a target is detected again after a trajectory is marked as lost, the false target detection and occlusion compensation process will be re-executed as a new target.

[0170] Example 2

[0171] Experiment 1: TSI False Target Detection Performance

[0172] There are 120 real trajectories and approximately 40 false trajectories, with a ratio of about 3:1. The false trajectories are generated from the secondary reflection signal when the target is close to the reflector, which is characterized by a strong signal (SNR ≥ 15 dB) but abnormal motion; and multipath reflection trajectories are generated from diffuse scattering noise between multiple reflectors, which is characterized by a weak signal (SNR < 10 dB) and irregular motion.

[0173] The target type of the above data was determined using three methods: the present invention (TSI+SNR dual-dimensional verification), traditional DBSCAN clustering, and a single SNR threshold (15dB). The determination rules for each method are as follows:

[0174] 1) This invention (TSI+SNR dual-dimensional verification): setting , , dB; when TSI > When TSI is ≤ 0, it is considered a false target; when TSI is ≤ 0, it is considered a false target. When it is determined to be a real target; when < TSI ≤ And SNR ≥ If it is identified as a suspicious target (retained), then... < TSI ≤ And SNR < If so, it is determined to be a false target.

[0175] 2) Traditional DBSCAN clustering: setting the neighborhood radius m, minimum number of points = 3, cluster radar point cloud reflection points with a spatial distance within 0.3 m and a number of no less than 3 into one class, retain the trajectory in the clustering result as the real target, and remove isolated noise points; for continuous trajectories (such as ghost shadows), since DBSCAN is only based on spatial distance clustering, it cannot distinguish false trajectories with abnormal motion characteristics, so all of them are retained.

[0176] 3) Single SNR threshold (15dB): Set the SNR threshold to 15 dB. When the average SNR of the trajectory is ≥ 15 dB, it is judged as a real target. When the average SNR of the trajectory is < 15 dB, it is judged as a false target and removed.

[0177] Table 10 shows a comparison between this invention and the traditional DBSCAN clustering method and the single SNR threshold method:

[0178] Table 10: Comparison of False Target Detection

[0179] index This invention (TSI+SNR) Traditional DBSCAN clustering Single SNR threshold (15dB) Correctly identify the real number 118 articles 102 articles 98 items Misjudging real numbers as fake numbers 2 18 articles 22 articles Correctly identify spurious numbers 39 articles 32 articles 28 articles Misjudging a false number as a true number 1 item 8 12 articles True target recognition rate 98.3% 85.0% 81.7% accuracy 98.1% 83.8% 78.8%

[0180] Note: True target recognition rate = (Number of correctly identified true targets / Total number of true trajectories) * 100%; Accuracy = (Number of correctly identified true targets + Number of correctly identified false targets) / Total number of trajectories * 100%.

[0181] Experiment 2: Verification of Occlusion Compensation Effect

[0182] Five sets of experimental data were selected for verification. The experimental results are shown in Table 11, covering different scenarios from short occlusion (3 frames) to long occlusion (10 frames). The experimental parameters are the same as in Example 1, including:

[0183] 1) Environmental complexity factor ,

[0184] 2) Basic process noise matrix ,

[0185] 3) Initial covariance matrix ,

[0186] 4) Sampling time interval s,

[0187] 5) Time-adaptive growth coefficient s⁻³,

[0188] 6) Speed ​​adaptive coefficient m⁻²s²,

[0189] 7) Location standard deviation threshold m.

[0190] Table 11: Verification of Occlusion Compensation

[0191] experimental group Obstruction of frames Duration of occlusion speed Last frame Uncompensated bias There is a compensation bias Deviation reduction rate 1 3 frames 0.3 seconds 1.00m / s 0.6 seconds 0.199 0.30 m 0.042 m 86.0% 2 4 frames 0.4 seconds 1.05m / s 0.55 seconds 0.240 0.42 m 0.068 m 83.8% 3 5 frames 0.5 seconds 0.98m / s 0.62 seconds 0.283 0.49 m 0.091 m 81.4% 4 6 frames 0.6 seconds 1.00m / s 0.6 seconds 0.329 0.60 m 0.135 m 77.5% 5 10 frames 1.0 second 1.00m / s 0.6 seconds >0.35 (starting from frame 7) 1.00 m 0.362 m 63.8%

[0192] Note: No compensation for deviation That is, the target moves at a speed during the duration of occlusion. Displacement in uniform motion; with compensation for deviation. ,in This is the location of the target first detected by the radar after the shielding ends. The last one in the virtual point sequence Location (e.g., the occlusion time in experimental group 5 exceeds) Seconds, the system in the 7th frame due to m stops generating virtual points. (The virtual point position in frame 6); Deviation reduction rate = (uncompensated deviation - compensated deviation) / uncompensated deviation * 100%.

[0193] The average deviation reduction rate of the 5 groups of experiments was 76.2%, and the occlusion duration did not exceed [a certain value]. The deviation reduction rate of the first four groups all exceeded 80%.

[0194] In summary, this invention forms a complete point cloud enhancement chain by sequentially executing false target detection and occlusion compensation processes: In false target detection, based on the physical constraints of human motion, the trajectory smoothness index is calculated by product aggregation of three dimensions: velocity change, acceleration change, and motion direction change. Combined with SNR dual-dimensional verification, the trajectory is classified into real targets, false targets, and suspicious targets, effectively eliminating false targets such as "ghosting" and multipath reflections, and solving the technical problem that traditional filters cannot distinguish continuous false targets. For real and suspicious targets that pass detection, the occlusion compensation process generates virtual points frame by frame based on the Kalman filter prediction step when occlusion occurs. The basic process noise is dynamically calculated through the environmental complexity factor, and a time-velocity dual adaptive adjustment mechanism is introduced. The position standard deviation is used as the reliability criterion to fill the trajectory gaps during occlusion while ensuring the reliability of virtual points, thus solving the technical problem of trajectory breakage caused by occlusion. The two steps are executed sequentially and in synergy—eliminating false trajectories and avoiding meaningless occlusion compensation for false trajectories; and providing continuity assurance for real targets, so that the final output trajectory data has both high purity and high continuity, effectively improving the perception reliability of millimeter-wave radar in complex indoor environments.

[0195] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the protection scope of the present invention.

[0196] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

[0197] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed by the present invention.

Claims

1. A method for enhancing millimeter-wave radar point clouds by evaluating trajectory smoothness and compensating for occlusion, characterized in that, Includes the following steps: S1: Physically Constrained TSI False Target Detection: S11. Perform clustering and inter-frame target association on the raw radar point cloud data to construct the target motion trajectory; S12. Calculate the inter-frame velocity change, acceleration change, and motion direction change for the target motion trajectory frame by frame. After normalization based on the physical constraint parameters of human motion, calculate the trajectory smoothness index TSI using the smoothness formula. S13. Combining TSI and SNR, classify the trajectory into real targets, false targets, or suspicious targets, and delete false targets; output trajectory data with classification labels. S2: Occlusion compensation based on state prediction: S21. Monitor trajectory data. When the number of consecutive frames lost by the target reaches the first preset threshold and the signal-to-noise ratio before the loss is continuously lower than the second preset threshold, determine that an occlusion event has occurred and record the target state of the last frame before the occlusion. S22. Based on the target state, virtual points and covariance matrices are generated frame by frame using Kalman filtering; wherein, the process noise covariance matrix of the virtual points is dynamically calculated through the environmental complexity factor and adaptively adjusted with occlusion time and target speed. S23. Calculate the initial confidence of the virtual point based on the label type in step S13 and the speed of the target state in step S21, and then calculate the confidence of the current virtual point through the speed adaptive exponential decay model. S24. Calculate the position standard deviation and effective compensation duration of the virtual point based on the confidence level. When the position standard deviation is less than or equal to the threshold and the occlusion duration is less than or equal to the effective compensation duration, insert the virtual point into the trajectory; otherwise, stop generating and mark the target as lost.

2. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S11, the clustering process uses the DBSCAN algorithm, which sets the neighborhood radius and the minimum number of points to cluster reflection points within the neighborhood radius and with a number not less than the minimum number of points into one class; the inter-frame target association uses the Hungarian algorithm, which uses the Euclidean distance between detection points in adjacent frames as the cost matrix element. When the distance between the detection point and the position of the last frame of the existing trajectory is less than a preset threshold, the detection point is associated with the corresponding motion trajectory.

3. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S12, the smoothness formula is: in, , , These are the changes in velocity. Change in acceleration and change in direction of motion Normalized values ​​in three dimensions, Maximum acceleration; Maximum acceleration; Maximum turning angular velocity; This represents the radar sampling time interval; The total number of frames contained in each trajectory; This is the sensitivity coefficient; Upper limit constraint on the value.

4. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S13, the verification includes: Set a threshold for identifying false targets Threshold for determining the true target Effective signal-to-noise ratio threshold ; when At that time, regardless of the SNR, the target is identified as a false target and deleted. when At that time, regardless of the SNR, it is determined to be a real target and retained; when <TSI≤ And SNR≥ If identified as a suspicious target and retained; <TSI≤ And SNR < If the target is false, it will be deleted.

5. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S21, the first preset threshold for the number of consecutive frames lost by the target is 3 frames, the number of frames with a signal-to-noise ratio continuously lower than the preset threshold before loss is 2 frames, and the second preset threshold for the signal-to-noise ratio is 15dB; the target state in the last frame before occlusion includes position coordinates, velocity components, and covariance matrix.

6. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S22, the prediction equation of the Kalman filter is: ； ； in, The predicted state of the virtual point in frame k contains four components. These are the X coordinate, Y coordinate, X-direction velocity, and Y-direction velocity, respectively. for The covariance matrix reflects the uncertainty of the predicted state; It is a 4×4 state transition matrix, describing the evolution of the uniform motion state; Let be the state vector at time (k-1); Let be the covariance matrix at time (k-1); Let be the process noise covariance matrix, where The duration of the current occlusion. The magnitude of the resultant velocity of the target before it is blocked; This represents the radar sampling time interval; The time step index for the Kalman filter is counted starting from the moment the occlusion occurs. The corresponding virtual point in the first frame after occlusion. The corresponding occlusion Frame virtual point; The number of frames for which the radar does not output target data is determined by counting the number of frames the radar does not output on a frame-by-frame basis, until the target reappears or the prediction becomes unreliable; For matrix The transpose of .

7. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 6, characterized in that, The process noise covariance matrix is ​​dynamically calculated using the environmental complexity factor and adaptively adjusted according to the occlusion time and target velocity. The adjustment formula is as follows: ； ； ； in, Based on the basic process noise matrix, through the environmental complexity factor Dynamic calculation; The last before the cover The average number of effective reflection points within the frame's preset radius. To calculate the number of frames in the window, if the number of trajectory frames is less than the upper limit of the window, then all accumulated frames are used. The reference reflection point density; The noise variance at the reference position, The reference velocity noise variance; This is the duration of the current occlusion; The time-adaptive growth coefficient represents the rate at which the noise in the control process increases with the duration of occlusion. The resultant velocity of the target in the last frame before the obstruction is determined by the radar Doppler measurement in the last frame before the obstruction and remains a fixed value throughout the entire obstruction event. The speed adaptive coefficient represents the rate at which the noise in the control process increases with the speed.

8. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S23, the initial confidence level is calculated using the following formula: in: As the initial confidence level, The factor for reducing the authenticity of the target. The velocity sensitivity coefficient; For reference speed; The sum of the velocity of the target in the last frame before occlusion; Covariance matrix The trace of the mid-position component reflects the uncertainty of the predicted position; This represents the maximum permissible location variance. The velocity adaptive exponential decay model is as follows: in, The duration of the occlusion is The speed before the obstruction is The confidence level of the target; This is the initial confidence level; The basic attenuation coefficient; The velocity sensitivity coefficient; For reference speed; The sum of the velocity of the target in the last frame before occlusion; The duration of occlusion is determined by starting the timer from the moment the occlusion is confirmed. ,in, For the occluded frame index, This represents the radar sampling time interval.

9. The millimeter-wave radar point cloud enhancement method for trajectory smoothness evaluation and occlusion compensation as described in claim 1, characterized in that, In step S24, the threshold σ_threshold = 0.35 m, and the position standard deviation σ_pos is calculated using the following formula: When the following conditions are met simultaneously Duration of occlusion , At a specified time, insert the virtual point into the trajectory; otherwise, stop generating virtual points and mark the target as lost; where, To effectively compensate for the duration, To meet The duration of occlusion; if the target is detected again after the trajectory is marked as lost, the false target detection and occlusion compensation process will be re-executed as a new target.

Images