Multi-level voxel adaptive laser radar pedestrian tracking method

Abstract

The invention discloses a multistage voxel adaptive laser radar pedestrian tracking method, which comprises the following steps: S1, acquiring laser radar point cloud data, and adding ground filtering and elevation filtering; s2, performing hierarchical voxel grid processing on the basis of the point cloud data after ground filtering and elevation filtering, inputting the point cloud data after hierarchical voxel grid processing into a DV-Det model to detect a point cloud pedestrian target of a current frame, and adding a Soft-NMS for post-processing optimization; s3, designing an unscented Kalman filter based on a pedestrian kinematics equation, predicting the motion state of a pedestrian target, and adding a joint probability data interconnection algorithm and an adaptive life cycle management strategy; and S4, the mass center of the pedestrian target is calculated and detected, the relative position and the movement speed of the pedestrian target are solved, and a display panel of pedestrian state information is added to a visual interface. In conclusion, the defects in the prior art are overcome, high-precision and low-delay pedestrian detection and tracking can be achieved in multiple indoor and outdoor scenes, and the method has high social use value.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian tracking, and in particular to a multi-level voxel adaptive laser radar pedestrian tracking method. Background Art

[0002] In the field of pedestrian monitoring in public places like airports, train stations, and smart campuses, environmental perception systems must be highly robust to cope with complex and changing outdoor conditions. However, traditional vision-based monitoring methods often face challenges such as decreased detection accuracy and increased false alarm rates in conditions with drastic changes in lighting conditions, inclement weather, and equipment contamination, making them difficult to meet the requirements for high-reliability monitoring. In comparison, LiDAR, with its advantages such as active ranging, immunity to ambient light interference, and high-precision three-dimensional perception, is an ideal sensor for pedestrian detection in complex environments. Mechanically rotating LiDAR, in particular, provides high-density point cloud data with a 360° horizontal field of view, wide coverage, and is unaffected by lighting conditions, ensuring stable operation around the clock. Furthermore, LiDAR's depth information directly reflects the spatial structure of the target, avoiding the scale variations associated with traditional vision-based methods due to perspective transformations and occlusions, making it particularly advantageous in tasks such as pedestrian detection, behavior analysis, and trajectory prediction.

[0003] However, early LiDAR-based pedestrian detection methods primarily relied on manually designed features (such as point cloud clustering, geometric shape analysis, and reflectivity threshold segmentation). These methods suffer from high false detection and missed detection rates, weak feature generalization, unstable dynamic target tracking, and sensitivity to environmental interference. To address these issues, there is an urgent need to design an efficient, robust, and complex-adaptive multi-level voxel-based LiDAR pedestrian tracking method that can achieve high-precision, low-latency pedestrian detection and tracking in multiple indoor and outdoor scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0005] Figure 1 It is the overall flow chart of the present invention. Summary of the Invention

[0006] In order to solve the problems mentioned in the above background technology, the present invention provides a multi-level voxel adaptive lidar pedestrian tracking method.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] The multi-level voxel adaptive lidar pedestrian tracking method includes the following steps:

[0009] S1. Obtain lidar point cloud data and add ground and elevation filters;

[0010] S2. Based on the ground-filtered and elevation-filtered point cloud data, a hierarchical voxel grid is applied to the processed point cloud data. This hierarchical voxel grid is then fed into the DV-Det model to detect pedestrian targets in the current frame point cloud. Soft-NMS post-processing optimization is then applied.

[0011] S3. Design an unscented Kalman filter based on the pedestrian kinematic equations to predict the motion state of pedestrian targets, and add a joint probabilistic data interconnection algorithm and an adaptive life cycle management strategy;

[0012] S4. Calculate the center of mass of the detected pedestrian target, solve the relative position and movement speed of the pedestrian target, and add a display panel of pedestrian status information to the visualization interface.

[0013] Preferably, the step S1 specifically includes: using a mechanical rotating laser radar to collect scene point cloud data, and using ground filtering and elevation filtering to remove some invalid point clouds.

[0014] Preferably, the specific contents of using hierarchical voxel grid processing based on the point cloud data after ground filtering and elevation filtering in step S2 include:

[0015] First, for a given input point cloud where p i ∈R 3 is the coordinate, f i ∈R c is characterized by;

[0016] For layer l, set the voxel resolution r l , and calculate the grid

[0017] For each key point Aggregation field points Where k represents the number of voxel divisions in each spatial dimension, and multi-level configuration is performed according to the level l, which can be obtained:

[0018]

[0019] where R l is the domain radius about level l, Indicates the key points Feature representation after voxelization of neighborhood points;

[0020] Extract multi-scale features through 3D sparse convolution and fuse the outputs of different layers:

[0021]

[0022] Preferably, in step S2, the point cloud data processed by the hierarchical voxel grid is input into the DV-Det model to detect pedestrian targets in the current frame point cloud, specifically:

[0023] For each candidate box B = (x, y, z, w, l, h, θ), where x, y, z are the center of mass coordinates of the target in the three-dimensional coordinate system, θ is the heading angle of the target, l, w, h are the three-dimensional dimensions of the target boundary, and the prediction confidence P is cls =σ(MLP(F out )) and regression offset ΔB = MLP reg (F out ), and calculate the prediction box B p With the real box B g 3D IoU:

[0024]

[0025] Among them, A is the intersection area between the prediction box and the true value, and H is the intersection height between the prediction box and the true value.

[0026] Preferably, in step S2, the specific content of adding Soft-NMS post-processing optimization includes:

[0027] Since traditional NMS will suppress overlapping frames, resulting in missed detection of dense pedestrians, the Soft-NMS optimization model is used in the post-processing process to reduce the confidence of overlapping frames through a continuous function instead of directly deleting them.

[0028]

[0029] Preferably, in step S3, an unscented Kalman filter based on the pedestrian kinematic equation is designed to predict the motion state of the pedestrian target, specifically including:

[0030] First, establish the state space equation of pedestrian motion:

[0031]

[0032] when When , the constant turning rate and velocity model degenerates into a uniform velocity model:

[0033]

[0034] Secondly, the pedestrian motion state space equation is used as input, and the unscented Kalman filter (UKF) is used to predict and update the pedestrian's motion trajectory. In the prediction stage, Sigma point sampling (proportionally corrected symmetric sampling) needs to be set:

[0035]

[0036] λ=α 2 (n+κ)-n

[0037] Among them, the scaling factor α = 0.01, the state dimension n = 5, P k-1 is the state covariance matrix of the previous moment; state prediction and covariance prediction are performed based on the Sigma point approximation of nonlinear distribution:

[0038]

[0039] Where Q is the covariance matrix of process noise, W i c is the covariance weight;

[0040] Then observe the Sigma point and the predicted value Z k|k-1 , calculate the Kalman gain K k , update the covariance matrix P k :

[0041]

[0042] Preferably, in step S3, adding the joint probabilistic data interconnection algorithm specifically includes: performing inter-frame data association calculation on the detected pedestrian target through joint probabilistic data interconnection (JPDA), assuming that the effective measurement gate (ellipse verification gate) is:

[0043]

[0044] Where γ is the gating threshold, determined by the chi-square distribution, and H is the observation matrix, which maps the state to the observation space. The calculation process of the joint association probability is as follows:

[0045]

[0046] in, is the square of the Mahalanobis distance between measurement t and target j, P D is the detection probability, λ FA is the false alarm rate;

[0047] According to the joint association probability β jt Perform a mixed state update to obtain the state estimate of measurement t associated with target j

[0048]

[0049] In order to improve the continuity of pedestrian target tracking and prevent the pedestrian ID from being updated after the target is blocked, adaptive life cycle management is added after the pedestrian target is successfully associated, and the target confidence C is calculated. k :

[0050]

[0051] Among them, the attenuation factor α=0.9 controls the weight of historical confidence.

[0052] Preferably, a life cycle decision is made, if C k ≥0.8, it is confirmed as the target of the new frame; if C k If the value is ≤0.2, the pedestrian target is considered to be out of the scene range and is deleted.

[0053] Preferably, the step S4 specifically includes: extracting the geometric center of the detection box as the target position according to the pedestrian target object detected and tracked in step S3. i =(x i ,y i ,z i ,w i ,l i ,h i ,θ i ), the geometric center of mass of the target is calculated as:

[0054]

[0055] Then calculate the instantaneous velocity by the change of the center of mass position of consecutive frames, assuming that C i (t) and C i (t+Δt) is the center of mass position of a single pedestrian target in adjacent frames during the movement process, and its velocity vector is:

[0056]

[0057] Finally, the system integrates and displays pedestrian detection boxes, trajectory lines, motion status information, and confidence labels through real-time interaction with the front-end visualization interface.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The system of the present invention can perform efficient point cloud pedestrian detection in a variety of complex environments, both indoors and outdoors. It can also ensure high detection accuracy and speed, especially in large-scale scenarios with high pedestrian density and large numbers. Through the constructed front-end visualization window, the system's operating status can be monitored in real time, and the detection status and multimodal information of pedestrian objects can be displayed in real time.

[0060] 2. The algorithm involved in the present invention first adopts a method combining ground filtering and elevation filtering based on the height range of pedestrians to filter out more than 50% of non-target point clouds, thereby improving the processing speed of subsequent algorithms. Then, the point cloud is processed using a hierarchical voxel grid, and the sparse point cloud feature extraction is enhanced by deep voxel encoding in conjunction with the DV-Det model, which increases the recall rate of pedestrian detection 15 meters away by 25%. At the same time, Soft-NMS uses Gaussian weighted suppression instead of hard thresholds to effectively improve the stability of pedestrian detection in dense crowd scenes. Then, a fusion tracking framework of UKF+JPDA+adaptive life cycle is constructed. The JPDA algorithm solves the measurement-trajectory fuzzy association in dense scenes through probability weighting, and dynamically adjusts the tracking cycle by integrating trajectory confidence and duration, thereby improving the tracking effect of multiple pedestrian targets and preventing the target ID from being frequently updated;

[0061] 3. Compared to traditional methods based on clustering and dynamic detection, the system of the present invention can effectively classify and identify pedestrian targets, significantly reducing interference from other dynamic objects. Compared to point cloud pedestrian detection methods based on VoxelNet, the method of the present invention has faster processing speed and lower false detection and missed detection rates.

[0062] In summary, the present invention overcomes the shortcomings of the existing technology and can achieve high-precision, low-latency pedestrian detection and tracking in multiple indoor and outdoor scenarios, with high social use value and application prospects. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0064] Example 1

[0065] This embodiment discloses a multi-level voxel adaptive laser radar pedestrian tracking method, including the following steps:

[0066] S1. Obtain lidar point cloud data and add ground and elevation filters;

[0067] S2. Based on the ground-filtered and elevation-filtered point cloud data, a hierarchical voxel grid is applied to the processed point cloud data. This hierarchical voxel grid is then fed into the DV-Det model to detect pedestrian targets in the current frame point cloud. Soft-NMS post-processing optimization is then applied.

[0068] S3. Design an unscented Kalman filter based on the pedestrian kinematic equations to predict the motion state of pedestrian targets, and add a joint probabilistic data interconnection algorithm and an adaptive life cycle management strategy;

[0069] S4. Calculate the center of mass of the detected pedestrian target, solve the relative position and movement speed of the pedestrian target, and add a display panel of pedestrian status information to the visualization interface.

[0070] In a specific embodiment, in step S1: a mechanical rotating laser radar is used to collect scene point cloud data, and ground filtering and elevation filtering are used to remove some invalid point clouds, thereby reducing the amount of point cloud data, alleviating the computational pressure of the algorithm in step S2, and improving the visualization presentation effect in S4.

[0071] In a specific embodiment, the specific content of step S2 includes the following steps:

[0072] S21. Apply hierarchical voxel grid processing to the ground-filtered and elevation-filtered point cloud data based on distance. Use fine-grained voxels for near scenes and gradually increase the voxel size for far scenes to reduce computational effort while retaining key features.

[0073] First, for a given input point cloud where p i ∈R 3 is the coordinate, f i ∈R c As a feature.

[0074] For layer l, set the voxel resolution r l , and calculate the grid

[0075] For each key point Aggregation field points Where k represents the number of voxel divisions in each spatial dimension, and multi-level configuration is performed according to the level l, which can be obtained:

[0076]

[0077] where R l is the domain radius about level l, Indicates the key points Feature representation of neighborhood points after voxelization.

[0078] Extract multi-scale features through 3D sparse convolution and fuse the outputs of different layers:

[0079]

[0080] S22. Input the point cloud data processed by the hierarchical voxel grid into the DV-Det model to detect pedestrian targets in the current frame point cloud:

[0081] For each candidate box B = (x, y, z, w, l, h, θ), predict the confidence P cls =σ(MLP(F out )) and regression offset ΔB = MLP reg (F out ), and calculate the prediction box B p With the real box B g 3D IoU:

[0082]

[0083] Among them, A is the intersection area between the prediction box and the true value, and H is the intersection height between the prediction box and the true value.

[0084] S23. Add Soft-NMS post-processing optimization

[0085] Since traditional NMS will suppress overlapping frames, resulting in missed detection of dense pedestrians, the Soft-NMS optimization model is used in the post-processing process to reduce the confidence of overlapping frames through a continuous function instead of directly deleting them.

[0086]

[0087] In a specific embodiment, the specific content of step S3 includes:

[0088] Design an unscented Kalman filter based on the pedestrian kinematic equation to predict the motion state of pedestrian targets;

[0089] First, establish the state space equation of pedestrian motion:

[0090]

[0091] when When , the constant turning rate and velocity model degenerates into a uniform velocity model:

[0092]

[0093] Secondly, the pedestrian's motion state space equation is used as input to predict and update the pedestrian's motion trajectory using the unscented Kalman filter (UKF). During the prediction phase, Sigma point sampling (scale-corrected symmetric sampling) needs to be set:

[0094]

[0095] λ=α 2 (n+κ)-n

[0096] Among them, the scaling factor α = 0.01, the state dimension n = 5, P k-1 is the state covariance matrix of the previous moment. State prediction and covariance prediction are performed based on the Sigma point approximation of nonlinear distribution:

[0097]

[0098] Where Q is the covariance matrix of process noise, W i c is the covariance weight.

[0099] Then observe the Sigma point and the predicted value Z k|k-1 , calculate the Kalman gain K k , update the covariance matrix P k :

[0100]

[0101] Then, the inter-frame data association calculation of the detected pedestrian targets is performed through the Joint Probabilistic Data Association (JPDA), assuming that the effective measurement gate (ellipse verification gate) is:

[0102]

[0103] Where γ is the gating threshold, determined by the chi-square distribution, and H is the observation matrix, which maps the state to the observation space. The calculation process of the joint association probability is as follows:

[0104]

[0105] in, is the square of the Mahalanobis distance between measurement t and target j, P D is the detection probability, λ FA is the false alarm rate.

[0106] According to the joint association probability β jt Perform a mixed state update to obtain the state estimate of measurement t associated with target j

[0107]

[0108] In order to improve the continuity of pedestrian target tracking and prevent the pedestrian ID from being updated after the target is blocked, adaptive life cycle management is added after the pedestrian target is successfully associated, and the target confidence C is calculated. k :

[0109]

[0110] Among them, the attenuation factor α=0.9 controls the weight of historical confidence.

[0111] Finally, make a life cycle decision. If C k ≥0.8, it is confirmed as the target of the new frame; if C k If the value is ≤0.2, the pedestrian target is considered to be out of the scene range and is deleted.

[0112] In a specific embodiment, the specific content of step S4 includes: according to the pedestrian target object detected and tracked in step S3, extracting the geometric center of the detection box as the target position. For each detected pedestrian 3D bounding box B i =(x i ,y i ,z i ,w i ,l i ,h i ,θ i ), the geometric center of mass of the target is calculated as:

[0113]

[0114] Then calculate the instantaneous velocity by the change of the center of mass position of consecutive frames, assuming that C i (t) and C i (t+Δt) is the center of mass position of a single pedestrian target in adjacent frames during the movement process, and its velocity vector is:

[0115]

[0116] Finally, the system integrates and displays pedestrian detection boxes, trajectory lines, motion status information, and confidence labels through real-time interaction with the front-end visualization interface.

[0117] The algorithm involved in the present invention first adopts a method combining ground filtering and elevation filtering based on the height range of pedestrians to filter out more than 50% of non-target point clouds, thereby improving the processing speed of subsequent algorithms. Then, hierarchical voxel grid processing is applied to the point cloud, and deep voxel encoding is used in conjunction with the DV-Det model to enhance the sparse point cloud feature extraction, which increases the recall rate of pedestrian detection 15 meters away by 25%. At the same time, Soft-NMS uses Gaussian weighted suppression instead of hard thresholds to effectively improve the stability of pedestrian detection in dense crowd scenes. Then, a fusion tracking framework of UKF+JPDA+adaptive life cycle is constructed. The JPDA algorithm solves the measurement-trajectory fuzzy association in dense scenes through probability weighting, and dynamically adjusts the tracking cycle by integrating trajectory confidence and duration, thereby improving the tracking effect of multiple pedestrian targets and preventing the target ID from being frequently updated.

[0118] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A multi-level voxel adaptive lidar pedestrian tracking method, characterized in that: The following steps are involved: S1. Obtain lidar point cloud data and add ground and elevation filters; S2. Based on the ground-filtered and elevation-filtered point cloud data, a hierarchical voxel grid is applied to the processed point cloud data. This hierarchical voxel grid is then fed into the DV-Det model to detect pedestrian targets in the current frame point cloud. Soft-NMS post-processing optimization is then applied. S3. Design an unscented Kalman filter based on the pedestrian kinematic equations to predict the motion state of pedestrian targets, and add a joint probabilistic data interconnection algorithm and an adaptive life cycle management strategy; S4. Calculate the center of mass of the detected pedestrian target, solve the relative position and movement speed of the pedestrian target, and add a display panel of pedestrian status information to the visualization interface.

2. The multi-level voxel adaptive lidar pedestrian tracking method according to claim 1, characterized in that: The step S1 specifically includes: using a mechanical rotating laser radar to collect scene point cloud data, and using ground filtering and elevation filtering to remove some invalid point clouds.

3. The multi-level voxel adaptive lidar pedestrian tracking method according to claim 1, characterized in that: The specific contents of the hierarchical voxel grid processing based on the point cloud data after ground filtering and elevation filtering in step S2 include: First, for a given input point cloud where p i ∈R 3 is the coordinate, f i ∈R c is characterized by; For layer l, set the voxel resolution r l , and calculate the grid For each key point Aggregation field points Where k represents the number of voxel divisions in each spatial dimension, and multi-level configuration is performed according to the level l, which can be obtained: where R l is the domain radius about level l, Indicates the key points Feature representation after voxelization of neighborhood points; Extract multi-scale features through 3D sparse convolution and fuse the outputs of different layers:

4. The multi-level voxel adaptive laser radar pedestrian tracking method according to claim 3, characterized in that: In step S2, the point cloud data processed by the hierarchical voxel grid is input into the DV-Det model to detect pedestrian targets in the current frame point cloud. The specific content is: For each candidate box B = (x, y, z, w, l, h, θ), predict the confidence P cls =σ(MLP(F out )) and regression offset ΔB = MLP reg (F out ), and calculate the prediction box B p With the real box B g 3D IoU: Among them, A is the intersection area between the prediction box and the true value, and H is the intersection height between the prediction box and the true value.

5. The multi-level voxel adaptive lidar pedestrian tracking method according to claim 4, characterized in that: In step S2, the specific contents of adding Soft-NMS post-processing optimization include: Since traditional NMS will suppress overlapping frames, resulting in missed detection of dense pedestrians, the Soft-NMS optimization model is used in the post-processing process to reduce the confidence of overlapping frames through a continuous function instead of directly deleting them.

6. The multi-level voxel adaptive laser radar pedestrian tracking method according to claim 1, characterized in that: In step S3, an unscented Kalman filter based on the pedestrian kinematic equation is designed to predict the motion state of the pedestrian target, specifically including: First, establish the state space equation of pedestrian motion: when When , the constant turning rate and velocity model degenerates into a uniform velocity model: Secondly, the pedestrian motion state space equation is used as input, and the unscented Kalman filter (UKF) is used to predict and update the pedestrian's motion trajectory. In the prediction stage, Sigma point sampling (proportionally corrected symmetric sampling) needs to be set: λ=a 2 (n+k)-n Among them, the scaling factor α = 0.01, the state dimension n = 5, P k-1 is the state covariance matrix of the previous moment; state prediction and covariance prediction are performed based on the Sigma point approximation of nonlinear distribution: Where Q is the covariance matrix of process noise, W i c is the covariance weight; Then observe the Sigma point and the predicted value Z k|k-1 , calculate the Kalman gain K k , update the covariance matrix P k :

7. The multi-level voxel adaptive laser radar pedestrian tracking method according to claim 6, characterized in that: In step S3, adding the joint probabilistic data interconnection algorithm specifically includes: performing inter-frame data association calculation on the detected pedestrian target through joint probabilistic data interconnection (JPDA), assuming that the effective measurement gate (ellipse verification gate) is: Where γ is the gating threshold, which is determined by the chi-square distribution, and H is the observation matrix, which maps the state to the observation space. The calculation process of the joint association probability is as follows: in, is the square of the Mahalanobis distance between measurement t and target j, P D is the detection probability, λ FA is the false alarm rate; According to the joint association probability β jt Perform a mixed state update to obtain the state estimate of measurement t associated with target j In order to improve the continuity of pedestrian target tracking and prevent the pedestrian ID from being updated after the target is blocked, adaptive life cycle management is added after the pedestrian target is successfully associated, and the target confidence C is calculated. k : Among them, the attenuation factor α=0.9 controls the weight of historical confidence.

8. The multi-level voxel adaptive laser radar pedestrian tracking method according to claim 7, characterized in that: Make life cycle decisions, if C k ≥0.8, it is confirmed as the target of the new frame; if C k If the value is ≤0.2, the pedestrian target is considered to be out of the scene range and is deleted.

9. The multi-level voxel adaptive laser radar pedestrian tracking method according to claim 1, characterized in that: The step S4 specifically includes: extracting the geometric center of the detection frame as the target position according to the pedestrian target object detected and tracked in step S3; for each detected pedestrian 3D bounding box B i =(x i ,y i ,z i ,w i ,l i ,h i ,θ i ), the geometric center of mass of the target is calculated as: Then calculate the instantaneous velocity by the change of the center of mass position of consecutive frames, assuming that C i (t) and C i (t+Δt) is the center of mass position of a single pedestrian target in adjacent frames during the movement process, and its velocity vector is: Finally, the system integrates and displays pedestrian detection boxes, trajectory lines, motion status information, and confidence labels through real-time interaction with the front-end visualization interface.

Images