Point cloud pedestrian identification and tracking method based on speed fluctuation characteristics, perception system and vehicle

Abstract

The invention relates to the technical field of automotive electronics, in particular to a point cloud pedestrian recognition and tracking method based on speed fluctuation characteristics, a sensing system and a vehicle, and the method comprises the following specific steps: S1, data acquisition and preprocessing; s2, estimating the overall movement speed of the target; s3, speed fluctuation feature extraction; s4, target classification based on multiple features; s5, an improved tracking management strategy; s6, performing tracking management on the classified targets; the sensing system comprises a memory and a processor, computer programs are stored in the memory, and the processor executes the computer programs and instructions of the computer programs; the vehicle is specifically a vehicle carrying the sensing system. A'micro-motion target 'is used as an independent tracking object, so that the problem that a tracking trajectory is lost and then restarted due to the fact that pedestrians are momentarily static is effectively avoided, a smoother and more continuous trajectory is formed, and more reliable information is provided for a downstream prediction and decision-making module.

Description

A point cloud-based pedestrian recognition and tracking method, sensing system, and vehicle based on velocity fluctuation features. Technical Field

[0001] This invention relates to the field of automotive electronics technology, specifically a point cloud-based pedestrian recognition and tracking method, perception system, and vehicle based on speed fluctuation characteristics. Background Technology

[0002] With the development of autonomous driving technology, the performance requirements of vehicle perception systems are increasing. Traditional millimeter-wave radar provides relatively sparse point clouds, mainly used for detecting large targets such as vehicles. In recent years, 4D imaging millimeter-wave radar technology has made significant progress. The point clouds it provides have greatly improved the pitch resolution and point cloud density in the three dimensions of angle, distance, and velocity, enabling it to depict more detailed contours of targets.

[0003] However, the increased point cloud density also presents significant challenges to in-vehicle computing platforms. Due to limited computing resources, it is impossible to perform full, point-level tracking and processing of all point clouds in a scene. The current industry practice is to first cluster the point cloud to form candidate targets; then, based on the overall motion velocity of these candidate targets (usually estimated using the centroid displacement method), they are divided into two main categories: "dynamic targets" and "static targets." Tracking resources are then primarily allocated to "dynamic targets," or "dynamic targets" plus "static targets" in specific areas (such as within a lane).

[0004] This binary "either dynamic or static" classification strategy has significant flaws, particularly in the perception of pedestrians: 1. Slow tracking initiation: Pedestrian movement is characterized by low speed, pauses, and agile turning. When a pedestrian is standing and waiting, walking slowly, or starting to move, their overall speed is likely to be lower than the set "dynamic target" threshold, thus being incorrectly classified as a "static target." Since tracking typically requires confirmation of a dynamic target across multiple frames before initiating a trajectory, the tracking trajectory for pedestrians starts slowly, or even fails to start at all.

[0005] 2. High risk of missing tracks: In complex urban scenarios, pedestrians frequently stop and start. The binary method based on overall speed will cause the tracking trajectory of pedestrians to frequently "terminate" and "restart," resulting in trajectory fragmentation and making it impossible to form a stable and continuous tracking, thus bringing the risk of misjudgment to the autonomous driving decision-making system.

[0006] Therefore, there is an urgent need in this field for a technical solution that can quickly and accurately identify and initiate pedestrian tracking under conditions of limited computing resources. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a point cloud-based pedestrian recognition and tracking method, a sensing system, and a vehicle based on velocity fluctuation characteristics.

[0008] A point cloud pedestrian recognition and tracking method based on velocity fluctuation features comprises the following steps: S1, Data acquisition and preprocessing: acquiring radar sequence point cloud data, clustering it into candidate targets, and performing preprocessing; S2, Overall target motion velocity estimation: calculating the overall motion velocity and velocity fluctuation features of each candidate target; S3, Velocity fluctuation feature extraction: based on the overall motion velocity and velocity fluctuation features, classifying the candidate targets into at least static targets, dynamic targets, and micro-moving targets; S4, Target classification based on multiple features: based on the overall motion velocity, calculating the velocity fluctuation features of the point cloud cluster, which characterizes the degree of dispersion or periodic change of the velocity of each point within the point cloud cluster relative to the overall motion velocity; S5, Improved tracking management strategy: the tracker receives the classification results from step S4 and performs differentiated tracking management; S6, Tracking management of classified targets: for point cloud clusters classified as micro-moving targets, the tracker immediately or with a lower latency threshold initiates a tracking trajectory for them.

[0009] The candidate target points in step S1 are the standard deviation, variance, and entropy of the radial velocity of all points within the cloud cluster, or the energy value in the 1-2.5Hz frequency band obtained based on frequency domain analysis.

[0010] The preprocessing in step S1 includes noise filtering, ground segmentation, and point cloud clustering.

[0011] The velocity fluctuation feature in step S2 is an indicator used to characterize the discreteness or periodic micro-movement of the velocity distribution of the point cloud inside the target.

[0012] The classification based on overall motion speed and speed fluctuation characteristics in step S3 specifically refers to: (1) if the overall motion speed is higher than the first threshold, it is a dynamic target; (2) if the overall motion speed is lower than the first threshold and the speed fluctuation characteristics are higher than the second threshold, it is a micro-motion target; (3) if the overall motion speed is lower than the first threshold and the speed fluctuation characteristics are lower than the second threshold, it is a static target.

[0013] The calculation method for velocity fluctuation characteristics in step S4 includes one or more of the following combinations: (1) Calculate the standard deviation or variance of the radial velocity of all points directly provided by the radar within the point cloud cluster; (2) Calculate the statistical quantity of the difference between the radial velocity of all points within the point cloud cluster and the projection of the overall motion velocity onto the radar radial direction; (3) Perform frequency domain transformation on the overall motion velocity sequence or velocity fluctuation sequence of the point cloud cluster in consecutive frames, and extract the energy in the 1.0Hz to 2.5Hz frequency band as periodic fluctuation characteristics.

[0014] The tracker in step S5 is based on a module of Kalman filtering and multiple hypothesis tracking.

[0015] In step S6, the tracking management specifically includes: (1) starting or maintaining a tracking trajectory for both dynamic and micro-moving targets; (2) assigning a "tentative" state to the tracking trajectory of a micro-moving target, and switching to a "confirmed" state after it is successfully associated for multiple consecutive frames and its features are stable; (3) for static targets, not starting tracking or only starting tracking in specific dangerous areas.

[0016] As an improvement to the present invention, a sensing system is proposed, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program and its instructions to implement the steps of the method described above.

[0017] As another improvement to the present invention, a vehicle equipped with the aforementioned sensing system is proposed.

[0018] The beneficial effects of this invention are: 1. Early identification and rapid tracking: By introducing the biomechanical characteristic of "speed fluctuation," it is possible to distinguish pedestrians from stationary obstacles, such as trees and stakes, when their overall speed is very low, such as when they are standing and moving slightly or starting slowly, and immediately start the tracking process, greatly shortening the start time of the pedestrian trajectory; 2. Improved tracking stability: Treating "slightly moving targets" as an independent tracking object effectively avoids the problem of losing the tracking trajectory due to the pedestrian's brief stillness and then restarting, forming a smoother and more continuous trajectory, providing more reliable information for downstream prediction and decision-making modules; 3. High computational efficiency: This invention only adds feature calculations for the internal points of clustered targets to the existing processing flow, without the need for intensive calculations on the entire scene point cloud, resulting in a small computational increment, which is very suitable for the actual situation of limited resources on vehicle computing platforms, achieving a good balance between performance and resources. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Figure 1 is a flowchart of the present invention. Detailed Implementation

[0021] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below.

[0022] Example 1: As shown in Figure 1, a point cloud pedestrian recognition and tracking method based on velocity fluctuation features can be deployed on an in-vehicle computing platform, such as ARM SoC or GPU. Its perception data comes from an in-vehicle 4D imaging millimeter-wave radar, which can provide high-density point cloud data. Each point contains at least the following information: three-dimensional coordinates (x, y, z), radial velocity (Vr), radar cross section (RCS), and point cloud intensity. The specific steps are as follows: S1, Data acquisition and preprocessing: The in-vehicle 4D millimeter-wave radar collects point cloud data of the surrounding environment at a fixed frequency, such as 10 Hz. After receiving the raw point cloud, the processor acquires radar sequence point cloud data, clusters it into candidate targets, and performs preprocessing; S2, Overall target motion velocity estimation: Calculate the overall motion velocity and velocity fluctuation characteristics of each candidate target; S3, Velocity fluctuation feature extraction: Based on the overall motion velocity and velocity fluctuation characteristics, classify the candidate targets into at least static targets, dynamic targets, and micro-moving targets; S4, Target classification based on multiple features: Based on the overall motion velocity, calculate the velocity fluctuation characteristics of the point cloud cluster, which is used to characterize the degree of dispersion or periodic change of the velocity of each point within the point cloud cluster relative to the overall motion velocity; S5, Improved tracking management strategy: The tracker receives the classification results from step S4 and performs differentiated tracking management; S6, Tracking management of classified targets: For point cloud clusters classified as micro-moving targets, the tracker immediately or with a lower latency threshold starts a tracking trajectory for them.

[0023] The candidate target points in step S1 are the standard deviation, variance, and entropy of the radial velocity of all points within the cloud cluster, or the energy value in the 1-2.5Hz frequency band obtained based on frequency domain analysis.

[0024] By introducing the biomechanical characteristic of "speed fluctuation," it is possible to distinguish pedestrians from stationary obstacles, such as trees and posts, when their overall speed is very low, such as when they are standing and moving slightly or starting slowly, and to immediately initiate the tracking process, which greatly shortens the start time of pedestrian trajectory.

[0025] The preprocessing in step S1 includes: (1) noise filtering: using a statistical outlier removal algorithm or an RCS / intensity-based thresholding method to filter out unreliable noise points.

[0026] (2) Ground segmentation: Methods such as Ray Ground Filter or planar model fitting (such as RANSAC) are used to separate and remove ground point clouds from non-ground point clouds, which significantly reduces the computational load of subsequent processing.

[0027] (3) Point cloud clustering: Clustering algorithms, such as Euclidean clustering (DBSCAN or its variants), are used for non-terrestrial point clouds to aggregate spatially adjacent point clouds into independent candidate target point cloud clusters. Each cluster is assigned a unique temporary ID and its three-dimensional bounding box and centroid coordinates are calculated.

[0028] The velocity fluctuation feature in step S2 is an index used to characterize the discreteness or periodic micro-motion of the velocity distribution of the point cloud inside the target. Specifically, for each candidate target point cloud cluster, it is necessary to estimate its overall motion velocity in the radar coordinate system. Here, two preferred implementation schemes are provided: Implementation scheme A, namely the centroid displacement method: by associating point cloud clusters of the same target in consecutive frames, the centroid displacement is calculated. Assuming the current frame time is t, the previous frame time is t-1, and the target centroid coordinates are C_t and C_{t-1}, the overall motion velocity V_overal can be estimated as:

[0029] This method is simple to implement but relies on stable and accurate inter-frame data correlation. Implementation scheme B, namely the Doppler velocity centroid method, is a better method. It utilizes the radial velocity information of each point directly provided by the 4D radar. For a point cloud cluster, a weighted average of the radial velocities of all points within it is calculated, for example, using RCS or intensity as weights, and this average is used as the projection of the overall velocity of the cluster onto the radar radial direction. Although this is a radial component, it is sufficient for a reliable feature to determine the target's dynamic state in a short time. This method does not require complex inter-frame correlation and has a faster response.

[0030] The classification based on overall motion speed and speed fluctuation characteristics in step S3 specifically refers to: (1) if the overall motion speed is higher than the first threshold, it is a dynamic target; (2) if the overall motion speed is lower than the first threshold and the speed fluctuation characteristics are higher than the second threshold, it is a micro-motion target; (3) if the overall motion speed is lower than the first threshold and the speed fluctuation characteristics are lower than the second threshold, it is a static target.

[0031] In step S3, for each candidate target point cloud cluster, while estimating its overall motion velocity, its velocity fluctuation feature (VFF) is calculated. This feature is used to quantify the micro-motions within the target. The following are several feasible calculation methods: Implementation scheme A, radial velocity standard deviation - simplest implementation: For a point cloud cluster containing N points, its velocity fluctuation feature VFF_std is calculated as follows: Where Vr_i is the radial velocity of the i-th point, and μ_Vr is the average radial velocity of all points in the cluster.

[0032] Advantages: Minimal computational cost; effectively captures the discreteness of the velocity field inside the target. Due to the swinging of limbs, the VFF_std value of a pedestrian is significantly higher than that of a rigid body such as a vehicle.

[0033] Implementation Plan B, based on fluctuations in overall velocity compensation: This method is more accurate. First, it assumes the target is a rigid body, whose ideal radial velocity field should be determined by the projection of the overall velocity V_overall onto the radar in all directions. Velocity fluctuations are the residuals between the actual observed values ​​and the ideal values.

[0034] The overall velocity vector V_overall of the target can be estimated by obtaining a three-dimensional vector through implementation scheme A of S2.

[0035] For each point i within the cluster, its unit direction vector is u_i, then: ideal radial velocity Vr_ideal_i = V_overall · u_i.

[0036] Calculate the speed residual: Residual_i = Vr_i - Vr_ideal_i.

[0037] The velocity fluctuation characteristic VFF_res can be defined as the standard deviation or mean absolute deviation of these residuals.

[0038] Advantages: It can better separate the overall motion from the internal micro-motions and has a strong ability to resist interference from the overall motion.

[0039] Implementation scheme C, frequency domain feature analysis: This method is suitable for analyzing targets in multiple consecutive frames, such as a 1-second sliding window with a total of 10 frames.

[0040] For the same target, extract its VFF_std for each frame within the window, from implementation scheme A or the overall radial velocity, to form a time series.

[0041] Perform a Fast Fourier Transform (FFT) on the sequence to obtain its spectrum.

[0042] The energy integral of the spectrum in the range of 1.0 Hz to 2.5 Hz, which is within the typical human gait frequency range, is calculated as the velocity fluctuation characteristic VFF_freq.

[0043] Advantages: It can directly capture the periodic gait characteristics of pedestrians with extremely high specificity.

[0044] In practical applications, the computationally simple VFF_std can be preferred and combined with the geometric features of the target, such as the bounding box height, to achieve the best balance between performance and efficiency.

[0045] Treating "micro-moving targets" as a separate tracking object effectively avoids the problem of tracking trajectories being lost due to pedestrians' brief periods of stillness, which then restart. This results in smoother, more continuous trajectories, providing more reliable information for downstream prediction and decision-making modules.

[0046] The calculation method for velocity fluctuation characteristics in step S4 includes one or more of the following combinations: (1) Calculate the standard deviation or variance of the radial velocity of all points directly provided by the radar within the point cloud cluster; (2) Calculate the statistical quantity of the difference between the radial velocity of all points within the point cloud cluster and the projection of the overall motion velocity onto the radar radial direction; (3) Perform frequency domain transformation on the overall motion velocity sequence or velocity fluctuation sequence of the point cloud cluster in consecutive frames, and extract the energy in the 1.0 Hz to 2.5 Hz frequency band as periodic fluctuation characteristics.

[0047] Using the features obtained from S2 and S3, candidate targets are finely classified by setting two thresholds: dynamic velocity threshold V_th_dynamic: for example, 0.5 m / s, to distinguish targets with obvious motion.

[0048] Fluctuation threshold F_th_micro: This threshold can be obtained statistically from a large dataset containing pedestrians and static objects. For example, it can be set to 2-3 times the standard deviation of the velocity of a typical static object (such as a telephone pole).

[0049] The classification logic is as follows: Condition 1: If If so, the target is classified as a dynamic target. Such targets are usually moving vehicles, fast-moving pedestrians, etc.

[0050] Condition 2: If If VFF >= F_th_micro, then the target is classified as a micro-moving target. Such targets are very likely to be non-rigid bodies such as pedestrians or cyclists who are stationary or moving at low speed.

[0051] Condition 3: If If VFF < F_th_micro, then the target is classified as a static target. Such targets are real static obstacles, such as guardrails, curbs, stationary vehicles, etc.

[0052] S5: Improved Tracking Management Strategy In step S5, the tracker receives the classification results from step S4 and performs differentiated tracking management: For dynamic targets: directly start a new "Confirmed" trajectory for it, or associate it with an existing "Confirmed" trajectory.

[0053] For micro-movement targets: This is key to improving pedestrian heel-taking performance.

[0054] Initiate a tentative trajectory immediately: Once a target is classified as a "micro-movement target," the tracker immediately creates a "tentative" trajectory for it. This trajectory will then be predicted and updated as normal.

[0055] Fast confirmation mechanism: A short confirmation period is set for such trajectories (e.g., successful association for 2-3 consecutive frames). Once this condition is met, the status is immediately raised to "Confirmed". This is significantly faster than the logic in traditional methods that requires waiting for the target to reach a dynamic threshold before starting tracking.

[0056] Model selection: Preferably, an interactive multi-model (IMM) filter is used for the trajectory of the "micro-moving target," which includes at least one constant velocity model and one constant position model. This allows the tracker to smoothly handle the transition of a pedestrian from stationary to moving or from moving to stationary.

[0057] For static targets: tracking is usually not initiated. However, exceptions can be set, such as static targets directly in front of the vehicle in its lane, in which case tracking should still be initiated for collision risk assessment.

[0058] The tracker in step S5 is based on a module of Kalman filtering and multiple hypothesis tracking.

[0059] In step S6, the tracking management specifically includes: (1) starting or maintaining a tracking trajectory for both dynamic and micro-moving targets; (2) assigning a "tentative" state to the tracking trajectory of a micro-moving target, and switching to a "confirmed" state after it is successfully associated for multiple consecutive frames and its features are stable; (3) for static targets, not starting tracking or only starting tracking in specific dangerous areas.

[0060] This invention adds only feature calculations for points inside clustered targets to the existing processing flow, without requiring intensive calculations on the entire scene point cloud. The computational increment is small, making it very suitable for the resource-constrained realities of in-vehicle computing platforms, achieving a good balance between performance and resources.

[0061] Example 2: A sensing system based on the point cloud pedestrian recognition and tracking method based on velocity fluctuation features in Example 1, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program and its instructions to implement the steps of the method.

[0062] Example 3: A vehicle equipped with the perception system described in Example 2 above.

[0063] A specific application is as follows: Suppose a pedestrian is waiting at a red light at an intersection, with an overall speed of -0 m / s, but their body is slightly swaying; S1, after the 4D radar acquires their point cloud: S2, their V_overall ≈ 0 m / s is calculated. Using traditional methods, this would be classified as a "static target" and ignored by the tracker.

[0064] S3. Calculate the radial velocity standard deviation of its point cloud cluster, VFF_std = 0.3 m / s, while the typical value for static obstacles is < 0.1 m / s.

[0065] S4, due to However, since VFF > F_th_micro (0.15 m / s), the pedestrian was correctly classified as a "micro-moving target".

[0066] S5. The tracker immediately starts a "trial" trajectory for it. When the red light turns green and the pedestrian begins to move, its V_overall increases rapidly, indicating that the tracker has locked onto the target in advance, achieving seamless, fast, and zero-latency trajectory tracking.

[0067] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely prisms of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A point cloud pedestrian recognition and tracking method based on velocity fluctuation features, characterized in that: The specific steps are as follows: S1, Data acquisition and preprocessing: acquire radar sequence point cloud data and cluster it into candidate targets and perform preprocessing; S2, Overall target motion velocity estimation: calculate the overall motion velocity and velocity fluctuation characteristics of each candidate target; S3, Velocity fluctuation feature extraction: based on the overall motion velocity and velocity fluctuation characteristics, classify the candidate targets into at least static targets, dynamic targets and micro-movement targets; S4. Target classification based on multiple features: Based on the overall motion velocity, calculate the velocity fluctuation feature of the point cloud cluster. This feature is used to characterize the degree of dispersion or periodic change of the velocity of each point within the point cloud cluster relative to the overall motion velocity. S5. Improved tracking management strategy: The tracker receives the classification results from step S4 and performs differentiated tracking management; S6. Tracking management of classified targets: For point cloud clusters classified as micro-moving targets, the tracker immediately or with a lower latency threshold starts a tracking trajectory for them.

2. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: The candidate target points in step S1 are the standard deviation, variance, and entropy of the radial velocity of all points within the cloud cluster, or the energy value in the 1-2.5Hz frequency band obtained based on frequency domain analysis.

3. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: The preprocessing in step S1 includes noise filtering, ground segmentation, and point cloud clustering.

4. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: The velocity fluctuation feature in step S2 is an indicator used to characterize the discreteness or periodic micro-movement of the velocity distribution of the point cloud inside the target.

5. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: The classification based on overall motion speed and speed fluctuation characteristics in step S3 specifically refers to: (1) if the overall motion speed is higher than the first threshold, it is a dynamic target; (2) if the overall motion speed is lower than the first threshold and the speed fluctuation characteristics are higher than the second threshold, it is a micro-motion target; (3) if the overall motion speed is lower than the first threshold and the speed fluctuation characteristics are lower than the second threshold, it is a static target.

6. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: The calculation method for velocity fluctuation characteristics in step S4 includes one or more of the following combinations: (1) Calculate the standard deviation or variance of the radial velocity of all points directly provided by the radar within the point cloud cluster; (2) Calculate the statistical quantity of the difference between the radial velocity of all points within the point cloud cluster and the projection of the overall motion velocity onto the radar radial direction; (3) Perform frequency domain transformation on the overall motion velocity sequence or velocity fluctuation sequence of the point cloud cluster in consecutive frames, and extract the energy in the 1.0Hz to 2.5Hz frequency band as periodic fluctuation characteristics.

7. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: The tracker in step S5 is based on a module of Kalman filtering and multiple hypothesis tracking.

8. The point cloud pedestrian recognition and tracking method based on velocity fluctuation features according to claim 1, characterized in that: In step S6, the tracking management specifically includes: (1) starting or maintaining a tracking trajectory for both dynamic and micro-moving targets; (2) assigning a "tentative" state to the tracking trajectory of a micro-moving target, and switching to a "confirmed" state after it is successfully associated for multiple consecutive frames and its features are stable; (3) for static targets, not starting tracking or only starting tracking in specific dangerous areas.

9. A sensing system, characterized in that: The method includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program and its instructions to implement the steps of the method according to any one of claims 1 to 8.

10. A vehicle, characterized in that: It is equipped with the sensing system as described in claim 9.

Images