Engineering vehicle multi-laser radar point cloud three-dimensional target detection and dynamic tracking method

Abstract

The invention provides an engineering vehicle multi-laser radar point cloud three-dimensional target detection and dynamic tracking method. The method comprises the following steps: multi-laser radar data acquisition; time synchronization and space synchronization of the multi-laser radar system; carrying out self dynamic local area filtering based on the vehicle turning angle; carrying out point cloud noise suppression and structured processing; carrying out unsupervised target detection; carrying out differentiated three-dimensional multi-target tracking based on a target scale; and outputting a dynamic target state and circularly updating the system. According to the method disclosed by the invention, a set of three-dimensional environment perception technology closed loop aiming at an engineering vehicle operation scene is constructed, and stable tracking and state output of a dynamic target are finally realized by starting from multi-source data fusion and through progressive data processing and analysis; the technical problem of real-time, robust and low-cost perception of the engineering vehicle in a complex dynamic environment is solved, and a core perception capability is provided for automatic driving and safety early warning.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point cloud. Background Technology

[0002] Three-dimensional detection and continuous tracking are key technologies for ensuring the safety of engineering vehicle operations, improving operational efficiency, and advancing automated control. However, existing technologies still face many challenges in engineering applications.

[0003] Existing technologies using LiDAR on engineering vehicles can generate point clouds ranging from hundreds of thousands to millions of points, resulting in sparse and unevenly distributed data. Irregular storage methods lead to high computational demands and poor real-time performance, limiting rapid response to dynamic scenes. Furthermore, during steering or operation, the vehicle body undergoes non-rigid deformation, causing changes in the point cloud data acquired by the LiDAR. This dynamic structural interference introduces anomalies, affecting the accuracy of 3D perception of the surrounding environment. Moreover, existing technologies largely rely on deep learning methods, typically requiring large amounts of labeled data, resulting in high training costs, limited model generalization ability, and insufficient interpretability, making reliable tracking difficult in unknown environments. Additionally, some methods depend on high-performance computing equipment or complex models, making stable operation in edge computing environments challenging.

[0004] Therefore, there is an urgent need to design a three-dimensional target detection and dynamic tracking method suitable for the complex operating environment of engineering vehicles to solve the current technical problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for 3D target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds. This method can achieve 360-degree 3D perception around the vehicle without training and has high interpretability, robustness, and real-time performance.

[0006] The technical solution of this invention is: a method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds, comprising the following steps: Multiple lidar data acquisition; Time and space synchronization of multiple lidar systems; Dynamic local area filtering based on vehicle turning angle; Point cloud noise suppression and structuring; Unsupervised target detection; Differentiated 3D multi-target tracking based on target scale; Dynamic target status output and system cyclic update.

[0007] Furthermore, the time synchronization and spatial synchronization of the multi-lidar system includes the following steps: Time synchronization: Based on the original timestamps, the nearest neighbor matching method is used to align the radar point clouds to a unified time reference; Spatial synchronization: Initial extrinsic parameters are obtained through manual measurement as coarse calibration to provide stable initial values; then, the generalized iterative nearest point algorithm is used for fine calibration to optimize the relative pose between radars frame by frame; finally, the calibration results of multiple frames are statistically fused to generate the final extrinsic parameters.

[0008] Furthermore, the self-dynamic local region filtering based on vehicle turning angle includes the following steps: The point cloud of the rear half of the vehicle is filtered out using a fixed rectangular region in the local coordinate system; The front half of the vehicle dynamically constructs a rotating rectangular region based on the current turning angle to filter out the corresponding point cloud.

[0009] Furthermore, the point cloud noise suppression and structuring process includes the following steps: Outlier removal: Based on the average distance statistics of the local neighborhood of the point cloud, isolated noise points are identified and removed; Radius consistency filtering: Count the number of points within a fixed radius neighborhood and remove outliers with insufficient local density; Height filtering: Based on the Z-axis height information of the point cloud, a height threshold is set to retain point clouds that are below the preset height range, so as to enhance the perception effect of targets near the ground. Voxel downsampling: Divide the three-dimensional space into fixed-size voxel units and replace the original point cloud with voxel-represented points to achieve a regularized representation of the point cloud and reduce the amount of data; Ground point removal: A random sampling consensus algorithm is used to identify and remove ground points, and the output non-ground point cloud is used for subsequent target detection and analysis.

[0010] Furthermore, the unsupervised target detection includes the following steps: DBSCAN Clustering: The DBSCAN algorithm is used to cluster point cloud data. Effective clustering screening: The clustering results are screened based on a threshold of the number of points. Only clusters with a number of points greater than or equal to the set threshold are judged as effective target detection results. Target size calculation and small target labeling: Based on the size information of the 3D bounding box, the detected targets are automatically classified, and small and large targets are labeled according to the preset size threshold; Target center and distance filtering: Calculate the distance between the center point of each target and the origin of the coordinate system, and filter based on this distance; Target size filtering: The target size is filtered. If the length and width of the target are within a certain range, the target will be considered valid. Target storage and output: Store targets that meet the criteria for use in subsequent target tracking tasks.

[0011] Furthermore, the DBSCAN algorithm divides dense point regions in the point cloud into different clusters based on the set neighborhood radius and minimum number of points parameters, while points that do not belong to any cluster are marked as noise.

[0012] Furthermore, the generation of the 3D bounding box's size information includes the following steps: For each cluster, principal component analysis (PCA) is performed in a two-dimensional plane to obtain the principal and secondary orientations of the target, thereby determining the orientation of the three-dimensional bounding box. The dimensions of the target are obtained by calculating the minimum bounding rectangle in the PCA coordinate system; In the Z-axis direction, the height of the target is calculated directly using the minimum and maximum values ​​of the point cloud clusters, and the dimensions of the 3D bounding box are finally obtained.

[0013] Furthermore, the target-scale-based differentiated 3D multi-target tracking includes the following steps: Target state prediction: The state of each tracked target is predicted using a Kalman filter to obtain the estimated position at the current time step; Current target measurement: By calculating the center of the three-dimensional bounding box of each target, the actual position of the target in the current frame is obtained, providing accurate measurement information for subsequent matching; Cost matrix construction and optimal matching: The cost matrix is ​​constructed based on the Euclidean distance between the prediction center and the measurement center, and the Hungarian algorithm is used to solve for the optimal matching to ensure the global optimal association between multiple objectives; Target state update: After matching is completed, the target state is updated by fusing prediction and measurement information using a Kalman filter; New target creation: For unmatched measurement targets, initialize them as new targets and assign them new Kalman filters; Stable target detection and classification: The stability of each tracked target is judged based on the number of consecutive tracking frames or motion smoothness index; Historical trajectory maintenance and smoothing: A historical state window is maintained for stable targets, and the motion trajectory is smoothed using the exponential moving average method.

[0014] Furthermore, in the stable target detection and classification process, small targets participate in tracking and output immediately after creation, while large targets need to undergo stability verification before participating in output.

[0015] Furthermore, the dynamic target state output and system cyclic update include the following steps: The system outputs the state of the target that is being stably tracked, and after completing the processing of a single frame, it proceeds to the processing of the point cloud in the next frame, thus achieving continuous 3D environment perception.

[0016] The beneficial effects of this invention are: (1) The method in this invention constructs a closed loop of three-dimensional environmental perception technology for engineering vehicle operation scenarios. Starting from multi-source data fusion, through progressive data processing and analysis, it finally achieves stable tracking and state output of dynamic targets, solves the technical problem of real-time, robust and low-cost perception of engineering vehicles in complex dynamic environments, and provides core perception capabilities for autonomous driving and safety warning. (2) The vehicle's own structure is differentiated at the front and rear, which can adapt to various driving states such as straight driving and turning. While accurately filtering out its own interference, it retains the effective environmental point cloud adjacent to the vehicle body to the maximum extent, which significantly improves the robustness and accuracy of the perception system during vehicle movement. (3) Unsupervised processing flow realizes end-to-end conversion from raw point cloud to structured target list. The whole process does not require any pre-collected scene data, manual annotation or model training. It overcomes the inherent defects of deep learning methods such as poor generalization, dependence on labeled data and weak interpretability. It directly perceives the physical world through geometric and statistical laws. It is particularly suitable for engineering vehicle operation environment with complex and changeable working conditions and difficulty in pre-defining all targets. It reflects the high adaptability, strong robustness and good interpretability of the method itself. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method for three-dimensional target detection and dynamic tracking of engineering vehicles using multiple lidar point clouds in this invention.

[0018] Figure 2 This is a flowchart of the time and space synchronization method for multiple lidar systems in this invention.

[0019] Figure 3 This is a flowchart of the method for filtering out local regions based on the vehicle's turning angle in this invention.

[0020] Figure 4 This is a flowchart of the point cloud noise suppression and structuring method in this invention.

[0021] Figure 5 This is a flowchart of the unsupervised target detection method in this invention.

[0022] Figure 6 This is a flowchart of the method for differentiated three-dimensional multi-target tracking based on target scale in this invention.

[0023] Figure 7 This is a schematic diagram of the visible area of ​​the lidar on one side of the engineering machinery in this invention.

[0024] Figure 8 This is a schematic diagram illustrating the visualization effect of multiple lidars after spatiotemporal synchronization in this invention.

[0025] Figure 9 This is a schematic diagram illustrating the division of the non-rigid body regions of the front and rear halves of the loader in this invention.

[0026] Figure 10 This is a schematic diagram illustrating the visualization effect of the vehicle's own dynamic local region filtering based on the vehicle's turning angle in this invention.

[0027] Figure 11 This is a comparison image showing the before and after effects of ground removal in this invention.

[0028] Figure 12 This is a comparison image of the original point cloud, ground removal, clustering, and ground / non-ground data in this invention.

[0029] Figure 13 This is a schematic diagram illustrating the visualization effect of ground point and non-ground point extraction in this invention.

[0030] Figure 14 This is a visualization diagram of the three-dimensional target detection of vehicles and pedestrians in this invention.

[0031] Figure 15 This is a schematic diagram of vehicle detection and pedestrian tracking in this invention. Detailed Implementation

[0032] Various exemplary embodiments of the invention will now be described in detail with reference to the accompanying drawings. The descriptions of the exemplary embodiments are merely illustrative and are in no way intended to limit the invention or its application or use. The invention can be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the invention thorough and complete, and to fully express the scope of the invention to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps, the composition of materials, numerical expressions, and values ​​set forth in these embodiments should be interpreted as merely exemplary and not as limiting.

[0033] The terms "first," "second," and similar words used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Words such as "including" or "comprising" mean that the element preceding the word encompasses the element listed after it, without excluding the possibility of encompassing other elements. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0034] like Figure 1As shown, a method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds is disclosed, including the following steps: S100, multi-LiDAR data acquisition; S200, time and space synchronization of multiple lidar systems; S300, based on the vehicle's own dynamic local area filtering at the turning angle; S400, point cloud noise suppression and structured processing; S500, unsupervised target detection; S600, a differential 3D multi-target tracking system based on target scale; S700, dynamic target status output and system cyclic update.

[0035] In step S100 of some embodiments, a multi-LiDAR system mounted on the roof of the vehicle acquires point clouds of the surrounding environment and transmits the data to a processing module, providing input for 3D target detection and tracking. For example... Figure 7 The image shows the visible area of ​​the lidar on one side of the engineering machinery.

[0036] In multi-lidar systems, the point clouds collected by different lidars have spatiotemporal discrepancies due to inconsistent sampling times. To ensure the fusion accuracy of point clouds from multiple lidars, time synchronization and spatial calibration are required.

[0037] In step S200, such as Figure 2 As shown, the time and space synchronization of a multi-LiDAR system includes the following steps: S201, Time Synchronization: Based on the original timestamp, the nearest neighbor matching method is used to align each radar point cloud to a unified time reference; S202, Spatial Synchronization: Initial extrinsic parameters are obtained through manual measurement as coarse calibration to provide stable initial values; then, the generalized iterative nearest point algorithm is used for fine calibration, optimizing the relative pose between radars frame by frame; finally, the calibration results of multiple frames are statistically fused to generate the final extrinsic parameters.

[0038] In step S201 of some embodiments, the original hardware timestamp carried by each lidar point cloud data is obtained. Since there may be slight drift in the clocks of each lidar, the timeline of the main lidar is used as a unified reference. For any slave lidar, for each frame of point cloud data it acquires, the system searches for the frame with the closest timestamp to the main lidar on the timeline. If the time difference is less than a preset threshold, the two frames are considered to correspond in time. Subsequently, based on the time difference and target motion estimation, linear interpolation is performed on the points in the slave lidar point cloud to further align them to a precise reference time. After completing the above matching and interpolation, a set of time-aligned multi-library point cloud data is output for subsequent spatial synchronization and fusion processing. Figure 8The image shows the visualization effect after multiple lidar sensors are synchronized in time and space.

[0039] In step S202 of some embodiments, with the vehicle stationary, the installation position and approximate orientation of each radar are manually measured using measuring tools. Based on these measurements, the initial extrinsic parameter matrix of each radar is calculated. Although this step has limited accuracy, it provides a stable initial value for subsequent automated fine calibration, preventing the algorithm from getting trapped in local optima due to poor initial values. The time-synchronized point cloud data output in step S201 is used for frame-by-frame automated fine calibration. The main radar point cloud is selected as the target point cloud, and the secondary radar point cloud as the source point cloud. The Generalized Iterative Closest Point (GICP) algorithm is employed. This algorithm not only considers point-to-point distance but also introduces local surface covariance information, resulting in stronger robustness for point cloud registration. The algorithm uses the coarse calibration extrinsic parameters as initial values ​​and iteratively optimizes to find the optimal relative pose that minimizes the matching error between the two point clouds. For each successfully processed frame of synchronized data, a set of optimized extrinsic parameter estimates for that frame is output. Multiple frames of data are continuously collected and processed to obtain a series of extrinsic parameter estimates. The mean and standard deviation of each component of the extrinsic parameters are calculated, and outlier frames exceeding ±3 standard deviations are removed. For the remaining clean extrinsic parameter estimates, the arithmetic mean of the rotation and translation components is calculated separately. The fused average value is used as the final calibration extrinsic parameter of the slave radar relative to the master radar and written into the system configuration file. During subsequent online perception, the system uses this final extrinsic parameter in real time to transform the point cloud data of all radars into a unified vehicle coordinate system, forming a complete 360° fused point cloud.

[0040] In step S300, such as Figure 3 As shown, the self-dynamic local region filtering based on vehicle turning angle includes the following steps: S301, the rear half of the vehicle is relatively fixed in the local coordinate system, and the point cloud of the rear half of the vehicle is filtered out using a fixed rectangular area in the local coordinate system. S302, the front half of the vehicle rotates with the turning angle, and the front half of the vehicle dynamically constructs a rotating rectangular region according to the current turning angle to filter out the corresponding point cloud.

[0041] In 3D point cloud perception of engineering vehicles, the vehicle's own structure, especially when turning, can interfere with sensor data acquisition. The aforementioned self-dynamic local region filtering method based on vehicle turning angle can adapt to the vehicle's driving and turning states, effectively eliminating interference from its own structure.

[0042] In step S301 of some embodiments, one or more three-dimensional rectangular regions tightly enclosing the rear half of the vehicle structure are defined in a unified vehicle coordinate system based on vehicle CAD drawings or actual measurements. For each point in the synchronized fused point cloud, it is determined whether its coordinates fall within any of the aforementioned fixed rectangular regions. If its coordinates fall within the aforementioned fixed rectangular regions, the point is determined to be part of the vehicle's own point cloud and is discarded. The boundary parameters of the rectangular regions can be stored in a configuration file to adapt to subtle deformations of different vehicle models or the same vehicle model under different load conditions, ensuring the accuracy of filtering.

[0043] In step S302 of some embodiments, the signals from the steering wheel angle sensor or wheel angle sensor are read in real time via the vehicle CAN bus to obtain the current front wheel steering angle δ of the vehicle. The steering angle δ is defined with the vehicle traveling straight as 0, left turns as positive, and right turns as negative. When the vehicle is traveling straight, δ=0, and a reference three-dimensional rectangular region tightly enclosing the front half of the vehicle in the vehicle coordinate system is defined. When the steering angle δ is not zero, this reference region needs to be rotated around the Z-axis of the vehicle coordinate system. For any representative point within the reference region, a rotation transformation is applied. By calculating the vertices of the rotated region, the spatial range of the dynamically rotated rectangular region at the current moment can be determined. To improve real-time performance, in implementation, each point to be judged can be directly subjected to a reverse rotation, transforming it back to the reference coordinate system, and then determining whether it is within the reference fixed region. Within each frame point cloud processing cycle, the system obtains the latest steering angle δ and dynamically constructs or equivalently judges and filters out regions according to the above method, performing real-time filtering of the current frame point cloud.

[0044] Taking a loader as an example, the division of the non-rigid body areas of the front and rear halves of the loader is as follows: Figure 9 As shown. The effect of filtering out the dynamic local region based on the vehicle's turning angle is as follows. Figure 10 As shown.

[0045] In step S400, such as Figure 4 As shown, point cloud noise suppression and structuring processing includes the following steps: S401, Outlier Removal: Based on the average distance statistics of the local neighborhood of the point cloud, isolated noise points are identified and removed; outlier removal depends on the statistical distance distribution and is somewhat dependent on the global density of the overall point cloud. S402, Radius Consistency Filter: Counts the number of points in a fixed radius neighborhood and removes outliers with insufficient local density; S403, Height Filtering: Based on the Z-axis height information of the point cloud, a height threshold is set to retain point clouds that are below the preset height range, so as to enhance the perception effect of targets near the ground. S404, Voxel downsampling: Divide the three-dimensional space into fixed-size voxel units and replace the original point cloud with voxel-represented points to achieve a regularized representation of the point cloud and reduce the amount of data; S405, Ground Point Removal: The random sampling consensus algorithm is used to identify and remove ground points, and the output non-ground point cloud is used for subsequent target detection and analysis.

[0046] In step S401 of some embodiments, a statistical outlier removal algorithm is employed. For each point in the point cloud, the algorithm calculates its distance to all outliers. The average distance between the nearest neighbors Then, the average distance for all points in the entire point cloud is calculated. mean and standard deviation Set a distance threshold. ,in, As a scale factor, if the average distance of a point > If a node is found to be an outlier, it is removed. This effectively filters out discrete noise, preventing it from interfering with subsequent clustering and structure analysis, and improving data cleanliness. As an example, the number of nearest neighbors... , scale factor These parameters can be configured according to the radar model and the typical noise level of the environment.

[0047] Step S402 is a supplement to step S401, focusing on removing anomalous sparse point clusters where the local point cloud density is significantly lower than that of the surrounding area. Such point clusters may be formed by invalid measurements or edge portions of objects of non-interest.

[0048] In step S402 of some embodiments, for each point in the point cloud Statistical analysis of its range within a specified radius The number of points in the spherical neighborhood Set a minimum number of points threshold. .like < If the density of the region containing that point is insufficient, the point is removed. Further cleaning of the point cloud enhances the continuity of the point cloud in potential target regions, creating conditions for stable clustering. As an example, the search radius... Minimum number of points threshold .

[0049] In step S403 of some embodiments, an effective height range is set in the vehicle coordinate system with the Z-axis vertically upward. Traverse the point cloud, retaining only the Z coordinate that satisfies... This significantly reduces the amount of data to be processed and preemptively eliminates interference from point clouds in a large number of irrelevant areas, allowing the system's computing resources to be concentrated on key sensing areas. Slightly below ground level to accommodate unevenness. Higher than the maximum expected target height. As an example, , .

[0050] In step S404 of some embodiments, the three-dimensional space is divided into sections with side lengths of... A regular voxel grid is used. For each non-empty voxel, all points within it are replaced by a representative point representing the centroid of that voxel or the geometric center of all points. This significantly reduces the data input for clustering, feature calculation, and other steps, greatly improving the speed of subsequent algorithms; it overcomes the density differences in the original point cloud caused by varying distances from the radar, making the distance-based algorithm parameters more stable; and it helps suppress minor noise. Voxel side length It can be set according to the original point cloud density and accuracy requirements. As an example, to This operation can reduce the number of point clouds by an order of magnitude.

[0051] In step S405 of some embodiments, the Random Sample Consensus (RANSAC) algorithm is used to fit the ground plane model. It is assumed that the ground can be approximated as a planar model: Randomly select three points from the point cloud to calculate an initial planar model, then calculate the distances from all points to this plane. Set a distance threshold. , distance less than Points are identified as interior points of the model. This random sampling and evaluation process is repeated multiple times. The planar model with the most interior points is selected as the final ground plane model. Based on the final model, all points are divided into ground points and non-ground points. The non-ground points are output as input for subsequent object detection. Accurate separation of ground and non-ground point clouds allows subsequent object detection algorithms to focus entirely on potential moving or stationary obstacles, greatly reducing false detections and computational waste. As an example, The number of iterations is 1000. In one example, a comparison of the effects before and after ground point removal is shown below. Figure 11 As shown in the diagram, the visualization effect of ground point and non-ground point extraction is as follows: Figure 13 As shown.

[0052] like Figure 5 As shown, in step S500, unsupervised target detection includes the following steps: S501, DBSCAN Clustering: The DBSCAN algorithm is used to cluster point cloud data; S502, Effective Clustering Screening: The clustering results are screened based on a threshold of the number of points. Only clusters with a number of points greater than or equal to the set threshold are judged as effective target detection results. S503, Target Size Calculation and Small Target Marking: Based on the size information of the 3D bounding box, the detected targets are automatically classified, and small and large targets are marked according to the preset size threshold; S504, Target Center and Distance Filtering: Calculate the distance between the center point of each target and the origin of the coordinate system, and perform filtering based on this distance; S505, Target Size Filtering: Targets are filtered by size. Targets whose length and width are within a certain range are considered valid. S506, Target Storage and Output: Stores targets that meet the criteria for use in subsequent target tracking tasks.

[0053] The preprocessed non-terrestrial point cloud still contains objects that are spatially discrete sets of points. The purpose of step S500 is to automatically identify and segment independent potential targets from these point clouds using geometric and statistical methods without relying on any pre-trained model, and to calculate their basic properties to provide initial input for the tracking module.

[0054] In step S501 of some embodiments, a density-based spatial clustering algorithm (DBSCAN) is used to segment the spatially aggregated point cloud into different clusters, each cluster corresponding to a potential physical target. The DBSCAN algorithm requires two core parameters: neighborhood search radius and neighborhood search radius. and minimum points The law traverses the point cloud; for a point, if its... The number of points in the radius neighborhood is not less than If a new cluster is created, all points with reachable density are recursively added to that cluster. Points that cannot belong to any cluster are marked as noise. This method effectively discovers clusters of arbitrary shapes and automatically filters out noise points, deconstructing the scene into independent object candidates. Taking the uniform point cloud distribution after voxel downsampling in step S404 as an example, we set... , These parameters can be adjusted based on the point cloud density and the minimum target size.

[0055] The initial clusters generated in step S501 may contain invalid clusters formed by residual noise or tiny fragments. Step S502 filters these clusters using a point count threshold to ensure that the targets for subsequent processing have a certain point cloud scale.

[0056] In step S502 of some embodiments, a minimum effective point threshold is set. Count the number of points contained in each cluster. Only when > Only when this happens is the cluster considered a valid target candidate and proceeds to the next step. Filtering out false, small clusters generated by the algorithm or environment improves system robustness and reduces unnecessary computation. As an example, It can be set to 10-20 points, depending on the point cloud density after downsampling and the requirement for the smallest detectable target.

[0057] In step S503 of some embodiments, the size information of the three-dimensional bounding box is generated. A size threshold for small targets is predefined. If the calculated length, width, and height of a target are all less than the corresponding threshold, it is marked as a small target; otherwise, it is marked as a large target. This obtains the precise geometric attributes of the target and achieves preliminary semantic-level coarse classification, providing a decisive input for differentiated processing in the tracking stage.

[0058] In step S504 of some embodiments, the two-dimensional horizontal distance D from the center point of each target bounding box to the origin of the vehicle coordinate system is calculated. The maximum effective tracking distance is then set. If D > If the target is not specified, then that target will be filtered. Different settings can be applied to small and large targets. This limits the system's effective sensing range, avoids processing too many irrelevant distant targets, saves computing resources, and allows the system to focus on security-related areas.

[0059] In step S505 of some embodiments, a reasonable range for the length and width of the target is set. For a target, if its length or width is not within the preset reasonable range, it is filtered out. Utilizing simple prior knowledge for logical filtering further improves the reliability of the detection results and reduces interference input to the tracking module.

[0060] In step S506 of some embodiments, a data structure is created for each final retained target to store its key attributes; wherein the key attributes include at least target ID, bounding box center coordinates, size, orientation angle, target category, and point cloud cluster reference.

[0061] Through steps S501 to S506, a series of unsupervised processing steps achieve an end-to-end transformation from raw point clouds to a structured target list. The entire process requires no pre-collected scene data, manual annotation, or model training, overcoming the inherent limitations of deep learning methods such as poor generalization, reliance on labeled data, and weak interpretability. It directly perceives the physical world through geometric and statistical laws, making it particularly suitable for the complex and variable working environments of engineering vehicles where it is difficult to predefine all targets. This demonstrates the method's high adaptability, strong robustness, and good interpretability.

[0062] In step S503 above, the generation of the 3D bounding box size information includes the following steps: For each cluster, principal component analysis (PCA) is performed in the two-dimensional plane (XY plane) to obtain the principal and secondary directions of the target and determine the orientation of the three-dimensional bounding box. The dimensions of the target are obtained by calculating the minimum bounding rectangle in the PCA coordinate system; In the Z-axis direction, the height of the target is calculated directly using the minimum and maximum values ​​of the point cloud clusters, ultimately obtaining the dimensions of the 3D bounding box; the dimensions of the 3D bounding box include length. ,Width Elevation, center point coordinates Orientation angle .

[0063] Specifically, all points in the current cluster are projected onto the XY plane to obtain a set of two-dimensional points. Calculate the two-dimensional mean center of this point set. Then, subtract the center value from the coordinates of all points to obtain a decentralized set of points. Calculate the decentralized set of nodes. covariance matrix Subsequently, the covariance matrix was analyzed. Eigenvalue decomposition yields two eigenvalues. , and its corresponding eigenvectors , Assuming The largest eigenvalue Corresponding feature vector This indicates the direction in which the point set is most dispersed on the horizontal plane, i.e., the main direction of the target. Second largest eigenvalue. Corresponding feature vector Then, the direction perpendicular to the main direction is the secondary direction. (The center of the point set is considered as the secondary direction.) With the origin, and Establish a temporary two-dimensional local coordinate system using the coordinate axes. Then, apply this system to all two-dimensional points. Transform to the PCA coordinate system described above to obtain the new coordinates. ,in shaft edge direction, shaft edge Direction. In this coordinate system, the calculation point set is in... Maximum value on the axis Minimum value and in Maximum value on the axis Minimum value Target length Target width The bounding box is centered on the horizontal plane. By centering in the PCA coordinate system The target's orientation angle is obtained by inverse transformation back to the vehicle coordinate system. That is, the feature vector The angle with the vehicle coordinate system's X-axis can be calculated using the vector dot product and the arctangent function. In one example, a comparison of the original point cloud, ground culling, clustering, and ground / non-ground effects is shown. Figure 12 As shown.

[0064] like Figure 6 As shown, in step S600, the differential 3D multi-target tracking based on target scale includes the following steps: S601, Target State Prediction: Use a Kalman filter to predict the state of each tracked target and obtain the estimated position at the current time step; S602, Current Target Measurement: By calculating the center of the three-dimensional bounding box of each target, the actual position of the target in the current frame is obtained, providing accurate measurement information for subsequent matching; S603, Cost Matrix Construction and Optimal Matching: The cost matrix is ​​constructed based on the Euclidean distance between the prediction center and the measurement center, and the Hungarian algorithm is used to solve for the optimal matching to ensure the global optimal association between multiple objectives; S604, Target State Update: After matching is complete, a Kalman filter is used. Update the target status by integrating prediction and measurement information; S605, New Target Creation: For unmatched measurement targets, initialize them as new targets and assign them a new Kalman filter; S606, Stable Target Detection and Classification Processing: Performs stability judgment on each tracked target, and determines whether the target is stable based on the number of consecutive tracking frames or motion smoothness index; S607, Historical Trajectory Maintenance and Smoothing: Maintain historical state windows for stable targets and use the exponential moving average method to smooth the motion trajectory.

[0065] In step S601 of some embodiments, a Kalman filter is used to predict the state of each tracked target to obtain the estimated center point of the current time step. The predicted state takes into account the motion information of the target in the previous frame, including position, velocity and acceleration, so that the algorithm can effectively estimate the target motion trend in a dynamic environment.

[0066] In some embodiments, step S602 directly uses the target 3D bounding box center coordinates provided by the detection module. As a measurement of the target in the current frame Each measurement is accompanied by a measurement noise covariance matrix. The value can be estimated based on sensor accuracy, target distance, and target size. Typically, small targets have less point cloud data and higher uncertainty in center localization, therefore a larger value can be assigned. Large target point clouds are stable, center positioning is more accurate, and smaller targets can be assigned... .

[0067] In step S603 of some embodiments, for all tracked targets (predicted states) and all current frame measurements, the correlation cost between them is calculated to form a cost matrix. Matrix elements express The tracking target and the first The matching cost between measurements is primarily used in this embodiment, employing the Mahalanobis or Euclidean distance between the predicted and measured locations. To reduce computational costs for impossible matches, an association threshold can be set, with costs exceeding this threshold set to infinity. The association threshold can be set according to the target size. Small targets are highly maneuverable, allowing for a more relaxed threshold; large targets exhibit predictable movement patterns, requiring a tighter threshold to reduce erroneous associations. The data association problem is transformed into a bipartite graph optimal allocation problem, solved using the Hungarian Algorithm. This algorithm can find the globally optimal matching scheme that minimizes the total matching cost in polynomial time, effectively handling the association ambiguity problem when multiple targets are close to each other, thereby reducing the probability of target loss and erroneous matching.

[0068] In step S604 of some embodiments, for a successfully matched tracking target, its state estimate is updated using its corresponding current frame measurement. For each matching pair, the standard update step of the Kalman filter is performed, and the Kalman gain is calculated. Furthermore, by integrating predicted and measured values, a more accurate posterior state estimate can be obtained. Covariance During updates, additional smoothing constraints can be imposed on the state estimates of small targets, such as smoothing their position or velocity using a first-order low-pass filter, to suppress observation jitter caused by sparse point clouds.

[0069] In step S605 of some embodiments, a new tracking target is created for each unmatched measurement. Its measurement location is used as the initial state, and a new Kalman filter is assigned to it. The initial velocity can be set to zero or estimated based on the first multi-frame observations. The newly created target is immediately assigned its size category label obtained during the detection phase, which will determine all subsequent differential tracking parameters.

[0070] In step S606 of some embodiments, during the stable target detection and classification process, small targets participate in tracking and output immediately after creation, while large targets must undergo stability verification before participating in output. Specifically, a stability score or tracking lifetime is maintained for each tracked target. Judgment criteria may include ensuring that the number of consecutive frames in which the target is successfully tracked exceeds a threshold. The target's position or velocity changes within the most recent frames must be below a certain threshold to indicate that its motion trajectory is reasonable and smooth, rather than noisy or jittery. For tracked objects marked as small targets, considering their potential rapid appearance or disappearance, a lower threshold can be set in the earlier frames after tracking begins. This involves outputting its status to ensure a rapid system response to potential threats. For tracked objects marked as large targets, to avoid misclassifying transient static objects as stable targets, they must meet stricter stability conditions (setting a higher threshold). Output is only possible after stricter motion smoothing metrics are applied. For tracked targets that have not been matched for a long time, it is determined that they have left the sensing area or disappeared, and they are removed from the tracking list.

[0071] In step S607 of some embodiments, for targets determined to be stable, their historical states are maintained and smoothed to output a more aesthetically pleasing and physically intuitive trajectory. A fixed-length queue of historical states, such as the states of the most recent 20 frames, is maintained for each stable target. The historical position series is smoothed using methods such as Exponential Moving Average (EMA). EMA assigns higher weight to recent data, and its formula is:

[0072] in It is the smoothed value at time t. It is the observation value at time t. It is a smoothing factor ( ). The larger the value, the more sensitive it is to recent observations. The smoothed target state is used as the final output of this tracking module; the smoothed target state includes the smoothed position, size, category, velocity, and unique tracking ID. In one example, the visualization of 3D target detection for vehicles and pedestrians is shown below. Figure 14 As shown; the visualization effect of vehicle detection and pedestrian tracking, such as Figure 15 As shown.

[0073] In step S700, the dynamic target state output and system cyclic update include the following steps: The system outputs the state of the target that is being stably tracked, and after completing the processing of a single frame, it proceeds to the processing of the point cloud in the next frame, thus achieving continuous 3D environment perception.

[0074] Specifically, a standardized data structure is generated for each stable target, containing information such as position, size, category, velocity, orientation, unique tracking ID, and tracking status confidence. The encapsulated target list is published to subscribers via a preset communication interface. The output frequency is synchronized with the system processing frame rate, typically 10-20Hz, ensuring downstream systems receive timely environmental status updates. After completing all processing and output for the current frame, the system must update its internal state to prepare for processing the next frame of point cloud data, achieving a closed-loop cycle. For all tracked targets, the predicted state of their Kalman filters naturally becomes the previous frame state in step S601 of the next frame processing cycle after the current frame update is completed. The system must ensure that the internal states of all trackers are correctly maintained. The intermediate data buffer used only for the current frame processing is released or reset. For targets confirmed to have disappeared and been deleted, the memory occupied by their corresponding trackers is reclaimed by the system to prevent memory leaks. The system's main loop listens for or waits for new multi-radar raw data packets. Once a new data packet arrives, or reaches a preset period, a new round of processing is immediately triggered, starting from step S100.

[0075] As an example of the aforementioned subscriber, the subscriber is a vehicle collision avoidance warning system that receives target information, calculates the collision time, and triggers audible and visual alarms or automatic braking; an autonomous driving decision planning module that serves as dynamic environmental input for global path replanning and local trajectory generation; or a human-machine interface that renders dynamic targets in real time on the vehicle's display screen to assist the driver in visibility.

[0076] This application embodiment also provides a readable storage medium storing a computer program, which is loaded and executed by a processor to implement the steps of the above-described method for three-dimensional target detection and dynamic tracking of multiple lidar point clouds for engineering vehicles. The specific implementation process can be referred to the description of the corresponding part of the above embodiment, and will not be repeated in this embodiment.

[0077] This application also proposes a computer program product or computer program including computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in various optional implementations of the above-described method for three-dimensional target detection and dynamic tracking of multi-LiDAR point clouds for engineering vehicles. Specific implementation processes can be referred to the descriptions of the corresponding embodiments above, and will not be repeated here.

[0078] The various embodiments of the present invention have now been described in detail. To avoid obscuring the concept of the invention, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.

[0079] The above embodiments only illustrate some implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds, characterized in that, Includes the following steps: Multiple lidar data acquisition; Time and space synchronization of multiple lidar systems; Dynamic local area filtering based on vehicle turning angle; Point cloud noise suppression and structuring; Unsupervised target detection; Differentiated 3D multi-target tracking based on target scale; Dynamic target status output and system cyclic update.

2. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 1, characterized in that, The time and space synchronization of the multi-lidar system includes the following steps: Time synchronization: Based on the original timestamps, the nearest neighbor matching method is used to align the radar point clouds to a unified time reference; Spatial synchronization: Initial extrinsic parameters are obtained through manual measurement as coarse calibration to provide stable initial values; then, the generalized iterative nearest point algorithm is used for fine calibration to optimize the relative pose between radars frame by frame; finally, the calibration results of multiple frames are statistically fused to generate the final extrinsic parameters.

3. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 1, characterized in that, The self-dynamic local region filtering based on vehicle turning angle includes the following steps: The point cloud of the rear half of the vehicle is filtered out using a fixed rectangular region in the local coordinate system; The front half of the vehicle dynamically constructs a rotating rectangular region based on the current turning angle to filter out the corresponding point cloud.

4. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 1, characterized in that, The point cloud noise suppression and structuring process includes the following steps: Outlier removal: Based on the average distance statistics of the local neighborhood of the point cloud, isolated noise points are identified and removed; Radius consistency filtering: Count the number of points within a fixed radius neighborhood and remove outliers with insufficient local density; Height filtering: Based on the Z-axis height information of the point cloud, a height threshold is set to retain point clouds that are below the preset height range, so as to enhance the perception effect of targets near the ground. Voxel downsampling: Divide the three-dimensional space into fixed-size voxel units and replace the original point cloud with voxel-represented points to achieve a regularized representation of the point cloud and reduce the amount of data; Ground point removal: A random sampling consensus algorithm is used to identify and remove ground points, and the output non-ground point cloud is used for subsequent target detection and analysis.

5. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 1, characterized in that, The unsupervised target detection includes the following steps: DBSCAN Clustering: The DBSCAN algorithm is used to cluster point cloud data. Effective clustering screening: The clustering results are screened based on a threshold of the number of points. Only clusters with a number of points greater than or equal to the set threshold are judged as effective target detection results. Target size calculation and small target labeling: Based on the size information of the 3D bounding box, the detected targets are automatically classified, and small and large targets are labeled according to the preset size threshold; Target center and distance filtering: Calculate the distance between the center point of each target and the origin of the coordinate system, and filter based on this distance; Target size filtering: The target size is filtered. If the length and width of the target are within a certain range, the target will be considered valid. Target storage and output: Store targets that meet the criteria for use in subsequent target tracking tasks.

6. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 5, characterized in that: The DBSCAN algorithm divides dense point regions in the point cloud into different clusters based on the set neighborhood radius and minimum number of points parameters, while points that do not belong to any cluster are marked as noise.

7. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 5, characterized in that, The generation of the dimensional information of the three-dimensional bounding box includes the following steps: For each cluster, principal component analysis (PCA) is performed in a two-dimensional plane to obtain the principal and secondary orientations of the target, thereby determining the orientation of the three-dimensional bounding box. The dimensions of the target are obtained by calculating the minimum bounding rectangle in the PCA coordinate system; In the Z-axis direction, the height of the target is calculated directly using the minimum and maximum values ​​of the point cloud clusters, and the dimensions of the 3D bounding box are finally obtained.

8. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 1, characterized in that, The target-scale-based differential 3D multi-target tracking includes the following steps: Target state prediction: The state of each tracked target is predicted using a Kalman filter to obtain the estimated position at the current time step; Current target measurement: By calculating the center of the three-dimensional bounding box of each target, the actual position of the target in the current frame is obtained, providing accurate measurement information for subsequent matching; Cost matrix construction and optimal matching: The cost matrix is ​​constructed based on the Euclidean distance between the prediction center and the measurement center, and the Hungarian algorithm is used to solve for the optimal matching to ensure the global optimal association between multiple objectives; Target state update: After matching is completed, the target state is updated by fusing prediction and measurement information using a Kalman filter; New target creation: For unmatched measurement targets, initialize them as new targets and assign them new Kalman filters; Stable target detection and classification: The stability of each tracked target is judged based on the number of consecutive tracking frames or motion smoothness index; Historical trajectory maintenance and smoothing: A historical state window is maintained for stable targets, and the motion trajectory is smoothed using the exponential moving average method.

9. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 8, characterized in that, In the stable target detection and classification process, small targets participate in tracking and output immediately after creation, while large targets need to undergo stability verification before participating in output.

10. The method for three-dimensional target detection and dynamic tracking of engineering vehicles using multi-LiDAR point clouds according to claim 1, characterized in that, The dynamic target state output and system cyclic update include the following steps: The system outputs the state of the target that is being stably tracked, and after completing the processing of a single frame, it proceeds to the processing of the point cloud in the next frame, thus achieving continuous 3D environment perception.

Images