Laser point cloud target detection method and device and electronic equipment

Abstract

The invention relates to the technical field of laser detection, in particular to a laser point cloud target detection method and device and electronic equipment. Comprising the steps of obtaining laser point cloud data of a closed area; performing distortion correction on the laser point cloud data to obtain original point cloud data; detecting the original point cloud data by using a pre-trained detection model to obtain a detection frame; and optimizing the detection frame by using a three-dimensional tracking algorithm to obtain a detection result. The problems of special point cloud distortion, detection model complexity and time sequence jitter in a closed area are solved. Wherein the distortion correction improves the data quality, the lightweight model guarantees the real-time detection efficiency, and the three-dimensional tracking algorithm strengthens the time sequence consistency, so that high-precision and high-stability target detection is realized in a closed area scene, and the sensing reliability of the automatic driving system is directly improved. Therefore, the actual application requirements of intelligent driving are met.

Description

Technical Field

[0001] This application relates to the field of laser detection technology, specifically to a laser point cloud target detection method, device, and electronic equipment. Background Technology

[0002] Enclosed areas specifically refer to confined spaces with clearly defined physical boundaries, restricted access, and relatively controllable environments. Typical scenarios include office parks, industrial plants, ports, and parking lots. The core characteristics of such areas are the high predictability of environmental elements (e.g., fixed road structures, regular obstacle distribution, stable traffic flow) and the dominance of static occlusion in perception conditions (building walls and facilities cause significant multipath reflection effects). Compared to the dynamic complexity of open roads, the stability of enclosed areas provides a foundation for scene-customized optimization of point cloud target detection, offering unique advantages, especially when dealing with low-speed targets, structured paths, and building occlusion.

[0003] Although existing laser point cloud target detection technology has been widely applied, it still has significant limitations in closed-area scenarios. Traditional laser point cloud target detection methods mainly fall into two categories: geometric feature analysis and clustering segmentation. Geometric feature-based methods achieve target recognition by calculating the differential properties of local point clouds. For example, they utilize differences in point density distribution (the contrast between uniform point cloud distribution in flat areas and local high density of protruding objects), normal vector consistency measures (distinguishing between the aggregation degree of normal vectors on continuous surfaces and the disorder of edges), and curvature feature stability (distinguishing between the curvature stability of regular objects and the oscillation characteristics of vegetation areas) to construct classification thresholds. While this method is computationally efficient, it is sensitive to point cloud quality, and its robustness significantly decreases in scenarios with severe occlusion or noise interference. Clustering-based methods segment targets based on spatial proximity. Euclidean clustering relies on kd-trees to accelerate neighborhood search and groups spatially close points into independent objects by setting a preset distance threshold (usually positively correlated with sensor resolution and point cloud sparsity), which is suitable for regular shape detection. Region growing clustering starts from the seed point with the lowest curvature and gradually expands the clustering region with the angle between the normal vectors and the curvature difference as growth constraints. It can effectively handle the segmentation problem of sticky targets, but has higher computational complexity.

[0004] Therefore, a more accurate laser point cloud target detection method is needed in closed areas. Summary of the Invention

[0005] In view of this, embodiments of this application provide a laser point cloud target detection method, apparatus, and electronic device, which improves the accuracy, real-time performance, and stability of laser point cloud target detection in closed areas, thereby meeting the practical application needs of intelligent driving.

[0006] The first aspect of this application provides a laser point cloud target detection method, including: Acquire laser point cloud data of a closed area; The laser point cloud data is subjected to distortion correction to obtain the original point cloud data; The pre-trained detection model is used to detect the original point cloud data to obtain detection boxes; The detection box is optimized using a 3D tracking algorithm to obtain the detection result.

[0007] A second aspect of this application provides a laser point cloud target detection device, comprising: The data acquisition module is used to acquire laser point cloud data of a closed area; The distortion correction module is used to correct the distortion of the laser point cloud data to obtain the original point cloud data. The model detection module is used to detect the original point cloud data using a pre-trained detection model to obtain detection boxes; The target tracking optimization module is used to optimize the detection box using a 3D tracking algorithm to obtain the detection result.

[0008] A third aspect of this application provides an electronic device including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the laser point cloud target detection method as provided in the first aspect of this application.

[0009] A fourth aspect of this application provides a computer program product including a computer program that, when run, causes the method described in the first aspect of this application to be performed.

[0010] The laser point cloud target detection method provided in the first aspect of this application acquires laser point cloud data of a closed region; performs distortion correction on the laser point cloud data to obtain raw point cloud data; uses a pre-trained detection model to detect the raw point cloud data to obtain detection boxes; and optimizes the detection boxes using a 3D tracking algorithm to obtain detection results. This method solves the problems of point cloud distortion, detection model complexity, and temporal jitter unique to closed regions. Distortion correction improves data quality, lightweight models ensure real-time detection efficiency, and the 3D tracking algorithm enhances temporal consistency, thereby achieving high-precision and high-stability target detection in closed region scenarios, directly improving the perception reliability of autonomous driving systems. This meets the practical application needs of intelligent driving.

[0011] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of a laser point cloud target detection method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the point cloud collected in this application; Figure 3 This is a schematic diagram of point cloud distortion in this application; Figure 4 This is a schematic diagram of the point cloud after correction in this application; Figure 5 This is a schematic diagram of the SUSTechPOINTS annotation interface in this application; Figure 6 This is the overall structure of the CenterPoint model in this application; Figure 7 This is a network structure diagram of the backbone network SECOND and the neck network SECONDFPN of this application; Figure 8 This is a network structure diagram of the CenterHead detection head in this application; Figure 9 This is a graph showing the changes in training loss for this application; Figure 10 This is a schematic diagram of the structure of the laser point cloud target detection device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0016] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0017] like Figure 1 As shown, the laser point cloud target detection method provided in this application includes the following steps S101 to S106: Step S101: Obtain laser point cloud data of the closed area; Step S102: Perform distortion correction on the laser point cloud data to obtain the original point cloud data; Step S103: Use the pre-trained detection model to detect the original point cloud data to obtain detection boxes; Step S104: Optimize the detection box using a 3D tracking algorithm to obtain the detection result.

[0018] In application, the method of this embodiment is executed by the vehicle. Laser point cloud data of an enclosed area can be acquired through the vehicle's laser sensor, and detection results are obtained through a detection model mounted in the vehicle's processor. The processor in each step can be a hardware module or a virtual functional module; it can be a single processor or multiple interactive processors.

[0019] This application embodiment acquires laser point cloud data of a closed area; performs distortion correction on the laser point cloud data to obtain raw point cloud data; uses a pre-trained detection model to detect the raw point cloud data, obtaining detection boxes; and optimizes the detection boxes using a 3D tracking algorithm to obtain detection results. This solves the problems of point cloud distortion, detection model complexity, and temporal jitter unique to closed areas. Distortion correction improves data quality, a lightweight model ensures real-time detection efficiency, and the 3D tracking algorithm enhances temporal consistency, thereby achieving high-precision and high-stability target detection in closed area scenarios, directly improving the perception reliability of autonomous driving systems. This meets the practical application needs of intelligent driving.

[0020] To obtain a dataset for training the model, a demonstration vehicle equipped with a forward-facing LiDAR was used to collect data within the office park. During the data collection, the vehicle maintained a speed of approximately 10 km / h. The vehicle traveled along a closed ring road within the office park, covering various locations including office buildings, a cafeteria, an open-air parking lot, workshops, a football field, and grassy areas. The primary targets observed were motor vehicles and pedestrians. Figure 2 This is an example of a point cloud image captured from a single frame. The frame contains clearly visible pedestrian and vehicle targets.

[0021] After driving the demonstration vehicle around the office park, the recorded images of the rosbag bag were analyzed, yielding 4800 frames of point cloud data. Further analysis of the point cloud data revealed that speed bumps and uneven areas on the park's roads did indeed cause distortion in the collected point cloud data. Figure 3 This example presents a typical distortion case, where a small angle is clearly visible between the ground point cloud and the X-axis of the vehicle's coordinate system. This type of distortion can potentially interfere with the accuracy of data annotation and the effectiveness of model training.

[0022] In one embodiment, to effectively reduce the adverse effects of distortion, step S102 includes: Step S1021: Fit the ground plane based on the random sampling consensus algorithm to obtain the ground plane normal vector.

[0023] In this application, a plane fitting technique based on the RANSAC (Random Sample Consensus) algorithm is used to process the collected point cloud data. RANSAC is a robust estimation technique that can filter the optimal set of points that conform to the geometric characteristics of the ground plane from massive datasets containing a large amount of noise and outliers through iterative sampling and hypothesis testing mechanisms. In each iteration, a set of points is randomly selected and assumed to be located on the ground plane. The parameters of the ground plane model are estimated using these points, and the number of interior points that conform to the model is counted. After multiple iterations, the model with the most interior points is selected as the final ground plane fitting model, thereby obtaining the normal vector of the ground plane. This method is highly robust to noise and outliers in the data and can effectively handle point cloud data in complex environments.

[0024] Step S1022: Based on the calculated ground plane normal vector and the preset reference vector, calculate the rotation angle to be applied.

[0025] In application, based on the spatial angle between the fitted ground plane normal vector and the ideal ground normal vector (usually the Z-axis direction vector in the vehicle coordinate system), the required rotation angle is calculated using the vector dot product formula. This angle accurately characterizes the degree of ground plane tilt in the point cloud data, providing a quantitative basis for subsequent rotation operations.

[0026] Step S1023: Rotate the laser point cloud data based on the rotation angle to obtain the original point cloud data.

[0027] In application, a corresponding three-dimensional rotation matrix is ​​constructed based on the calculated rotation angle parameters. The original acquired point cloud is rigidly rotated around a specified rotation axis (usually the Y-axis) to restore the distorted ground plane to an ideal state parallel to the real ground, thereby eliminating systematic errors caused by uneven road surface features and ensuring that subsequent data annotation and model training processes are based on accurate point cloud data with geometric topological relationships.

[0028] This application's embodiments effectively eliminate point cloud geometric distortion caused by road surface unevenness through a robust plane fitting mechanism. The algorithm iteratively samples a set of points to estimate ground plane parameters and selects interior points. Combined with a preset reference vector, it calculates the precise rotation angle, ultimately restoring the ideal geometric relationship to the rotated point cloud. This significantly improves data annotation accuracy and model training performance, providing low-noise, highly consistent input data for subsequent detection, and indirectly enhancing the robustness of the entire detection system.

[0029] This application constructs a dedicated laser point cloud target detection dataset for office park scenarios and uses the RANSAC algorithm to accurately correct the collected point cloud data, thereby addressing point cloud distortion caused by road surface features and improving data quality. By constructing a dedicated dataset and point cloud correction method, point cloud distortion caused by factors such as uneven road surfaces is effectively eliminated, improving the accuracy of data annotation and the effectiveness of model training. This provides a more accurate and reliable data foundation for subsequent target detection and enhances the performance of the entire detection system.

[0030] Figure 4 shows the effect of correcting the point cloud in Figure 3 according to the aforementioned steps. It can be seen intuitively that after the correction process, the original angle between the point cloud ground and the X-axis of the vehicle body coordinate system has been effectively eliminated, and the spatial geometric relationship of the point cloud data has been significantly improved.

[0031] After point cloud distortion correction is completed, ground truth labels for targets within the point cloud need to be obtained. The open-source annotation tool SUSTechPOINTS is used to annotate the position (x, y, z axis coordinates), size (length, width, height), and orientation angle of motor vehicles and pedestrians appearing in the point cloud. Figure 5 This is a schematic diagram of the interface for annotating point clouds using SUSTechPOINTS.

[0032] After annotation, 3266 frames containing the target point cloud were retained to build the dataset. To standardize the training process, the labeled files were converted from .txt format to the .json format of the KITTI dataset. To verify the model's generalization ability, the 3266 frame point clouds were randomly shuffled, and 2000 frames were used to form the training set, while the remaining 1266 frames formed the test set. The target category distributions in the training and test sets are shown in Table 1. As can be seen from Table 1, the target distributions of the two categories in the training and test sets are similar.

[0033] Table 1. Distribution of target categories in the dataset In one embodiment, the pre-trained detection model is implemented based on the CenterPoint architecture, including a voxelization module, a feature encoding layer, a feature extraction backbone network, a neck network, and a detection head; The detection head includes a shared convolutional layer and pedestrian / motor vehicle task branches. The pedestrian task branch and the motor vehicle task branch each include a regression sub-branch, a heatmap sub-branch, a height sub-branch, a rotation angle sub-branch, and a dimension sub-branch. Each sub-branch includes two convolutional layers, and the number of output channels of the second convolutional layer is the number of variables that the current branch needs to output.

[0034] In applications, CenterPoint is an advanced deep learning-based 3D object detection model designed specifically for processing LiDAR point cloud data, suitable for most LiDAR point cloud object detection tasks. It efficiently processes sparse voxelized point cloud data through sparse convolutional networks, extracting rich feature information. CenterPoint employs a multi-task learning framework, predicting the orientation of objects while detecting their position and size, thus improving detection accuracy and robustness. After thorough training, CenterPoint can accurately identify and locate various objects, such as vehicles and pedestrians, and possesses good real-time performance and adaptability, effectively handling complex scenes and data variations.

[0035] The overall structure of the CenterPoint model implemented in this embodiment is as follows: Figure 6 As shown, it includes a point cloud voxelization module, a Pillar feature encoding layer, a point cloud feature extraction backbone network SECOND, a neck network SECONDFPN, ​​and a detection head CenterHead.

[0036] The point cloud voxelization module divides the input point cloud into regular voxels of 0.2x0.2x8, retaining a maximum of 20 points per voxel. The Pillar feature encoding network maps the point cloud features within the voxels to a high-dimensional space, transforming the sparse Pillar features into a dense BEV feature map with a resolution of 512x512. The backbone network SECOND further extracts features from the BEV feature map through three stages of convolutional modules, while the neck network SECONDFPN fuses the features extracted by the backbone network through convolutional and deconvolutional layers. The structure is as follows: Figure 7 As shown.

[0037] The fused features output from the SECONDFPN neck network are then fed into the detection head, CenterHead, to obtain target-level output. Given the specific distribution characteristics of target types in office park scenarios, this embodiment of the application has specifically restructured the CenterHead component in the CenterPoint model. After in-depth analysis of target attributes in the park environment, redundant task branches in the original CenterHead were simplified, and some detection task modules with low relevance in this scenario were removed. While maintaining core functionality, the number of task branches was successfully reduced. This improvement effectively reduces the network parameter scale, simplifies the computation process, achieves a lightweight network structure, and improves the model's operating efficiency in specific scenarios, making it more suitable for the actual needs of laser point cloud target detection in enclosed areas of office parks.

[0038] The structure of CenterHead is as follows: Figure 8 As shown, the process includes a shared convolutional layer and pedestrian / vehicle task branches. The shared convolutional layer contains one convolutional layer to obtain 64 channels of features. Then, based on the detection category, it is divided into two task branches with identical structures: a Regression Branch, a Heatmap Branch, a Height Branch, a Rotation Branch, and a Dimension Branch. Each sub-branch contains two convolutional layers, with the number of output channels of the second convolutional layer equal to the number of variables to be output by the current branch. Finally, the output of each task branch is encoded by the post-processing module Box Coder to obtain the target bounding box, including the x, y, and z coordinates of the target box's center point, the box's length, width, and height, and the rotation angle.

[0039] This application's embodiments achieve network lightweighting and task-specificity through a detection model structure based on the CenterPoint architecture and optimized detection head design. The detection head includes shared convolutional layers and classification task branches. Each branch integrates sub-branches such as regression and heatmaps and simplifies the output channels. This reduces redundant parameters and computational complexity, improving model operating efficiency while maintaining core detection functions. This makes it more suitable for resource-constrained vehicle environments, increasing real-time target detection speed without sacrificing accuracy.

[0040] This application implements a point cloud target detection model based on the CenterPoint architecture, but with targeted improvements to the detection head. By considering the characteristics of targets in closed regions, the number of task branches is reduced to achieve network lightweighting, unlike the general detection head design of existing CenterPoint models. The improved CenterHead structure achieves network lightweighting, reducing computational complexity and improving model efficiency while maintaining target detection accuracy. This makes it more suitable for real-time target detection tasks on resource-constrained vehicle-mounted devices, enhancing the system's real-time performance and practicality.

[0041] In one embodiment, it also includes: Acquire laser point cloud data of a closed area; The laser point cloud data is subjected to distortion correction to obtain the original point cloud data; The model is built based on the CenterPoint architecture, and the original point cloud data is used to train the model to obtain the detection model; The detection model is deployed on the vehicle.

[0042] In this application, after constructing the CenterPoint model, it was trained using raw point cloud data. During training, the Adam optimizer was employed, combined with a cosine annealing learning rate adjustment algorithm to optimize model performance. The entire training process was performed on a single NVIDIA GeForce RTX 2080Ti graphics card, completing 200 training rounds and a total of 200,000 iterations. The changes in the loss function during training are shown below. Figure 9 As shown in the figure, the loss function tends to converge after about 200 training rounds, indicating that the model training has reached a relatively stable state.

[0043] After completing model training, the model from the final training round was used for inference and evaluation on the test set. To fully evaluate the model's detection performance, precision, recall, and F1 score were used as evaluation metrics, and the results are shown in Table 2. The precision, recall, and F1 score for both classes were almost all above 0.8, indicating that the model has high accuracy and robustness in pedestrian and motor vehicle detection tasks, effectively identifying targets and accurately predicting their locations.

[0044] Table 2 Evaluation metrics results on the test set This application embodiment trains the CenterPoint model using a dedicated dataset and deploys it on the vehicle, ensuring that the model is customized for the characteristics of closed areas, eliminating the performance degradation problem of general models in specific scenarios, thereby improving detection accuracy and generalization ability. At the same time, the vehicle deployment realizes a closed loop from training to application, enhancing the practicality and engineering feasibility of the technical solution.

[0045] In one embodiment, deploying the detection model on the vehicle includes: The detection model is converted into an intermediate representation in ONNX format and then serialized using the TensorRT tool before being deployed on the vehicle.

[0046] In this application, ONNX serves as an intermediate format to ensure cross-platform compatibility, while TensorRT serialization accelerates the inference process, reduces computational latency, and improves the execution efficiency of the model on the vehicle processing unit. This solves the problems of difficulty in ensuring temporal consistency in high-frequency detection scenarios on the vehicle side and the deployment bottleneck of deep learning models in real-time systems, directly supporting the real-time requirements of high-frequency point cloud processing.

[0047] In one embodiment, step S103 includes: Step S1031: Input the original point cloud data into the detection model deployed on the vehicle.

[0048] Step S1032: Use the detection model to perform point cloud voxelization on the original point cloud data to obtain point cloud voxel features.

[0049] In application, this step aims to perform preliminary voxelization on the input raw point cloud data to meet the format requirements of the model input, while reducing data dimensionality and improving processing efficiency. The point cloud voxelization logic on the vehicle side is as follows: The point cloud data is divided into a 3D mesh according to a specified voxel size and coordinate range. The coordinates of each point are calculated, and after filtering out points that exceed the limits, the points are assigned to the point list of the corresponding voxels. At the same time, the maximum number of voxels and the maximum number of points within each voxel are limited. Finally, voxels or points exceeding the limits are truncated, and the set of valid voxels is returned. Finally, the average value of the points in each voxel is calculated as the final point cloud voxel feature.

[0050] Step S1033: Use the detection model deployed on the vehicle to infer the voxel features of the point cloud and obtain the inference result.

[0051] In the application, the pre-trained model is used for inference on the vehicle. Since models under the PyTorch framework cannot directly perform inference on the vehicle, the pre-trained model needs to be converted into an intermediate representation in ONNX format. Subsequently, the model is further serialized using the TensorRT tool to optimize its running efficiency on the vehicle. For the serialized model, the input is voxelized point cloud data, and the output includes the output results of five sub-branches corresponding to two task branches.

[0052] In this embodiment, the voxelization module divides the point cloud into a regular grid and extracts features, reducing data dimensionality. During the inference stage, it directly outputs the results of key branches, optimizes the utilization of computing resources, and accelerates the generation of detection boxes. This avoids the computational bottleneck of traditional methods under high-density point clouds, ensuring the smoothness and response speed of real-time detection on the vehicle.

[0053] In one embodiment, after step S1033, post-processing of the inference result is further included, and the post-processing flow is as follows: Extract the indices of the top K highest scores from the heatmap; Based on the index, obtain the corresponding scores, categories, and coordinates from the outputs of the regression sub-branch, heatmap sub-branch, height sub-branch, rotation angle sub-branch, and dimension sub-branch; The coordinates are adjusted by regression features, the rotation angle is calculated and height and dimension information is collected, and the coordinates are transformed to the physical coordinate system. A comprehensive mask is generated based on a score threshold and a preset spatial range to filter out invalid detections; Output the 3D detection bounding boxes that meet the conditions, their corresponding scores, and category labels.

[0054] In application, since the model inference results do not directly correspond to the final target-level detection boxes, further post-processing is required to generate accurate detection boxes. The post-processing logic deployed on the vehicle side is as follows: First, the indices of the top K high scores are extracted from the heatmap. Based on these indices, the corresponding scores, categories, and coordinates are extracted from the outputs of other branches. Then, the coordinates are adjusted through regression features, the rotation angle is calculated, height and dimension information are collected, and the coordinates are transformed to the physical coordinate system. Next, a comprehensive mask is generated based on the score threshold and a preset spatial range (such as the minimum and maximum values ​​of X / Y / height) to filter invalid detections. Finally, the qualified 3D detection boxes (including position, size, and rotation angle), corresponding scores, and category labels are output. The entire process realizes 3D target decoding and filtering from feature maps to physical space.

[0055] The post-processing workflow of this application embodiment accurately parses the model output and filters invalid detections. By extracting high-score features through indexing, adjusting coordinates, transforming the physical coordinate system, and masking, this scheme generates high-quality 3D detection boxes and eliminates noise interference. This improves the accuracy and consistency of the detection boxes, solves the problems of false positives and false negatives in the model output, and provides more reliable target information for downstream decision-making.

[0056] In one embodiment, step S104 includes: Obtain the list of detection boxes output by the detection model; The current frame target box state of each active tracker in the detection box list is predicted using the Kalman filter method; Construct the IOU matrix between predicted bounding boxes and detected bounding boxes; The IOU matrix is ​​correlated using the Hungarian algorithm, and the set of matching pairs, the set of unmatched detection boxes, and the set of unmatched trackers are output. For successfully matched trackers, update the Kalman state with the detection box and reset the loss counter; increment the loss count for unmatched trackers and remove them from the tracker list when the number of consecutive lost frames exceeds a threshold; initialize unmatched detection boxes with the required confidence level as new trackers and add them to the tracker list.

[0057] In application, in order to ensure that the target bounding box results generated by the point cloud target detection model in frame-by-frame detection maintain as much continuity as possible between adjacent frames and avoid visual inconsistency caused by inter-frame jitter or flicker, this application embodiment introduces the SORT (Simple Online and Realtime Tracking) tracking algorithm to optimize the detection bounding box.

[0058] The SORT algorithm is a simple and efficient target tracking algorithm suitable for real-time applications. Its core idea is to predict the target's motion state using Kalman filtering and solve the correlation between target detection and tracking using the Hungarian algorithm. In the field of target detection, existing SORT algorithms are mainly applied to 2D target detection, with relatively limited application in 3D target detection. Therefore, this application, based on in-depth research of the 2D SORT algorithm and combined with the characteristics and requirements of 3D target detection, makes targeted improvements and extensions, designing a SORT algorithm suitable for 3D scenarios. Table 3 shows the pseudocode of this algorithm, which includes six stages. First, the target detector obtains the detection box of the current frame; in this application, this is the detection box obtained through the CenterPoint model. Then, Kalman filtering is used to predict the target box state of all active trackers in the current frame. Unlike the 2D SORT algorithm, the target box state here includes seven parameters (x, y, z, l, w, h, r), representing the x, y, z coordinates of the target box center point, the length, width, and height of the target box, and the yaw angle, respectively. Next, an IOU matrix is ​​constructed between the predicted bounding box and the detection bounding box. The IOU calculation method here differs from the 2D SORT algorithm, as it needs to consider the yaw angle of the target bounding box from the bird's-eye view. Then, the Hungarian algorithm is used for data association, dividing the trackers into matched pairs, unmatched trackers, and unmatched detections. The size of the state transition matrix involved in this process also changes with the state parameters of the target bounding box. Trackers that are successfully matched update their Kalman state with the detection bounding box and reset the loss counter. Unmatched trackers accumulate the number of lost frames and are removed after exceeding a threshold. Finally, unmatched high-confidence detections are initialized as new trackers. This algorithm, through the synergy of Kalman filtering (motion modeling) and the Hungarian algorithm (data association), dynamically maintains the target ID and trajectory with linear complexity, achieving efficient and real-time multi-target tracking.

[0059] Table 3. Pseudocode of 3D SORT Target Tracking Algorithm This application's embodiments enhance the temporal continuity and stability of the detection boxes. By using Kalman filtering to predict the target state, constructing an IOU matrix, and associating data using the Hungarian algorithm, this scheme dynamically maintains the target ID and trajectory, reducing inter-frame jitter and flickering. This directly optimizes the robustness of target tracking, ensures a smooth transition of detection results in complex environments with closed areas, and improves the planning accuracy of the autonomous driving system.

[0060] In terms of target tracking, this application extends the classic 2D SORT algorithm to implement a 3D SORT tracking algorithm suitable for laser point clouds. This optimizes the real-time detection boxes generated by the model, enhancing the temporal continuity and accuracy of the detection boxes. The aforementioned 3D SORT algorithm effectively optimizes the continuity of real-time detection boxes, solving the problem of discontinuous detection box timing in complex environments with closed areas. This improves the stability and accuracy of target tracking, providing more precise target detection and tracking information for autonomous driving decision-making and planning, and enhancing the overall performance and reliability of the system.

[0061] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0062] Table 4 shows three detection examples in the test set. The first column is a point cloud diagram after the targets are labeled using SUSTechPOINTS; the second column is a point cloud diagram after the detected targets are visualized using real-time vehicle-side detection; and the third column is the performance index for each category. In all three examples, the Precision and Recall scores for two categories are no less than 0.8, indicating that the model can accurately detect pedestrians and vehicles within closed areas and has high practical value.

[0063] Table 4 Test Set Detection Examples This application also provides a laser point cloud target detection device for performing the steps described in the above-described laser point cloud target detection method embodiments. The laser point cloud target detection device can be a virtual device within an electronic device, run by the electronic device's processor, or it can be the electronic device itself.

[0064] like Figure 10 As shown, the laser point cloud target detection device 100 provided in this application embodiment includes: Data acquisition module 101 is used to acquire laser point cloud data of a closed area; The distortion correction module 102 is used to perform distortion correction on the laser point cloud data to obtain the original point cloud data. Model detection module 103 is used to detect the original point cloud data using a pre-trained detection model to obtain detection boxes; The target tracking optimization module 104 is used to optimize the detection box using a three-dimensional tracking algorithm to obtain the detection result.

[0065] In applications, the modules in the laser point cloud target detection device can be software program modules, or they can be implemented through different logic circuits integrated in the processor, or they can be implemented through multiple distributed processors.

[0066] like Figure 11 As shown, this application embodiment also provides an electronic device 200, including: at least one processor 201 ( Figure 11 The diagram shows only one processor, memory 202, and computer program 203 stored in memory 202 and executable on at least one processor 201. When processor 201 executes computer program 203, it implements the steps in the various method embodiments described above.

[0067] In applications, electronic devices may include, but are not limited to, processors and memory. Those skilled in the art will understand that... Figure 11 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0068] In applications, the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0069] In applications, memory can be an internal storage unit of an electronic device in some embodiments, such as a hard drive or RAM. In other embodiments, memory can be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal and external storage units of the electronic device. Memory is used to store operating systems, applications, bootloaders, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0070] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0071] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0072] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0073] This application provides a computer program product, including a computer program, which, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0075] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0076] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0077] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A laser point cloud target detection method, characterized in that, include: Acquire laser point cloud data of a closed area; The laser point cloud data is subjected to distortion correction to obtain the original point cloud data; The pre-trained detection model is used to detect the original point cloud data to obtain detection boxes; The detection box is optimized using a 3D tracking algorithm to obtain the detection result.

2. The laser point cloud target detection method as described in claim 1, characterized in that, The distortion correction of the laser point cloud data includes: The ground plane normal vector is obtained by fitting the ground plane using a random sampling consensus algorithm. Based on the calculated ground plane normal vector and the preset reference vector, the required rotation angle is calculated; The laser point cloud data is rotated based on the rotation angle to obtain the original point cloud data.

3. The laser point cloud target detection method as described in claim 1, characterized in that, The pre-trained detection model is implemented based on the CenterPoint architecture, including a voxelization module, a feature encoding layer, a feature extraction backbone network, a neck network, and a detection head; The detection head includes a shared convolutional layer and pedestrian / motor vehicle task branches. The pedestrian task branch and the motor vehicle task branch each include a regression sub-branch, a heatmap sub-branch, a height sub-branch, a rotation angle sub-branch, and a dimension sub-branch. Each sub-branch includes two convolutional layers, and the number of output channels of the second convolutional layer is the number of variables that the current branch needs to output.

4. The laser point cloud target detection method as described in claim 3, characterized in that, Also includes: Acquire laser point cloud data of a closed area; The laser point cloud data is subjected to distortion correction to obtain the original point cloud data; The model is built based on the CenterPoint architecture, and the original point cloud data is used to train the model to obtain the detection model; The detection model is deployed on the vehicle.

5. The laser point cloud target detection method as described in claim 4, characterized in that, Deploying the detection model on the vehicle includes: The detection model is converted into an intermediate representation in ONNX format and then serialized using the TensorRT tool before being deployed on the vehicle.

6. The laser point cloud target detection method as described in claim 1, characterized in that, The detection model obtained through pre-training is used to detect the corrected laser point cloud data to obtain detection boxes, including: The raw point cloud data is input into the detection model deployed on the vehicle. The original point cloud data is voxelized using the detection model to obtain point cloud voxel features; The detection model deployed on the vehicle is used to infer the voxel features of the point cloud to obtain the inference result.

7. The laser point cloud target detection method as described in claim 6, characterized in that, After obtaining the reasoning result, the process also includes: Extract the indices of the top K highest scores from the heatmap; Based on the index, obtain the corresponding scores, categories, and coordinates from the outputs of the regression sub-branch, heatmap sub-branch, height sub-branch, rotation angle sub-branch, and dimension sub-branch; The coordinates are adjusted by regression features, the rotation angle is calculated and height and dimension information is collected, and the coordinates are transformed to the physical coordinate system. A comprehensive mask is generated based on a score threshold and a preset spatial range to filter out invalid detections; Output the 3D detection bounding boxes that meet the conditions, their corresponding scores, and category labels.

8. The laser point cloud target detection method as described in claim 1, characterized in that, The optimization of the detection box using a 3D tracking algorithm to obtain the detection result includes: Obtain the list of detection boxes output by the detection model; The current frame target box state of each active tracker in the detection box list is predicted using the Kalman filter method; Construct the IOU matrix between predicted bounding boxes and detected bounding boxes; The IOU matrix is ​​correlated using the Hungarian algorithm, and the set of matching pairs, the set of unmatched detection boxes, and the set of unmatched trackers are output. For successfully matched trackers, update the Kalman state with the detection box and reset the loss counter; increment the loss count for unmatched trackers and remove them from the tracker list when the number of consecutive lost frames exceeds a threshold; initialize unmatched detection boxes with the required confidence level as new trackers and add them to the tracker list.

9. A laser point cloud target detection device, characterized in that, include: The data acquisition module is used to acquire laser point cloud data of a closed area; The distortion correction module is used to correct the distortion of the laser point cloud data to obtain the original point cloud data. The model detection module is used to detect the original point cloud data using a pre-trained detection model to obtain detection boxes; The target tracking optimization module is used to optimize the detection box using a 3D tracking algorithm to obtain the detection result.

10. An electronic device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, the electronic device performs the method as described in any one of claims 1-8.

Images