A method for enriching a small amount of lidar point cloud data

Abstract

This application discloses a method for enriching a small amount of LiDAR point cloud data. The steps are as follows: constructing a vehicle sample library grouped by distance from the original point cloud data containing a small number of vehicles; constructing a ground point cloud base map based on a semi-enclosed scene and generating deployment intervals with different radii; selecting vehicle point cloud samples from the vehicle sample library that meet the distances of each deployment interval and randomly placing them within the deployment intervals; correcting the orientation angle of the placed vehicle point cloud samples and transforming and fusing them into the ground point cloud base map; performing occlusion simulation on each vehicle point cloud sample fused into the ground point cloud base map to generate the final training point cloud data frame and its corresponding annotation information. The beneficial effects of this application are: by modeling the geometric structure, spatial distribution, and radar scanning characteristics of real vehicle samples, this invention significantly expands the number of training samples without introducing additional manual annotation costs.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method for enriching a small amount of lidar point cloud data. Background Technology

[0002] In recent years, fixed-deployment LiDAR has gradually become an important sensor for vehicle monitoring and environmental perception in scenarios such as service areas, parking lots, urban roads, and highways due to its all-weather operation capability, high spatial measurement accuracy, and strong resistance to light interference. Compared with traditional vision-based perception solutions, LiDAR can directly acquire dense and high-precision 3D point cloud data, and can still stably achieve target detection and behavior tracking under low light or complex lighting conditions.

[0003] However, in target detection and tracking algorithms based on laser point clouds, the model training effect largely depends on the quantity and annotation quality of the training data. In practical engineering applications, the acquisition of LiDAR point cloud data is usually more difficult than that of image data. Furthermore, the manual annotation process for point cloud data is complex, time-consuming, and costly, resulting in a long and costly acquisition cycle for high-quality annotated point cloud samples, making it difficult to construct a sufficiently large training dataset in a short period. When only a small amount of high-quality point cloud data is used for model training, the model is prone to overfitting due to insufficient sample quantity, homogeneous sample distribution, or insufficient differences between samples. This leads to insufficient recognition accuracy and poor generalization ability in practical applications, ultimately affecting the stability and reliability of the overall vehicle recognition system.

[0004] Currently, sample augmentation methods for 3D point cloud data mainly focus on randomly perturbing or transforming existing point cloud data, such as random rotation, translation, scaling, mirroring, noise injection, and point cloud pruning and resampling. While these methods improve the model's robustness to pose changes and noise interference to some extent, they are essentially still based on augmenting the spatial distribution already existing in the original point cloud data. They cannot generate new samples with real physical meaning and fail to fundamentally solve the problems of insufficient training sample quantity and insufficient sample diversity. Summary of the Invention

[0005] One objective of this application is to provide a method for enriching a small amount of lidar point cloud data that can overcome at least one of the deficiencies in the aforementioned background art.

[0006] To achieve at least one of the above objectives, the technical solution adopted in this application is: a method for enriching a small amount of lidar point cloud data, comprising the following steps:

[0007] S100: From the raw point cloud data containing a small number of vehicles, extract the 3D bounding box of each vehicle as a vehicle point cloud sample based on the annotation information, and construct a vehicle sample library grouped by distance based on the different distances of each vehicle to the radar center point.

[0008] S200: Construct a ground point cloud base map based on the semi-enclosed scene, and determine the deployable area in the ground point cloud base map; within the obtained deployable area, generate deployment intervals with different radii according to the distance from the radar center point.

[0009] S300: Select vehicle point cloud samples that match the distance of each deployment interval from the vehicle sample library and place them randomly within the deployment interval; the number of vehicle point cloud samples placed in each deployment interval increases as the radius of the deployment interval increases.

[0010] S400: Based on the orientation angle and azimuth angle of the vehicle point cloud sample in the original point cloud data, and combined with the new azimuth angle of the vehicle point cloud sample at the placement position, the orientation angle of the placed vehicle point cloud sample is corrected and transformed and fused into the ground point cloud base map.

[0011] S500: Based on the view of the radar center point in the ground point cloud base map, occlusion simulation is performed on each vehicle point cloud sample fused in the ground point cloud base map to generate the final training point cloud data frame and its corresponding annotation information.

[0012] Preferably, in step S100, the vehicle's annotation information includes the coordinates of the vehicle's center point in the radar coordinate system, its size parameters, and its rotation angle around the vertical axis; based on the vehicle's annotation information, the coordinates of the vehicle's eight three-dimensional corner points in the radar coordinate system are obtained, and a three-dimensional bounding box is constructed; all point cloud data inside the three-dimensional bounding box are extracted from the original point cloud data in the form of a three-dimensional mask, and the data is then decentralized to form the required vehicle point cloud sample.

[0013] Preferably, in step S200, if the semi-enclosed scene is the current acquisition scene of the original point cloud data, the construction of the ground point cloud base map includes the following process: removing all labeled vehicle point cloud samples from the original point cloud data, and selecting one frame of background point cloud where vehicle removal has been completed as the initial base map; performing ground normalization processing on the initial base map to uniformly transform the background point cloud to a standard coordinate system based on the ground; determining the local blank areas in the background point cloud according to the positions of the removed vehicles in the initial base map; cropping and supplementing the local point cloud data of the corresponding local blank areas in other frames into the background point cloud of the initial base map to obtain a complete ground point cloud base map.

[0014] Preferably, if valid point cloud data for the corresponding local blank area location cannot be obtained in other frames, the supplementation of the local blank area in the initial base map includes the following process: In the initial base map, extract the corner coordinates of the local blank area on the ground plane, and calculate the radial distance and azimuth information of the corner relative to the radar center point; In the initial base map, search for ground areas with similar radial distance and azimuth distribution characteristics in the vicinity of the local blank area, and map the local point cloud data corresponding to the ground area to the local blank area for supplementation.

[0015] Preferably, in step S200, if the semi-enclosed scene is the current acquisition scene that is not the original point cloud data, the horizontal angular resolution and vertical angular distribution of the LiDAR in the semi-enclosed scene are back-calculated; based on the angle parameters obtained by back-calculation, a ground point cloud base map of the semi-enclosed scene is simulated and generated at the same installation height as the LiDAR marked in the original point cloud data.

[0016] Preferably, the acquisition of the ground point cloud base map specifically includes the following process: Based on a frame of point cloud data from a LiDAR in a semi-enclosed scene, calculate the lateral and longitudinal angles of each point cloud relative to the radar coordinate system according to the projection relationship between the point cloud and the horizontal plane and the three-dimensional spatial position of the point cloud, and construct angle sequences respectively; perform noise reduction processing on the angle sequences and sort them according to the size of the angles; obtain the lateral angular resolution by statistically analyzing the difference distribution between adjacent lateral angles; obtain the longitudinal angle distribution by clustering or mean statistical analysis of the longitudinal angles; construct a virtual radar with reference to the installation height of the LiDAR in the original point cloud data; construct the unit direction vectors corresponding to all the back-calculated combinations of lateral and longitudinal angles based on the virtual radar, calculate the intersection positions of each unit direction vector ray and the ground plane, summarize all ground intersections, and obtain the required ground point cloud base map.

[0017] Preferably, in step S200, the radial space of the deployable area is divided into multiple deployment intervals according to a fixed step size based on the distance to the radar center point; when placing vehicle point cloud samples in step S300, a single vehicle point cloud sample is allowed to be placed in its own deployment interval and adjacent deployment intervals at the same time.

[0018] Preferably, in step S400, the process of correcting the orientation angle and transforming the coordinates of the placed vehicle point cloud sample is as follows: the difference between the orientation angle and azimuth angle of the vehicle point cloud sample in the original point cloud data is added to the new azimuth angle of the vehicle point cloud sample at the placement position to obtain the new orientation angle of the vehicle point cloud sample at the placement position; a rotation matrix is ​​constructed based on the difference between the new orientation angle of the vehicle point cloud sample and the rotation angle in the original point cloud data; based on the obtained rotation matrix and combined with the coordinates of the vehicle point cloud sample at the placement position, the decentralized vehicle point cloud sample is rotated and translated so that the placed vehicle point cloud sample is transformed into the ground point cloud base map.

[0019] Preferably, after placing the vehicle point cloud sample, it is necessary to fine-tune the vehicle point cloud sample in the height direction. Specifically, the process includes the following steps: based on the 3D bounding box after transforming the vehicle point cloud sample to the ground point cloud base map, extract the four corner points of its bottom surface and project them onto the ground plane to obtain the ground projection area; calculate the average height of all ground point clouds in the ground projection area, and translate the placed vehicle point cloud sample in the height direction based on the obtained average height.

[0020] Preferably, in step S500, the occlusion simulation of the vehicle point cloud sample includes the following process: extracting two corner points at the top of the three-dimensional bounding box of the vehicle point cloud sample facing the radar center point; constructing rays from the radar center point as the origin to the two extracted corner points respectively; deleting all point clouds located behind the vehicle point cloud sample and below the ray height from the ground point cloud base map in the occlusion area formed by the ray and the ground.

[0021] Compared with the prior art, the beneficial effects of this application are as follows:

[0022] (1) This application significantly expands the number of training samples by modeling the geometric structure, spatial distribution, and radar scanning characteristics of real vehicle samples without introducing additional manual annotation costs. The generated data is highly consistent with the real collected data in terms of spatial structure and point cloud morphology, effectively alleviating the problem of insufficient model training under small sample conditions. This method is particularly suitable for practical engineering scenarios where point cloud data acquisition and annotation costs are high.

[0023] (2) By introducing vehicle orientation angle adjustment and occlusion modeling mechanism based on radar viewpoint, the generated vehicle point cloud conforms to the real lidar imaging law in terms of spatial position, orientation relationship and occlusion effect. Compared with the traditional random perturbation or simple copying method, this application avoids the problem of sample generation that does not conform to physical constraints, thereby improving the credibility and practical value of training data.

[0024] (3) By introducing changes in the number of vehicles, differences in spatial distribution, and multi-target occlusion relationships during the data generation process, this application constructs a dataset that covers more complex and high-risk real-world application scenarios. The model trained on this dataset can learn the spatial structural features and occlusion patterns of vehicles more fully in actual operation, thereby reducing missed detections and false detections and improving the overall stability and reliability of the system.

[0025] (4) This application is based on standard point cloud coordinate calculation and geometric transformation operations, and can be directly embedded into existing LiDAR data processing and model training pipelines. It can quickly build large-scale training datasets without making significant modifications to the original hardware architecture or algorithm framework, and has good engineering application value and promotion prospects. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall working steps of this application. Detailed Implementation

[0027] The present application will now be further described in conjunction with specific embodiments. It should be noted that, in the description of this specification, the use of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicates that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0028] In the description of this application, it should be noted that the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., which indicate the orientation and positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of this application.

[0029] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0030] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0031] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0032] The terms “comprising” and “having”, and any variations thereof, in the specification and claims of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0033] One preferred embodiment of this application, such as Figure 1 As shown, a method for enriching a small amount of lidar point cloud data includes the following steps:

[0034] S100: From the raw point cloud data containing a small number of vehicles, extract the 3D bounding box of each vehicle as a vehicle point cloud sample based on the annotation information, and construct a vehicle sample library grouped by distance according to the different distances of each vehicle to the radar center point.

[0035] Understandably, the limited number of vehicle samples in the original point cloud data makes it difficult to support diverse training of the model for real-world scenarios. Therefore, in this step, we first extract the 3D bounding box of each vehicle as a vehicle point cloud sample from the already labeled original point cloud data containing a small number of vehicle samples, based on the existing precise annotation information. Subsequently, in order to preserve the scanning characteristics of vehicle targets at different distances by the LiDAR (such as point cloud density and morphological features), this step constructs a vehicle sample library grouped by distance, based on the different spatial distances from the center point of each vehicle sample to the center point of the LiDAR.

[0036] Compared to traditional methods, this step can extract a "digital twin of the vehicle" with a realistic physical form from limited raw data and organize it according to the distance dimension, laying the data foundation for subsequent large-scale, highly realistic sample generation. This allows the subsequent sample generation process to no longer depend on the original position of the vehicle in the raw data, realizing the separation and reuse of vehicle samples from the background scene.

[0037] S200: Construct a ground point cloud base map based on the semi-enclosed scene and determine the deployable area in the ground point cloud base map; within the obtained deployable area, generate deployment intervals with different radii according to the distance from the radar center point.

[0038] Understandably, before generating LiDAR point cloud training samples, it is necessary to first determine the target mode for sample generation, i.e., whether to generate point cloud training samples that highly match a specific scene, or to generate general point cloud training samples for general scenes. Since general scenes do not need to consider specific scene geometry and only randomly place vehicles on an idealized flat ground, the generated point cloud data will lack the "spatial constraint features" unique to real scenes, making it difficult to use in specific semi-enclosed scenes such as parking lots and service areas. Therefore, this application focuses on model training for semi-enclosed scenes.

[0039] Therefore, in this step, we can first construct a ground point cloud base map with no vehicle obstruction and a complete background based on the semi-enclosed scene, and then determine the deployable area where vehicles are allowed to appear based on prior scene knowledge in this base map. Finally, within this deployable area, we divide it into multiple annular deployment intervals with different radii according to their distance from the radar center point.

[0040] Compared to traditional methods, this step can quickly construct a structurally complete ground point cloud base map with clear spatial constraints for any given semi-enclosed scene, providing a crucial foundation for the subsequent physical and reasonable placement of vehicle samples. It significantly improves the scene realism and consistency of the generated point cloud data, avoiding the physical inconsistencies caused by randomly placing vehicles in traditional methods.

[0041] S300: Select vehicle point cloud samples that match the distance of each deployment interval from the vehicle sample library and place them randomly within the deployment interval; the number of vehicle point cloud samples placed in each deployment interval increases as the radius of the deployment interval increases.

[0042] Understandably, after completing the construction of the scene base map, in order to realistically simulate the diverse vehicle driving modes in a specific semi-closed scene, this step can select vehicle point cloud samples that meet the distance requirements of each deployment interval from the vehicle sample library constructed in step S100, and randomly sample and place them in the corresponding deployment interval.

[0043] Compared to traditional methods, this step enables on-demand, proportional, and cross-regional random deployment of vehicle samples, maximizing the spatial diversity of generated samples while ensuring physical rationality. It can generate diverse vehicle distribution scenarios covering different distances and densities, effectively mitigating the risk of model overfitting to specific spatial distributions.

[0044] S400: Based on the orientation angle and azimuth angle of the vehicle point cloud sample in the original point cloud data, and combined with the new azimuth angle of the vehicle point cloud sample at the placement position, the orientation angle of the placed vehicle point cloud sample is corrected and transformed and fused into the ground point cloud base map.

[0045] It is understandable that since vehicles have an inherent orientation during the initial data collection (e.g., the front of the vehicle is facing the radar), when they are randomly placed in a new position, their orientation must be adjusted according to the azimuth angle of the new position relative to the radar; otherwise, the generated point cloud shape will deviate significantly from the actual scanning results. This step is precisely to solve this technical problem.

[0046] Compared to traditional methods, this step addresses the issue of vehicle pose adaptation in new scenes, ensuring that the generated point cloud conforms to real-world physics in both orientation and height. The generated vehicle point cloud exhibits realistic morphology and accurate spatial positioning, providing high-quality training data for the model to learn realistic target geometric features.

[0047] S500: Based on the view of the radar center point in the ground point cloud base map, occlusion simulation is performed on each vehicle point cloud sample fused in the ground point cloud base map to generate the final training point cloud data frame and its corresponding annotation information.

[0048] Understandably, in real LiDAR scanning, foreground objects can occlude background objects, resulting in missing point cloud data in the occluded areas. To simulate this key physical phenomenon and make the generated data more realistic, this step simulates occlusion for each vehicle point cloud sample in the fused point cloud base map.

[0049] Compared to traditional methods, this step simulates the physical imaging mechanism of LiDAR, generating point cloud data containing complex occlusion relationships. This enables the target detection and tracking model trained on this data to better learn and understand some appearance features of targets under mutual occlusion conditions, thereby significantly reducing the probability of missed detections and false detections in practical applications and improving the robustness of the model.

[0050] It should be understood that the technical solution of this application aims to solve the technical problems of overfitting, poor generalization ability and low recognition accuracy of target detection and tracking models based on laser point clouds in semi-enclosed scenarios such as service areas and parking lots, where the high cost and long cycle of acquiring high-quality labeled point cloud data lead to insufficient training samples and a single sample distribution.

[0051] This application significantly expands the number of training samples by modeling the geometric structure, spatial distribution, and radar scanning characteristics of real vehicle samples without introducing additional manual annotation costs. The generated data is highly consistent with the real-world data in terms of spatial structure and point cloud morphology, effectively alleviating the problem of insufficient model training under small sample conditions. This method is particularly suitable for practical engineering scenarios where point cloud data acquisition and annotation costs are high.

[0052] This application introduces a vehicle orientation angle adjustment and a radar-viewpoint-based occlusion modeling mechanism, resulting in vehicle point clouds that conform to the real-world LiDAR imaging patterns in terms of spatial location, orientation relationship, and occlusion effect. Compared to traditional random perturbation or simple copying methods, this application avoids the problem of generating samples that do not conform to physical constraints, thereby improving the credibility and practical value of the training data.

[0053] This application incorporates variations in vehicle quantity, spatial distribution differences, and multi-target occlusion relationships during the data generation process, constructing a dataset that covers more complex and high-risk real-world application scenarios. Models trained on this dataset can more fully learn the spatial structural features and occlusion patterns of vehicles during actual operation, thereby reducing missed and false detections and improving the overall stability and reliability of the system.

[0054] This application, based on standard point cloud coordinate calculation and geometric transformation operations, can be directly embedded into existing LiDAR data processing and model training pipelines. It allows for the rapid construction of large-scale training datasets without significant modifications to existing hardware architectures or algorithm frameworks, demonstrating strong engineering application value and promising prospects for widespread adoption. Furthermore, it is not dependent on specific LiDAR models or parameter configurations, nor is it limited by specific point cloud target detection or tracking network structures, making it applicable to various LiDAR configurations and semi-enclosed scenarios such as service areas or parking lots of different sizes and layouts.

[0055] To facilitate understanding of the technical solution of this application, the various steps of this application will be described in detail below.

[0056] In this embodiment, during step S100, each labeled vehicle target corresponds to a set of three-dimensional spatial labeling parameters, i.e., the vehicle's labeling information; specifically, it includes the center point coordinates (x, y, x) of any vehicle i in the radar coordinate system. i y i , zi ), Dimensions (width w) i , length l i high h i and the rotation angle r about the vertical axis (z-axis) i Based on the vehicle's annotation information, the coordinates of its eight 3D corner points in the radar coordinate system can be accurately obtained, thus constructing a 3D bounding box that completely encloses the vehicle's point cloud. Then, using a 3D mask, all point cloud data within this bounding box are extracted from the original point cloud data, and these point cloud data are decentralized (i.e., their center point coordinates are subtracted), ultimately forming independent vehicle point cloud samples that can be used subsequently. This method ensures that the extracted vehicle point cloud samples are spatially complete and facilitates subsequent geometric transformations such as rotation and translation.

[0057] For all extracted vehicle samples, it is necessary to group the vehicles according to the principle of proximity, combining the original vehicle annotation information and simulated vehicle occlusion relationships. Specifically, based on the center point coordinates of vehicle i, the radial distance d from the vehicle point cloud sample to the radar center point can be calculated. i And the azimuth angle θ between the vehicle point cloud sample and the ground principal axis of the radar coordinate system. i The specific calculation method is well known to those skilled in the art, and therefore will not be described in detail here.

[0058] After obtaining the radial distance and angle parameters of each vehicle point cloud sample, a vehicle sample library can be constructed. Specifically, based on the distance from the vehicle to the radar center point, the radial space corresponding to the radar's field of view can be divided into fixed step sizes. d. Divide the space into intervals; for example, in one preferred embodiment, the maximum radial spatial range of the radar is 30m. If d=10m, the radial space of the radar can be divided into three distance intervals: [0, 10 m), [10 m, 20 m), and [20 m, 30 m). Then, based on the calculated radial distance, all labeled vehicle point cloud samples are grouped according to their respective distance spaces. This grouping aims to constrain the density and morphological characteristics of vehicle point clouds within different distance ranges, making the generated samples more consistent with the actual scanning characteristics of a specific lidar at different ranges.

[0059] In this embodiment, when executing step S200, for the semi-enclosed scene sample generation mode, it is necessary to further distinguish whether the sample generation is based on the original point cloud data collected by the LiDAR in the current target scene or on the migration generation based on the original point cloud data of other scenes. That is, when constructing the ground point cloud base map, if the semi-enclosed scene is the current acquisition scene of the original point cloud data obtained in step S100, then the original point cloud data can be directly used to obtain the required ground point cloud base map; if the semi-enclosed scene is not the acquisition scene of the original point cloud data, then the lateral angular resolution and longitudinal angular distribution of the LiDAR in the semi-enclosed scene can be inversely calculated; based on the inversely calculated angle parameters, the ground point cloud base map of the semi-enclosed scene is simulated and generated at the same installation height as the LiDAR marked in the original point cloud data. For ease of understanding, the construction process of the ground point cloud base map in the above two cases will be described in detail below through two specific examples.

[0060] Example 1: If the semi-enclosed scene is the current scene for acquiring raw point cloud data, the construction of the ground point cloud base map includes the following process:

[0061] First, all labeled vehicle point cloud samples are removed from the original point cloud data to obtain a multi-frame background point cloud containing only fixed structures such as ground, roads, and buildings. Then, one frame of the background point cloud that has been processed to remove vehicles is selected as the initial base map.

[0062] Next, ground normalization processing is performed on the obtained initial base map. Specifically, principal component analysis (PCA) and ground extraction processing are performed on the initial base map to eliminate the influence of ground tilt caused by lidar installation height, installation attitude errors, or terrain undulations. Ground extraction can be performed using a plane fitting method based on Random Sample Consensus (RANSAC) to fit the ground plane equation and construct a rotation matrix to uniformly transform the background point cloud to a standard coordinate system based on the ground, thereby eliminating the influence of lidar installation tilt angle or terrain undulations. At the same time, the rotation matrix of the coordinate transformation is recorded and used to perform inverse transformation on the point cloud data after subsequent sample generation to ensure that the final generated point cloud is still in the original lidar coordinate system, thus ensuring the consistency between the generated data and the actual acquired data in the spatial coordinate system.

[0063] Next, based on the positions of the removed vehicles in the initial base map, local blank areas in the background point cloud are determined. Specifically, for each removed vehicle bounding box in the initial base map, its eight corner points are projected onto the ground plane to form a corresponding two-dimensional envelope region, representing the vehicle's occlusion of the ground in the top-down direction. Simultaneously, eight rays are constructed from the radar center point as the origin, pointing to the eight corner points of the vehicle's three-dimensional bounding box; the largest area enclosed by these eight rays and the ground is the vehicle's view occlusion region. The vehicle's two-dimensional envelope region and the view occlusion region together constitute the local blank area, representing the missing point cloud range caused by occlusion of the vehicle in the current frame.

[0064] Finally, by leveraging the complementary nature of information over time, local point cloud data from the corresponding blank areas of the initial base map in other background point cloud frames where vehicle removal has been completed are cropped and added to the background point cloud of the initial base map to obtain a complete ground point cloud base map.

[0065] It is important to note that in this embodiment, for other time frames where vehicle removal has been completed, it is necessary to first search for valid background point cloud data in the corresponding blank areas of the initial base map. If background point cloud data exists in the blank areas in other time frames, the point cloud data in those areas is cropped and added to the background point cloud of the initial base map. This utilizes the temporal complementarity between point clouds in multiple frames to fill in the background missing areas in a single frame point cloud caused by vehicle occlusion. However, in some extreme cases, due to prolonged occlusion or sparse point cloud acquisition, valid point cloud data may not exist in the background point cloud of the corresponding blank areas of the initial base map in a particular time frame. To address this, this embodiment introduces a spatial neighborhood-based completion strategy, which will be described in detail below.

[0066] Specifically, in this example, if valid point cloud data for the corresponding local blank area location cannot be obtained in other frames, the supplementation of the local blank area in the initial base map includes the following process: First, in the initial base map, the corner coordinates of the local blank area on the ground plane are extracted, and the radial distance and azimuth information of the corner point relative to the radar center point are calculated. Then, based on this, ground areas with similar radial distance and azimuth distribution characteristics are searched within the neighborhood of the local blank area, and the local point cloud data corresponding to the ground area is mapped to the local blank area to achieve approximate completion of the local blank area. Through the above neighborhood completion mechanism, even if there is no completely corresponding background data in the point cloud of multiple frames, the continuity and integrity of the generated scene base map point cloud in the overall structure can be guaranteed.

[0067] Example 2: If the semi-enclosed scene is the current acquisition scene that is not the original point cloud data, the construction of the ground point cloud base map includes the following process:

[0068] First, based on a frame of point cloud data from a LiDAR in a semi-enclosed scene, the horizontal angle and vertical angle of each point cloud relative to the radar coordinate system's x-axis are calculated according to the projection relationship between the point cloud and the horizontal plane and the point cloud's 3D spatial position. Then, the horizontal and vertical angles corresponding to all point clouds are summarized to construct a corresponding angle sequence. Wherein, any point cloud p... k Corresponding horizontal angle and longitudinal angle The specific calculation expression is as follows:

[0069] ; .

[0070] In the formula, (x k y k , z k ) represents the point cloud p k Coordinates in the radar coordinate system.

[0071] Then, the obtained angle sequence is denoised and sorted according to the size of the included angle; by statistically analyzing the difference distribution between adjacent lateral included angles, the lateral angular resolution of the lidar can be calculated; by clustering or averaging the longitudinal included angles, the longitudinal angular distribution of the lidar can be calculated.

[0072] Then, based on the horizontal and vertical angle parameters obtained from the above back calculation, a virtual radar can be constructed under the assumption of no obstruction, referring to the installation height h of the lidar in the original point cloud data.

[0073] Finally, based on the virtual radar (center point coordinates are o, installation height is h), all inversely calculated combinations of lateral and longitudinal angles (θ) are constructed. h θ v The unit direction vectors d(θ) corresponding to ) h θ v ), and calculate the position p of the intersection point between each unit direction vector ray and the ground plane. ground By summing up all the ground intersection points, the required ground point cloud base map is obtained. The intersection point location p... ground The calculation expression is:

[0074] p ground =o+t·d(θ h θ v ); t=h / (-sin(θ) v )).

[0075] In the formula, t represents the ray parameter.

[0076] In this embodiment, after obtaining the semi-closed scene base map point cloud that can be used for sample generation, the current scene needs to be spatially divided to limit the deployable area of ​​vehicle targets and avoid generating vehicle samples in locations where vehicles are obviously unlikely to appear. The purpose of scene division is to introduce prior scene constraints so that the spatial distribution of subsequently generated vehicle samples conforms to the traffic and usage patterns of real-world scenarios.

[0077] Specifically, in the obtained ground point cloud base map, spatial areas where vehicles are allowed to appear are selected manually or semi-automatically, taking into account the actual structure of the scene. These deployable areas mainly include parking areas and vehicle driving areas. Locations that clearly do not meet the conditions for vehicle parking or passage, such as streetlights, green belts, barriers, and building foundations, are not considered for vehicle deployment. In actual implementation, multiple key points can be selected in the top-view projection plane of the point cloud, and a two-dimensional polygon can be constructed using these key points as vertices to describe the spatial areas where vehicles can be deployed.

[0078] By employing the above method, the entire ground point cloud base map can be divided into deployable and non-deployable regions. This allows for spatial sampling and deployment of vehicle targets only within the deployable regions during subsequent vehicle sample generation. Specifically, during vehicle deployment, the set of boundary points can be extracted from the 2D polygons corresponding to the deployable regions. This set of boundary points defines the spatial boundary conditions for vehicle target generation, preventing generated vehicle targets from exceeding the deployable region or overlapping with non-deployable regions. This approach effectively constrains vehicle generation positions without introducing complex scene semantic parsing models, providing a spatial basis for the rational deployment of subsequent vehicle samples.

[0079] As can be understood from the foregoing, based on the grouping distance when constructing the vehicle sample library, the deployable area can be divided into multiple deployment areas according to the distance range of the lidar; that is, according to the distance to the center point of the lidar, the radial space of the deployable area is divided into multiple deployment intervals according to a fixed step size.

[0080] In this embodiment, during step S300, based on the divided deployment intervals, vehicle point cloud samples that meet the distance requirements of each deployment interval are selected from the vehicle sample library constructed in step S100, and randomly sampled and placed within the corresponding deployment intervals. To simulate the variation of vehicle density with distance in a real scene (sparse near, dense far), the number of vehicle point cloud samples placed in each deployment interval is set to increase with the radius of the deployment interval. In one specific example, the number N of vehicle point cloud samples placed within the deployment area is... k The radius R from the deployment area to the radar center k The relation is:

[0081] .

[0082] It is important to note that, due to the relatively small number of vehicle samples in the original point cloud data, to avoid missing samples in some deployment intervals, a single vehicle point cloud sample is allowed to be placed in both its own deployment interval and adjacent deployment intervals simultaneously. Specifically, if the radial distance d corresponding to vehicle point cloud sample i is... i Located in the k-th deployment interval Then the vehicle point cloud sample can be simultaneously assigned to the (k-1), k, and k+1th deployment intervals. Due to the span of the deployment intervals ( d) The vehicle is relatively small compared to its own size (around 5m), which means that the migration of vehicle point cloud samples within a limited range will not cause significant distortion to the spatial features of the vehicle point cloud.

[0083] In this embodiment, during step S400, the process of correcting the orientation angle and transforming the coordinates of the placed vehicle point cloud sample is as follows: First, based on the randomly generated vehicle center point position within the deployment interval, calculate its new azimuth angle γ relative to the radar center point. Then, add the difference between the orientation angle α and azimuth angle β of the vehicle point cloud sample in the original point cloud data to the new azimuth angle γ of the vehicle point cloud sample at the placement position to obtain the new orientation angle r* = α - β + γ of the vehicle point cloud sample at the placement position. Then, based on the difference between the calculated new orientation angle r* and the rotation angle r in the original point cloud data... r = r * - r, construct the rotation matrix R; the specific expression is as follows:

[0084] .

[0085] Finally, based on the obtained rotation matrix R and the coordinates c* of the vehicle point cloud sample at the placement position, the decentralized vehicle point cloud sample is rotated and translated, so that the placed vehicle point cloud sample is transformed into the ground point cloud base map. At this time, the coordinates of the center point of the randomly placed vehicle point cloud sample in the ground point cloud base map are p*=R·p+c*; where p represents the center point coordinates of the vehicle point cloud sample in the standard coordinate system with the ground as the reference.

[0086] In this embodiment, after placing and correcting the orientation of the vehicle point cloud sample, it is necessary to fine-tune the vehicle point cloud sample in the height direction to adapt to the slight undulations of the real ground. Specifically, this includes the following process: First, based on the transformed 3D bounding box of the vehicle point cloud sample, extract the four corner points of its bottom surface and project them onto the ground plane to obtain the ground projection area. Then, calculate the average height of all ground point clouds within this area. Finally, translate the entire vehicle point cloud sample along the z-axis so that the bottom surface of its bounding box aligns with the calculated average ground height, thereby ensuring a physically reasonable contact relationship between the vehicle and the ground. For ease of understanding, a specific example will be used to illustrate this in detail below.

[0087] In a specific example, assuming the vehicle point cloud sample is placed at a random location, the heights of the ground point cloud within its bottom projection area are 0.52 meters, 0.55 meters, 0.48 meters, and 0.53 meters, respectively, with a calculated average height of 0.52 meters. The original z-coordinate of the vehicle's bounding box bottom is 0. Therefore, all the vehicle's point clouds can be translated upwards by 0.52 meters along the z-axis, so that the vehicle's "tires" just touch the "ground."

[0088] In this embodiment, when performing step S500, the occlusion simulation of the vehicle point cloud sample includes the following process: First, for each vehicle point cloud sample, extract the two corner points at the top of its 3D bounding box facing the radar center point. Then, with the radar center point as the origin, construct two rays to these two corner points respectively; the area enclosed by these two rays and the ground is the occlusion area of ​​the vehicle. Finally, within this occlusion area, identify all point clouds located behind the vehicle point cloud sample (i.e., further away from the radar) and with a height lower than the rays, and delete them from the ground point cloud base map. After performing the above operations sequentially on all placed vehicles, a frame of simulated lidar point cloud data with a true front-to-back occlusion relationship can be obtained.

[0089] To make it easier to understand, a specific example will be used to illustrate this in detail below.

[0090] In a specific example, suppose in a frame of generated data, a truck is placed 20 meters directly in front of the radar (20, 0), and a car is placed 30 meters behind the truck (50, 0). During occlusion simulation, the two corner points of the truck's top facing the radar (e.g., the corner points at coordinates (20, 1, 2.5) and (20, -1, 2.5)) are extracted, forming two rays with the origin (0, 0, 0). Calculations show that the car is located within the occlusion area enclosed by these two rays, and the height of most of the car's point cloud (especially the lower part) is lower than the height of the rays at the same horizontal distance. Therefore, the parts of the car obscured by the truck (e.g., its front, roof, etc.) are automatically deleted by the system, and the final point cloud only shows the unobstructed rear of the car. This is consistent with the imaging results of a real LiDAR system.

[0091] In summary, by repeatedly performing the above-mentioned vehicle deployment and occlusion simulation process, multiple frames of highly diverse simulated LiDAR point cloud data can be generated in a short time, thereby significantly expanding the training sample size and effectively improving the recognition accuracy and generalization ability of point cloud-based target detection and tracking models.

[0092] The basic principles, main features, and advantages of this application have been described above. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely the principles of this application. Various changes and modifications can be made to this application without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claims. The scope of protection claimed by this application is defined by the appended claims and their equivalents.

Claims

1. A method for enriching a small amount of lidar point cloud data, characterized in that, Includes the following steps: S100: From the raw point cloud data containing a small number of vehicles, extract the 3D bounding box of each vehicle as a vehicle point cloud sample based on the annotation information, and construct a vehicle sample library grouped by distance based on the different distances of each vehicle to the radar center point. S200: Construct a ground point cloud base map based on the semi-enclosed scene, and determine the deployable area in the ground point cloud base map; within the obtained deployable area, generate deployment intervals with different radii according to the distance from the radar center point. S300: Select vehicle point cloud samples that match the distance of each deployment interval from the vehicle sample library and place them randomly within the deployment interval; the number of vehicle point cloud samples placed in each deployment interval increases as the radius of the deployment interval increases. S400: Based on the orientation angle and azimuth angle of the vehicle point cloud sample in the original point cloud data, and combined with the new azimuth angle of the vehicle point cloud sample at the placement position, the orientation angle of the placed vehicle point cloud sample is corrected and transformed and fused into the ground point cloud base map. S5 00: Based on the perspective of the radar center point in the ground point cloud base map, occlusion simulation is performed on each vehicle point cloud sample fused in the ground point cloud base map to generate the final training point cloud data frame and its corresponding annotation information. Specifically, when executing step S200, if the semi-enclosed scene is not the current acquisition scene of the original point cloud data, the acquisition of the ground point cloud base map of the semi-enclosed scene includes the following process: Based on a frame of point cloud data from a LiDAR in a semi-enclosed scene, the lateral and longitudinal angles of each point cloud relative to the radar coordinate system are calculated according to the projection relationship between the point cloud and the horizontal plane and the three-dimensional spatial position of the point cloud, and angle sequences are constructed respectively. The angle sequences are then denoised and sorted according to the size of the angles. The lateral angular resolution is obtained by statistically analyzing the difference distribution between adjacent lateral angles. The longitudinal angle distribution is obtained by clustering or mean analysis of the longitudinal angles. Based on the installation height of the lidar in the original point cloud data, a virtual lidar is constructed. Based on the virtual lidar, the unit direction vectors corresponding to all combinations of horizontal and vertical angles are constructed, and the intersection points of each unit direction vector ray and the ground plane are calculated. All ground intersection points are summarized to obtain the required ground point cloud base map.

2. The method for enriching a small amount of lidar point cloud data as described in claim 1, characterized in that, In step S100, the vehicle's annotation information includes the coordinates of the vehicle's center point in the radar coordinate system, its size parameters, and its rotation angle around the vertical axis. Based on the vehicle's annotation information, the coordinates of the vehicle's eight three-dimensional corner points in the radar coordinate system are obtained, and a three-dimensional bounding box is constructed. All point cloud data inside the three-dimensional bounding box are extracted from the original point cloud data in the form of a three-dimensional mask, and the data is then decentralized to form the required vehicle point cloud sample.

3. The method for enriching a small amount of lidar point cloud data as described in claim 1, characterized in that, In step S200, if the semi-enclosed scene is the current acquisition scene of the original point cloud data, the construction of the ground point cloud base map includes the following process: Remove all labeled vehicle point cloud samples from the original point cloud data, and select one frame of background point cloud that has already undergone vehicle removal processing as the initial base map. The initial base map is subjected to ground normalization processing to uniformly transform the background point cloud to a standard coordinate system based on the ground. Based on the locations of the removed vehicles in the initial base map, determine the local blank areas in the background point cloud; The local point cloud data of the corresponding blank areas in other frames are cropped and added to the background point cloud of the initial base map to obtain a complete ground point cloud base map.

4. The method for enriching a small amount of lidar point cloud data as described in claim 3, characterized in that, If valid point cloud data for the corresponding local blank area location cannot be obtained in other frames, the supplementation of the local blank area in the initial base map includes the following process: In the initial base map, the coordinates of the corner points of the local blank areas on the ground plane are extracted, and the radial distance and azimuth information of the corner points relative to the radar center point are calculated. In the initial base map, ground regions with similar radial distance and azimuth distribution characteristics are searched within the vicinity of the local blank areas, and the local point cloud data corresponding to the ground regions are mapped to the local blank areas to supplement them.

5. The method for enriching a small amount of lidar point cloud data as described in claim 1, characterized in that, In step S200, the radial space of the deployable area is divided into multiple deployment intervals according to a fixed step size based on the distance to the radar center point; in step S300, when placing vehicle point cloud samples, a single vehicle point cloud sample is allowed to be placed in its own deployment interval and adjacent deployment intervals at the same time.

6. The method for enriching a small amount of lidar point cloud data as described in any one of claims 1-5, characterized in that, In step S400, the process of correcting the orientation angle and transforming the coordinates of the placed vehicle point cloud samples is as follows: The difference between the orientation angle and azimuth angle of the vehicle point cloud sample in the original point cloud data is added to the new azimuth angle of the vehicle point cloud sample at the placement position to obtain the new orientation angle of the vehicle point cloud sample at the placement position. Construct a rotation matrix based on the difference between the new orientation angle of the vehicle point cloud sample and the rotation angle in the original point cloud data; Based on the obtained rotation matrix and the coordinates of the vehicle point cloud sample at the placement position, the decentralized vehicle point cloud sample is rotated and translated, so that the placed vehicle point cloud sample is transformed into the ground point cloud base map.

7. The method for enriching a small amount of lidar point cloud data as described in claim 6, characterized in that, After placing the vehicle point cloud samples, it is necessary to fine-tune the vehicle point cloud samples in the height direction, which includes the following process: Based on the 3D bounding box after transforming the vehicle point cloud sample to the ground point cloud base map, extract the four corner points of its bottom surface and project them onto the ground plane to obtain the ground projection area. Calculate the average height of all ground point clouds within the ground projection area, and translate the placed vehicle point cloud samples in the height direction based on the obtained average height.

8. The method for enriching a small amount of lidar point cloud data as described in claim 1, characterized in that, In step S500, the occlusion simulation of the vehicle point cloud samples includes the following process: Extract the two corner points at the top of the 3D bounding box of the vehicle point cloud sample, which face the radar center point. With the radar center point as the origin, construct rays to the two extracted corner points respectively; In the area obscured by the ray and the ground, all point clouds located behind the vehicle point cloud samples and below the height of the ray are deleted from the ground point cloud base map.

Images