Point cloud completion method, device and equipment, and computer program product
By employing a point cloud completion strategy based on target motion models and local geometric features, combined with lightweight optimization, the problems of LiDAR point cloud completion technology in terms of computational resource consumption, dynamic motion compensation accuracy, and adaptability to scenarios with missing data were solved, achieving high-precision and real-time environmental perception effects.
Patent Information
- Application Number
- CN202511076747.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-01
- Publication Date
- 2025-11-11
AI Technical Summary
Existing LiDAR point cloud completion technology has many problems in terms of computing resource consumption, dynamic motion compensation accuracy, ability to distinguish between static and dynamic targets, and adaptability to scenarios with missing data, making it difficult to meet the needs of autonomous driving and intelligent transportation for high-precision, real-time environmental perception.
A point cloud completion strategy based on target motion model and local geometric features is adopted, combined with point cloud lightweight optimization. Dynamic missing data is completed by using the motion model of dynamic targets, static details are repaired by local geometric features, and lightweight processing of point cloud data is performed.
It improves the accuracy and efficiency of point cloud data completion, reduces data storage and transmission costs, and meets the needs of real-time processing and efficient transmission at both vehicle and roadside levels.
Smart Images
Figure CN120931701A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent transportation technology, and in particular to a point cloud completion method, apparatus and equipment, and computer program product. Background Technology
[0002] In the fields of autonomous driving and intelligent transportation, LiDAR (Light Detection and Ranging) has become one of the core sensors due to its advantages such as high precision and long-range detection. Its point cloud data can accurately depict the geometry and spatial distribution of the surrounding environment, providing crucial information for key aspects of vehicle environmental perception, decision-making, planning, and control execution. However, due to various factors, LiDAR point cloud data often suffers from incomplete data, severely impacting the accuracy of environmental perception. Therefore, point cloud completion technology has become a research hotspot and key technology in this field.
[0003] Currently, mainstream LiDAR point cloud completion techniques primarily employ deep learning methods, such as Generative Adversarial Networks (GANs) or diffusion models. These methods train the model using large-scale labeled datasets, enabling it to learn the distribution characteristics and inherent patterns of point cloud data, and then complete the missing point cloud during the inference stage.
[0004] However, existing LiDAR point cloud completion technology has many problems in terms of computing resource consumption, dynamic motion compensation accuracy, ability to distinguish between static and dynamic targets, and adaptability to scenarios with missing data, making it difficult to meet the needs of autonomous driving and intelligent transportation for high-precision, real-time environmental perception. Summary of the Invention
[0005] This application provides a point cloud completion method, apparatus, device, and computer program product to improve the accuracy and efficiency of point cloud data completion.
[0006] The embodiments of this application adopt the following technical solutions:
[0007] In a first aspect, embodiments of this application provide a point cloud completion method, the point cloud completion method comprising:
[0008] Obtain the point cloud data to be completed;
[0009] The point cloud data to be completed is completed using the first point cloud completion strategy to obtain the first point cloud completion result. The first point cloud completion strategy is implemented based on the target motion model.
[0010] The first point cloud completion result is completed using a second point cloud completion strategy to obtain a second point cloud completion result. The second point cloud completion strategy is based on the local geometric features of the point cloud.
[0011] The second point cloud completion result is optimized using a point cloud lightweight optimization strategy to obtain optimized point cloud data.
[0012] Optionally, the point cloud data to be completed is time-series point cloud data, and the step of completing the point cloud data to be completed using the first point cloud completion strategy to obtain the first point cloud completion result includes:
[0013] The time-series point cloud data is aligned to obtain aligned time-series point cloud data;
[0014] Based on the aligned temporal point cloud data, a velocity field of the dynamic target is generated using a velocity field estimation algorithm.
[0015] Based on the velocity field of the dynamic target, a motion consistency verification mechanism is used to verify it, and a valid dynamic target is obtained.
[0016] Based on the velocity field of the effective dynamic target, the point cloud data of the effective dynamic target is completed using a linear interpolation algorithm to obtain the first point cloud completion result.
[0017] Optionally, the step of verifying the velocity field of the dynamic target using a motion consistency verification mechanism to obtain a valid dynamic target includes:
[0018] Based on the velocity field of the dynamic target, the motion trajectory error of the target in consecutive frames is calculated.
[0019] If the motion trajectory error is less than a preset trajectory error threshold, then the target is determined to be a valid dynamic target;
[0020] Otherwise, the target is determined to be a static target.
[0021] Optionally, the step of using a second point cloud completion strategy to complete the first point cloud completion result to obtain a second point cloud completion result includes:
[0022] The spherical neighborhood search algorithm is used to search in the first point cloud completion result to obtain the missing area of the point cloud to be completed;
[0023] Extract the local geometric features of the missing region in the point cloud to be completed;
[0024] Based on the local geometric features of the missing point cloud region to be completed, a point cloud completion algorithm is used to complete the missing point cloud region to obtain the second point cloud completion result.
[0025] Optionally, the local geometric features include the curvature of the points and the normal vector of the local plane. The step of using a point cloud completion algorithm to complete the missing point cloud region based on the local geometric features of the missing region to be completed, and obtaining the second point cloud completion result, includes:
[0026] The missing point cloud region to be completed is filtered based on the normal vector of the local plane to obtain the filtered missing point cloud region to be completed.
[0027] Based on the curvature of the points, planar fitting and / or surface fitting are performed on the missing regions of the filtered point cloud to be completed to obtain the fitting results;
[0028] Based on the fitting results, a uniform sampling algorithm is used to generate completion points in the filtered missing point cloud region to obtain the second point cloud completion result.
[0029] Optionally, the point cloud lightweight optimization strategy includes a point cloud downsampling strategy. The optimization of the second point cloud completion result using the point cloud lightweight optimization strategy to obtain optimized point cloud data includes:
[0030] The point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result is determined based on the local geometric features of the point cloud in the second point cloud completion result;
[0031] The point cloud completion result is downsampled using the point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result to obtain downsampled point cloud data.
[0032] Optionally, the point cloud lightweight optimization strategy includes a point cloud data compression strategy. The optimized point cloud data obtained by optimizing the second point cloud completion result using the point cloud lightweight optimization strategy includes:
[0033] The downsampled point cloud data is compressed using a point cloud data compression strategy to obtain compressed point cloud data.
[0034] Secondly, embodiments of this application also provide a point cloud completion device, the point cloud completion device comprising:
[0035] The acquisition unit is used to acquire the point cloud data to be completed;
[0036] The first completion unit is used to complete the point cloud data to be completed using the first point cloud completion strategy to obtain the first point cloud completion result. The first point cloud completion strategy is implemented based on the target motion model.
[0037] The second completion unit is used to complete the first point cloud completion result using the second point cloud completion strategy to obtain the second point cloud completion result. The second point cloud completion strategy is implemented based on the local geometric features of the point cloud.
[0038] The lightweight unit is used to optimize the second point cloud completion result using a point cloud lightweight optimization strategy to obtain optimized point cloud data.
[0039] Thirdly, embodiments of this application also provide an apparatus, comprising:
[0040] A processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform any of the aforementioned point cloud completion methods.
[0041] Fourthly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement any of the aforementioned point cloud completion methods.
[0042] The point cloud completion method adopted in this application embodiment can achieve the following beneficial effects: First, the point cloud data to be completed is acquired; then, a first point cloud completion strategy is used to complete the point cloud data to be completed, resulting in a first point cloud completion result. The first point cloud completion strategy is implemented based on a target motion model; then, a second point cloud completion strategy is used to complete the first point cloud completion result, resulting in a second point cloud completion result. The second point cloud completion strategy is implemented based on the local geometric features of the point cloud; finally, a point cloud lightweight optimization strategy is used to optimize the second point cloud completion result, resulting in optimized point cloud data. The point cloud completion method in this application embodiment uses the motion model of a dynamic target to complete dynamic missing data and local geometric features to repair static details, effectively addressing the dual challenges of dynamic target occlusion and sparse static background in autonomous driving scenarios, improving the accuracy and efficiency of point cloud data completion; through lightweight point cloud data processing, data storage and transmission costs are reduced, meeting the needs of real-time processing and efficient transmission at the vehicle and roadside levels. Attached Figure Description
[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0044] Figure 1 This is a flowchart illustrating a point cloud completion method in an embodiment of this application;
[0045] Figure 2 This is a schematic diagram of the structure of a point cloud completion device in an embodiment of this application;
[0046] Figure 3 This is a schematic diagram of the structure of a device according to an embodiment of this application. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0048] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0049] Existing LiDAR point cloud completion technologies mainly suffer from the following problems:
[0050] (1) Deep learning relies heavily on computational resources.
[0051] Current mainstream solutions heavily rely on deep learning models, requiring large-scale labeled datasets for training. Acquiring and labeling this data is not only costly but also time-consuming and labor-intensive. For example, building a high-quality labeled dataset covering multiple scenarios and target types may require significant human and material resources, as well as professional annotators and rigorous quality control processes.
[0052] During the inference phase, deep learning models typically require GPU-accelerated point cloud completion to achieve relatively efficient operation. However, even with GPU-accelerated point cloud completion, the latency is generally still greater than the 50ms latency of point cloud completion itself, making it difficult to meet automotive-grade real-time requirements. In autonomous driving scenarios, vehicles need to perceive their surroundings in real time and make rapid decisions; excessive latency may prevent the vehicle from responding to emergencies in a timely manner, increasing safety risks.
[0053] The model fails to adequately capture the motion characteristics of dynamic targets. Dynamic targets exhibit complex characteristics such as acceleration and abrupt trajectory changes during motion, which existing models often struggle to accurately model and predict, resulting in limited completion accuracy. For example, when a vehicle accelerates rapidly or makes a sharp turn, the completed point cloud may not accurately reflect the vehicle's actual position and shape.
[0054] (2) Insufficient accuracy of dynamic motion compensation
[0055] Traditional interpolation algorithms, such as iterative nearest point (ICP)-based temporal registration methods, have significant limitations in high-speed motion scenes. These methods do not adequately consider the independent displacement vectors of dynamic targets, and are prone to introducing motion blur errors when dealing with high-speed moving targets.
[0056] Taking the existing frustum-based feature extraction method as an example, although this method can compress the amount of data to a certain extent, the target trajectory deviation in the completed point cloud exceeds 0.5m (measured data) because it does not incorporate dynamic target motion prediction. This deviation is not negligible in autonomous driving scenarios and may lead to the vehicle misjudging the position of surrounding targets, affecting driving safety.
[0057] (3) Weak ability to distinguish between static and dynamic targets
[0058] Existing technologies typically do not distinguish between dynamic targets and static backgrounds during downsampling. This leads to the easy loss of key information (such as detailed features of dynamic targets like pedestrian edges and vehicle outlines) during downsampling, while redundant point clouds of static backgrounds consume significant computational resources.
[0059] Due to the lack of a mechanism to prioritize the preservation of dynamic targets, the downsampled point cloud data cannot accurately reflect the characteristics of dynamic targets, ultimately affecting the completion effect. Real-world testing data shows an error rate exceeding 15%, which poses a significant challenge for autonomous driving systems requiring high-precision environmental perception.
[0060] (4) Poor adaptability to data missing scenarios
[0061] In scenarios with severe occlusion or low-line LiDAR (such as 16 lines), existing geometric fitting methods, such as the Random Sample Consensus (RANSAC) plane fitting method, lack robustness in completing unstructured environments (such as curved guardrails, irregular vegetation, etc.).
[0062] These methods are susceptible to noise interference, resulting in outliers, with outliers accounting for more than 8% of the total. The presence of outliers can interfere with subsequent environmental perception and decision-making algorithms, reducing the reliability and stability of the system.
[0063] The aforementioned problems mainly stem from the limitations of existing technological approaches:
[0064] (1) Deep learning models are highly dependent on hardware computing power and data annotation. Complex deep learning models require a lot of computing resources for training and inference, while the acquisition and annotation of large-scale labeled datasets are costly, which limits the widespread application of the technology and the improvement of real-time performance.
[0065] (2) Motion compensation does not take into account the independent motion characteristics of dynamic targets. Traditional methods often use a uniform motion model when dealing with dynamic targets, ignoring the motion differences of different dynamic targets, resulting in insufficient motion compensation accuracy.
[0066] (3) Downsampling strategies lack a mechanism to prioritize the preservation of dynamic targets. Existing downsampling methods are usually based on global or local statistical features, without taking into account the difference in importance between dynamic targets and static backgrounds, which can easily lead to the loss of key information.
[0067] (4) Geometric completion does not utilize the fusion of multiple features in the local neighborhood. Existing geometric fitting methods only utilize a single feature of the point cloud (such as position information) and do not fully consider other features in the local neighborhood (such as curvature, normal vector, etc.), resulting in insufficient robustness of completion in unstructured environments.
[0068] Based on this, embodiments of this application provide a point cloud completion method, such as... Figure 1 The diagram provided illustrates a point cloud completion method according to an embodiment of this application. The point cloud completion method includes at least the following steps S110 to S140:
[0069] Step S110: Obtain the point cloud data to be completed.
[0070] In autonomous driving scenarios, the point cloud completion method of this application embodiment can complete point cloud data for both vehicle-side and road-side locations. Vehicle-side point cloud data can be acquired in real time using vehicle-mounted LiDAR and includes point cloud data of dynamic targets such as vehicles, pedestrians, and obstacles, as well as static backgrounds such as roads and buildings. Road-side point cloud data can be acquired using roadside fixed LiDAR and includes point cloud data of the entire traffic scene.
[0071] Step S120: The point cloud data to be completed is completed using the first point cloud completion strategy to obtain the first point cloud completion result. The first point cloud completion strategy is implemented based on the target motion model.
[0072] After acquiring the point cloud data to be completed, a first point cloud completion strategy can be used to complete the data. This strategy is primarily based on the motion characteristics of dynamic targets. For example, a mass motion model can be built for the dynamic target, and its position in occluded or missing frames can be inferred from the model. This allows for the completion of missing point cloud data for the dynamic target, yielding the first point cloud data completion result. Compared to deep learning methods, this approach reduces reliance on GPUs and labeled data while maintaining the effectiveness of point cloud data completion.
[0073] Step S130: The first point cloud completion result is completed using the second point cloud completion strategy to obtain the second point cloud completion result. The second point cloud completion strategy is based on the local geometric features of the point cloud.
[0074] Based on the first point cloud completion result, a second point cloud completion strategy is further used to complete the point cloud. The second point cloud completion strategy mainly uses the local geometric features of the point cloud data to further repair the details missing in the scene due to sensor resolution or distance, improve the geometric realism of the completion result, and thus output the second point cloud completion result.
[0075] Step S140: The second point cloud completion result is optimized using a point cloud lightweight optimization strategy to obtain optimized point cloud data.
[0076] After obtaining the completed point cloud data, a lightweight point cloud optimization strategy is further used to perform lightweight processing on the completed point cloud data. For example, downsampling and compression processing of the point cloud data can be performed. The aim is to reduce data storage and transmission costs while maintaining the geometric accuracy of the completion results, and to support real-time processing on the vehicle and efficient transmission to the cloud.
[0077] The point cloud completion method in this application uses the motion model of the dynamic target to complete the dynamic missing parts and local geometric features to repair static details. It effectively addresses the dual challenges of dynamic target occlusion and static background sparsity in autonomous driving scenarios, and improves the accuracy and efficiency of point cloud data completion. Through lightweight point cloud data processing, it reduces data storage and transmission costs, and meets the needs of real-time processing and efficient transmission at the vehicle and roadside.
[0078] In some embodiments of this application, the point cloud data to be completed is temporal point cloud data. The step of completing the point cloud data to be completed using a first point cloud completion strategy to obtain a first point cloud completion result includes: aligning the temporal point cloud data to obtain aligned temporal point cloud data; generating a velocity field of a dynamic target using a velocity field estimation algorithm based on the aligned temporal point cloud data; verifying the velocity field of the dynamic target using a motion consistency verification mechanism to obtain a valid dynamic target; and completing the point cloud data of the valid dynamic target using a linear interpolation algorithm based on the velocity field of the valid dynamic target to obtain the first point cloud completion result.
[0079] The point cloud data to be completed in this embodiment is temporal point cloud data, such as three consecutive frames of LiDAR point cloud data (with a time interval ≤ 100ms) to ensure that the inter-frame timestamp alignment error is less than a certain threshold, such as 1ms, to avoid motion artifacts caused by time asynchrony. The pose information (position + attitude) of each frame's point cloud is obtained through IMU / GPS or SLAM algorithms, and all frame point clouds are transformed to the same world coordinate system or vehicle coordinate system to achieve background alignment. Then, voxel mesh filtering (voxel size, for example, 0.1m) is applied. 3The point cloud is downsampled to remove outliers (such as those with abnormal reflection intensity or isolated points), retaining an effective point cloud density of over 95%, thus improving the efficiency of subsequent processing. These preprocessing operations can eliminate inter-frame pose differences, providing spatiotemporally consistent input data for velocity field estimation.
[0080] Based on the processed temporal point cloud data, a particle motion model is established for dynamic targets (such as vehicles and pedestrians) in the point cloud data. Assuming that their motion conforms to uniform velocity or uniform acceleration, the instantaneous velocity vector can be calculated based on the displacement of the same point cloud in adjacent frames.
[0081]
[0082] Where, Δp → This represents the spatial displacement of the same point between two frames.
[0083] KD-Tree is used to accelerate nearest neighbor search. Corresponding point clouds (such as points on the surface of the same vehicle) are matched between adjacent frames. Inter-frame displacement vectors are calculated, and outliers with matching errors exceeding a certain threshold (e.g., 0.1m, such as mismatches caused by occlusion or noise) are removed, retaining reliable displacement data. Based on the effective displacement vectors, the instantaneous velocity vectors of the dynamic target at each location are calculated, constructing the velocity field (velocity magnitude + direction) of the dynamic target.
[0084] Based on the velocity field of the aforementioned dynamic targets, a motion consistency verification mechanism is further used to verify the motion of the dynamic targets, identify valid dynamic targets, thereby reducing the false completion rate of static obstacles and improving the reliability of the completion results.
[0085] Based on the displacement vector predicted by the velocity field, the missing point cloud regions of the effective dynamic targets in the current frame are determined (such as holes caused by occlusion or sensor blind spots). Bidirectional linear interpolation is then performed on these missing point cloud regions to generate a complete point cloud. This process allows adjustment of the number of interpolation points to restore the point cloud density of the completed region to a certain proportion, such as over 90%, of the original data, maintaining geometric continuity.
[0086] Motion consistency verification reduces the false completion rate of static obstacles, significantly improving completion accuracy. Velocity field constraints combined with linear interpolation greatly restore the point cloud density of the completed region. Compared to traditional deep learning methods, this reduces reliance on GPUs and labeled data, improving the efficiency and accuracy of point cloud data completion.
[0087] In some embodiments of this application, the step of verifying the dynamic target's velocity field using a motion consistency check mechanism to obtain a valid dynamic target includes: calculating the target's motion trajectory error over multiple consecutive frames based on the dynamic target's velocity field; if the motion trajectory error is less than a preset trajectory error threshold, then the target is determined to be a valid dynamic target; otherwise, the target is determined to be a static target.
[0088] Based on the velocity field, the displacement vector of each dynamic target is predicted between each of the three consecutive frames (e.g., frame 1 → frame 2, frame 2 → frame 3). The predicted displacement is calculated as the velocity vector multiplied by the time interval (e.g., 100ms). The actual displacement vector of the same dynamic target between adjacent frames (i.e., the spatial displacement of the corresponding point cloud) is calculated by point cloud matching (e.g., KD-Tree nearest neighbor search).
[0089] Calculate the Euclidean distance difference between the predicted displacement and the measured displacement, and average the error between two consecutive frames to obtain the average trajectory error e of the dynamic target. If the average trajectory error e of the dynamic target is less than 0.2m, it is determined to be a valid dynamic target (the motion conforms to the velocity field model, and may be a vehicle, pedestrian, etc.). If e ≥ 0.2m, it is determined to be a static target or noise (such as a stationary vehicle being falsely detected as dynamic, or a sudden change in target motion causing the velocity field to fail).
[0090] By quantizing trajectory errors and comparing them with thresholds, only valid dynamic targets are completed, reducing redundant computation, improving point cloud completion efficiency, reducing the false completion rate of static obstacles, significantly reducing invalid completion, adapting to complex scenarios (such as multi-target interaction, rapid acceleration / deceleration), and improving the robustness of dynamic target recognition.
[0091] In some embodiments of this application, the step of using a second point cloud completion strategy to complete the first point cloud completion result to obtain a second point cloud completion result includes: using a spherical neighborhood search algorithm to search in the first point cloud completion result to obtain the missing point cloud region to be completed; extracting the local geometric features of the missing point cloud region to be completed; and using a point cloud completion algorithm to complete the missing point cloud region to be completed based on the local geometric features of the missing point cloud region to be completed, thereby obtaining the second point cloud completion result.
[0092] When using the spherical neighborhood search algorithm to search within the first point cloud completion result, neighborhood search parameters are first set. For example, a spherical neighborhood with a radius of 0.5m is defined, centered on the center of the missing region in the first point cloud completion result. The point cloud is pre-segmented using an Octree spatial index structure, dividing the search space into multi-level cubic units. Hierarchical traversal is used to quickly locate the target neighborhood, ensuring a single-point search time of a certain threshold, such as 0.1ms. The point cloud within the spherical neighborhood is traversed, and regions with a density below the threshold (e.g., fewer than 10 points in the neighborhood) or containing holes are marked as missing regions to be completed. The spatial index accelerates the search, efficiently locating locally missing regions and providing a precise range for subsequent feature extraction.
[0093] After identifying the missing region to be filled, local geometric features are extracted from it. These features may include, for example, the curvature of points and the normal vectors of local planes. By quantizing the geometric structure surrounding the missing region, more constraint information is provided for the completion algorithm.
[0094] Based on the extracted local geometric features, further point cloud completion algorithms, such as uniform sampling algorithms, are used for targeted point cloud completion processing, which further restores the point cloud density in the missing areas and improves the completion accuracy of unstructured environments.
[0095] By employing spherical neighborhood search and Octree acceleration, missing regions are accurately located, reducing point cloud completion errors and improving completion efficiency. Completion is driven by local geometric features (curvature, normal vectors), adapting to the completion needs of complex scenes (such as vehicle edges and pedestrian limbs).
[0096] In some embodiments of this application, the local geometric features include the curvature of points and the normal vector of a local plane. The step of using a point cloud completion algorithm to complete the missing point cloud region based on the local geometric features of the missing region to obtain the second point cloud completion result includes: filtering the missing point cloud region to be completed based on the normal vector of the local plane to obtain a filtered missing point cloud region to be completed; performing plane fitting and / or surface fitting on the filtered missing point cloud region to be completed based on the curvature of the points to obtain a fitting result; and generating completion points in the filtered missing point cloud region to be completed using a uniform sampling algorithm based on the fitting result to obtain the second point cloud completion result.
[0097] (1) Curvature calculation of a point: Calculate the covariance matrix C of the points in the neighborhood and perform eigenvalue decomposition. The curvature of the point is calculated as follows:
[0098]
[0099] Where λ1, λ2, and λ3 are the eigenvalues of the covariance matrix, and the curvature values reflect the degree of surface unevenness (e.g., curvature ≈ 0 in planar regions and curvature > 0.1 in edge regions).
[0100] (2) Calculation of local plane normal vector:
[0101] Principal component analysis (PCA) is used to fit a local plane, and the eigenvector corresponding to the smallest eigenvalue is used as the normal vector direction of the local plane. If the angle between the normal vector and the direction of gravity (negative Z-axis) is less than a certain angle threshold, such as 15°, and the height is less than a certain threshold, such as 0.5m, it is determined to be a ground point and can be excluded and completed in subsequent steps.
[0102] Based on the angle between the normal vector and the direction of gravity and the height threshold, ground point clouds are removed from the missing regions of the point cloud to be completed, reducing invalid completion calculations. The final filtered missing regions of the point cloud to be completed are then obtained.
[0103] For the missing regions of the filtered point cloud to be completed, planar fitting / surface fitting is performed based on the curvature of the points:
[0104] (1) Plane fitting (low curvature region):
[0105] The least squares method is used to fit the plane equation ax + by + cz + d = 0 to the point cloud of the planar region. The coefficients (a, b, c, d) are solved by minimizing the sum of squared residuals to obtain the fitted plane. Uniform sampling is performed within the boundary of the missing region of the fitted plane to generate a point cloud that conforms to the plane equation. The sampling density is consistent with the surrounding point cloud. The density of the surrounding point cloud can be determined based on the point cloud density at the corresponding position of the missing region in the current frame in the completed point cloud data of the previous frame.
[0106] (2) Surface fitting (high curvature region):
[0107] Using the quadratic surface equation z = ax 2 +by 2 The fitted surface is obtained by applying the formula cxy+dx+ey+f, and outliers are removed using the RANSAC algorithm before the parameters are solved. Layered sampling is then performed along the surface normal direction within the boundaries of the missing regions of the fitted surface to generate a smooth point cloud that conforms to the curvature variation.
[0108] To further improve the accuracy of point cloud completion, the Mahalanobis distance DM from the completed point to the fitted model can be calculated. If DM > 3σ (σ is the standard deviation of the model residuals), it is identified as an outlier and removed.
[0109] Curvature classification and geometric fitting reduced the completion error in planar and curved regions. RANSAC and Mahalanobis distance filtering of outliers further improved the completion accuracy of point cloud data.
[0110] In some embodiments of this application, the point cloud lightweight optimization strategy includes a point cloud downsampling strategy. The step of optimizing the second point cloud completion result using the point cloud lightweight optimization strategy to obtain optimized point cloud data includes: determining the point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result based on the local geometric features of the point cloud in the second point cloud completion result; and downsampling the second point cloud completion result using the point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result to obtain downsampled point cloud data.
[0111] The point cloud lightweight optimization strategy in this application includes a point cloud downsampling strategy. The principle of the point cloud downsampling strategy is to prioritize the preservation of dynamic targets and downsample the static background.
[0112] For example, points with curvature > 0.3 (typically vehicle edges and pedestrian contours) retain a 100% sampling rate, meaning no downsampling is performed. This preserves key dynamic information in autonomous driving perception (such as obstacle contours), avoiding the impact of downsampling on target detection and tracking accuracy. For regions with curvature ≤ 0.3 (the average curvature of all points in the region), a random sampling rate, such as 30%, is used. This significantly reduces the number of point clouds in static background regions while retaining sufficient geometric information for environment modeling.
[0113] The above-mentioned point cloud downsampling processing strategy reduces the number of point clouds in the static background area while ensuring the accuracy of point cloud data, reduces the transmission bandwidth requirement, and improves the efficiency of point cloud data transmission and processing, thereby meeting the real-time operation requirements of vehicle-side and road-side embedded devices.
[0114] In some embodiments of this application, the point cloud lightweight optimization strategy includes a point cloud data compression strategy. The step of using the point cloud lightweight optimization strategy to optimize the second point cloud completion result to obtain optimized point cloud data includes: using the point cloud data compression strategy to compress the downsampled point cloud data to obtain compressed point cloud data.
[0115] The point cloud lightweight optimization strategy in this application embodiment may also include a point cloud data compression strategy, which is to compress the overall data volume of the point cloud data after downsampling in the aforementioned embodiment.
[0116] For example, an octree spatial partitioning and encoding method can be used. Using the point cloud bounding box as the root node, it is recursively partitioned into 8 sub-cubes (octree structure) until the number of points within a sub-node is less than or equal to a preset threshold (e.g., 10 points / node) or the maximum depth is reached (e.g., 16 layers). Nodes containing edge points are prohibited from merging to ensure complete preservation of local geometric details. Adjacent nodes containing only non-edge points are merged into a parent node, and their average coordinates are calculated to reduce redundant data storage. Point cloud coordinates are converted from floating-point numbers to integers and stored as the index and offset of the octree node. Through spatial hierarchical storage and adaptive merging, the local structure of dynamic targets is preserved while compressing the static background. Subsequently, the octree structure is recursively reconstructed based on the encoded index to recover the node coordinates and point distribution.
[0117] By employing point cloud data compression strategies, efficient compression of point cloud data and lossless preservation of key information are achieved. While reducing transmission and storage overhead, the perception performance of low-beam LiDAR in complex scenarios is significantly improved, providing a low-cost, highly reliable real-time point cloud processing solution for autonomous driving systems.
[0118] This application also provides a point cloud completion device 200, such as... Figure 2 As shown, a schematic diagram of a point cloud completion device according to an embodiment of this application is provided. The point cloud completion device 200 includes: an acquisition unit 210, a first completion unit 220, a second completion unit 230, and a lightweight unit 240, wherein:
[0119] Acquisition unit 210 is used to acquire the point cloud data to be completed;
[0120] The first completion unit 220 is used to complete the point cloud data to be completed using a first point cloud completion strategy to obtain a first point cloud completion result. The first point cloud completion strategy is implemented based on the target motion model.
[0121] The second completion unit 230 is used to complete the first point cloud completion result using a second point cloud completion strategy to obtain a second point cloud completion result. The second point cloud completion strategy is implemented based on the local geometric features of the point cloud.
[0122] The lightweight unit 240 is used to optimize the second point cloud completion result using a point cloud lightweight optimization strategy to obtain optimized point cloud data.
[0123] In some embodiments of this application, the point cloud data to be completed is temporal point cloud data, and the first completion unit 220 is specifically used for: aligning the temporal point cloud data to obtain aligned temporal point cloud data; generating a velocity field of a dynamic target using a velocity field estimation algorithm based on the aligned temporal point cloud data; verifying the velocity field of the dynamic target using a motion consistency verification mechanism to obtain a valid dynamic target; and completing the point cloud data of the valid dynamic target using a linear interpolation algorithm based on the velocity field of the valid dynamic target to obtain the first point cloud completion result.
[0124] In some embodiments of this application, the first completion unit 220 is specifically used to: calculate the motion trajectory error of the target in multiple consecutive frames based on the velocity field of the dynamic target; if the motion trajectory error is less than a preset trajectory error threshold, then the target is determined to be a valid dynamic target; otherwise, the target is determined to be a static target.
[0125] In some embodiments of this application, the second completion unit 230 is specifically used to: search in the first point cloud completion result using a spherical neighborhood search algorithm to obtain the missing point cloud region to be completed; extract the local geometric features of the missing point cloud region to be completed; and complete the missing point cloud region to be completed using a point cloud completion algorithm based on the local geometric features of the missing point cloud region to be completed, thereby obtaining the second point cloud completion result.
[0126] In some embodiments of this application, the local geometric features include the curvature of a point and the normal vector of a local plane. The second completion unit 230 is specifically used to: filter the missing region of the point cloud to be completed according to the normal vector of the local plane to obtain the filtered missing region of the point cloud to be completed; perform plane fitting and / or surface fitting on the filtered missing region of the point cloud to be completed according to the curvature of the point to obtain the fitting result; and generate completion points in the filtered missing region of the point cloud to be completed using a uniform sampling algorithm according to the fitting result to obtain the second point cloud completion result.
[0127] In some embodiments of this application, the point cloud lightweight optimization strategy includes a point cloud downsampling strategy. The lightweight unit 240 is specifically used to: determine the point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result based on the local geometric features of the point cloud in the second point cloud completion result; and perform downsampling processing on the second point cloud completion result using the point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result to obtain downsampled point cloud data.
[0128] In some embodiments of this application, the point cloud lightweight optimization strategy includes a point cloud data compression strategy, and the lightweight unit 240 is specifically used to: compress the downsampled point cloud data using the point cloud data compression strategy to obtain compressed point cloud data.
[0129] It is understood that the above-mentioned point cloud completion device can realize all the steps of the point cloud completion method provided in the foregoing embodiments. The relevant explanations of the point cloud completion method are applicable to the point cloud completion device, and will not be repeated here.
[0130] Figure 3 This is a schematic diagram of the structure of a device according to an embodiment of this application. For example... Figure 3 As shown, the device includes one or more processors (or processing units), and may also include one or more memories coupled to the processors, and may also include a communication module coupled to the processors.
[0131] A communication module can be used to communicate with other devices or apparatuses, such as sending or receiving data and / or signals. A communication module may have at least one communication module for communication. A communication module may include any interface necessary for communicating with other devices. Exemplarily, a communication module may be a transceiver, circuit, bus, module, or other type of communication module.
[0132] The processor may include, but is not limited to, one or more of the following: a general-purpose computer, a special-purpose computer, a microcontroller, a digital signal processor (DSP), or a controller-based multi-core controller architecture. The device may have multiple processors, such as application-specific integrated circuit (ASIC) chips, which are time-dependent on a clock synchronized with the main processor.
[0133] The memory may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, at least one of the following: read-only memory (ROM), electrically programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD), or other magnetic and / or optical storage. Examples of volatile memories include, but are not limited to, at least one of the following: random access memory (RAM), or other volatile memories that do not persist during the duration of a power outage.
[0134] A computer program consists of computer-executable instructions that are executed by an associated processor. Programs can be stored in ROM. A processor can perform any appropriate action and processing by loading the program into RAM.
[0135] Possible implementations of this application can be achieved through a program, enabling the communication device to execute any of the processes discussed in the foregoing embodiments. Possible implementations of this application can also be achieved through hardware or a combination of software and hardware.
[0136] In some implementations, the program may be tangibly contained in a computer-readable storage medium, which may include in a device (such as in memory) or other storage device accessible by the device. The program may be loaded from the computer-readable storage medium into RAM for execution. The computer-readable storage medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc.
[0137] This application also provides a computer-readable storage medium storing computer instructions or program code thereon, which, when executed by a processor, causes the processor to perform the methods and functions involved in any of the above embodiments. A computer-readable medium can be any tangible medium that contains or stores a program for or relating to an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. More detailed examples of computer-readable storage media include electrical connections with one or more wires, magnetic media (e.g., disks, floppy disks, hard disks, magnetic tapes, magnetic storage devices), optical media (e.g., optical storage devices, DVDs), semiconductor media (e.g., solid-state drives), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), or any suitable combination thereof.
[0138] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. Embodiments of this application also provide at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. This computer program product includes one or more computer-executable instructions, such as instructions included in a program module, which execute in a device on a target real or virtual processor to perform the processes, methods, and functions involved in any of the above embodiments. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0139] This application also proposes a computer program product, including a computer program or instructions that, when run on a computer, cause the computer to perform the processes, methods, and functions described in the above embodiments. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided as needed. The machine-executable instructions for the program modules can be executed locally or in a distributed device. In a distributed device, the program modules can reside in both local and remote storage media.
[0140] Generally, the various embodiments of this application can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are shown and described as block diagrams, flowcharts, or represented using some other illustration, it should be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.
[0141] It should be noted that although embodiments of this application have been described above with reference to the accompanying drawings, these embodiments are not independent of each other, and they can be combined to obtain other embodiments. The methods, situations, categories, and classifications of embodiments in this application are only for the convenience of description and should not constitute a special limitation. Various methods, categories, situations, and features in embodiments can be combined with each other if logically consistent. The various embodiments of this application can be arbitrarily combined to achieve different technical effects. The embodiments of this application will not list various combinations.
[0142] Furthermore, although the operation of the methods of this disclosure is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps. It should also be noted that the features and functions of two or more devices according to this disclosure may be embodied in one device. Conversely, the features and functions of one device described above may be further divided and embodied by multiple devices.
[0143] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0144] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A point cloud completion method, characterized in that, The point cloud completion method includes: Obtain the point cloud data to be completed; The point cloud data to be completed is completed using the first point cloud completion strategy to obtain the first point cloud completion result. The first point cloud completion strategy is implemented based on the target motion model. The first point cloud completion result is completed using a second point cloud completion strategy to obtain a second point cloud completion result. The second point cloud completion strategy is based on the local geometric features of the point cloud. The second point cloud completion result is optimized using a point cloud lightweight optimization strategy to obtain optimized point cloud data.
2. The point cloud completion method according to claim 1, characterized in that, The point cloud data to be completed is time-series point cloud data. The step of completing the point cloud data to be completed using the first point cloud completion strategy to obtain the first point cloud completion result includes: The time-series point cloud data is aligned to obtain aligned time-series point cloud data; Based on the aligned temporal point cloud data, a velocity field of the dynamic target is generated using a velocity field estimation algorithm. Based on the velocity field of the dynamic target, a motion consistency verification mechanism is used to verify it, and a valid dynamic target is obtained. Based on the velocity field of the effective dynamic target, the point cloud data of the effective dynamic target is completed using a linear interpolation algorithm to obtain the first point cloud completion result.
3. The point cloud completion method according to claim 2, characterized in that, The velocity field based on the dynamic target is verified using a motion consistency verification mechanism to obtain valid dynamic targets, including: Based on the velocity field of the dynamic target, the motion trajectory error of the target in consecutive frames is calculated. If the motion trajectory error is less than a preset trajectory error threshold, then the target is determined to be a valid dynamic target; Otherwise, the target is determined to be a static target.
4. The point cloud completion method according to claim 1, characterized in that, The step of using a second point cloud completion strategy to complete the first point cloud completion result to obtain the second point cloud completion result includes: The spherical neighborhood search algorithm is used to search in the first point cloud completion result to obtain the missing area of the point cloud to be completed; Extract the local geometric features of the missing region in the point cloud to be completed; Based on the local geometric features of the missing point cloud region to be completed, a point cloud completion algorithm is used to complete the missing point cloud region to obtain the second point cloud completion result.
5. The point cloud completion method according to claim 4, characterized in that, The local geometric features include the curvature of points and the normal vector of local planes. The step of using a point cloud completion algorithm to complete the missing point cloud region based on the local geometric features of the region to be completed, to obtain the second point cloud completion result, includes: The missing point cloud region to be completed is filtered based on the normal vector of the local plane to obtain the filtered missing point cloud region to be completed. Based on the curvature of the points, planar fitting and / or surface fitting are performed on the missing regions of the filtered point cloud to be filled to obtain the fitting results; Based on the fitting results, a uniform sampling algorithm is used to generate completion points in the filtered missing point cloud region to obtain the second point cloud completion result.
6. The point cloud completion method according to claim 1, characterized in that, The point cloud lightweight optimization strategy includes a point cloud downsampling strategy. The optimized point cloud data obtained by optimizing the second point cloud completion result using this strategy includes: The point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result is determined based on the local geometric features of the point cloud in the second point cloud completion result; The point cloud completion result is downsampled using the point cloud downsampling strategy corresponding to the point cloud in the second point cloud completion result to obtain downsampled point cloud data.
7. The point cloud completion method according to claim 6, characterized in that, The point cloud lightweight optimization strategy includes a point cloud data compression strategy. The optimized point cloud data obtained by optimizing the second point cloud completion result using the point cloud lightweight optimization strategy includes: The downsampled point cloud data is compressed using a point cloud data compression strategy to obtain compressed point cloud data.
8. A point cloud completion device, characterized in that, The point cloud completion device includes: The acquisition unit is used to acquire the point cloud data to be completed; The first completion unit is used to complete the point cloud data to be completed using the first point cloud completion strategy to obtain the first point cloud completion result. The first point cloud completion strategy is implemented based on the target motion model. The second completion unit is used to complete the first point cloud completion result using the second point cloud completion strategy to obtain the second point cloud completion result. The second point cloud completion strategy is implemented based on the local geometric features of the point cloud. The lightweight unit is used to optimize the second point cloud completion result using a point cloud lightweight optimization strategy to obtain optimized point cloud data.
9. An apparatus comprising: processor; And a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the point cloud completion methods of claims 1 to 7.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements any one of the point cloud completion methods described in claims 1 to 7.