A method and device for registering point cloud data of a UAV, and a point cloud registration device

Abstract

The application relates to a UAV point cloud data registration method and device and a point cloud registration equipment, and belongs to the technical field of point cloud data processing. The method comprises the following steps: acquiring and storing UAV point cloud data based on an octree structure; performing linear interpolation compensation on the UAV point cloud data to obtain first point cloud data and second point cloud data; extracting corner points of the first point cloud data and the second point cloud data at multiple scales based on a Harris 3D algorithm, performing matching and fusion, and obtaining a first key point set and a second key point set; determining feature point descriptions of the first key point set and the second key point set based on a neighborhood weighting algorithm and an FPFH feature description algorithm; determining a target conversion matrix based on an ICP point cloud registration algorithm, and registering the UAV point cloud data of a current frame with the UAV point cloud data of a previous frame of the current frame based on the target conversion matrix. The application realizes efficient and accurate registration of UAV point cloud data.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, and in particular to a method, apparatus, and point cloud registration device for registering UAV point cloud data. Background Technology

[0002] With the continuous advancement of drone technology, drones are being used more and more widely in various fields, such as environmental monitoring, terrain mapping, and traffic monitoring. Drones equipped with lidar can efficiently collect three-dimensional point cloud data of the ground or targets.

[0003] However, during drone flight, the collected point cloud data is often distorted due to the drone's movement and the scanning method of LiDAR, which affects the accuracy of the data and the subsequent point cloud registration effect. At the same time, point cloud data is usually characterized by massive volume, high dimensionality, and high noise. When facing large-scale and complex scenarios, the poor computational efficiency of traditional point cloud registration algorithms often fails to meet practical needs.

[0004] Therefore, how to achieve efficient and accurate registration of UAV point cloud data has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, apparatus and point cloud registration device for UAV point cloud data registration, so as to achieve efficient and accurate UAV point cloud data registration.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a registration method for UAV point cloud data, comprising: Get the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure. Linear interpolation compensation is performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and linear interpolation compensation is performed on the stored UAV point cloud data of the previous frame to obtain the second point cloud data. The corner points of the first and second point cloud data at multiple scales are extracted based on the Harris3D algorithm. The corner points of the first point cloud data at multiple scales are matched and fused to obtain the first key point set. The corner points of the second point cloud data at multiple scales are matched and fused to obtain the second key point set. The feature point descriptions of the first key point set and the second key point set are determined based on the neighborhood weighting algorithm and the FPFH feature description algorithm. The ICP point cloud registration algorithm is used to determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set, and the UAV point cloud data of the current frame is registered with the UAV point cloud data of the previous frame based on the target transformation matrix.

[0007] In one possible implementation, the step of performing linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data includes: Distortion compensation, data completion, and smoothing are performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data.

[0008] In one possible implementation, the step of performing distortion compensation, data completion, and smoothing processing on the stored UAV point cloud data of the current frame to obtain the first point cloud data includes: Based on the drone's motion information and acquisition timestamp, linear interpolation is performed on the stored drone point cloud data of the current frame in time. Linear interpolation is performed on the data points in the adjacent regions of the missing region to complete the data points in the missing region; Based on the linear relationship between adjacent data points, the stored UAV point cloud data of the current frame is denoised.

[0009] In one possible implementation, the extraction of corner points from the first and second point cloud data at multiple scales based on the Harris3D algorithm includes: Based on the Gaussian pyramid, we construct multi-scale representations of the first point cloud data and the second point cloud data. Based on the Harris3D algorithm, the corner points of the first point cloud data at each scale are extracted from the representation of the first point cloud data at multiple scales, thus obtaining the corner points of the first point cloud data at multiple scales. The corner points of the second point cloud data at multiple scales are extracted from the representation of the second point cloud data at multiple scales using the Harris3D algorithm.

[0010] In one possible implementation, the matching and fusion of corner points of the first point cloud data at multiple scales to obtain a first set of key points includes: Based on the Euclidean distance between corner points in the same region at two adjacent scales, corner points of the first point cloud data are matched at multiple scales, and the matched corner point pairs are weighted and fused to obtain the first key point set.

[0011] In one possible implementation, matching corner points of the first point cloud data across multiple scales based on the Euclidean distance between corner points of the same region at two adjacent scales includes: If the Euclidean distance between corner points in the same region at two adjacent scales is less than the distance threshold and the feature response values ​​are in the same direction, then the corner points in the same region at two adjacent scales are identified as a matching pair. The weighted fusion of the matched corner point pairs includes: The weight of the corresponding corner point is determined based on the response value or scale of the scale level where the corner point is located, and the matched corner point pairs are then weighted and fused.

[0012] In one possible implementation, the determination of the feature point description of the first keypoint set based on the neighborhood weighting algorithm and the FPFH feature description algorithm includes: Determine the local neighborhood point set of the target key point, where the target key point is any key point in the first key point set; Determine the geometric feature tuples and Euclidean distance between each neighboring point in the local neighborhood point set and the target key point, and determine the weighted weight of each neighboring point in the local neighborhood point set based on the Euclidean distance between each neighboring point in the local neighborhood point set and the target key point. Based on the geometric feature tuples between each neighboring point in the local neighborhood point set and the target key point, and the weighted weight of each neighboring point in the local neighborhood point set, the FPFH feature description of the first key point set is determined as the feature point description of the first key point set.

[0013] In one possible implementation, the target transformation matrix includes a rotation matrix and a translation vector.

[0014] On the other hand, the present invention also provides a registration device for UAV point cloud data, comprising: The acquisition module is used to acquire the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure. The compensation module is used to perform linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and to perform linear interpolation compensation on the stored UAV point cloud data of the previous frame to obtain the second point cloud data. The extraction module is used to extract corner points of the first point cloud data and the second point cloud data at multiple scales based on the Harris3D algorithm. It matches and fuses the corner points of the first point cloud data at multiple scales to obtain the first key point set, and matches and fuses the corner points of the second point cloud data at multiple scales to obtain the second key point set. The determination module is used to determine the feature point descriptions of the first key point set and the second key point set based on the neighborhood weighting algorithm and the FPFH feature description algorithm. The registration module is used to determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set based on the ICP point cloud registration algorithm, and to register the UAV point cloud data of the current frame with the UAV point cloud data of the previous frame based on the target transformation matrix.

[0015] Secondly, the present invention also provides a point cloud registration device, including a data acquisition unit, a memory, and a processor, wherein, The data acquisition device is communicatively connected to the processor and is used to acquire UAV point cloud data of the current frame and UAV point cloud data of the previous frame. The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the UAV point cloud data registration method described in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: The UAV point cloud data registration method, apparatus, and point cloud registration device provided by this invention firstly utilizes an octree topology structure for spatial partitioning, which efficiently manages point cloud data and improves the processing efficiency of subsequent point cloud data, thereby improving the registration efficiency of point cloud data. Next, linear interpolation compensation is applied to the point cloud data to improve its accuracy, thus enhancing the accuracy of subsequent point cloud registration. Then, the Harris3D algorithm is used to extract corner points at multiple scales and perform matching and fusion to extract key points of the point cloud. Following this, the neighborhood weighting algorithm and the FPFH feature description algorithm are used to determine the feature point descriptions of the first and second key point sets, thereby improving the accuracy of feature description. Finally, the ICP point cloud registration algorithm is used to determine the target transformation matrix from the feature point descriptions of the first and second key point sets. Then, based on the target transformation matrix, the UAV point cloud data of the current frame is registered with the UAV point cloud data of the previous frame, achieving unmanned point cloud data registration. This invention achieves efficient and accurate registration of UAV point cloud data. Attached Figure Description

[0017] Figure 1 A schematic flowchart of an embodiment of the registration method for UAV point cloud data provided by the present invention; Figure 2 A schematic flowchart of an embodiment of the UAV point cloud data registration process provided by the present invention; Figure 3 A schematic flowchart of an embodiment of the point cloud distortion preprocessing process based on linear interpolation compensation provided by the present invention; Figure 4 A schematic flowchart of an embodiment of the Harris3D algorithm based on multi-scale improvement provided by the present invention; Figure 5 A schematic flowchart of an embodiment of the FPFH feature description algorithm based on neighborhood weighting provided by the present invention; Figure 6 A schematic diagram of an embodiment of the registration device for UAV point cloud data provided by the present invention; Figure 7 This is a schematic diagram of an embodiment of the point cloud registration device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a method, apparatus, and point cloud registration device for UAV point cloud data, which will be described below.

[0023] Figure 1 A schematic flowchart of an embodiment of the registration method for UAV point cloud data provided by the present invention is shown below. Figure 1 As shown, the registration method for UAV point cloud data includes: S101. Obtain the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure.

[0024] It should be noted that the registration method for UAV point cloud data provided by this invention can be applied to three-dimensional point cloud data registration scenarios, especially UAV three-dimensional point cloud data registration scenarios.

[0025] When registering UAV point cloud data, the point cloud registration device (such as a desktop or portable computer) first acquires the UAV point cloud data of the current frame and the UAV point cloud data of the previous frame. Then, it stores the UAV point cloud data of the current frame and the previous frame using an octree structure. Spatial partitioning through the octree topology allows for efficient management of point cloud data, improving the processing efficiency of subsequent point cloud data and thus enhancing the registration efficiency.

[0026] S102. Perform linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and perform linear interpolation compensation on the stored UAV point cloud data of the previous frame to obtain the second point cloud data.

[0027] It should be noted that in order to improve the accuracy of point cloud data and thus improve the accuracy of subsequent point cloud registration, linear interpolation compensation can be performed on the UAV point cloud data of the current frame to obtain the first point cloud data, and linear interpolation compensation can be performed on the UAV point cloud data of the previous frame to obtain the second point cloud data.

[0028] S103. Based on the Harris3D algorithm, extract the corner points of the first point cloud data and the second point cloud data at multiple scales. Match and fuse the corner points of the first point cloud data at multiple scales to obtain the first key point set. Match and fuse the corner points of the second point cloud data at multiple scales to obtain the second key point set.

[0029] It should be noted that: Next, the Harris3D algorithm can be used to extract corner points from the first and second point cloud data at multiple scales. The corner points of the first point cloud data at multiple scales are then matched and fused to obtain the first set of keypoints. Similarly, the corner points of the second point cloud data at multiple scales are matched and fused to obtain the second set of keypoints. This achieves the extraction of keypoints from both the first and second point cloud data.

[0030] S104. Determine the feature point descriptions of the first key point set and the second key point set based on the neighborhood weighting algorithm and the FPFH feature description algorithm.

[0031] It should be noted that: Next, the feature point descriptions of the first keypoint set and the second keypoint set can be determined by the neighborhood weighting algorithm and the FPFH feature description algorithm, thereby improving the accuracy of feature description.

[0032] S105. Based on the ICP point cloud registration algorithm, determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set, and register the UAV point cloud data of the current frame with the UAV point cloud data of the previous frame based on the target transformation matrix.

[0033] It should be noted that: Finally, the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set can be determined by the ICP point cloud registration algorithm. Then, the UAV point cloud data of the current frame is registered with the UAV point cloud data of the previous frame according to the target transformation matrix, so as to achieve unmanned point cloud data registration.

[0034] In summary, the UAV point cloud data registration method provided in this embodiment of the invention first uses an octree topology structure for spatial partitioning, which can efficiently manage point cloud data and improve the processing efficiency of subsequent point cloud data, thereby improving the registration efficiency of point cloud data. Next, linear interpolation compensation is performed on the point cloud data to improve the accuracy of the point cloud data, thereby improving the accuracy of subsequent point cloud registration. Then, the Harris3D algorithm is used to extract corner points at multiple scales and perform matching and fusion to achieve the extraction of key points in the point cloud. Next, the feature point descriptions of the first key point set and the second key point set are determined by the neighborhood weighting algorithm and the FPFH feature description algorithm, thereby improving the accuracy of feature description. Finally, the ICP point cloud registration algorithm is used to determine the target transformation matrix from the feature point descriptions of the first key point set to the feature point descriptions of the second key point set. Then, the UAV point cloud data of the current frame is registered with the UAV point cloud data of the previous frame according to the target transformation matrix, realizing UAV-level point cloud data registration. This invention achieves efficient and accurate registration of UAV point cloud data.

[0035] In some embodiments of the present invention, the step of performing linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data includes: Distortion compensation, data completion, and smoothing are performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data.

[0036] It should be noted that when performing linear interpolation compensation on the stored UAV point cloud data of the current frame, distortion compensation, data completion, and smoothing can be performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data. The second point cloud data can also be obtained through the same processing method.

[0037] In some embodiments of the present invention, the step of performing distortion compensation, data completion, and smoothing processing on the stored UAV point cloud data of the current frame to obtain the first point cloud data includes: Based on the drone's motion information and acquisition timestamp, linear interpolation is performed on the stored drone point cloud data of the current frame in time. Linear interpolation is performed on the data points in the adjacent regions of the missing region to complete the data points in the missing region; Based on the linear relationship between adjacent data points, the stored UAV point cloud data of the current frame is denoised.

[0038] It should be noted that when performing distortion compensation, data completion, and smoothing on the stored drone point cloud data of the current frame, linear interpolation can be performed on the stored drone point cloud data of the current frame in time based on the drone's motion information and acquisition timestamp. Linear interpolation can also be performed on the data points of adjacent regions of the missing regions to complete the data points of the missing regions. Finally, noise reduction processing can be performed on the stored drone point cloud data of the current frame based on the linear relationship between adjacent data points.

[0039] In some embodiments of the present invention, the extraction of corner points of the first point cloud data and the second point cloud data at multiple scales based on the Harris3D algorithm includes: Based on the Gaussian pyramid, we construct multi-scale representations of the first point cloud data and the second point cloud data. Based on the Harris3D algorithm, the corner points of the first point cloud data at each scale are extracted from the representation of the first point cloud data at multiple scales, thus obtaining the corner points of the first point cloud data at multiple scales. The corner points of the second point cloud data at multiple scales are extracted from the representation of the second point cloud data at multiple scales using the Harris3D algorithm.

[0040] It should be noted that when extracting corner points of the first and second point cloud data at multiple scales using the Harris 3D algorithm, a multi-scale representation of both the first and second point cloud data can be constructed using a Gaussian pyramid. Then, the corner points at each scale of the first point cloud data are extracted using the Harris 3D algorithm, resulting in the corner points of the first point cloud data at multiple scales. Similarly, the corner points at each scale of the second point cloud data are extracted using the Harris 3D algorithm, resulting in the corner points of the second point cloud data at multiple scales.

[0041] In some embodiments of the present invention, the step of matching and fusing corner points of the first point cloud data at multiple scales to obtain a first set of key points includes: Based on the Euclidean distance between corner points in the same region at two adjacent scales, corner points of the first point cloud data are matched at multiple scales, and the matched corner point pairs are weighted and fused to obtain the first key point set.

[0042] It should be noted that when matching and fusing corner points of the first point cloud data at multiple scales, the corner points of the first point cloud data at multiple scales can be matched based on the Euclidean distance between corner points in the same region at two adjacent scales. The matched corner point pairs are then weighted and fused to obtain the first set of key points. The determination of the second set of key points is done in a similar way.

[0043] In some embodiments of the present invention, the matching of corner points of the first point cloud data at multiple scales based on the Euclidean distance between corner points of the same region at two adjacent scales includes: If the Euclidean distance between corner points in the same region at two adjacent scales is less than the distance threshold and the feature response values ​​are in the same direction, then the corner points in the same region at two adjacent scales are identified as a matching pair. The weighted fusion of the matched corner point pairs includes: The weight of the corresponding corner point is determined based on the response value or scale of the scale level where the corner point is located, and the matched corner point pairs are then weighted and fused.

[0044] It should be noted that when matching corner points of the first point cloud data across multiple scales based on the Euclidean distance between corner points in the same region at two adjacent scales, if the Euclidean distance between corner points in the same region at two adjacent scales is less than a distance threshold and the feature response values ​​are in the same direction, then the corner points in the same region at two adjacent scales can be determined as a matching pair. When performing weighted fusion on the matched corner point pairs, the weight of the corresponding corner point can be determined based on the response value or scale of the scale level in which the corner point is located, and then the matched corner point pairs can be weighted and fused.

[0045] In some embodiments of the present invention, the determination of feature point descriptions for the first set of key points based on the neighborhood weighting algorithm and the FPFH feature description algorithm includes: Determine the local neighborhood point set of the target key point, where the target key point is any key point in the first key point set; Determine the geometric feature tuples and Euclidean distance between each neighboring point in the local neighborhood point set and the target key point, and determine the weighted weight of each neighboring point in the local neighborhood point set based on the Euclidean distance between each neighboring point in the local neighborhood point set and the target key point. Based on the geometric feature tuples between each neighboring point in the local neighborhood point set and the target key point, and the weighted weight of each neighboring point in the local neighborhood point set, the FPFH feature description of the first key point set is determined as the feature point description of the first key point set.

[0046] It should be noted that when determining the feature point description of the first keypoint set based on the neighborhood weighting algorithm and the FPFH feature description algorithm, the local neighborhood point set of the target keypoint (any keypoint in the first keypoint set) can be determined first. Then, the geometric feature tuple and Euclidean distance between each neighboring point in the local neighborhood point set and the target keypoint can be determined. The weighting weight of each neighboring point in the local neighborhood point set is then determined based on the Euclidean distance between them. Finally, the FPFH feature description of the first keypoint set is determined as its feature point description based on the geometric feature tuple between each neighboring point in the local neighborhood point set and the target keypoint, and the weighting weight of each neighboring point in the local neighborhood point set.

[0047] In some embodiments of the present invention, the target transformation matrix includes a rotation matrix and a translation vector.

[0048] It should be noted that the target transformation matrix determined by the ICP algorithm can include rotation matrix and translation vector.

[0049] The following describes the registration method for UAV point cloud data provided by the present invention through specific embodiments.

[0050] Combination Figure 2 The registration process for UAV point cloud data specifically includes the following steps: 1. By improving the data structure, a more efficient data structure is used to store the point cloud data of the previous and current frames acquired by the UAV-borne point cloud scanner, thereby accelerating the search and matching speed of subsequent point clouds and improving the processing efficiency of point cloud data.

[0051] A more efficient data structure refers to the octree structure. By partitioning space based on the octree topology, point cloud data can be managed efficiently, improving processing efficiency.

[0052] 2. Perform distortion preprocessing on the stored point cloud data of the previous frame and the point cloud data of the current frame respectively.

[0053] Combination Figure 3 From this perspective, distortion preprocessing refers to using linear interpolation to compensate for and reconstruct point cloud data.

[0054] Linear interpolation compensation refers to estimating and compensating for incomplete or distorted point cloud data caused by various reasons (sensor motion, occlusion) by calculating the linear relationship between adjacent point cloud data. This includes distortion compensation, data completion, and smoothing.

[0055] Distortion compensation refers to the process of linearly interpolating the point cloud data over time using the drone's motion information (IMU data) and the acquisition timestamp when the drone collects point cloud data. This estimates the accurate location of each point cloud data point in the real world, thus compensating for distortion. Data completion addresses the issue of missing data in certain areas of point cloud data due to occlusion, specular reflection, or sensor resolution limitations. It estimates the missing data points based on adjacent data points using linear interpolation, thus completing the data. Smoothing reduces fluctuations and noise between data points by calculating the linear relationship between adjacent data points and making appropriate adjustments, resulting in smoother and more continuous point cloud data. Therefore, by improving the data structure, using a more efficient data structure to store and process point cloud data, and utilizing point cloud reconstruction technology with linear interpolation compensation, the accuracy of point cloud data is improved.

[0056] 3. Using the pre-processed distortion point cloud data from the previous frame and the current frame, extract their respective point cloud feature points using the improved feature extraction algorithm. The improved feature extraction algorithm refers to the multi-scale improved Harris3D algorithm.

[0057] Combination Figure 4 The Harris3D algorithm based on multi-scale improvement refers to constructing a multi-scale spatial representation using a Gaussian pyramid. At each scale, the Harris3D algorithm is used to calculate the features of each point in the point cloud, including the covariance matrix and its eigenvalues. By comparing these eigenvalues, points with corner characteristics can be identified. Finally, by matching and fusing the identified corner points at different scales, the final keypoint detection result can be obtained. Specifically, it includes the following sub-steps: 1) The Gaussian pyramid constructs a multi-scale space.

[0058] Given the original point cloud ,in First, a Gaussian pyramid is constructed to obtain point cloud representations at L scales:

[0059] in This represents the original point cloud (at the finest scale). For the s-th layer... By analyzing the point cloud layer above The result is obtained by performing Gaussian filtering and downsampling. The specific steps are as follows: Gaussian filtering: for each point Calculate the weighted average position of its neighborhood points, with weights determined by a three-dimensional Gaussian kernel. Decision, among which Let be the scale parameter of the s-th layer (usually taken as ). (where k is the initial scale and k is the scale factor).

[0060] Downsampling: In the filtered point cloud, uniform downsampling is performed according to the voxel grid or distance threshold to reduce the point cloud resolution layer by layer, thereby simulating the structural changes from fine to coarse.

[0061] After the above processing, a set of point cloud sequences from high resolution to low resolution is obtained, reflecting geometric features at different scales.

[0062] 2) Harris3D corner detection at each scale.

[0063] For each scale s, in the point cloud The Harris3D corner detection is performed independently. The specific process is as follows: Calculate the covariance matrix for each point: for point Determine its neighborhood (For example, all points within radius rs or the nearest Ks points). Calculate the centroid of the neighborhood points. Then construct the covariance matrix:

[0064] It is a 3×3 symmetric positive semi-definite matrix.

[0065] Solving for eigenvalues: Eigenvalue decomposition yields three eigenvalues. , , And satisfy The magnitude of the eigenvalue reflects the degree of dispersion of the point cloud along the principal direction at that point. like This indicates that the point cloud is distributed in a linear pattern (edge ​​points); like This indicates that the point cloud is distributed in a planar pattern (planar points); like This indicates that the point cloud is evenly distributed in all directions (scattered points or corner points).

[0066] Calculate corner response values: Define the Harris3D corner response function as follows:

[0067] in:

[0068]

[0069] α is an empirical constant (usually taken as 0.001). 0.01).

[0070] Non-maximum suppression: In the neighborhood of each point, retain the point with the largest local response value and remove the rest to obtain the candidate corner point set K(s) at this scale.

[0071] 3) Cross-scale corner point matching and fusion.

[0072] Corner points detected at different scales may correspond to the same physical corner point at different resolutions. In order to obtain scale-invariant keypoints, matching and fusion between adjacent scales are required.

[0073] Scale-space backtracking: for corner points at a fine scale s Based on the correspondence during downsampling (e.g., recording the source point index of each downsampling point), find its associated region in the coarse scale s+1, and calculate the spatial distance between the corner point ks+1 detected at the coarse scale and ks.

[0074] Matching strategy: If the Euclidean distance between two corner points ks and ks+1 is less than a threshold If the feature response values ​​are aligned (typically twice the size of adjacent scale voxels), they are considered a matching pair.

[0075] Fusion and final keypoint generation: For a set of matching corner pairs The final keypoint location is calculated using a weighted average method, with the weights proportional to the response value or the inverse of the scale level, thus preserving the geometric center across multiple scales.

[0076] If a corner point has no matching object in adjacent scales and its response value is higher than the global threshold of that scale, it is still retained as a key point (representing a feature unique to that scale).

[0077] The final set of key points It contains multi-scale feature points ranging from coarse to fine, and has scale invariance and strong noise resistance.

[0078] 4. Extract the feature points from the point cloud data of the previous frame and the point cloud data of the current frame, and perform feature point description based on neighborhood weighting. Feature point description based on neighborhood weighting refers to describing feature points using the FPFH (Fast Point Feature Histograms) feature description algorithm based on neighborhood weighting.

[0079] Combination Figure 5 The improved FPFH feature description algorithm based on neighborhood weighting is an improvement on the FPFH algorithm. It further enhances the accuracy and robustness of feature description by introducing a neighborhood weighting mechanism. The specific implementation steps are as follows: 1) Establish the local neighborhood of the query point.

[0080] For the extracted key points (Feature points in the point cloud of the current or previous frame) determine their local neighborhood point set. The neighborhood can be searched based on a fixed radius r, or based on the number of nearest neighbors k (k=50 is recommended). 100). Let the number of neighborhood points be m, i.e. Simultaneously, it is necessary to obtain the normal vector ni for each point. The normal vector can be calculated in the local neighborhood using principal component analysis (PCA).

[0081] 2) Calculate the geometric feature tuples between the neighboring points and the center point.

[0082] For each neighboring point Calculate its distance from the center point The three geometric features between them form the basis of FPFH:

[0083]

[0084]

[0085] in , , .

[0086] 3) Calculate the weights of neighboring points (the core of the neighborhood weighting mechanism).

[0087] For each neighboring point Calculate its distance from the center point Euclidean distance between them:

[0088] A Gaussian weighting function is used to assign weights to each point:

[0089] in This is a bandwidth parameter, typically taken as 1 / 3 of the neighborhood radius or adaptively determined based on the point cloud density.

[0090] Normalized weights: To ensure the sum of the weights is a constant and facilitate threshold comparison, the weights can be normalized.

[0091] Because the weights are accumulated values ​​in the subsequent histogram statistics, normalization only affects the absolute value of the feature vector, not the relative distribution.

[0092] 4) Construct a weighted simplified point feature histogram (SPFH).

[0093] Will Each neighborhood point is divided into b intervals (b=11), and the 3D histogram H is initialized to 0. For each neighborhood point, according to... Determine the index and accumulate weights:

[0094] After traversal, L2 normalization yields .

[0095] 5) Expand the neighborhood to calculate the final weighted FPFH.

[0096]

[0097] in It needs to be pre-calculated recursively for each neighborhood point. The output is a weighted FPFH feature vector.

[0098] The neighborhood weighting mechanism refers to using a Gaussian weighting function to calculate weights based on the Euclidean distance between neighboring points and the center point. This effectively utilizes the information of neighboring points to enhance the feature description of the center point, which can not only improve the accuracy of feature description but also enhance the robustness of the algorithm to noise and occlusion.

[0099] 5. Match the point cloud data of the current frame with the point cloud data of the previous frame to obtain the transformation matrix. Using the feature points extracted and described from the point cloud data of the current frame and the point cloud data of the previous frame, the ICP point cloud registration algorithm is used to quickly configure the feature points of the two point clouds, obtaining the rotation matrix and translation vector.

[0100] 6. Using the obtained rotation matrix and translation vector, the point cloud data of the current frame is registered with the point cloud data of previous frames, preparing data for subsequent UAV operations. By replacing the entire point cloud registration with feature point registration, timeliness is improved while avoiding point cloud registration getting stuck in local optima, thus improving the accuracy of point cloud registration.

[0101] Compared with the prior art, the beneficial effects of the present invention are as follows: To address the distortion of UAV point cloud data caused by complex and ever-changing scenarios, this invention employs multi-scale improved feature extraction techniques, such as edge and corner point extraction, to distinguish different objects from the background. Utilizing these features not only accurately identifies and removes distorted portions of the point cloud data but also preserves the true geometric shape and structural features of objects, improving the accuracy and robustness of feature extraction. Addressing the issue that existing point cloud distortion removal methods cannot simultaneously achieve both real-time performance and accuracy, this invention improves the data structure by using a more efficient data structure for storing and processing point cloud data. Simultaneously, it utilizes linear interpolation compensation point cloud reconstruction technology to enhance the accuracy of the point cloud data. Finally, addressing the poor computational efficiency of traditional point cloud registration algorithms, this invention replaces full point cloud registration with feature point registration, improving efficiency while avoiding point cloud registration getting trapped in local optima, thus improving the accuracy of point cloud registration.

[0102] To better implement the UAV point cloud data registration method in this embodiment of the invention, based on the UAV point cloud data registration method, correspondingly, as follows: Figure 6 As shown, this embodiment of the invention also provides a registration device for UAV point cloud data. The UAV point cloud data registration device 600 includes: The acquisition module 601 is used to acquire the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure. The compensation module 602 is used to perform linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and to perform linear interpolation compensation on the stored UAV point cloud data of the previous frame to obtain the second point cloud data. The extraction module 603 is used to extract corner points of the first point cloud data and the second point cloud data at multiple scales based on the Harris3D algorithm, match and fuse the corner points of the first point cloud data at multiple scales to obtain a first key point set, and match and fuse the corner points of the second point cloud data at multiple scales to obtain a second key point set. The determination module 604 is used to determine the feature point descriptions of the first key point set and the second key point set based on the neighborhood weighting algorithm and the FPFH feature description algorithm. The registration module 605 is used to determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set based on the ICP point cloud registration algorithm, and to register the UAV point cloud data of the current frame with the UAV point cloud data of the previous frame based on the target transformation matrix.

[0103] The UAV point cloud data registration device 600 provided in the above embodiments can realize the technical solutions described in the above UAV point cloud data registration method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above UAV point cloud data registration method embodiments, and will not be repeated here.

[0104] like Figure 7 As shown, the present invention also provides a point cloud registration device 700. The point cloud registration device 700 includes a processor 701, a memory 702, a display 703, and a data acquisition unit 704. Figure 7 Only some components of the point cloud registration device 700 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0105] In some embodiments, processor 701 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 702 or process data, such as the registration method for UAV point cloud data in this invention.

[0106] In some embodiments, processor 701 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 701 may be local or remote. In some embodiments, processor 701 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, etc., or any combination thereof.

[0107] In some embodiments, the memory 702 may be an internal storage unit of the point cloud registration device 700, such as a hard disk or memory of the point cloud registration device 700. In other embodiments, the memory 702 may also be an external storage device of the point cloud registration device 700, such as a pluggable hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the point cloud registration device 700.

[0108] Furthermore, the memory 702 may include both internal storage units of the point cloud registration device 700 and external storage devices. The memory 702 is used to store the application software and various types of data for which the point cloud registration device 700 is installed.

[0109] In some embodiments, display 703 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an organic light-emitting diode (OLED) touchscreen. Display 703 is used to display information from the point cloud registration device 700 and to display a user interface for visualization. Components 701-703 of the point cloud registration device 700 communicate with each other via a system bus.

[0110] In some embodiments, the data acquisition device 704 may be a chip or integrated circuit with data transmission and reception functions.

[0111] In one embodiment, when the processor 701 executes the registration program for UAV point cloud data in the memory 702, the following steps can be implemented: Get the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure. Linear interpolation compensation is performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and linear interpolation compensation is performed on the stored UAV point cloud data of the previous frame to obtain the second point cloud data. The corner points of the first and second point cloud data at multiple scales are extracted based on the Harris3D algorithm. The corner points of the first point cloud data at multiple scales are matched and fused to obtain the first key point set. The corner points of the second point cloud data at multiple scales are matched and fused to obtain the second key point set. The feature point descriptions of the first key point set and the second key point set are determined based on the neighborhood weighting algorithm and the FPFH feature description algorithm. The ICP point cloud registration algorithm is used to determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set, and the UAV point cloud data of the current frame is registered with the UAV point cloud data of the previous frame based on the target transformation matrix.

[0112] It should be understood that when the processor 701 executes the registration program for the UAV point cloud data in the memory 702, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0113] Furthermore, this embodiment of the invention does not specifically limit the type of point cloud registration device 700 mentioned. The point cloud registration device 700 can be a portable electronic device such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, the point cloud registration device 700 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0114] The above provides a detailed description of the registration method, apparatus, and point cloud registration device for UAV point cloud data provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A registration method for UAV point cloud data, characterized in that, include: Get the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure. Linear interpolation compensation is performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and linear interpolation compensation is performed on the stored UAV point cloud data of the previous frame to obtain the second point cloud data. The corner points of the first and second point cloud data at multiple scales are extracted based on the Harris3D algorithm. The corner points of the first point cloud data at multiple scales are matched and fused to obtain the first key point set. The corner points of the second point cloud data at multiple scales are matched and fused to obtain the second key point set. The feature point descriptions of the first key point set and the second key point set are determined based on the neighborhood weighting algorithm and the FPFH feature description algorithm. The ICP point cloud registration algorithm is used to determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set, and the UAV point cloud data of the current frame is registered with the UAV point cloud data of the previous frame based on the target transformation matrix.

2. The registration method for UAV point cloud data according to claim 1, characterized in that, The step of performing linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data includes: Distortion compensation, data completion, and smoothing are performed on the stored UAV point cloud data of the current frame to obtain the first point cloud data.

3. The registration method for UAV point cloud data according to claim 2, characterized in that, The process of performing distortion compensation, data completion, and smoothing on the stored UAV point cloud data of the current frame to obtain the first point cloud data includes: Based on the drone's motion information and acquisition timestamp, linear interpolation is performed on the stored drone point cloud data of the current frame in time. Linear interpolation is performed on the data points in the adjacent regions of the missing region to complete the data points in the missing region; Based on the linear relationship between adjacent data points, the stored UAV point cloud data of the current frame is denoised.

4. The registration method for UAV point cloud data according to claim 1, characterized in that, The extraction of corner points from the first and second point cloud data at multiple scales based on the Harris3D algorithm includes: Based on the Gaussian pyramid, we construct multi-scale representations of the first point cloud data and the second point cloud data. Based on the Harris3D algorithm, the corner points of the first point cloud data at each scale are extracted from the representation of the first point cloud data at multiple scales, thus obtaining the corner points of the first point cloud data at multiple scales. The corner points of the second point cloud data at multiple scales are extracted from the representation of the second point cloud data at multiple scales using the Harris3D algorithm.

5. The registration method for UAV point cloud data according to claim 1, characterized in that, The process of matching and fusing corner points of the first point cloud data at multiple scales to obtain a first set of key points includes: Based on the Euclidean distance between corner points in the same region at two adjacent scales, corner points of the first point cloud data are matched at multiple scales, and the matched corner point pairs are weighted and fused to obtain the first key point set.

6. The registration method for UAV point cloud data according to claim 5, characterized in that, The matching of corner points of the first point cloud data at multiple scales based on the Euclidean distance between corner points of the same region at two adjacent scales includes: If the Euclidean distance between corner points in the same region at two adjacent scales is less than the distance threshold and the feature response values ​​are in the same direction, then the corner points in the same region at two adjacent scales are identified as a matching pair. The weighted fusion of the matched corner point pairs includes: The weight of the corresponding corner point is determined based on the response value or scale of the scale level where the corner point is located, and the matched corner point pairs are then weighted and fused.

7. The registration method for UAV point cloud data according to claim 1, characterized in that, The feature point descriptions for determining the first set of key points based on the neighborhood weighting algorithm and the FPFH feature description algorithm include: Determine the local neighborhood point set of the target key point, where the target key point is any key point in the first key point set; Determine the geometric feature tuples and Euclidean distance between each neighboring point in the local neighborhood point set and the target key point, and determine the weighted weight of each neighboring point in the local neighborhood point set based on the Euclidean distance between each neighboring point in the local neighborhood point set and the target key point. Based on the geometric feature tuples between each neighboring point in the local neighborhood point set and the target key point, and the weighted weight of each neighboring point in the local neighborhood point set, the FPFH feature description of the first key point set is determined as the feature point description of the first key point set.

8. The registration method for UAV point cloud data according to claim 1, characterized in that, The target transformation matrix includes a rotation matrix and a translation vector.

9. A registration device for UAV point cloud data, characterized in that, include: The acquisition module is used to acquire the drone point cloud data of the current frame and the drone point cloud data of the previous frame, and store the drone point cloud data of the current frame and the drone point cloud data of the previous frame based on an octree structure. The compensation module is used to perform linear interpolation compensation on the stored UAV point cloud data of the current frame to obtain the first point cloud data, and to perform linear interpolation compensation on the stored UAV point cloud data of the previous frame to obtain the second point cloud data. The extraction module is used to extract corner points of the first point cloud data and the second point cloud data at multiple scales based on the Harris3D algorithm. It matches and fuses the corner points of the first point cloud data at multiple scales to obtain the first key point set, and matches and fuses the corner points of the second point cloud data at multiple scales to obtain the second key point set. The determination module is used to determine the feature point descriptions of the first key point set and the second key point set based on the neighborhood weighting algorithm and the FPFH feature description algorithm. The registration module is used to determine the target transformation matrix from the feature point description of the first key point set to the feature point description of the second key point set based on the ICP point cloud registration algorithm, and to register the UAV point cloud data of the current frame with the UAV point cloud data of the previous frame based on the target transformation matrix.

10. A point cloud registration device, characterized in that, Includes a data acquisition unit, memory, and processor, among which, The data acquisition device is communicatively connected to the processor and is used to acquire UAV point cloud data of the current frame and UAV point cloud data of the previous frame. The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the registration method for UAV point cloud data according to any one of claims 1 to 7.

Images