A cross-source point cloud registration method based on visual projection assistance

Abstract

The application belongs to the technical field of computer vision, and particularly relates to a cross-source point cloud registration method based on visual projection assistance, aiming to solve the problem of low registration accuracy and efficiency of the existing cross-source point cloud registration method affected by data heterogeneity, noise interference and redundant data in non-overlapping areas. The method first performs multi-scale downsampling and feature extraction on the source point cloud and the target point cloud through a KPConv-FPN backbone network, and divides the densest point cloud and the coarse-grained super point cloud; then, a projection mapping from three-dimensional point cloud to two-dimensional image is constructed based on camera calibration parameters, and effective candidate super points are screened to eliminate redundant data; then, PPF features are calculated based on the local densest point cloud, and feature aggregation is completed by combining self-attention and cross-domain attention mechanism, and accurate point pairs are obtained through super point and densest point double-layer matching; finally, the LGR estimator is used to solve the rotation and displacement matrix, and accurate point cloud registration is realized. The application can effectively suppress noise and reduce computational complexity, significantly improve the registration accuracy and robustness of cross-source point cloud, and is suitable for real scene applications such as autonomous driving, robot navigation and three-dimensional reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a cross-source point cloud registration method based on visual projection assistance. Background Technology

[0002] Cross-source point cloud registration is a fundamental core task in the field of 3D computer vision. Its core objective is to achieve accurate spatial alignment of 3D point cloud data collected by different types of sensors, providing key technical support for fields such as autonomous driving environment perception, mobile robot autonomous navigation, 3D scene reconstruction, and industrial precision inspection. It is an important link in realizing the fusion of multi-source 3D data information and constructing a complete 3D spatial cognition, and has important research value and practical application significance in many fields such as intelligent equipment research and development and smart city construction.

[0003] Current research on point cloud registration techniques focuses on same-source point cloud registration scenarios, forming two main categories: traditional registration methods represented by Iterative Closest Point (ICP) and its improved algorithms, and feature extraction and matching registration methods based on deep learning models. Although there has been some initial progress in research on cross-source point cloud registration, most of the related solutions are simple improvements to same-source registration methods. Some methods attempt to improve the robustness of cross-source feature matching by extracting local geometric features such as FPFH and PPF, while others use deep learning networks to learn high-dimensional semantic features of cross-source point clouds to achieve alignment. Moreover, most of the existing research and validation of cross-source point cloud registration are based on public benchmark datasets such as 3DCGS and KITTI-CrossSource. However, the aforementioned datasets have significant limitations. 3DCGS only provides small-scale indoor scene point clouds obtained based on Kinect and the Structure of Motion (SfM) method, which is difficult to support the training and optimization of data-driven deep registration models. The point cloud data of KITTI-CrossSource consists of LiDAR scan point clouds and MonoRec sequence reconstructed point clouds. Both its source and target point clouds are generated by image sequence reconstruction and are not directly collected by real physical sensors. This results in significant differences from the point cloud data characteristics of actual application scenarios, limiting the applicability of related registration methods in real cross-source scenarios.

[0004] Compared to point cloud registration from the same source, cross-source point cloud registration faces more significant technical challenges, and existing methods are insufficient to meet the application requirements of real-world scenarios. On the one hand, the acquisition principles of different types of scanning devices are fundamentally different, resulting in significant differences in data density and structural patterns in cross-source point clouds. For example, point clouds acquired by rotating LiDAR exhibit a sparse ring structure, while those acquired by semi-solid-state LiDAR show a fan-shaped distribution. Furthermore, the noise levels, outlier distributions, and missing point cloud regions also vary considerably among different sensors. These heterogeneous characteristics easily lead to numerous feature matching errors, severely restricting registration performance. On the other hand, existing publicly available cross-source point cloud datasets do not fully cover key issues commonly found in real-world environments, such as inherent sensor noise, acquisition outliers, and heterogeneous data structure patterns. Models trained on such datasets have poor generalization capabilities in real-world cross-source scenarios. Furthermore, existing cross-source point cloud registration methods do not specifically address redundant data in non-overlapping point cloud regions, which not only increases the computational load of the model but also introduces invalid interference, further reducing the accuracy and efficiency of registration. Moreover, existing methods cannot effectively enhance the geometric consistency between cross-source point clouds, resulting in a significant decrease in registration accuracy in real cross-source point cloud data processing. There is an urgent need for a cross-source point cloud registration method that can accurately locate overlapping regions of point clouds, effectively suppress noise interference, and enhance cross-source geometric consistency. Summary of the Invention

[0005] To address the problems existing in the background art, one aspect of the present invention provides a cross-source point cloud registration method based on visual projection assistance, comprising:

[0006] S1: Input the source point cloud and the target point cloud into the multi-scale feature extraction module respectively, and perform downsampling and feature extraction in sequence to obtain multi-level source point cloud and target point cloud;

[0007] S2: The source point cloud and target point cloud of the first level are designated as the densest source point cloud and the densest target point cloud, respectively; the source point cloud and target point cloud of the last level are designated as the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, respectively.

[0008] S3: Based on the camera calibration parameters, the projection mask generation module filters out candidate superpoints in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud;

[0009] S4: Based on the candidate superpoints in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, the densest source point cloud and the densest target point cloud are divided by the nearest neighbor algorithm to obtain the local densest point cloud corresponding to each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud.

[0010] S5: Compute the PPF features of each candidate superpoint in the coarse-grained super-source point cloud and coarse-grained super-target point cloud based on the local densest point cloud corresponding to each candidate superpoint.

[0011] S6: Perform similarity matching based on the PPF features of each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud to obtain superpoint matching pairs; perform similarity matching on the densest points in the local densest point clouds corresponding to the two superpoints in the superpoint matching pairs to obtain the densest point matching pairs.

[0012] S7: Calculate the rotation and displacement matrices between the source and target point clouds using the LGR estimator based on the densest point matching pairs, and register the source and target point clouds using the rotation and displacement matrices between them.

[0013] Another aspect of the present invention provides a cross-source point cloud registration system based on visual projection assistance, the system including a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the cross-source point cloud registration method based on visual projection assistance.

[0014] Another aspect of the present invention provides a computer storage medium storing a computer program, which, when executed by a processor, implements the aforementioned cross-source point cloud registration method based on visual projection assistance.

[0015] The present invention has at least the following beneficial effects

[0016] This invention constructs a mapping relationship between 2D images and 3D point clouds through camera calibration parameters. It utilizes a projection mask generation module to accurately filter candidate superpoints in coarse-grained superpoint clouds, effectively eliminating redundant point cloud data in non-overlapping regions. This suppresses noise interference and significantly reduces the computational load of the model. Simultaneously, a multi-scale feature extraction module is employed to achieve hierarchical feature extraction of the point cloud. Based on a block segmentation strategy combined with PPF feature calculation, the geometric feature information of cross-source point clouds is fully explored, enhancing the geometric consistency between them. High-quality point pair matching relationships are obtained through two-layer similarity matching between superpoints and the densest points. Combined with an LGR estimator, the rotation and displacement matrices are accurately calculated. Ultimately, this significantly improves the accuracy and robustness of point cloud registration in real cross-source scenarios, effectively solving the problem of poor registration performance of existing methods when facing density differences, structural heterogeneity, and noise interference in cross-source point clouds. It adapts to the processing needs of cross-source point cloud data collected by real physical sensors, providing more reliable technical support for practical applications such as autonomous driving and robot navigation. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0018] Figure 2 This is a schematic diagram of the multi-scale feature extraction module of the present invention;

[0019] Figure 3 This is a projection visualization of the target point cloud of the present invention;

[0020] Figure 4 This is a visualization of the registration results of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 and Figure 2 One aspect of the present invention provides a cross-source point cloud registration method based on visual projection assistance, comprising:

[0023] S1: Input the source point cloud and the target point cloud into the multi-scale feature extraction module respectively, and perform downsampling and feature extraction in sequence to obtain multi-level source point cloud and target point cloud;

[0024] Preferably, the multi-scale feature extraction module adopts a KPConv-FPN backbone network, which includes M sequentially cascaded feature extraction stages; each feature extraction stage includes a KPConv convolution and a downsampling module; wherein, the KPConv convolution is used to extract features from the input point cloud data; the downsampling module selects representative points from the point cloud data processed by the KPConv convolution through farthest point sampling to obtain the output point cloud data of the feature extraction stage; the M feature extraction stages sequentially output the source point cloud or target point cloud of the first to Mth levels.

[0025] In this embodiment, the source point cloud is source domain data, and the target point cloud is target domain data. This solution provides a cross-domain point cloud registration scheme. The source point cloud and the target point cloud are 3D point cloud data collected by different types of physical sensors. They are cross-source heterogeneous point cloud data, collected by scanning / sensing devices with different principles, such as rotating LiDAR, semi-solid-state LiDAR, and visual depth cameras (Kinect, etc.). The acquisition scenario is adapted to practical application scenarios such as autonomous driving, robot navigation, and 3D scene reconstruction. The two are point cloud data from different frames in the same 3D scene, with spatial overlap. In this embodiment, the source point cloud and target point cloud are input into the multi-scale feature extraction module, and downsampling and feature extraction are performed sequentially to obtain multi-level source point cloud and target point cloud. This step is the basic feature processing link for cross-source point cloud registration. The core relies on the KPConv-FPN backbone network to realize the hierarchical extraction of multi-scale point cloud features. The KPConv-FPN backbone network contains M sequentially cascaded feature extraction stages. Each feature extraction stage consists of KPConv convolutional units and downsampling units. The multi-scale feature transfer and fusion are realized by feature cascading between stages. The original source point cloud and target point cloud are then processed. The points are fed into the network as input, sequentially through the first to Mth feature extraction stages. In each stage, a KPConv convolutional unit is used to extract spatial features from the input point cloud. This unit is designed for the unstructured nature of point clouds, enabling effective aggregation of neighborhood features without voxelization, accurately capturing the local geometric structure and semantic information of the point cloud. Then, a downsampling unit employs a farthest point sampling (FPS) strategy to select representative point cloud data from the KPConv convolution-processed point cloud. This reduces the number of points and subsequent computational load while preserving the point cloud's characteristics to the greatest extent possible. The core spatial features form the output point cloud for this stage. After M feature extraction stages, the network will output source and target point clouds at the 1st to Mth levels in sequence. The point clouds in the preceding stages retain the dense details of the original point clouds due to fewer downsampling times. The point clouds in the subsequent stages, after multiple downsampling, have reduced density and coarser granularity, highlighting the overall global features of the point clouds. This ultimately forms a multi-level point cloud feature system from dense to sparse and from fine to coarse. The multi-scale feature extraction module completes the hierarchical feature extraction and downsampling processing of the source and target point clouds, achieving the dual effect of detail preservation and global control.

[0026] S2: The source point cloud and target point cloud of the first level are designated as the densest source point cloud and the densest target point cloud, respectively; the source point cloud and target point cloud of the last level are designated as the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, respectively.

[0027] In this embodiment, the first-level source point cloud and target point cloud output by the multi-scale feature extraction module are defined as the densest source point cloud and the densest target point cloud, respectively. This level of point cloud is the point cloud data after initial feature extraction and downsampling by the KPConv-FPN backbone network. It undergoes only the fewest downsampling processes, preserving the point cloud density and detailed features of the original source and target point clouds to the greatest extent, and contains rich local geometric information. The last-level source point cloud and target point cloud output by the multi-scale feature extraction module are defined as the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, respectively. This level of point cloud undergoes progressive downsampling and feature aggregation through M feature extraction stages, significantly reducing the point cloud density and the number of points. It retains only the global spatial structure and core semantic features of the original point cloud, forming a coarser-grained super-point set, which serves as the operational vehicle for subsequent rapid filtering of overlapping point cloud regions and reducing invalid computations. By accurately defining the densest point cloud and coarse-grained superpoint cloud in multi-scale hierarchical point clouds, the operation objects of coarse and fine registration in the point cloud registration process are separated. This provides a low-computational operation carrier for quickly screening overlapping candidate regions and eliminating non-overlapping redundant data through coarse-grained superpoint clouds, significantly reducing the computational overhead of subsequent projection screening and feature matching. It also preserves the densest point cloud data with rich detailed features for fine-grained point pair accurate matching. At the same time, it establishes the spatial relationship between coarse-grained global features and fine-grained local features, enabling the subsequent registration process to achieve progressive matching from coarse to fine and from global to local. This lays a dual data foundation for improving registration efficiency and registration accuracy.

[0028] S3: Based on the camera calibration parameters, the projection mask generation module filters out candidate superpoints in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud;

[0029] Preferably, step S3 includes:

[0030] S31: Expand the coordinates of superpoints in the coarse-grained super-source point cloud and coarse-grained super-target point cloud into homogeneous coordinates in the lidar coordinate system;

[0031] S32: Use the camera extrinsic matrix to convert the homogeneous coordinates of superpoints in the coarse-grained supersource point cloud and coarse-grained supertarget point cloud in the lidar coordinate system to the coordinates in the camera coordinate system;

[0032] S33: In the camera coordinate system, filter out superpoints with depth values ​​less than or equal to 0 in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud to obtain the intermediate coarse-grained super-source point cloud and the intermediate coarse-grained super-target point cloud.

[0033] S34: Expand the coordinates of the superpoints in the intermediate coarse-grained supersource point cloud and the intermediate coarse-grained supertarget point cloud in the camera coordinate system to homogeneous coordinates;

[0034] S35: Convert the homogeneous coordinates of the superpoints in the intermediate coarse-grained super-source point cloud and the intermediate coarse-grained super-target point cloud in the camera coordinate system into homogeneous pixel coordinates on the given image plane by using the camera intrinsic parameter matrix, and then divide the homogeneous pixel coordinates by the homogeneous scale factor to obtain the standard pixel coordinates.

[0035] S36: Check whether the standard pixel coordinates of the superpoints in the intermediate coarse-grained super-source point cloud and the intermediate coarse-grained super-target point cloud are within the range of the given image. If so, mark the mask of the corresponding superpoint as 1 and take the superpoint as a candidate superpoint.

[0036] In this embodiment, candidate superpoints in the coarse-grained super-source point cloud and coarse-grained super-target point cloud are selected by the projection mask generation module based on camera calibration parameters. It establishes a spatial mapping relationship between the 3D point cloud and the 2D image based on camera intrinsic and extrinsic calibration parameters. Through multi-step coordinate transformation and mask selection, superpoints within the effective viewpoint of the image in the coarse-grained super-point cloud are accurately identified, and invalid superpoints are eliminated. The camera calibration parameters achieve a precise spatial mapping of the coarse-grained super-point cloud from the LiDAR coordinate system to the camera coordinate system and then to the image pixel coordinate system. Through multiple rounds of coordinate transformation and layer-by-layer selection, not only are candidate superpoints within the effective viewpoint of the image in the coarse-grained super-point cloud accurately identified, but also the overlapping candidate regions of the cross-source point cloud are accurately located. Redundant superpoints in non-overlapping regions are effectively eliminated, reducing the amount of invalid data for subsequent feature calculation and matching from the source, significantly reducing the overall computational cost of the model. Furthermore, by filtering superpoints without effective depth values, the interference of noisy superpoints on the subsequent registration process is suppressed, providing high-quality operation objects for subsequent superpoint feature calculation and similarity matching, thus improving the accuracy and robustness of the subsequent registration process.

[0037] Please see Figure 3 , Figure 3 This is a projection visualization of the target point cloud. The image intuitively presents the projection mapping effect from 3D point cloud to 2D image achieved by the projection mask generation module based on camera calibration parameters in this solution. It clearly shows the pixel distribution state of the coarse-grained super-target point cloud in the image plane after transformation from the LiDAR coordinate system, camera coordinate system to the image pixel coordinate system. At the same time, it can clearly distinguish the effective projection area marked as candidate super points by the mask and the invalid projection area that has been removed. It intuitively demonstrates the accurate positioning effect of the visual projection filtering strategy on the candidate area of ​​overlapping cross-source point clouds, and also verifies that the projection mask generation module can effectively filter out invalid point cloud data outside the camera view.

[0038] S4: Based on the candidate superpoints in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, the densest source point cloud and the densest target point cloud are divided by the nearest neighbor algorithm to obtain the local densest point cloud corresponding to each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud.

[0039] Preferably, the local densest point cloud corresponding to each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud includes:

[0040]

[0041]

[0042] in, Indicating the first coarse-grained supersource point cloud The local densest point cloud corresponding to each candidate superpoint; Indicates the most dense source point cloud. The most densely packed point; This indicates the first selection in the coarse-grained supersource point cloud. One candidate superpoint; This indicates the set distance parameter; Indicates the first coarse-grained super-target point cloud The local densest point cloud corresponding to each candidate superpoint; Indicates the number of densest target point clouds. The most densely packed point; This indicates the first selection in the coarse-grained super-target point cloud. One candidate superpoint; This represents the L2 norm.

[0043] In this embodiment, based on the selection of candidate superpoints, a spatial association is established between the coarse-grained superpoint cloud and the densest point cloud. The nearest neighbor algorithm is used to divide the region of the densest point cloud, defining a precise local point cloud range for subsequent PPF feature calculation of candidate superpoints. Each local point cloud range can be considered a local block. The nearest neighbor algorithm is selected as the core algorithm for region division. Using all candidate superpoints in the coarse-grained supersource point cloud and coarse-grained supertarget point cloud as spatial reference points, nearest neighbor retrieval calculation is performed on each point cloud data point in the densest source point cloud and the densest target point cloud. For any densest point in the densest source point cloud... Calculate its relationship with all candidate superpoints in the coarse-grained supersource point cloud. Two-dimensional Euclidean distance This will satisfy the condition that the distance is less than the set distance parameter. The densest point is assigned to the local point cloud set corresponding to the candidate superpoint, thus obtaining the unique local densest source point cloud corresponding to each candidate superpoint in the coarse-grained supersource point cloud. This method achieves the regional division of the densest source point cloud, and similarly completes the regional division of the densest target point cloud. After completing the regional division, the basic geometric features such as spatial coordinates and normal vectors of each local densest point cloud are retained, and a one-to-one mapping relationship between the local densest point cloud and the corresponding candidate superpoint is established. Through the nearest neighbor algorithm, the most dense point cloud is accurately divided into regions based on coarse-grained candidate superpoints. This not only establishes the spatial mapping relationship between coarse-grained superpoints and fine-grained densest points, allowing the feature calculation of candidate superpoints to be carried out based on their surrounding local densest point clouds, ensuring the locality and specificity of feature calculation, but also decomposes the global densest point cloud into multiple local point cloud sets, transforming the subsequent global feature calculation into local feature calculation, which greatly reduces the computational complexity of PPF features and improves the efficiency and accuracy of feature calculation.

[0044] S5: Compute the PPF features of each candidate superpoint in the coarse-grained super-source point cloud and coarse-grained super-target point cloud based on the local densest point cloud corresponding to each candidate superpoint.

[0045] Preferably, the PPF features of each candidate superpoint in the coarse-grained supersource point cloud and the coarse-grained supertarget point cloud include:

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064] in, express The geometric center; This represents the local densest point cloud corresponding to each candidate superpoint in the coarse-grained supersource point cloud or coarse-grained supertarget point cloud; Represents the source domain; Indicates the target domain; This represents the index of candidate superpoints in a coarse-grained supersource point cloud; This represents the index of candidate superpoints in a coarse-grained supertarget point cloud; express The number of the most dense points; express The Middle The most densely packed point; express The average normal vector; express The normal vector; express and The included angle; Represents the inverse cosine function; express The model; express The model; express The model; express and The included angle; express and The included angle; express and The distance between them; Represents the L2 norm; express PPF characteristics; Indicates average pooling; Represents a multilayer perceptron; This represents the global features of candidate superpoints in a coarse-grained supersource point cloud or a coarse-grained supertarget point cloud. This represents the global feature of all candidate superpoints in a coarse-grained supersource point cloud. This represents the global feature of all candidate hyperpoints in the coarse-grained hypertarget point cloud; and This indicates the number of candidate superpoints in the coarse-grained supersource point cloud and the coarse-grained supertarget point cloud; , , , , and This represents a trainable weight matrix; This represents the features of all candidate superpoints in a coarse-grained supersource point cloud. , and This represents the query, key, and value vectors corresponding to the coarse-grained super-source point cloud; , and This represents the query, key, and value vectors corresponding to the coarse-grained supertarget point cloud; Indicates the activation function; Indicates the scaling factor; Indicates transpose; This represents the preliminary aggregated features of all candidate superpoints in a coarse-grained supersource point cloud. This represents the initial aggregated features of all candidate superpoints in the coarse-grained supertarget point cloud; This represents the features of all candidate hyperpoints in a coarse-grained hypertarget point cloud. , , , and Represents trainable weight parameters; , and This represents the relevant queries, key, and value vectors corresponding to the source domain point cloud; , and This represents the relevant queries, key, and value vectors corresponding to the target domain point cloud; This represents the PPF feature of all candidate superpoints in a coarse-grained supersource point cloud; This represents the PPF feature of all candidate superpoints in the coarse-grained supertarget point cloud.

[0065] In this embodiment, based on the local densest point cloud partitioning, the PPF features of each candidate superpoint in the coarse-grained supersource point cloud and coarse-grained supertarget point cloud are calculated. This step uses the local densest point cloud corresponding to each candidate superpoint as the computational unit. Through multi-stage operations such as geometric feature statistics, single-point pair feature extraction, feature encoding, and cross-domain aggregation, the geometric information of the local densest point cloud is transformed into global PPF features of candidate superpoints with strong discriminativeness and high robustness. This is achieved through a hierarchical computation strategy from basic geometric features to single-point pair features, and then to the aggregation of local and global features, combined with self-attention and cross-domain attention mechanisms. The feature fusion method not only fully explores the geometric consistency features of local regions in cross-source point clouds, effectively weakening the feature interference caused by the heterogeneity of cross-source point clouds in terms of density and structure, but also improves the discriminativeness and robustness of PPF features through nonlinear coding and attention aggregation. This allows the generated candidate superpoint PPF features to contain both local fine geometric information and global and cross-domain feature correlations, while avoiding the influence of point cloud order and local noise on features. This provides a highly discriminative and adaptable feature foundation for subsequent accurate similarity matching between superpoints and the densest points, significantly improving the accuracy of cross-source point cloud feature matching.

[0066] S6: Perform similarity matching based on the PPF features of each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud to obtain superpoint matching pairs; perform similarity matching on the densest points in the local densest point clouds corresponding to the two superpoints in the superpoint matching pairs to obtain the densest point matching pairs.

[0067] S7: Calculate the rotation and displacement matrices between the source and target point clouds using the LGR estimator based on the densest point matching pairs, and register the source and target point clouds using the rotation and displacement matrices between them.

[0068] In this embodiment, a two-layer similarity matching strategy of coarse-to-fine is adopted. First, coarse-grained superpoint matching is completed based on the highly robust PPF features. Then, fine-grained point pair matching is completed by focusing on the local densest point cloud corresponding to the matching superpoint. This achieves a progressive screening from coarse-grained global matching to fine-grained local matching, effectively filtering out erroneous matching pairs caused by cross-source point cloud heterogeneity and significantly improving the accuracy and effectiveness of point pair matching. The LGR estimator is used to solve high-quality densest point matching pairs, accurately obtaining the rotation and displacement matrices between the source and target point clouds. The spatial alignment of cross-source point clouds is completed through this transformation matrix. Compared with traditional solution methods, the LGR estimator can better suppress the interference of a small amount of noise in the matching pair, ensuring the accuracy of the registration transformation matrix. Finally, high-precision registration of cross-source point clouds is achieved, effectively solving the problem of low registration accuracy of existing methods under cross-source point cloud density differences, structural heterogeneity, and noise interference, and significantly improving the overall robustness and spatial alignment accuracy of cross-source point cloud registration.

[0069] Please see Figure 4 , Figure 4 This is a visualization of the cross-source point cloud registration results based on the present invention. The diagram displays the target point cloud, source point cloud, actual registration effect, point pair correspondence predicted by the present solution, and the predicted registration effect of the present solution in different regions, forming a multi-dimensional comparison of registration effects. It can be clearly observed that the most dense point pair correspondence matched by the present solution is highly consistent with the point cloud correspondence of the real scene. Moreover, the final predicted registration effect achieves accurate spatial alignment between the source point cloud and the target point cloud, effectively solving the registration deviation problem caused by density differences, structural heterogeneity, and noise interference in cross-source point clouds. It intuitively verifies the technical advantages of the present solution in cross-source point cloud registration tasks, namely high precision and strong robustness.

[0070] Another aspect of the present invention provides a cross-source point cloud registration system based on visual projection assistance, the system including a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the cross-source point cloud registration method based on visual projection assistance.

[0071] Another aspect of the present invention provides a computer storage medium storing a computer program, which, when executed by a processor, implements the aforementioned cross-source point cloud registration method based on visual projection assistance.

[0072] In summary, this invention constructs a mapping relationship between 2D images and 3D point clouds through camera calibration parameters. It utilizes a projection mask generation module to accurately filter candidate superpoints in coarse-grained superpoint clouds, effectively eliminating redundant point cloud data in non-overlapping regions. This suppresses noise interference and significantly reduces the computational load of the model. Simultaneously, a multi-scale feature extraction module is employed to achieve hierarchical feature extraction of the point cloud. Based on a block segmentation strategy combined with PPF feature calculation, the geometric feature information of cross-source point clouds is fully explored, enhancing the geometric consistency between them. Furthermore, high-quality point pair matching relationships are obtained through two-layer similarity matching between superpoints and the densest points. Combined with an LGR estimator, the rotation and displacement matrices are accurately calculated. Ultimately, this significantly improves the accuracy and robustness of point cloud registration in real cross-source scenarios, effectively solving the problem of poor registration performance of existing methods when facing density differences, structural heterogeneity, and noise interference in cross-source point clouds. It adapts to the processing needs of cross-source point cloud data collected by real physical sensors, providing more reliable technical support for practical applications such as autonomous driving and robot navigation.

[0073] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A cross-source point cloud registration method based on visual projection assistance, characterized in that, include: S1: Input the source point cloud and the target point cloud into the multi-scale feature extraction module respectively, and perform downsampling and feature extraction in sequence to obtain multi-level source point cloud and target point cloud; S2: The source point cloud and target point cloud of the first level are designated as the densest source point cloud and the densest target point cloud, respectively; the source point cloud and target point cloud of the last level are designated as the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, respectively. S3: Based on the camera calibration parameters, the projection mask generation module filters out candidate superpoints in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud; S4: Based on the candidate superpoints in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud, the densest source point cloud and the densest target point cloud are divided by the nearest neighbor algorithm to obtain the local densest point cloud corresponding to each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud. S5: Compute the PPF features of each candidate superpoint in the coarse-grained super-source point cloud and coarse-grained super-target point cloud based on the local densest point cloud corresponding to each candidate superpoint. S6: Perform similarity matching based on the PPF features of each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud to obtain superpoint matching pairs; perform similarity matching on the densest points in the local densest point clouds corresponding to the two superpoints in the superpoint matching pairs to obtain the densest point matching pairs. S7: Calculate the rotation and displacement matrices between the source and target point clouds using the LGR estimator based on the densest point matching pairs, and register the source and target point clouds using the rotation and displacement matrices between them.

2. The cross-source point cloud registration method based on visual projection assistance according to claim 1, characterized in that, The multi-scale feature extraction module adopts the KPConv-FPN backbone network, which includes M sequentially cascaded feature extraction stages. Each feature extraction stage includes a KPConv convolution and a downsampling module; wherein, the KPConv convolution is used to extract features from the input point cloud data; the downsampling module selects representative points from the point cloud data processed by the KPConv convolution through farthest point sampling to obtain the output point cloud data of the feature extraction stage; the M feature extraction stages sequentially output the source point cloud or target point cloud of the first to Mth levels.

3. The cross-source point cloud registration method based on visual projection assistance according to claim 1, characterized in that, Step S3 includes: S31: Expand the coordinates of superpoints in the coarse-grained super-source point cloud and coarse-grained super-target point cloud in the lidar coordinate system to homogeneous coordinates; S32: Use the camera extrinsic matrix to convert the homogeneous coordinates of superpoints in the coarse-grained supersource point cloud and coarse-grained supertarget point cloud in the lidar coordinate system to coordinates in the camera coordinate system; S33: In the camera coordinate system, filter out superpoints with depth values ​​less than or equal to 0 in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud to obtain the intermediate coarse-grained super-source point cloud and the intermediate coarse-grained super-target point cloud. S34: Extend the coordinates of the superpoints in the intermediate coarse-grained supersource point cloud and the intermediate coarse-grained supertarget point cloud in the camera coordinate system to homogeneous coordinates; S35: Convert the homogeneous coordinates of the superpoints in the intermediate coarse-grained super-source point cloud and the intermediate coarse-grained super-target point cloud in the camera coordinate system into homogeneous pixel coordinates on the given image plane by using the camera intrinsic parameter matrix, and then divide the homogeneous pixel coordinates by the homogeneous scale factor to obtain the standard pixel coordinates. S36: Check whether the standard pixel coordinates of the superpoints in the intermediate coarse-grained super-source point cloud and the intermediate coarse-grained super-target point cloud are within the range of the given image. If so, mark the mask of the corresponding superpoint as 1 and take the superpoint as a candidate superpoint.

4. The cross-source point cloud registration method based on visual projection assistance according to claim 1, characterized in that, The local densest point cloud corresponding to each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud includes: in, Indicating the first coarse-grained supersource point cloud The local densest point cloud corresponding to each candidate superpoint; Indicates the most dense source point cloud The most densely packed point; This indicates the first selection in the coarse-grained supersource point cloud. One candidate superpoint; This indicates the set distance parameter; Indicates the first coarse-grained super-target point cloud The local densest point cloud corresponding to each candidate superpoint; Indicates the number of the densest target point cloud The most densely packed point; This indicates the first selection in the coarse-grained super-target point cloud. One candidate superpoint; This represents the L2 norm.

5. The cross-source point cloud registration method based on visual projection assistance according to claim 1, characterized in that, The PPF features of each candidate superpoint in the coarse-grained super-source point cloud and the coarse-grained super-target point cloud include: in, express The geometric center; This represents the local densest point cloud corresponding to each candidate superpoint in the coarse-grained supersource point cloud or coarse-grained supertarget point cloud; Represents the source domain; Indicates the target domain; This represents the index of candidate superpoints in a coarse-grained supersource point cloud; This represents the index of candidate superpoints in a coarse-grained supertarget point cloud; express The number of the most dense points; express The Middle The most densely packed point; express The average normal vector; express The normal vector; express and The included angle; Represents the inverse cosine function; express The model; express The model; express The model; express and The included angle; express and The included angle; express and The distance between them; Represents the L2 norm; express PPF characteristics; Indicates average pooling; Represents a multilayer perceptron; This represents the global features of candidate superpoints in a coarse-grained supersource point cloud or a coarse-grained supertarget point cloud. This represents the global feature of all candidate superpoints in a coarse-grained supersource point cloud. This represents the global feature of all candidate hyperpoints in the coarse-grained hypertarget point cloud; and This indicates the number of candidate superpoints in the coarse-grained supersource point cloud and the coarse-grained supertarget point cloud; , , , , and This represents a trainable weight matrix; This represents the features of all candidate superpoints in a coarse-grained supersource point cloud. , and This represents the query, key, and value vectors corresponding to the coarse-grained super-source point cloud; Indicates the activation function; Indicates the scaling factor; Indicates transpose; This represents the preliminary aggregated features of all candidate superpoints in a coarse-grained supersource point cloud. This represents the initial aggregated features of all candidate superpoints in the coarse-grained supertarget point cloud; This represents the features of all candidate hyperpoints in a coarse-grained hypertarget point cloud. , , , and Represents trainable weight parameters; , and This represents the relevant queries, key, and value vectors corresponding to the source domain point cloud; , and This represents the relevant queries, key, and value vectors corresponding to the target domain point cloud; This represents the PPF feature of all candidate superpoints in a coarse-grained supersource point cloud; This represents the PPF feature of all candidate superpoints in the coarse-grained supertarget point cloud.

6. A cross-source point cloud registration system based on visual projection assistance, characterized in that, The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute a cross-source point cloud registration method based on visual projection assistance as described in any one of claims 1 to 5.

7. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements a cross-source point cloud registration method based on visual projection assistance as described in any one of claims 1 to 5.

Images