Color point cloud registration method and system based on feature fusion

Abstract

The invention discloses a color point cloud registration method and system based on feature fusion, and the method comprises the steps: obtaining a source point cloud and a target point cloud; extracting first geometric key points and first texture key points of the source point cloud, and fusing the first geometric key points and the first texture key points to obtain a first key point set; a second geometric key point and a second texture key point of the target point cloud are fused to obtain a second key point set; respectively extracting a first feature descriptor of each point in the first key point set and a second feature descriptor of each point in the second key point set; according to the first feature descriptor and the second feature descriptor, performing coarse registration by adopting a random sampling consistency algorithm to obtain an initial rigid transformation matrix; constructing a loss function according to geometric distances and color distances of corresponding points between the source point cloud and the target point cloud, and performing iterative optimization on the initial rigid transformation matrix to obtain a final rigid transformation matrix; and performing registration splicing on the source point cloud and the target point cloud according to the final rigid transformation matrix. And the registration precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a color point cloud registration method and system based on feature fusion. Background Art

[0002] 3D point cloud is an effective form for perceiving 3D scene information. It is a set of discrete points distributed in 3D space collected by structured light cameras or 3D laser scanners. Point cloud data stores rich 3D spatial structure information, which greatly makes up for the lack of spatial structure information in 2D images. It has a unique advantage in expressing the appearance of complex scenes and objects, so it is widely used in unmanned driving, measurement, remote sensing, virtual reality and other fields.

[0003] Point cloud registration is a key step in obtaining complete information about the surface of an object, and plays an important role in 3D reconstruction, 3D measurement, and posture estimation. Due to the influence of the angle of the scanning device, the shape of the scanned object, and environmental factors, the physical scanning and collection of point cloud data cannot be completed in one go. In order to obtain complete 3D information about the surface of an object, it is necessary to collect data in blocks from multiple angles of the object, and then splice the 3D point clouds collected from different angles to obtain complete 3D information contained in the same coordinate system. At present, most point cloud registration algorithms first extract feature information such as normals and curvatures of the points in the point cloud, and then find the corresponding points between the point clouds to be registered based on this information, and finally calculate the transformation matrix based on these corresponding points. These methods may be unstable in the presence of noise interference or insufficient geometric features, resulting in inaccurate alignment.

[0004] There is currently no effective solution to the problem of inaccurate registration and splicing in the prior art. Summary of the invention

[0005] To solve the above problems, the present invention provides a color point cloud registration method and system based on feature fusion, which increases the calculation accuracy by combining color texture features on the basis of extracting point cloud geometric features to determine the transformation matrix, so as to solve the problem of inaccurate point cloud registration splicing in the prior art.

[0006] To achieve the above object, the present invention provides a color point cloud registration method based on feature fusion, including: acquiring a source point cloud and a target point cloud; extracting first geometric key points of the source point cloud and second geometric key points of the target point cloud according to the ISS algorithm; extracting first texture key points of the source point cloud and second texture key points of the target point cloud according to color features; fusing the first geometric key points and the first texture key points according to a preset distance range to obtain a first key point set; fusing the second geometric key points and the second texture key points according to a preset distance range to obtain a second key point set; respectively extracting a first feature descriptor of each point in the first key point set and a second feature descriptor of each point in the second key point set; according to the first feature descriptor and the second feature descriptor, using the random sample consensus algorithm to perform rough registration on the first key point set and the second key point set to obtain an initial rigid transformation matrix; constructing a loss function according to the geometric distance and color distance of corresponding points between the source point cloud and the target point cloud, and iteratively optimizing the initial rigid transformation matrix to obtain a final rigid transformation matrix; registering and splicing the source point cloud and the target point cloud according to the final rigid transformation matrix to obtain a spliced point cloud.

[0007] Further optionally, the extracting the first texture key points of the source point cloud according to color features includes: converting the source point cloud from the RGB color space to the Lab color space; in the Lab color space, searching for neighborhood points within a predetermined distance for each point in the source point cloud; calculating a color covariance matrix according to the three-dimensional Lab color vectors of each point and its neighborhood points; performing eigenvalue decomposition on the color covariance matrix to obtain color eigenvalues, and calculating a generalized variance value of the color covariance matrix according to the color eigenvalues; screening the first texture key points according to the generalized variance value.

[0008] Further optionally, the extracting the first feature descriptor of each point in the first key point set includes: for each first key point in the first key point set, calculating its corresponding FPFH feature descriptor and color feature descriptor in the Lab space; splicing the FPFH feature descriptor and the color feature descriptor to obtain a first feature descriptor corresponding to each first key point.

[0009] Further optionally, performing rough registration on the first key point set and the second key point set by using the random sample consensus algorithm according to the first feature descriptor and the second feature descriptor to obtain an initial rigid transformation matrix includes: randomly selecting at least three non - collinear target first key points from the first key point set, and searching for the target second key points closest to each target first key point in the second key point set according to the first feature descriptor and the second feature descriptor; calculating a candidate rigid transformation matrix by using the singular value decomposition method for the target first key points and the target second key points; performing coordinate transformation on the first key points in the first key point set according to the candidate rigid transformation matrix to obtain a transformed key point set, calculating the spatial distance between each point in the transformed key point set and the matching points in the second key point set, and taking the points with spatial distances lower than a preset distance threshold as inliers; repeating the above steps until a preset number of iterations is reached, and taking the candidate rigid transformation matrix with the largest number of inliers as the initial rigid transformation matrix.

[0010] Further optionally, constructing a loss function according to the geometric distance and color distance between corresponding points in the source point cloud and the target point cloud, and performing iterative optimization on the initial rigid transformation matrix to obtain a final rigid transformation matrix includes: performing coordinate transformation on each source point in the source point cloud according to the initial rigid transformation matrix to obtain a transformed point cloud; for each point in the transformed point cloud, searching for its corresponding target point in the target point cloud to construct a point pair set; calculating a geometric error term according to the spatial distance of each point pair in the point pair set; calculating a color error term according to the estimated color value of the source point on the tangent plane where the corresponding target point is located and the original color value; constructing a loss function by weighting the geometric error term and the color error term, and using the Gauss - Newton optimization algorithm to perform iteration with the initial rigid transformation matrix as the initial value until the loss function converges to obtain the final rigid transformation matrix.

[0011] On the other hand, the present invention also provides a color point cloud registration system based on feature fusion, including: a data acquisition module for acquiring a source point cloud and a target point cloud; a key point extraction module for extracting the first geometric key points of the source point cloud and the second geometric key points of the target point cloud according to the ISS algorithm; extracting the first texture key points of the source point cloud and the second texture key points of the target point cloud according to color features; a fusion module for fusing the first geometric key points with the first texture key points according to a preset distance range to obtain a first key point set; fusing the second geometric key points with the second texture key points according to a preset distance range to obtain a second key point set; a feature extraction module for respectively extracting the first feature descriptors of each point in the first key point set and the second feature descriptors of each point in the second key point set; a rough registration module for performing rough registration on the first key point set and the second key point set by using the random sample consensus algorithm according to the first feature descriptor and the second feature descriptor to obtain an initial rigid transformation matrix; a matrix optimization module for constructing a loss function according to the geometric distance and color distance of corresponding points between the source point cloud and the target point cloud, and iteratively optimizing the initial rigid transformation matrix to obtain a final rigid transformation matrix; a registration and splicing module for registering and splicing the source point cloud and the target point cloud according to the final rigid transformation matrix to obtain a spliced point cloud.

[0012] Further optionally, the fusion module includes: a color space conversion sub-module for converting the source point cloud from the RGB color space to the Lab color space; a neighborhood search sub-module for searching for neighborhood points within a predetermined distance for each point in the source point cloud in the Lab color space; a matrix calculation sub-module for calculating a color covariance matrix according to the three-dimensional Lab color vectors of each point and its neighborhood points; a variance value calculation sub-module for performing eigenvalue decomposition on the color covariance matrix to obtain color eigenvalues, and calculating the generalized variance value of the color covariance matrix according to the color eigenvalues; a screening sub-module for screening to obtain the first texture key points according to the generalized variance value.

[0013] Further optionally, the feature extraction module includes: a feature descriptor calculation sub-module for calculating the corresponding FPFH feature descriptor and the color feature descriptor in the Lab space for each first key point in the first key point set; a feature descriptor fusion sub-module for splicing the FPFH feature descriptor and the color feature descriptor to obtain the first feature descriptor corresponding to each first key point.

[0014] Further optionally, the rough registration module includes: a point selection sub-module, configured to randomly select at least three non-collinear target first key points from the first key point set, and find the target second key point closest to each target first key point in the second key point set according to the first feature descriptor and the second feature descriptor; a solution sub-module, configured to calculate a candidate rigid transformation matrix by using the singular value decomposition method for the target first key point and the target second key point; a statistics sub-module, configured to perform coordinate transformation on the first key points in the first key point set according to the candidate rigid transformation matrix to obtain a transformed key point set, calculate the spatial distance between each point in the transformed key point set and the matching points in the second key point set, and use the points with the spatial distance lower than the preset distance threshold as inliers; an iteration sub-module, configured to repeat the steps of the point selection sub-module, the solution sub-module, and the statistics sub-module until the preset iteration number is reached, and use the candidate rigid transformation matrix with the largest number of inliers as the initial rigid transformation matrix.

[0015] Further optionally, the matrix optimization module includes: a coordinate transformation sub-module, configured to perform coordinate transformation on each source point in the source point cloud according to the initial rigid transformation matrix to obtain a transformed point cloud; a point pair construction sub-module, configured to find the corresponding target point of each point in the transformed point cloud in the target point cloud and construct a point pair set; a geometric error term construction sub-module, configured to calculate a geometric error term according to the spatial distance of each point pair in the point pair set; a color error term construction sub-module, configured to calculate a color error term according to the estimated color value and the original color value of the source point on the tangent plane where the corresponding target point is located; a final rigid transformation matrix determination sub-module, configured to weight and construct a loss function for the geometric error term and the color error term, and use the Gauss-Newton optimization algorithm to perform iteration with the initial rigid transformation matrix as the initial value until the loss function converges to obtain the final rigid transformation matrix.

[0016] The above technical solution has the following beneficial effects: by combining the distance model with the Lab color model to extract the key points with significant changes in the surface geometry and texture features of the point cloud, the representativeness of the key points is improved; by fusing the geometric feature descriptor and the color feature descriptor of the point cloud to describe the features of the key points, the color information is taken into account while extracting the geometric features of the point cloud, thereby improving the registration accuracy; by using the random sample consensus algorithm and the colored iterative closest point algorithm to estimate the transformation matrix, it is beneficial to improve the registration accuracy of the colored point cloud model with insignificant geometric features. Description of the Drawings

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 is the flowchart of the color point cloud registration method based on feature fusion provided by the embodiment of the present invention;

[0019] Figure 2 is the flowchart of the texture key point extraction method provided by the embodiment of the present invention;

[0020] Figure 3 is the flowchart of the feature descriptor extraction method provided by the embodiment of the present invention;

[0021] Figure 4 is the flowchart of the FPFH algorithm neighbor point search method provided by the embodiment of the present invention;

[0022] Figure 5 is the schematic diagram of the local reference coordinate system between point pairs provided by the embodiment of the present invention;

[0023] Figure 6 is the schematic diagram of feature histogram splicing provided by the embodiment of the present invention;

[0024] Figure 7 is the flowchart of the rough registration method provided by the embodiment of the present invention;

[0025] Figure 8 is the flowchart of the final rigid transformation matrix calculation method provided by the embodiment of the present invention;

[0026] Figure 9 is the structural schematic diagram of the color point cloud registration system based on feature fusion provided by the embodiment of the present invention;

[0027] Figure 10 is the structural schematic diagram of the key point extraction module provided by the embodiment of the present invention;

[0028] Figure 11 is the structural schematic diagram of the feature extraction module provided by the embodiment of the present invention;

[0029] Figure 12 is the structural schematic diagram of the rough registration module provided by the embodiment of the present invention;

[0030] Figure 13 is the structural schematic diagram of the matrix optimization module provided by the embodiment of the present invention.

[0031] Reference numerals: 100 - data acquisition module; 200 - key point extraction module; 2001 - color space conversion sub - module; 2002 - neighborhood search sub - module; 2003 - matrix calculation sub - module; 2004 - variance value calculation sub - module; 2005 - screening sub - module; 300 - fusion module; 400 - feature extraction module; 4001 - feature descriptor calculation sub - module; 4002 - feature descriptor fusion sub - module; 500 - rough registration module; 5001 - point selection sub - module; 5002 - solution sub - module; 5003 - statistics sub - module; 5004 - iteration sub - module; 600 - matrix optimization module; 6001 - coordinate transformation sub - module; 6002 - point pair construction sub - module; 6003 - geometric error term construction sub - module; 6004 - color error term construction sub - module; 6005 - final rigid transformation matrix determination sub - module; 700 - registration and stitching module. Detailed implementation manners

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] To solve the problem of inaccurate point cloud registration in the prior art, an embodiment of the present invention provides a method for registering colored point clouds based on feature fusion. Figure 1 is a flowchart of the method for registering colored point clouds based on feature fusion provided by an embodiment of the present invention. As Figure 1 shown, the method includes:

[0034] S1. Obtain the source point cloud and the target point cloud;

[0035] When scanning the target area, a lidar is used to collect point cloud data at different angles. These point clouds are all point clouds to be registered and need to be registered into the same coordinate system to obtain the complete three - dimensional information of the target area.

[0036] The target point cloud is the reference point cloud, that is, its coordinate system is the reference coordinate system, and other point clouds to be registered need to perform coordinate transformation based on this coordinate system.

[0037] The source point cloud is the point cloud to be registered, that is, each point of it needs to be transformed to the coordinate system of the target point cloud through a transformation matrix. The source point cloud can be one or more.

[0038] In the specific implementation process, a 3D scanning device with color image acquisition capabilities can be used, such as RGB-D cameras (e.g., Kinect, RealSense), structured light scanners, or lidars equipped with color cameras and other sensors to collect color point cloud data of the same object or scene from multiple perspectives respectively. Each point cloud data contains the geometric coordinate information (X, Y, Z) of the points in the three-dimensional space and the corresponding color information (R, G, B).

[0039] To improve the efficiency of point cloud processing and the registration accuracy, optionally, operations such as noise reduction, filtering, resampling, or normal vector estimation are performed on the source point cloud and the target point cloud to provide a basis for subsequent geometric and texture feature extraction.

[0040] S2. Extract the first geometric key points of the source point cloud and the second geometric key points of the target point cloud according to the ISS algorithm; extract the first texture key points of the source point cloud and the second texture key points of the target point cloud according to the color features;

[0041] Use the ISS (Intrinsic Shape Signature) algorithm to extract the first geometric key points and the second geometric key points from the source point cloud and the target point cloud respectively. This algorithm mainly identifies points with significant shapes based on the spatial structure changes in the local neighborhoods of the point cloud, has rotational invariance and robustness, and is suitable for processing complex three-dimensional surfaces.

[0042] The logic of the ISS algorithm is as follows:

[0043] For each point in the point cloud P , set a search radius , and search for its neighboring points within the spherical neighborhood centered at the point with the radius , and calculate an Euclidean distance weight value for each neighboring point:

[0044]

[0045] In the above formula is any neighboring point within the region centered at with the radius . Then calculate the covariance matrix of each point :

[0046]

[0047] Since the calculated covariance matrix is a -dimensional real symmetric matrix, the eigenvalue decomposition is performed on each covariance matrix to obtain three eigenvalues, which are sorted from largest to smallest as , , , and the ratio between the eigenvalues is less than the set threshold and The points are taken as candidate key points, as shown in the following formula:

[0048]

[0049] in and is a threshold set in advance, and , For the candidate key points obtained, according to their minimum eigenvalue Perform non-maximum suppression, that is, retain candidate key points with large changes along the minimum main direction as geometric feature key points in a certain proportion. In fact, only the points with the most significant features in the local area are retained within a certain range, and a set of key points with obvious geometric features is obtained, namely the first geometric key point and the second geometric key point (both are point sets).

[0050] In addition, the color covariance matrix in the local area of the point cloud is calculated in the Lab color space, and the generalized variance of the obtained color covariance matrix is used to measure the local texture saliency. The larger points scattered in the color space are taken as the texture feature key points to be extracted, that is, the first texture key points of the source point cloud and the second texture key points of the target point cloud (both are point sets).

[0051] The extraction of the above geometric key points and the extraction of texture key points, the feature extraction of the source point cloud and the feature extraction of the target point cloud can all be obtained through multi-threaded parallel computing.

[0052] S3, according to the preset distance range, the first geometric key point is merged with the first texture key point to obtain a first key point set; according to the preset distance range, the second geometric key point is merged with the second texture key point to obtain a second key point set;

[0053] Based on the spatial proximity relationship, texture key points with a predetermined spatial distance are added to the geometric key points, so that a key point set with joint perception of geometry and color is constructed by fusing the information of the two types of key points, further improving the key point coverage and description ability of point cloud registration. a ,r b ] are added to the key point set, and the range can be set according to the actual situation; or the geometric key points are used as the benchmark, the texture key points within the predetermined range are eliminated, and the geometric key points and the filtered texture key points are added to the key point set.

[0054] The source point cloud P and the target point cloud Q are fused with geometric and color key points to obtain the first key point set. and the second keypoint set 。

[0055] S4. Extract the first feature descriptors of each point in the first key point set and the second feature descriptors of each point in the second key point set respectively;

[0056] Feature extraction is performed on each point in the first key point set and each point in the second key point set to obtain the corresponding first feature descriptors and second feature descriptors respectively. For each key point, a composite feature vector is extracted by analyzing its geometric relationship and color characteristics in the local space of the point cloud, forming a high-dimensional representation for subsequent accurate matching between key points.

[0057] S5. According to the first feature descriptors and the second feature descriptors, use the random sample consensus algorithm to perform rough registration on the first key point set and the second key point set to obtain an initial rigid transformation matrix;

[0058] Use the random sample consensus (RANSAC) algorithm combined with feature descriptors for matching, eliminate mismatched point pairs therefrom, and estimate the initial spatial transformation relationship between the source point cloud and the target point cloud, that is, the initial rigid transformation matrix.

[0059] The initial rigid transformation matrix obtained in this step can achieve a rough level of point cloud alignment, serve as the initial solution for fine registration, provide a reasonable convergence starting point for subsequent optimization iterations, and avoid falling into local optimal solutions or divergence problems.

[0060] S6. Construct a loss function based on the geometric distance and color distance of corresponding points between the source point cloud and the target point cloud, and perform iterative optimization on the initial rigid transformation matrix to obtain a final rigid transformation matrix;

[0061] Use the spatial position error (geometric distance) and color value error (color distance) between point pairs after registration by the initial rigid transformation matrix to jointly construct a loss function, and use the initial rigid transformation matrix obtained by rough registration as the initial value for optimization to iteratively solve the final registration matrix.

[0062] The geometric distance can be used to measure the closeness between the transformed source point cloud and the surface of the target point cloud based on the "point-to-plane distance", while the color distance can use the color gradient of points in the Lab space to evaluate the difference between the projected color of the transformed point on the tangent plane of the target point and the original color. By introducing a weight factor, the two types of errors are reasonably weighted to form an optimization objective that balances accuracy and robustness.

[0063] Iteratively update the rigid transformation matrix until the loss function converges to obtain the final optimal rigid transformation result, that is, the final rigid transformation matrix, to achieve higher-precision point cloud registration.

[0064] S7. Register and splice the source point cloud and the target point cloud according to the final rigid transformation matrix to obtain a spliced point cloud.

[0065] Apply the finally obtained rigid transformation matrix to the source point cloud, transform its coordinates to the unified coordinate system where the target point cloud is located, and merge the two point clouds into an integrated point cloud model to form complete spliced point cloud data. In subsequent use, this final rigid transformation matrix can be used for subsequent point cloud splicing.

[0066] As an alternative implementation Figure 2 is a flowchart of the texture key point extraction method provided by an embodiment of the present invention. As shown in Figure 2 Extract the first texture key points of the source point cloud according to color features, including:

[0067] S201. Convert the source point cloud from the RGB color space to the Lab color space;

[0068] The point cloud image obtained by the lidar is generally in the RGB color space. When performing feature extraction, it is first necessary to convert the point cloud from the RGB color space to the Lab color space, which can be achieved through the following method:

[0069]

[0070]

[0071]

[0072] In the above formula , , are the RGB color values of the original point cloud, , , are the RGB color values after normalization and Gamma correction, , , are the color values in the XYZ color space.

[0073] Then convert the color values of the point cloud in the XYZ color space to the Lab color space:

[0074]

[0075]

[0076] In the above formula , , , , , are the three-channel values in the Lab color space.

[0077] S202. In the Lab color space, search for neighborhood points within a predetermined distance for each point in the source point cloud;

[0078] After that, for each point in the point cloud P (source point cloud or target point cloud) , also set a search radius , and search for all neighboring points within the spherical neighborhood with the point as the center and as the radius, and calculate the distance weight value for each neighborhood point: wherein,

[0079]

[0080] is the distance weight value of the corresponding point.

[0081] S203. Calculate the color covariance matrix according to the three-dimensional Lab color vectors of each point and its neighborhood points;

[0082] Different from the covariance matrix based on geometric shape features, in the key point extraction algorithm based on texture attributes, in order to statistically analyze the local point cloud color features, according to the three-dimensional Lab color vector of the point and the three-dimensional Lab color vector of its neighboring points , calculate the color covariance matrix of each point :

[0083]

[0084] Perform subsequent calculations based on this color covariance matrix.

[0085] S204. Perform eigenvalue decomposition on the color covariance matrix to obtain color eigenvalues, and calculate the generalized variance value of the color covariance matrix according to the color eigenvalues;

[0086] Perform eigenvalue decomposition on each color covariance matrix to obtain three color eigenvalues sorted from large to small , , , and calculate the generalized variance value of the color covariance matrix of the point according to the three color eigenvalues. This generalized variance value is equal to the matrix determinant or the product of eigenvalues, that is:

[0087]

[0088] After calculating the generalized variance values of all points, select texture key points according to them.

[0089] ​S205. Select the first texture key points according to the generalized variance value.

[0090] Points with relatively large generalized variance values at a predetermined ratio can be selected as the first texture key points. For example, the points are sorted in descending order of the generalized variance value, and the first 80% of the points are selected as the first texture key points; alternatively, a variance threshold can be preset in advance, and the points corresponding to the generalized variance values greater than the variance threshold are used as the first texture key points.

[0091] The target point cloud obtains the second texture key points in the same way as above.

[0092] As an alternative implementation manner, Figure 3 is the flowchart of the feature descriptor extraction method provided by the embodiment of the present invention. As Figure 3 shown, extracting the first feature descriptor of each point in the first key point set includes:

[0093] S401. For each first key point in the first key point set, calculate its corresponding FPFH feature descriptor and color feature descriptor in the Lab space;

[0094] The FPFH algorithm depends on the Simplified Point Signature Histogram (SPSH). Assume that the point is a key point in the key point set and its normal vector is , as Figure 4 shown, search for its neighboring key points in the spherical neighborhood with the point as the center and R as the radius, and construct the local coordinate values with as the origin:

[0095]

[0096] As Figure 5 shown, then calculate the three angular features of the point , that is:

[0097]

[0098] Divide the value ranges of the three angular feature values into 11 equal intervals, count the number of feature values of the neighboring points falling in each sub-interval to form an 11-dimensional histogram, and finally connect the three histograms together to obtain the 33-dimensional SPFH of the point . Calculate the SPSH of each neighboring point and perform weighted statistics to obtain the 33-dimensional FPFH feature descriptor of the key point :

[0099]

[0100] In the above formula, is the FPFH value of the point , and are the SPSH values of the point and the point respectively, is the Euclidean distance from the point to the point , is the number of neighboring points of the point .

[0101] In order to fuse the FPFH feature descriptor with the color feature descriptor, it is necessary to first convert the key points of the colored point cloud from the RGB color space to the Lab color space to obtain the Lab color values of each key point and its neighborhood points , which are respectively denoted as , . The Lab color Euclidean distance between them is used to represent the color similarity between points:

[0102]

[0103] In order to balance the point cloud color and geometric features, a suitable maximum Lab distance is selected, which is divided into 33 equally spaced intervals, and the number of points in each interval is counted to form a 33-dimensional two-dimensional color histogram .

[0104] S402. Concatenate the FPFH feature descriptor and the color feature descriptor to obtain the first feature descriptor corresponding to each first key point.

[0105] As Figure 6 shown, finally, the two feature histograms are concatenated to obtain a 66-dimensional feature histogram that fuses color and geometric information of the key points. The feature fusion vector of a certain key point is denoted as:

[0106]

[0107] Calculate the fusion feature descriptors of each key point in the first key point set and the second key point set according to the above steps, and finally obtain the first feature descriptor set and the second feature descriptor set .

[0108] As an alternative implementation, Figure 7 is the flowchart of the rough registration method provided by the embodiment of the present invention. As Figure 7As shown, according to the first feature descriptor and the second feature descriptor, the random sample consensus (RANSAC) algorithm is used to perform rough registration on the first key point set and the second key point set to obtain an initial rigid transformation matrix, including:

[0109] S501. Randomly select at least three non - collinear target first key points from the first key point set, and search for the target second key point closest to each target first key point in the second key point set according to the first feature descriptor and the second feature descriptor;

[0110] S502. Calculate the candidate rigid transformation matrix by using the singular value decomposition method for the target first key point and the target second key point;

[0111] S503. Perform coordinate transformation on the first key points in the first key point set according to the candidate rigid transformation matrix to obtain a transformed key point set, calculate the spatial distance between each point in the transformed key point set and the matching point in the second key point set, and regard the points with spatial distance lower than the preset distance threshold as inliers;

[0112] S504. Repeat steps S501 - 503 until the preset number of iterations is reached, and regard the candidate rigid transformation matrix with the largest number of inliers as the initial rigid transformation matrix.

[0113] Use the random sample consensus (RANSAC) algorithm to perform outlier rejection and rough registration of point clouds to obtain the initial transformation matrix between the point clouds to be registered. The specific steps are as follows:

[0114] (1) Randomly select at least 3 non - collinear points from the first key point set of the source point cloud and search for the points with the closest feature descriptors in the second key point set of the target point cloud. This set of matching data is the sampled data;

[0115] (2) Based on the sampled data, use the singular value decomposition method to estimate the candidate rigid transformation matrix between the matching data ;

[0116] (3) Use the candidate rigid transformation matrix to perform coordinate transformation on the first key point set of the source point cloud to obtain a transformed key point set , and calculate the spatial distance between each point in and its matching point in

[0117] If the distance error of a certain point is lower than the set threshold, it is regarded as a screened matching point and added to the inlier set, otherwise it is regarded as an outlier;

[0117] (4) Repeat the above three steps (1) - (3). When the set preset number of iterations is reached, stop the iterative calculation;

[0118] (5) The candidate rigid transformation matrix with the largest number of inlier points is the optimal initial rigid transformation matrix.

[0119] As an alternative implementation, Figure 8 is a flowchart of the final rigid transformation matrix calculation method provided by the embodiments of the present invention. As shown in Figure 8 , a loss function is constructed based on the geometric distance and color distance between the corresponding points of the source point cloud and the target point cloud, and the initial rigid transformation matrix is iteratively optimized to obtain the final rigid transformation matrix, including:

[0120] S601. Perform coordinate transformation on each source point in the source point cloud according to the initial rigid transformation matrix to obtain a transformed point cloud;

[0121] S602. For each point in the transformed point cloud, find its corresponding target point in the target point cloud to construct a set of point pairs;

[0122] S603. Calculate the geometric error term according to the spatial distance of each point pair in the set of point pairs;

[0123] After transforming each source point with the initial rigid transformation matrix, find the nearest target point according to the spatial distance, and form a set of point pairs with the transformed source point and the corresponding target point.

[0124] Geometric error term is defined in the same way as the optimization objective of the point-to-plane ICP algorithm, that is, to minimize the distance between the new position of the point in the source point cloud after transformation and the plane where the corresponding point in the target point cloud is located:

[0125]

[0126] In the above formula, the point is any point in the source point cloud , T is the current rigid transformation matrix, the point is the nearest point in the target point cloud to it, and a set of corresponding points in the current iteration is constructed using these point pairs, is the normal of the point .

[0127] S604. Calculate the color error term according to the estimated color value of the source point on the tangent plane where the corresponding target point is located and the original color value;

[0128] Due to the discrete nature of the colors of point clouds, it becomes difficult to calculate gradients, optimize registration, and perform other geometric operations during iteration. To overcome this problem, a continuous and differentiable color function needs to be defined to represent the color variation of point clouds within a local region. First, a discrete function is used to represent the color values of each point in the target point cloud . Then, a virtual orthogonal camera is introduced at each point , and the imaging plane of this camera is the tangent plane passing through and perpendicular to its normal vector. On this tangent plane, a virtual image can be defined, and a continuous color function is used to represent the color information, where is the vector starting from point along the tangent plane direction. The function is linearly transformed within a small range and can be approximately represented using the first-order Taylor expansion:

[0129]

[0130] In the above formula, is the color gradient, representing the rate of change of color with respect to direction, which can be estimated by least squares fitting. Subsequently, the color error term can be used to measure the difference between the color of point and the color of its projection on the tangent plane of point , that is:

[0131]

[0132] In the above formula represents the original color value of point , is the color estimate of the projection point of the transformed point at , is the function that projects the 3D point onto the tangent plane of point , defined as:

[0133]

[0134] The color error term is calculated in the above manner.

[0135] S605. Weight the geometric error term and the color error term to construct a loss function, and use the Gauss-Newton optimization algorithm to perform iteration with the initial rigid transformation matrix as the initial value until the loss function converges, obtaining the final rigid transformation matrix.

[0136] Construct an error function based on the geometric error term and the color error term:

[0137]

[0138] In the above formula, T is the transformation matrix to be estimated, and correspond to the color error term and the geometric error term respectively, is the weight for balancing these two terms. After defining the error function, the Gauss-Newton method is used to iteratively update the transformation matrix , until the set convergence condition is met, then the transformation matrix between the final point clouds to be registered can be determined.

[0139] An embodiment of the present invention also provides a color point cloud registration system based on feature fusion, Figure 9 is a schematic structural diagram of the color point cloud registration system based on feature fusion provided by an embodiment of the present invention, as Figure 9 shown, the system includes:

[0140] A data acquisition module 100, configured to acquire a source point cloud and a target point cloud;

[0141] When scanning the target area, a lidar is used to collect point cloud data at different angles. These point clouds are all point clouds to be registered and need to be registered in the same coordinate system to obtain the complete three-dimensional information of the target area.

[0142] The target point cloud is the reference point cloud, that is, its coordinate system is the reference coordinate system, and other point clouds to be registered need to perform coordinate transformation based on this coordinate system.

[0143] The source point cloud is the point cloud to be registered, that is, each point of it needs to be transformed to the coordinate system of the target point cloud through the transformation matrix. The source point cloud can be one or more.

[0144] In the specific implementation process, a three-dimensional scanning device with the ability to collect color images can be used, such as an RGB-D camera (such as Kinect, RealSense), a structured light scanner, or a lidar paired with a color camera and other sensors, to collect color point cloud data of the same object or scene from multiple perspectives respectively. Each point cloud data contains the geometric coordinate information (X, Y, Z) of the point in the three-dimensional space and the corresponding color information (R, G, B).

[0145] Optionally, in order to improve the point cloud processing efficiency and registration accuracy, operations such as noise reduction, filtering, resampling, or normal vector estimation are performed on the source point cloud and the target point cloud to provide a basis for subsequent geometric and texture feature extraction.

[0146] The key point extraction module 200 is used to extract the first geometric key points of the source point cloud and the second geometric key points of the target point cloud according to the ISS algorithm; and extract the first texture key points of the source point cloud and the second texture key points of the target point cloud according to the color features.

[0147] Use the ISS (Intrinsic Shape Signature) algorithm to extract the first geometric key points and the second geometric key points from the source point cloud and the target point cloud respectively. This algorithm mainly identifies the points with significant shapes based on the spatial structure changes in the local neighborhood of the point cloud, has rotational invariance and robustness, and is suitable for processing complex three-dimensional surfaces.

[0148] The logic of the ISS algorithm is as follows:

[0149] For each point in the point cloud P , set a search radius , and search for its neighboring points within the spherical neighborhood centered at the point with as the radius, and calculate an Euclidean distance weight value for each neighboring point:

[0150]

[0151] In the above formula is any neighboring point within the region centered at with as the radius. Then calculate the covariance matrix of each point :

[0152]

[0153] Since the calculated covariance matrix is a -dimensional real symmetric matrix, the eigenvalue decomposition of each covariance matrix can obtain three eigenvalues, which are sorted from large to small as , , , and at the same time, the points with the ratio between the eigenvalues less than the set thresholds and are used as candidate key points, as shown in the following formula:

[0154]

[0155] Where and are the thresholds set in advance, and , . For the obtained candidate key points, according to their minimum eigenvalue Perform non-maximum suppression, that is, retain candidate key points with large changes along the minimum main direction as geometric feature key points in a certain proportion. In fact, only the points with the most significant features in the local area are retained within a certain range, and a set of key points with obvious geometric features is obtained, namely the first geometric key point and the second geometric key point (both are point sets).

[0156] In addition, the color covariance matrix in the local area of the point cloud is calculated in the Lab color space, and the generalized variance of the obtained color covariance matrix is used to measure the local texture saliency. The larger points scattered in the color space are taken as the texture feature key points to be extracted, that is, the first texture key points of the source point cloud and the second texture key points of the target point cloud (both are point sets).

[0157] The extraction of the above geometric key points and the extraction of the texture key points, the feature extraction of the source point cloud and the feature extraction of the target point cloud can all be obtained through multi-threaded parallel computing.

[0158] A fusion module 300 is used to fuse the first geometric key point with the first texture key point according to a preset distance range to obtain a first key point set; and fuse the second geometric key point with the second texture key point according to the preset distance range to obtain a second key point set;

[0159] Based on the spatial proximity relationship, texture key points with a predetermined spatial distance are added to the geometric key points, so that a key point set with joint perception of geometry and color is constructed by fusing the information of the two types of key points, further improving the key point coverage and description ability of point cloud registration. a ,r b ] are added to the key point set, and the range can be set according to the actual situation; or the geometric key points are used as the benchmark, the texture key points within the predetermined range are eliminated, and the geometric key points and the filtered texture key points are added to the key point set.

[0160] The source point cloud P and the target point cloud Q are fused with geometric and color key points to obtain the first key point set. and the second keypoint set .

[0161] A feature extraction module 400, used to extract a first feature descriptor of each point in the first key point set and a second feature descriptor of each point in the second key point set;

[0162] Feature extraction is performed on each point in the first key point set and each point in the second key point set to obtain corresponding first and second feature descriptors respectively. Each key point extracts a composite feature vector by analyzing its geometric relationship and color characteristics in the local space of the point cloud, forming a high-dimensional representation for subsequent precise matching between key points.

[0163] The rough registration module 500 is used to perform rough registration on the first key point set and the second key point set according to the first and second feature descriptors by using the random sample consensus algorithm to obtain an initial rigid transformation matrix.

[0164] The random sample consensus (RANSAC) algorithm is used in combination with feature descriptors for matching, eliminating mismatched point pairs, and estimating the initial spatial transformation relationship between the source point cloud and the target point cloud, that is, the initial rigid transformation matrix.

[0165] The initial rigid transformation matrix obtained in this step can achieve a rough level of point cloud alignment, serving as the initial solution for fine registration, providing a reasonable convergence starting point for subsequent optimization iterations, and avoiding falling into local optimal solutions or divergence problems.

[0166] The matrix optimization module 600 is used to construct a loss function based on the geometric distance and color distance between corresponding points of the source point cloud and the target point cloud, and iteratively optimize the initial rigid transformation matrix to obtain the final rigid transformation matrix.

[0167] A loss function is jointly constructed using the spatial position error (geometric distance) and color value error (color distance) between point pairs after registration by the initial rigid transformation matrix, and the initial rigid transformation matrix obtained by rough registration is used as the initial value for optimization to iteratively solve the final registration matrix.

[0168] The geometric distance can be used to measure the closeness between the transformed source point cloud and the surface of the target point cloud based on the "point-to-plane distance", while the color distance can use the color gradient of points in the Lab space to evaluate the difference between the projected color of the transformed point on the tangent plane of the target point and the original color. By introducing a weight factor, the two types of errors are reasonably weighted to form an optimization objective that balances accuracy and robustness.

[0169] The rigid transformation matrix is iteratively updated until the loss function converges to obtain the final optimal rigid transformation result, that is, the final rigid transformation matrix, to achieve a higher-precision point cloud registration.

[0170] The registration and stitching module 700 is used to register and stitch the source point cloud and the target point cloud according to the final rigid transformation matrix to obtain the stitched point cloud.

[0171] Apply the finally obtained rigid transformation matrix to the source point cloud, transform its coordinates to the unified coordinate system where the target point cloud is located, and merge the two point clouds into an integrated point cloud model to form complete spliced point cloud data. In subsequent use, this finally obtained rigid transformation matrix can be used for subsequent point cloud splicing.

[0172] As an alternative implementation Figure 10 is a schematic structural diagram of the key point extraction module provided by an embodiment of the present invention, as Figure 10 shown, the key point extraction module 200 includes:

[0173] A color space conversion sub-module 2001, configured to convert the source point cloud from the RGB color space to the Lab color space;

[0174] The point cloud image obtained by the lidar is generally in the RGB color space. When performing feature extraction, it is first necessary to convert the point cloud from the RGB color space to the Lab color space, which can be achieved through the following method:

[0175]

[0176]

[0177]

[0178] In the above formula , , are the RGB color values of the original point cloud, , , are the RGB color values after normalization and gamma correction, , , are the color values in the XYZ color space.

[0179] Then convert the color values of the point cloud in the XYZ color space to the Lab color space:

[0180]

[0181]

[0182] In the above formula , , , , , are the three-channel values in the Lab color space.

[0183] The neighborhood search sub-module 2002 is used to search for neighborhood points within a predetermined distance for each point in the source point cloud in the Lab color space;

[0184] After that, for each point in the point cloud P (source point cloud or target point cloud) , a search radius is also set , and search for all neighboring points within the spherical neighborhood with the point as the center and as the radius, and calculate the distance weight for each neighborhood point:

[0185]

[0186] where is the distance weight of the corresponding point.

[0187] The matrix calculation sub-module 2003 is used to calculate the color covariance matrix according to the three-dimensional Lab color vectors of each point and its neighborhood points;

[0188] Different from the covariance matrix based on geometric shape features, in the key point extraction algorithm based on texture attributes, in order to statistically analyze the local point cloud color features, according to the three-dimensional Lab color vector of the point and the three-dimensional Lab color vector of its neighboring points , calculate the color covariance matrix of each point :

[0189]

[0190] Perform subsequent calculations based on this color covariance matrix.

[0191] The variance value calculation sub-module 2004 is used to perform eigenvalue decomposition on the color covariance matrix to obtain color eigenvalues, and calculate the generalized variance value of the color covariance matrix according to the color eigenvalues;

[0192] Perform eigenvalue decomposition on each color covariance matrix to obtain three color eigenvalues sorted from large to small , , , calculate the generalized variance value of the color covariance matrix of the point , and this generalized variance value is equal to the matrix determinant or the product of eigenvalues, that is:

[0193]

[0194] After calculating the generalized variance values of all points, select texture key points based on them.

[0195] The screening sub-module 2005 is used to screen the first texture key points according to the generalized variance value.

[0196] Points with relatively large generalized variance values in a predetermined proportion can be selected as the first texture key points. For example, the points are sorted in descending order of the generalized variance value, and the first 80% of the points are selected as the first texture key points; alternatively, a variance threshold can be preset in advance, and the points corresponding to the generalized variance values greater than the variance threshold are used as the first texture key points.

[0197] The target point cloud obtains the second texture key points in the same manner as above.

[0198] As an alternative implementation manner, Figure 11 is a schematic structural diagram of the feature extraction module provided by an embodiment of the present invention. As Figure 11 shown, the feature extraction module 400 includes:

[0199] The feature descriptor calculation sub-module 4001 is used to calculate the corresponding FPFH feature descriptor and the color feature descriptor in the Lab space for each first key point in the first key point set.

[0200] The FPFH algorithm depends on the Simplified Point Feature Histogram (SPSH). Assume that the point is a key point in the key point set and its normal vector is . As Figure 4 shown, within the spherical neighborhood with the point as the center and R as the radius, search for its neighboring key points , and construct the local coordinate values with as the origin:

[0201]

[0202] As Figure 5 shown, then calculate the three angular features of the point , that is:

[0203]

[0204] The value ranges of the three angular feature values are all equally divided into 11 intervals, count the number of feature values of the neighboring points falling within each sub-interval to form an 11-dimensional histogram, and finally connect the three histograms together to obtain the 33-dimensional SPFH of the point . Calculate the SPSH of each neighboring point and perform weighted statistics to obtain the 33-dimensional FPFH feature descriptor of the key point :

[0205]

[0206] In the above formula, is the FPFH value of point , and are the SPSH values of points and respectively, is the Euclidean distance from point to point , is the number of neighboring points of point .

[0207] In order to fuse the FPFH feature descriptor with the color feature descriptor, it is necessary to first convert the key points of the colored point cloud from the RGB color space to the Lab color space to obtain the Lab color values of each key point and its neighborhood points , which are respectively denoted as , . The Lab color Euclidean distance between them is used to represent the color similarity between points:

[0208]

[0209] In order to balance the point cloud color and geometric features, a suitable maximum Lab distance is selected, which is divided into 33 equally spaced intervals, and the number of points in each interval is counted to form a 33-dimensional two-dimensional color histogram .

[0210] The feature descriptor fusion sub-module 4002 is used to splice the FPFH feature descriptor and the color feature descriptor to obtain the first feature descriptor corresponding to each first key point.

[0211] As Figure 6 shown, finally, the two feature histograms are spliced to obtain a 66-dimensional feature histogram that fuses color and geometric information of the key points. The feature fusion vector of a certain key point is denoted as:

[0212]

[0213] Calculate the fusion feature descriptors of each key point in the first key point set and the second key point set according to the above steps, and finally obtain the first feature descriptor set and the second feature descriptor set .

[0214] As an optional implementation manner, Figure 12 is the structural schematic diagram of the rough registration module provided by the embodiment of the present invention. As Figure 12 shown, the rough registration module 500 includes:

[0215] The point selection sub-module 5001 is used to randomly select at least three non-collinear target first key points from the first key point set, and search for the target second key points closest to each target first key point in the second key point set according to the first feature descriptor and the second feature descriptor;

[0216] The solution sub-module 5002 is used to calculate the candidate rigid transformation matrix by using the singular value decomposition method for the target first key point and the target second key point;

[0217] The statistics sub-module 5003 is used to perform coordinate transformation on the first key points in the first key point set according to the candidate rigid transformation matrix to obtain a transformed key point set, calculate the spatial distance between each point in the transformed key point set and the matching points in the second key point set, and use the points with spatial distances lower than the preset distance threshold as inliers;

[0218] The iteration sub-module 5004 is used to repeat the steps of the point selection sub-module, the solution sub-module, and the statistics sub-module until the preset number of iterations is reached, and use the candidate rigid transformation matrix with the largest number of inliers as the initial rigid transformation matrix.

[0219] Use the Random Sample Consensus (RANSAC) algorithm to perform false match rejection and rough point cloud registration to obtain the initial transformation matrix between the point clouds to be registered. The specific steps are as follows:

[0220] (1) Randomly select at least 3 non-collinear points from the first key point set of the source point cloud and search for the points with the closest feature descriptors in the second key point set of the target point cloud. This set of matching data is the sampled sample;

[0221] (2) Based on the sampled sample, use the singular value decomposition method to estimate the candidate rigid transformation matrix between the matching data ;

[0222] (3) Use the candidate rigid transformation matrix to perform coordinate transformation on the first key point set of the source point cloud to obtain a transformed key point set , and calculate the spatial distance between each point in and its matching point in

[0223] If the distance error of a certain point is lower than the set threshold, then regard it as a screened matching point and add this point to the inlier set, otherwise regard it as an outlier;

[0224] (5) The candidate rigid transformation matrix with the largest number of interior points is the optimal initial rigid transformation matrix.

[0225] As an alternative implementation, Figure 13 is a schematic structural diagram of the matrix optimization module provided by the embodiments of the present invention. As shown in Figure 13 shown, the matrix optimization module 600 includes:

[0226] A coordinate transformation sub-module 6001, configured to perform coordinate transformation on each source point in the source point cloud according to the initial rigid transformation matrix to obtain a transformed point cloud;

[0227] A point pair construction sub-module 6002, configured to find the corresponding target point of each point in the transformed point cloud in the target point cloud and construct a point pair set;

[0228] A geometric error term construction sub-module 6003, configured to calculate a geometric error term according to the spatial distance of each point pair in the point pair set;

[0229] After transforming each source point using the initial rigid transformation matrix, find the target point closest to it according to the spatial distance, and form a point pair set with the transformed source point and the corresponding target point.

[0230] Geometric error term is defined to be consistent with the optimization objective of the point-to-plane ICP algorithm, that is, to minimize the distance between the new position of the point in the source point cloud after transformation and the plane where the corresponding point in the target point cloud is located:

[0231]

[0232] In the above formula, the point is an arbitrary point in the source point cloud , T is the current rigid transformation matrix, and the point is the point closest to it in the target point cloud . Use these point pairs to construct a set of corresponding points in the current iteration , is the normal vector of the point .

[0233] A color error term construction sub-module 6004, configured to calculate a color error term according to the estimated color value of the source point on the tangent plane where the corresponding target point is located and the original color value;

[0234] Since the color of the point cloud is usually discrete, this makes it difficult to calculate gradients, optimize registration, and perform other geometric operations during iteration. To overcome this problem, it is necessary to define a continuous and differentiable color function to represent the color change of the point cloud in the local area. First, use the discrete function to represent the target point cloud Each point in has a color value. Then, a virtual orthogonal camera is introduced at each point . The imaging plane of this camera is the tangent plane passing through and perpendicular to its normal vector . On this tangent plane, a virtual image can be defined, and a continuous color function is used to represent color information, where is the vector starting from point and along the tangent plane direction. The function is linearly transformed within a small range and can be approximately represented by the first-order Taylor expansion:

[0235]

[0236] In the above formula, is the color gradient, representing the rate of change of color with respect to the direction and can be estimated by least squares fitting. Subsequently, the color error term is used to measure the difference between the color of point and the color of its projection on the tangent plane of point , that is:

[0237]

[0238] In the above formula, represents the original color value of point , is the color estimate of the projection point of the transformed point at , is the function that projects the three-dimensional point onto the tangent plane of point , defined as:

[0239]

[0240] The color error term is calculated in the above manner.

[0241] The final rigid transformation matrix determination sub-module 6005 is used to construct a loss function by weighting the geometric error term and the color error term, and adopts the Gauss-Newton optimization algorithm to iterate with the initial rigid transformation matrix as the initial value until the loss function converges to obtain the final rigid transformation matrix.

[0242] Construct an error function based on the geometric error term and the color error term:

[0243]

[0244] In the above formula, T is the transformation matrix to be estimated, and correspond to the color error term and the geometric error term respectively, is the weight for balancing these two terms. After defining the error function, the Gauss-Newton method is used for iterative updating of the transformation matrix , and the transformation matrix between the finally to-be-registered point clouds can be determined until the set convergence condition is satisfied.

[0245] The above technical solution has the following beneficial effects: By combining the distance model with the Lab color model to extract the key points with significant changes in the geometric and texture features of the point cloud surface, the representativeness of the key points is improved; By fusing the geometric feature descriptor and the color feature descriptor of the point cloud to describe the features of the key points, the color information is taken into account while extracting the geometric features of the point cloud, thereby improving the registration accuracy; By using the random sample consensus algorithm and the colored iterative closest point algorithm to estimate the transformation matrix, it is beneficial to improve the registration accuracy of the colored point cloud model with insignificant geometric features.

[0246] The specific implementation manners of the above invention further elaborate on the purpose, technical solution and beneficial effects of the present invention. It should be understood that the above content is only the specific implementation manners of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A color point cloud registration method based on feature fusion, characterized in that Including: Obtaining a source point cloud and a target point cloud; Extracting first geometric key points of the source point cloud and second geometric key points of the target point cloud according to the ISS algorithm; Extracting first texture key points of the source point cloud and second texture key points of the target point cloud according to color features; Fusing the first geometric key points and the first texture key points according to a preset distance range to obtain a first key point set; Fusing the second geometric key points and the second texture key points according to a preset distance range to obtain a second key point set; Respectively extracting a first feature descriptor for each point in the first key point set and a second feature descriptor for each point in the second key point set; According to the first feature descriptor and the second feature descriptor, using the random sample consensus algorithm to perform rough registration on the first key point set and the second key point set to obtain an initial rigid transformation matrix; Constructing a loss function according to the geometric distance and color distance of corresponding points between the source point cloud and the target point cloud, and iteratively optimizing the initial rigid transformation matrix to obtain a final rigid transformation matrix; Performing registration and splicing on the source point cloud and the target point cloud according to the final rigid transformation matrix to obtain a spliced point cloud.

2. The method for color point cloud registration based on feature fusion according to claim 1, wherein The extracting of the first texture key points of the source point cloud according to color features includes: Converting the source point cloud from the RGB color space to the Lab color space; In the Lab color space, searching for neighborhood points within a predetermined distance for each point in the source point cloud; Calculating a color covariance matrix according to the three-dimensional Lab color vectors of each point and its neighborhood points; Performing eigenvalue decomposition on the color covariance matrix to obtain color eigenvalues, and calculating the generalized variance value of the color covariance matrix according to the color eigenvalues; Screening to obtain first texture key points according to the generalized variance value.

3. The color point cloud registration method based on feature fusion according to claim 1, wherein The extracting of the first feature descriptor for each point in the first key point set includes: For each first key point in the first key point set, calculating its corresponding FPFH feature descriptor and color feature descriptor in the Lab space; Splicing the FPFH feature descriptor and the color feature descriptor to obtain the first feature descriptor corresponding to each first key point.

4. The method for color point cloud registration based on feature fusion according to claim 1, wherein The performing of rough registration on the first key point set and the second key point set according to the first feature descriptor and the second feature descriptor using the random sample consensus algorithm to obtain an initial rigid transformation matrix includes: Randomly selecting at least three non-collinear target first key points from the first key point set, and searching for the target second key point closest to each target first key point in the second key point set according to the first feature descriptor and the second feature descriptor; Calculating a candidate rigid transformation matrix by using the singular value decomposition method for the target first key point and the target second key point; Performing coordinate transformation on the first key points in the first key point set according to the candidate rigid transformation matrix to obtain a set of transformed key points, calculating the spatial distance between each point in the set of transformed key points and the matching points in the second key point set, and taking the points with the spatial distance lower than a preset distance threshold as inliers; Repeat the above steps until the preset number of iterations is reached, and use the candidate rigid transformation matrix with the largest number of inlier points as the initial rigid transformation matrix.

5. The method for color point cloud registration based on feature fusion according to claim 1, wherein The method of constructing a loss function based on the geometric distance and color distance between corresponding points in the source point cloud and the target point cloud, and iteratively optimizing the initial rigid transformation matrix to obtain the final rigid transformation matrix includes: Perform coordinate transformation on each source point in the source point cloud according to the initial rigid transformation matrix to obtain a transformed point cloud; For each point in the transformed point cloud, find its corresponding target point in the target point cloud and construct a point pair set; Calculate the geometric error term according to the spatial distance of each point pair in the point pair set; Calculate the color error term according to the estimated color value of the source point on the tangent plane where the corresponding target point belongs and the original color value; Weight the geometric error term and the color error term to construct a loss function, and use the Gauss-Newton optimization algorithm to perform iteration with the initial rigid transformation matrix as the initial value until the loss function converges to obtain the final rigid transformation matrix.

6. A color point cloud registration system based on feature fusion, characterized in that, It includes: A data acquisition module for acquiring a source point cloud and a target point cloud; A key point extraction module for extracting the first geometric key points of the source point cloud and the second geometric key points of the target point cloud according to the ISS algorithm; Extract the first texture key points of the source point cloud and the second texture key points of the target point cloud according to color features; A fusion module for fusing the first geometric key points and the first texture key points according to a preset distance range to obtain a first key point set; Fuse the second geometric key points and the second texture key points according to a preset distance range to obtain a second key point set; A feature extraction module for respectively extracting the first feature descriptors of each point in the first key point set and the second feature descriptors of each point in the second key point set; A rough registration module for roughly registering the first key point set and the second key point set according to the first feature descriptor and the second feature descriptor by using the random sample consensus algorithm to obtain an initial rigid transformation matrix; A matrix optimization module for constructing a loss function based on the geometric distance and color distance between corresponding points in the source point cloud and the target point cloud, and iteratively optimizing the initial rigid transformation matrix to obtain the final rigid transformation matrix; A registration and stitching module for registering and stitching the source point cloud and the target point cloud according to the final rigid transformation matrix to obtain a stitched point cloud.

7. The color point cloud registration system based on feature fusion according to claim 6, wherein The fusion module includes: A color space conversion sub-module for converting the source point cloud from the RGB color space to the Lab color space; A neighborhood search sub-module for searching for neighborhood points within a predetermined distance for each point in the source point cloud in the Lab color space; A matrix calculation sub-module for calculating the color covariance matrix according to the three-dimensional Lab color vectors of each point and its neighborhood points; A variance value calculation sub-module for performing eigenvalue decomposition on the color covariance matrix to obtain color eigenvalues, and calculating the generalized variance value of the color covariance matrix according to the color eigenvalues; A screening sub-module for screening to obtain the first texture key points according to the generalized variance value.

8. The color point cloud registration system based on feature fusion according to claim 6, characterized in that The feature extraction module includes: A feature descriptor calculation sub-module, which is used to calculate the corresponding FPFH feature descriptor and the color feature descriptor in the Lab color space for each first key point in the first key point set; A feature descriptor fusion sub-module, which is used to splice the FPFH feature descriptor and the color feature descriptor to obtain a first feature descriptor corresponding to each first key point.

9. The color point cloud registration system based on feature fusion according to claim 6, characterized in that The rough registration module includes: A point selection sub-module, which is used to randomly select at least three non-collinear target first key points from the first key point set, and find the target second key point closest to each target first key point in the second key point set according to the first feature descriptor and the second feature descriptor; A solution sub-module, which is used to calculate a candidate rigid transformation matrix by using the singular value decomposition method for the target first key point and the target second key point; A statistics sub-module, which is used to perform coordinate transformation on the first key points in the first key point set according to the candidate rigid transformation matrix to obtain a set of transformed key points, calculate the spatial distance between each point in the set of transformed key points and the matching points in the second key point set, and use the points with spatial distances lower than the preset distance threshold as inliers; An iteration sub-module, which is used to repeat the steps of the point selection sub-module, the solution sub-module, and the statistics sub-module until the preset number of iterations is reached, and use the candidate rigid transformation matrix with the largest number of inliers as the initial rigid transformation matrix.

10. The color point cloud registration system based on feature fusion according to claim 6, wherein, The matrix optimization module includes: A coordinate transformation sub-module, which is used to perform coordinate transformation on each source point in the source point cloud according to the initial rigid transformation matrix to obtain a transformed point cloud; A point pair construction sub-module, which is used to find the corresponding target point in the target point cloud for each point in the transformed point cloud and construct a set of point pairs; A geometric error term construction sub-module, which is used to calculate geometric error terms according to the spatial distances of each point pair in the set of point pairs; A color error term construction sub-module, which is used to calculate color error terms according to the estimated color value and the original color value of the source point on the tangent plane where the corresponding target point is located; A final rigid transformation matrix determination sub-module, which is used to construct a loss function by weighting the geometric error terms and the color error terms, and use the Gauss-Newton optimization algorithm to perform iterations with the initial rigid transformation matrix as the initial value until the loss function converges to obtain the final rigid transformation matrix.

Images