Building scene point cloud registration method and device

Abstract

The invention belongs to the technical field of three-dimensional point cloud matching, and relates to a building scene point cloud registration method and device. The method comprises the following steps: S1, dividing a building scene point cloud into a plurality of voxels, and determining a curvature space descriptor of each voxel; s2, inputting the curvature space descriptor into a pre-trained multi-layer perceptron network feature classifier, and outputting a geometric feature type corresponding to each voxel; s3, determining geometric features and model parameters of each geometric feature type according to the original point cloud; s4, performing geometric feature matching on different geometric feature types of the two frames of building scene point clouds to be matched; and S5, calculating a transformation matrix of each geometric feature type based on a matching result, and fusing the two frames of building scene point clouds according to the transformation matrix. The registration precision of the point cloud can be improved, and the registration efficiency of the large-scale point cloud can be obviously improved.

Description

Technical Field

[0001] This application belongs to the technical field of three-dimensional point cloud matching, and particularly relates to a method and device for point cloud registration in a building scene. Background Art

[0002] Point cloud registration technology refers to the ability to stitch or align two or more point clouds with certain overlapping attributes to a common reference coordinate system, thereby achieving the function of expanding or fusing point clouds into a complete map. A unified and efficient large-scale building scene point cloud geometric feature extraction and registration technology is one of the core technologies for large-scale three-dimensional reconstruction and robot positioning and mapping. The geometric feature-based method has broad application and development prospects in point cloud registration technology. The application of three-dimensional reconstruction technology in the field of structured buildings has the following challenges: rich features but extremely high symmetry; the number of single feature matches is small and not sufficient to constrain the registration transformation equation; geometric features are prone to false matches, resulting in phenomena such as matching failure.

[0003] Point cloud registration technology is mainly divided into two categories: geometric feature-based registration methods and learning-based end-to-end registration methods. Geometric feature-based methods usually rely on the inherent information of point clouds and achieve point cloud registration through the processes of feature extraction, feature matching, and registration transformation solution. They have the advantages of low computational cost and being unaffected by the computing platform. However, when the number of available geometric features in the point cloud scene is small and the feature symmetry is high, point cloud registration may be affected by low feature extraction accuracy and feature false matching, resulting in low registration accuracy or even registration failure, especially for small-scale point clouds. Using geometric feature-based registration methods usually fails. Learning-based end-to-end registration methods require more computational and training efforts due to the end-to-end nature. They are usually trained with an odometry dataset composed of pose relationships, lacking a large-scale large transformation dataset to support the performance of the end-to-end model, which affects the generalization and expansion ability of the model. Especially for large-scale building scenes, the lack of corresponding datasets to strengthen the model leads to the limited application of learning-based end-to-end registration methods in large-scale building scenes. For example, the Chinese invention patent with the application number 202010717508.9 discloses a registration method for a three-dimensional matching model combining an attention mechanism and a three-dimensional graph convolutional network. Its architecture is a three-branch Siamese architecture, including a Detector model and a Descriptor model, which are used to extract depth features and descriptor matches in the point cloud respectively, and then use the least squares idea to achieve the point cloud registration function. However, this patent is only applicable to the learned point cloud registration scenario.

[0004] None of the existing technologies can solve the problem of point cloud registration for large-scale structured building scenes. To address the three key issues faced in the point cloud registration technology for structured building scenes, a unified and efficient method and device for rapid extraction and registration of geometric features of point clouds in large-scale building scenes are proposed. Summary of the Invention

[0005] To solve at least one of the above technical problems, the present application provides a method and device for point cloud registration in a building scene, particularly applicable to the field of point cloud registration for large-scale structured building scenes.

[0006] The first aspect of the present application provides a method for point cloud registration in a building scene, mainly including:

[0007] Step S1: Divide the point cloud of the building scene into multiple voxels, and determine the curvature space descriptor of each voxel. The curvature space descriptor refers to an ordered set of the curvature of the voxel and the curvatures of multiple neighboring voxels around the voxel.

[0008] Step S2: Input the curvature space descriptor into a pre-trained multi-layer perceptron network feature classifier, and output the geometric feature types corresponding to each voxel. The geometric feature types include corner points, straight lines, and planes.

[0009] Step S3: Determine the geometric features and model parameters of each geometric feature type according to the original point cloud.

[0010] Step S4: For two frames of point clouds of the building scene to be matched, perform geometric feature matching for different geometric feature types respectively.

[0011] Step S5: Calculate the transformation matrix of each geometric feature type based on the matching results, and fuse the two frames of point clouds of the building scene according to the transformation matrix.

[0012] Preferably, step S1 further includes:

[0013] Step S11: Perform voxelization operation with a fixed size on the point cloud of the building scene. When the number of original point clouds contained in a voxel with a fixed size exceeds the threshold, use the principal component analysis method to calculate the curvature of the voxel. When the number of original point clouds contained in a voxel with a fixed size does not exceed the threshold and is greater than 0, the curvature value of the voxel is assigned 0. When the number of original point clouds in the voxel with a fixed size is 0, the curvature of the voxel is assigned -1.

[0014] Step S12: For each voxel, take it as the central voxel, and use the other voxels within the range of extending two steps along the mutually orthogonal coordinate axes in the three-dimensional space as neighboring voxels. Concatenate all the curvatures of the central voxel and each neighboring voxel in the order of the coordinate system to form a column vector with a fixed dimension, and record this column vector as the curvature space descriptor of the central voxel.

[0015] Preferably, in step S11, the fixed size is 0.1 m - 0.5 m, and the threshold is not less than 3.

[0016] Preferably, step S3 further includes:

[0017] When the geometric feature type is a corner point, the average value of the original point cloud within the voxel corresponding to the corner point is used as the geometric feature of the corner point;

[0018] When the geometric feature type is a straight line, the voxels corresponding to multiple straight lines are clustered, and for each group of clustered voxels, the direction and endpoint coordinates of the corresponding straight line are calculated and integrated based on the point clouds within each group of voxels;

[0019] When the geometric feature type is a plane, the voxels corresponding to multiple planes are clustered, and for each group of clustered voxels, the parametric equation of the corresponding plane is fitted based on the point clouds within each group of voxels.

[0020] Preferably, step S5 further includes:

[0021] Step S51: Extract the parameter set of the rotation matrix in the transformation matrix involved in the corner points, straight lines, and plane features respectively from two frames of building scene point clouds;

[0022] Step S52: Jointly construct a covariance matrix based on the parameter sets of the corner points, straight lines, and planes;

[0023] Step S53: Solve the rotation matrix by decomposing the covariance matrix;

[0024] Step S54: Calculate the translation matrix based on the rotation matrix using the least squares method.

[0025] The second aspect of the present application provides a building scene point cloud registration device, mainly including:

[0026] A local curvature space descriptor generation module, configured to divide the building scene point cloud into multiple voxels, and determine the curvature space descriptor of each voxel, where the curvature space descriptor refers to an ordered set of the curvature of the voxel and the curvatures of multiple surrounding voxels;

[0027] A multi - feature extraction module, configured to input the curvature space descriptor into a pre - trained multi - layer perceptron network feature classifier, and output the geometric feature type corresponding to each voxel, where the geometric feature type includes corner points, straight lines, and planes;

[0028] A geometric feature calculation module, configured to determine the geometric features and model parameters of each geometric feature type according to the original point cloud;

[0029] A geometric feature matching module, which is used to perform geometric feature matching on two frames of building scene point clouds to be matched for different geometric feature types respectively;

[0030] A multi-type feature synchronous registration module, which is used to calculate the transformation matrix of each geometric feature type based on the matching result, and fuse the two frames of building scene point clouds according to the transformation matrix.

[0031] Preferably, the local curvature space descriptor generation module includes:

[0032] A voxel curvature calculation unit, which is used to perform voxelization operation with a fixed size on the building scene point cloud. When the number of original point clouds contained in a voxel with a fixed size exceeds the threshold, the principal component analysis method is used to calculate the curvature of the voxel. When the number of original point clouds contained in a voxel with a fixed size does not exceed the threshold and is greater than 0, the curvature value of the voxel is assigned 0. When the number of original point clouds in a voxel with a fixed size is 0, the curvature of the voxel is assigned -1;

[0033] A curvature space descriptor calculation unit, which is used for each voxel. Taking it as the central voxel, other voxels within the range of extending two steps along the directions of the three mutually orthogonal coordinate axes in three-dimensional space are used as neighborhood voxels. The curvatures of the central voxel and all neighborhood voxels are concatenated in the order of the coordinate system to form a column vector containing a fixed dimension, and this column vector is denoted as the curvature space descriptor of the central voxel.

[0034] Preferably, in the voxel curvature calculation unit, the fixed size is 0.1m - 0.5m, and the threshold is not less than 3.

[0035] Preferably, the geometric feature calculation module includes:

[0036] A corner point geometric feature calculation unit, which is used when the geometric feature type is a corner point, and takes the average value of the original point clouds in the voxel corresponding to the corner point as the geometric feature of the corner point;

[0037] A line geometric feature calculation unit, which is used when the geometric feature type is a line, clusters the voxels corresponding to multiple lines, and for each group of clustered voxels, calculates and integrates the direction and endpoint coordinates of the corresponding line according to the point clouds in each group of voxels;

[0038] A plane geometric feature calculation unit, which is used when the geometric feature type is a plane, clusters the voxels corresponding to multiple planes, and for each group of clustered voxels, fits the parametric equation of the corresponding plane according to the point clouds in each group of voxels.

[0039] Preferably, the multi-type feature synchronous registration module includes:

[0040] A parameter set extraction unit for extracting the parameter sets related to the rotation matrix in the transformation matrix from the corner points, straight lines, and plane features in two frames of building scene point clouds respectively;

[0041] A covariance matrix construction unit for jointly constructing a covariance matrix based on the parameter sets of corner points, straight lines, and planes;

[0042] A rotation matrix solving unit for solving the rotation matrix by decomposing the covariance matrix;

[0043] A translation matrix solving unit for calculating the translation matrix based on the least squares method according to the rotation matrix.

[0044] In a third aspect of the present application, a computer device includes a processor, a memory, and a computer program stored on the memory and executable on the processor. The processor executes the computer program to implement the building scene point cloud registration method as described above.

[0045] In a fourth aspect of the present application, a readable storage medium stores a computer program, and the computer program is used to implement the building scene point cloud registration method as described above when executed by a processor.

[0046] The present application can improve the registration accuracy of point clouds and significantly improve the registration efficiency of large-scale point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of a preferred embodiment of the building scene point cloud registration method of the present application.

[0048] Figure 2 For the present application Figure 1 The multi-layer perceptron architecture diagram for feature classification of the illustrated embodiment.

[0049] Figure 3 It is a schematic structural diagram of a computer device of a terminal or a server suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] To make the purpose, technical solutions, and advantages of the present application more clear, the following will describe the technical solutions in the embodiments of the present application in more detail in combination with the accompanying drawings in the embodiments of the present application. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation to the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. The following will describe the embodiments of the present application in detail in combination with the accompanying drawings.

[0051] The present application provides a unified and efficient method for quickly extracting and registering geometric features of point clouds in large-scale building scenes. The main research object is the point clouds of large-scale building scenes, that is, at least including the position coordinate information of the point clouds, and its main source is the common multi-line lidar and three-dimensional laser point cloud acquisition device terminals in the field of view.

[0052] In the first aspect of the present application, a method for registering point clouds of building scenes is provided, as Figure 1 shown, which specifically includes the following steps:

[0053] Step S1: Divide the point clouds of the building scene into multiple voxels, and determine the curvature space descriptor of each voxel. The curvature space descriptor refers to an ordered set of the curvature of the voxel and the curvatures of multiple voxels around the voxel.

[0054] This step is used to calculate the curvature space descriptor, and a vector is formed by combining the curvature of the central voxel and the curvatures of its neighboring voxels. This vector serves as the curvature space descriptor of the central voxel.

[0055] In some optional embodiments, step S1 further includes:

[0056] Step S11: Perform voxelization operation on the point clouds of the building scene with a fixed size. When the number of original point clouds contained in the voxel with a fixed size exceeds the threshold, use the principal component analysis method to calculate the curvature of the voxel. When the number of original point clouds contained in the voxel with a fixed size does not exceed the threshold and is greater than 0, assign the curvature value of the voxel to 0. When the number of original point clouds in the voxel with a fixed size is 0, assign the curvature of the voxel to -1;

[0057] Step S12: For each voxel, taking it as the central voxel, the other voxels within the range of extending two steps along the directions of the mutually orthogonal coordinate axes in the three-dimensional space are used as neighboring voxels. The curvatures of the central voxel and all neighboring voxels are concatenated in the order of the coordinate system to form a column vector with a fixed dimension, and this column vector is denoted as the curvature space descriptor of the central voxel.

[0058] In this embodiment, first, a voxelization operation of a fixed size is performed on the point cloud P a In some alternative embodiments, the size of the voxelization operation is estimated according to the scale of the point cloud, usually selected between 0.1 m and 0.5 m. Then, the curvature of each voxel is calculated. If the voxel is empty, the corresponding curvature value is assigned -1. If the number of point clouds in the voxel is less than the threshold τ v , the corresponding curvature is assigned 0. In addition, to calculate the curvature of the voxel point cloud, the principal component analysis method can be used, or the method of principal curvature can be selected; the threshold τ v of the number of point clouds in the voxel is evaluated according to the density of the point cloud, but it is necessary to ensure the effectiveness and reliability of the calculation. Therefore, in some alternative embodiments, the threshold is not less than 3.

[0059] After the curvature of each voxel is calculated, the curvatures of multiple voxels are synthesized as the curvature space descriptor of the central voxel. That is, for each valid voxel, the curvatures of the voxels in its neighborhood are combined. For example, in the directions of the mutually orthogonal coordinate axes in the three-dimensional space, the classification descriptor of the current voxel is constructed respectively according to the ranges of positive and negative two steps, so as to better identify the geometric feature type corresponding to the central voxel. By concatenating all curvature values in the order of the coordinate system, a column vector with a fixed dimension can be obtained, which is denoted as the curvature space descriptor, and corresponding descriptors can be constructed for all voxels. In some alternative embodiments, the neighborhood space range of the voxel is selected according to the scale of the point cloud and the voxel size, and is usually symmetric, generally 5×5×5 or 7×7×7.

[0060] Step S2: Input the curvature space descriptor into a pre-trained multi-layer perceptron network feature classifier to output the geometric feature types corresponding to each voxel, and the geometric feature types include corner points, straight lines, and planes.

[0061] As Figure 2 shown, a shallow multi-layer perceptron can be used as the feature recognizer in this step, or a feature recognizer with deep learning capabilities can also be used. In addition, it should be noted that in order to obtain actual geometric features, a confidence threshold τ for the output of the classifier can be defined. If the probability of the classifier outputting the corresponding feature type is higher than this threshold, the current classification result is credible.

[0062] Step S3: Determine the geometric features and model parameters of each geometric feature type according to the original point cloud.

[0063] This step is directed to three geometric types, namely corner points, straight lines, and planes, and determines their geometric features based on the original point cloud data respectively. In some alternative embodiments, step S3 further includes:

[0064] When the geometric feature type is a corner point, the average value of the original point cloud within the voxel corresponding to the corner point is used as the geometric feature of the corner point;

[0065] When the geometric feature type is a straight line, the voxels corresponding to multiple straight lines are clustered, and for each group of voxels after clustering, the direction and endpoint coordinates of the corresponding straight line are calculated and integrated based on the point clouds within each group of voxels;

[0066] When the geometric feature type is a plane, the voxels corresponding to multiple planes are clustered, and for each group of voxels after clustering, the parametric equations of the corresponding plane are fitted based on the point clouds within each group of voxels.

[0067] In the above embodiments, the corner point feature directly uses the mean value of the point cloud within the voxel as the coordinate feature information. For the straight line and plane features, the random sample consensus method is used to integrate the straight line and plane voxels with similar attributes and separately integrate them into independent instance features, that is, the straight line voxels and plane voxels are separated individually, and the voxels with similar attributes are clustered using the idea of random sample consensus. For the clustered straight line voxels, the direction and endpoint coordinates of the integrated straight line are calculated respectively, and the SVD method and the principal component analysis method can be used; for the clustered plane voxels, the parametric equations of the plane are fitted using the corresponding original point cloud respectively, and specific algorithms can use the fitting variants of the random sample consensus model, such as MLESAC and M-SAC.

[0068] Step S4: For two frames of building scene point clouds to be matched, geometric feature matching is performed for different geometric feature types respectively.

[0069] In this application, through steps S1 - S3, the geometry within the building scene point cloud is accurately identified and feature calculations are performed. Then, in step S4, the two frames of point clouds P a and P b can be subjected to geometric feature matching calculations.

[0070] Step S5: Based on the matching results, transformation matrices for each geometric feature type are calculated, and the two frames of building scene point clouds are fused according to the transformation matrices.

[0071] In some alternative embodiments, step S5 further includes:

[0072] Step S51: Parameter sets related to the rotation matrix in the transformation matrix are respectively extracted from the two frames of building scene point clouds for corner points, straight lines, and plane features;

[0073] Step S52: Jointly construct a covariance matrix based on the parameter sets of corner points, lines, and planes;

[0074] Step S53: Solve for the rotation matrix according to the covariance matrix decomposition;

[0075] Step S54: Calculate the translation matrix based on the least squares method according to the rotation matrix.

[0076] In step S51, for the corner point feature, the corner point feature sets of two frames of building scene point clouds P a and P b are defined as p a and p b , and the corresponding centroid coordinates are respectively represented as and Calculate the decentralized corner point sets as and Derive the rotation equation for corner points as where n c represents the number of all corresponding corner point features. It can be seen that the parameter set of the rotation matrix R in the transformation matrix is the corner point sets and Here, it is briefly denoted as K a and K b .

[0077] Similarly, for the line feature, the rotation matrix is generated from the direction vector a in the coordinate system corresponding to the point cloud P and the direction vector b in the coordinate system corresponding to the point cloud P where n e represents the number of line matches. Derive the rotation equation for line features as where n l represents the number of all corresponding line features. It can be seen that the parameter set of the rotation matrix R in the transformation matrix is the above two line sets, here briefly denoted as W a and W b . For the plane feature, the mathematical model of the plane features of the point clouds P a and P b is Pl: a i x + b i y + c i z + d i = 0, where is the normal vector of the plane, and d is the truncation parameter of the plane equation. The rotation matrix R is composed of the normal vector sets a of the plane features in the coordinate system of the point cloud P and the normal vector sets b of the plane features in the coordinate system of the point cloud P Generate between. Derive the rotation equation for planar features: where n pl represents the number of plane correspondences. It can be seen that the parameter set of the rotation matrix R in the transformation matrix is the above two plane sets, here briefly denoted as O a and O b .

[0078] After that, in step S52, the constructed covariance matrix is expressed as:

[0079]

[0080] Step S53 is used to solve the rotation matrix. The specific steps are that the covariance matrix S can be subjected to matrix singular value decomposition S = U∑V T , and the rotation matrix variable is derived as R = UV T .

[0081] Finally, in step S54, the translation matrix is calculated. Specifically, the corner feature constrains the translation matrix equation as K b = T + R·K a , and the line feature is constrained by the three-dimensional collinearity principle , and the registration equation for each group of line features can be derived as Denote all the correct matching line feature parameters Since the endpoints of the line feature can simultaneously constrain rotation and translation Substitute it into the registration equation and abbreviate it as is the set of endpoints g of the line feature. The planar feature's constraint equations for rotation and translation are The registration constraint equation for the planar feature is derived as: O a ·R·T = d b -d b .

[0082] The equation for solving the registration transformation translation matrix by fusing corner, line, and planar features using the linear fusion method is:

[0083]

[0084] where I is the identity matrix with the same dimension as the matching corner features.

[0085] The above embodiments infer a registration solution model that integrates all features based on the least squares idea. This idea can flexibly adapt to different types and quantities of feature combinations, even those containing only one or two feature types. During the implementation process, since the geometric features (points, lines, and planes) contained in the large-scale point clouds in the construction field are usually rich and intuitive, it is relatively easy to match a large number of point-line-plane geometric features. This module can adapt to different types and quantities of matching features, effectively improve the accuracy of point cloud registration by leveraging the advantages of statistics, and the non-iterative method can also improve the registration efficiency.

[0086] The multi-geometric feature extraction and synchronous registration method used in this application has the following advantages:

[0087] 1) A feature classifier constructed using a multi-layer perceptron based only on the CPU platform can extract corner points, straight lines, and plane features in the building scene at one time through simple calculations.

[0088] 2) The rich multi-geometric features extracted are sufficient to support feature association and subsequent registration solution. There is no need to attach additional marker points to the point cloud, and effective feature information can be extracted only from the point cloud itself.

[0089] 3) Synchronous registration can combine the advantages of the types and quantities included in multiple features, significantly improving the registration accuracy of the point cloud.

[0090] 4) The proposed multi-feature extraction and registration method is especially suitable for large-scale building point clouds and can significantly improve the registration efficiency of large-scale point clouds.

[0091] The second aspect of this application provides a building scene point cloud registration device corresponding to the above method, mainly including:

[0092] A local curvature space descriptor generation module for dividing the building scene point cloud into multiple voxels and determining the curvature space descriptor of each voxel. The curvature space descriptor refers to the ordered set of the curvature of the voxel and the curvatures of multiple surrounding voxels.

[0093] A multi-feature extraction module for inputting the curvature space descriptor into a pre-trained multi-layer perceptron network feature classifier and outputting the geometric feature types corresponding to each voxel. The geometric feature types include corner points, straight lines, and planes.

[0094] A geometric feature calculation module for determining the geometric features and model parameters of each geometric feature type according to the original point cloud.

[0095] A geometric feature matching module for performing geometric feature matching on different geometric feature types for two frames of building scene point clouds to be matched.

[0096] A multi-type feature synchronous registration module is used to calculate the transformation matrix of each geometric feature type based on the matching result, and fuse the two-frame building scene point clouds according to the transformation matrix.

[0097] In some alternative embodiments, the local curvature space descriptor generation module includes:

[0098] A voxel curvature calculation unit is used to perform voxelization operation with a fixed size on the building scene point cloud. When the number of original point clouds contained in a voxel with a fixed size exceeds the threshold, the principal component analysis method is used to calculate the curvature of the voxel. When the number of original point clouds contained in a voxel with a fixed size does not exceed the threshold and is greater than 0, the curvature value of the voxel is assigned 0. When the number of original point clouds in a voxel with a fixed size is 0, the curvature of the voxel is assigned -1;

[0099] A curvature space descriptor calculation unit is used for each voxel. Taking it as the central voxel, other voxels within the range of extending two steps along the directions of the three mutually orthogonal coordinate axes in three-dimensional space are used as neighboring voxels. The curvatures of the central voxel and all neighboring voxels are concatenated in the order of the coordinate system to form a column vector containing a fixed dimension, and this column vector is denoted as the curvature space descriptor of the central voxel.

[0100] In some alternative embodiments, in the voxel curvature calculation unit, the fixed size is 0.1m - 0.5m, and the threshold is not less than 3.

[0101] In some alternative embodiments, the geometric feature calculation module includes:

[0102] A corner geometric feature calculation unit is used when the geometric feature type is a corner, and the average value of the original point clouds in the voxel corresponding to the corner is used as the geometric feature of the corner;

[0103] A line geometric feature calculation unit is used when the geometric feature type is a line. Cluster the voxels corresponding to multiple lines, and for each group of clustered voxels, calculate and integrate the direction and endpoint coordinates of the corresponding line according to the point clouds in each group of voxels;

[0104] A plane geometric feature calculation unit is used when the geometric feature type is a plane. Cluster the voxels corresponding to multiple planes, and for each group of clustered voxels, fit the parametric equation of the corresponding plane according to the point clouds in each group of voxels.

[0105] In some alternative embodiments, the multi-type feature synchronous registration module includes:

[0106] A parameter set extraction unit is used to extract the parameter sets related to the rotation matrix in the transformation matrix from the corner, line, and plane features in two frames of building scene point clouds respectively;

[0107] A covariance matrix construction unit for jointly constructing a covariance matrix based on a parameter set of corner points, straight lines, and planes;

[0108] A rotation matrix solving unit for solving a rotation matrix by decomposing the covariance matrix;

[0109] A translation matrix solving unit for calculating a translation matrix based on the least squares method according to the rotation matrix.

[0110] In a third aspect of the present application, a computer device includes a processor, a memory, and a computer program stored on the memory and executable on the processor. The processor executes the computer program to implement the building scene point cloud registration method as described above.

[0111] In a fourth aspect of the present application, a readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the building scene point cloud registration method as described above. The computer-readable storage medium may be included in the device described in the above embodiments; or it may exist alone without being assembled into the device. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed by the device, the data is processed according to the above method.

[0112] The computer program for running the building scene point cloud registration method of the present application may be set on the mobile robot chip or on a computer device remotely connected to the mobile robot. When it is installed on the remote computer device, refer to Figure 3 , which shows a schematic structural diagram of a computer device 400 suitable for implementing the embodiments of the present application. Figure 3 The shown computer device is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0113] As Figure 3 shown, the computer device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage section 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0114] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as required. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 410 as required so that a computer program read therefrom is installed into the storage section 408 as required.

[0115] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0117] The modules or units described in the embodiments of the present application can be implemented in software or in hardware. The described modules or units can also be provided in a processor, and the names of these modules or units do not in some cases constitute a limitation to the modules or units themselves.

[0118] As described above, the foregoing are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for registering point clouds in a building scene, characterized in that, Including: Step S1: Divide the building scene point cloud into multiple voxels, and determine the curvature space descriptor of each voxel. The curvature space descriptor refers to an ordered set of the curvature of the voxel and the curvatures of multiple voxels around the voxel; Step S2: Input the curvature space descriptor into a pre-trained multi-layer perceptron network feature classifier, and output the geometric feature types corresponding to each voxel. The geometric feature types include corner points, straight lines, and planes; Step S3: Determine the geometric features and model parameters of each geometric feature type according to the original point cloud; Step S4: For two frames of building scene point clouds to be matched, perform geometric feature matching for different geometric feature types respectively; Step S5: Calculate the transformation matrix of each geometric feature type based on the matching results, and fuse the two frames of building scene point clouds according to the transformation matrix.

2. The method for registering point clouds of a building scene according to claim 1, wherein Step S1 further includes: Step S11: Perform voxelization operation with a fixed size on the building scene point cloud. When the number of original point clouds contained in the voxel with a fixed size exceeds the threshold, use the principal component analysis method to calculate the curvature of the voxel. When the number of original point clouds contained in the voxel with a fixed size does not exceed the threshold and is greater than 0, assign the curvature value of the voxel to 0. When the number of original point clouds in the voxel with a fixed size is 0, assign the curvature of the voxel to -1; Step S12: For each voxel, take it as the central voxel, and take the other voxels within the range of extending two steps along the mutually orthogonal coordinate axes in the three-dimensional space as neighborhood voxels. Concatenate all the curvatures of the central voxel and each neighborhood voxel in the order of the coordinate system to form a column vector with a fixed dimension, and record this column vector as the curvature space descriptor of the central voxel.

3. The method for registering point clouds of a building scene according to claim 2, wherein, In step S11, the fixed size is 0.1m - 0.5m, and the threshold is not less than 3.

4. The method for registering point clouds of a building scene according to claim 1, wherein, Step S3 further includes: When the geometric feature type is a corner point, take the average value of the original point clouds in the voxel corresponding to the corner point as the geometric feature of the corner point; When the geometric feature type is a straight line, cluster the voxels corresponding to multiple straight lines. For each group of voxels after clustering, calculate and integrate the direction and endpoint coordinates of the corresponding straight line according to the points in each group of voxels respectively; When the geometric feature type is a plane, cluster the voxels corresponding to multiple planes. For each group of voxels after clustering, fit the parametric equation of the corresponding plane according to the points in each group of voxels respectively.

5. The method for registering point clouds of a building scene according to claim 1, characterized in that, Step S5 further includes: Step S51: Extract the parameter set of the rotation matrix in the transformation matrix from the two frames of building scene point clouds for corner points, straight lines, and plane features respectively; Step S52: Jointly construct a covariance matrix based on the parameter sets of corner points, straight lines, and planes; Step S53: Solve the rotation matrix by decomposing the covariance matrix; Step S54: Calculate the translation matrix based on the least squares method according to the rotation matrix.

6. An architectural scene point cloud registration device, characterized in that, Including: Local curvature space descriptor generation module, which is used to divide the building scene point cloud into multiple voxels, and determine the curvature space descriptor of each voxel. The curvature space descriptor refers to an ordered set of the curvature of the voxel and the curvatures of multiple voxels around the voxel; A multi-feature extraction module, which is used to input the curvature space descriptor into a pre-trained multi-layer perceptron network feature classifier, and output the geometric feature types corresponding to each voxel, where the geometric feature types include corner points, straight lines, and planes; A geometric feature calculation module, which is used to determine the geometric features and model parameters of each geometric feature type according to the original point cloud; A geometric feature matching module, which is used to perform geometric feature matching on two frames of building scene point clouds to be matched for different geometric feature types; A multi-type feature synchronous registration module, which is used to calculate the transformation matrix of each geometric feature type based on the matching results, and fuse the two frames of building scene point clouds according to the transformation matrix.

7. The building scene point cloud registration device according to claim 6, characterized in that, The local curvature space descriptor generation module includes: A voxel curvature calculation unit, which is used to perform voxelization operation with a fixed size on the building scene point cloud. When the number of original point clouds contained in a voxel with a fixed size exceeds the threshold, the principal component analysis method is used to calculate the curvature of the voxel. When the number of original point clouds contained in a voxel with a fixed size does not exceed the threshold and is greater than 0, the curvature value of the voxel is assigned 0. When the number of original point clouds in a voxel with a fixed size is 0, the curvature of the voxel is assigned -1; A curvature space descriptor calculation unit, which is used for each voxel. Taking it as the central voxel, the other voxels within the range of extending two steps along the three mutually orthogonal coordinate axes in the three-dimensional space are used as neighborhood voxels. The curvatures of the central voxel and all neighborhood voxels are concatenated in the order of the coordinate system to form a column vector with a fixed dimension, and this column vector is recorded as the curvature space descriptor of the central voxel.

8. The building scene point cloud registration device according to claim 7, characterized in that, In the voxel curvature calculation unit, the fixed size is 0.1m - 0.5m, and the threshold is not less than 3.

9. The building scene point cloud registration device according to claim 6, characterized in that, The geometric feature calculation module includes: A corner point geometric feature calculation unit, which is used to, when the geometric feature type is a corner point, take the average value of the original point cloud in the voxel corresponding to the corner point as the geometric feature of the corner point; A straight line geometric feature calculation unit, which is used to, when the geometric feature type is a straight line, cluster the voxels corresponding to multiple straight lines. For the clustered multiple groups of voxels, the direction and endpoint coordinates of the corresponding straight line are calculated and integrated according to the points in each group of voxels respectively; A plane geometric feature calculation unit, which is used to, when the geometric feature type is a plane, cluster the voxels corresponding to multiple planes. For the clustered multiple groups of voxels, the parametric equations of the corresponding planes are fitted according to the point clouds in each group of voxels respectively.

10. The building scene point cloud registration device according to claim 6, characterized in that, The multi-type feature synchronous registration module includes: A parameter set extraction unit, which is used to extract the parameter sets of the rotation matrix in the transformation matrix involved in the corner point, straight line, and plane features from two frames of building scene point clouds respectively; A covariance matrix construction unit, which is used to jointly construct a covariance matrix based on the parameter sets of the corner point, straight line, and plane; A rotation matrix solving unit, which is used to solve the rotation matrix by decomposing the covariance matrix; A translation matrix solving unit, which is used to calculate the translation matrix based on the least squares method according to the rotation matrix.

Images