Three-dimensional point cloud registration method, device and electronic equipment

Abstract

This disclosure provides a three-dimensional point cloud registration method, apparatus, and electronic device. The method includes: extracting at least one first plane from a first three-dimensional point cloud; extracting at least one second plane from a second three-dimensional point cloud; obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; matching the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that satisfies preset conditions; calculating transformation parameters in a four-dimensional planar space according to the plane matching pair, the transformation parameters representing parameters related to registering the first plane and the second plane; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the transformation parameters.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and more specifically, to a three-dimensional point cloud registration method, apparatus, and electronic device. Background Technology

[0002] With the development of information technology and high-precision sensors, point cloud images have gradually become the preferred format for representing the three-dimensional world. However, due to the limitations of sensor perspective, a directly acquired point cloud image often only contains data from a portion of the scene. Therefore, to obtain a complete 3D scene, point cloud registration technology is needed to register and fuse multiple acquired point cloud images. Point cloud registration technology studies the transformation estimation problem between two or more point cloud images. First, a transformation matrix is ​​obtained through feature matching, and then the transformation matrix is ​​used to merge partial point cloud data of the same scene into a complete scene. It has important applications in fields such as 3D surface reconstruction, autonomous driving, industrial robots, and mixed reality.

[0003] Traditional 3D point cloud registration techniques have achieved remarkable results, mostly based on local feature matching and then restoring rigid transformation parameters to minimize the geometric projection error between two point clouds.

[0004] These traditional 3D point cloud registration methods extract key points and local features within the 3D point neighborhood. While they achieve good results on dense and clean point clouds, they still have significant limitations. First, they are difficult to scale to large-scale scenes, mainly because computing the matching relationships of point pairs in large-scale scenes is extremely costly, and excessive point cloud sampling cannot guarantee the accuracy of the results. Second, these methods are highly sensitive to noise, outliers, uneven density, or missing data, easily leading to a large number of false matches.

[0005] In the field of point cloud registration, deep learning-based point cloud registration algorithms can be divided into two main categories: feature-based methods and end-to-end methods. The former aims to establish a reliable deep feature matching relationship and then obtain the transformation matrix based on the traditional pose optimization algorithm. The latter implements the process of solving the differentiable transformation matrix using a deep neural network, thereby directly restoring the transformation matrix.

[0006] Therefore, how to provide a method to solve the above problems (different data sources, densities, sampling and various transformations) is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, this disclosure provides a three-dimensional point cloud registration method, apparatus and electronic device.

[0008] One aspect of this disclosure provides a three-dimensional point cloud registration method, comprising: extracting at least one first plane from a first three-dimensional point cloud; extracting at least one second plane from a second three-dimensional point cloud; obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; matching the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that satisfies preset conditions; calculating a transformation parameter in a four-dimensional planar space according to the plane matching pair, the transformation parameter representing a parameter related to registering the first plane and the second plane; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the transformation parameter.

[0009] Another aspect of this disclosure provides a three-dimensional point cloud registration apparatus, comprising: a first plane extraction module for extracting at least one first plane from a first three-dimensional point cloud; a second plane extraction module for extracting at least one second plane from a second three-dimensional point cloud; a plane feature histogram acquisition module for acquiring a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; a matching module for matching the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that satisfies preset conditions; a transformation parameter calculation module for calculating transformation parameters in a four-dimensional planar space according to the plane matching pair, the transformation parameters representing parameters related to registering the first plane and the second plane; and a registration module for registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the transformation parameters.

[0010] Another aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the three-dimensional point cloud registration method of this disclosure.

[0011] According to embodiments of this disclosure, a novel registration framework utilizing planar structures is proposed for the registration and fusion of 3D point clouds. This method employs the following techniques: extracting at least one first plane from a first 3D point cloud; extracting at least one second plane from a second 3D point cloud; obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; matching the first plane and the second plane based on the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets preset conditions; calculating transformation parameters in a four-dimensional planar space based on the plane matching pair, where the transformation parameters characterize parameters related to registering the first plane and the second plane; and registering the first 3D point cloud and the second 3D point cloud based on the transformation parameters. This method is universal and can solve the registration problem in 3D point space under different data sources, densities, and various transformations, and can achieve efficient registration. Attached Figure Description

[0012] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0013] Figure 1 This illustration schematically shows an exemplary system architecture to which a three-dimensional point cloud registration method can be applied according to embodiments of the present disclosure;

[0014] Figure 2 A flowchart illustrating a three-dimensional point cloud registration method according to an embodiment of the present disclosure is shown schematically.

[0015] Figure 3 A schematic diagram illustrating the construction of a descriptor according to an embodiment of the present disclosure is shown.

[0016] Figure 4 This schematically illustrates an overall flowchart of a three-dimensional point cloud registration method according to an embodiment of the present disclosure;

[0017] Figure 5 The illustration schematically shows the visual comparison results of registration of two partial visibility scenes based on different methods according to embodiments of the present disclosure;

[0018] Figure 6A The illustration schematically shows the visualization result of indoor scene point cloud registration implemented according to the three-dimensional point cloud registration method according to an embodiment of the present disclosure;

[0019] Figure 6B The illustration schematically shows the visualization result of point cloud registration of an outdoor scene implemented according to the three-dimensional point cloud registration method of the present disclosure;

[0020] Figure 7A The illustration shows recall curves for different inter-point overlap rates using a three-dimensional point cloud registration method according to embodiments of the present disclosure.

[0021] Figure 7B The illustration schematically shows recall curves for different planar overlap rates using a three-dimensional point cloud registration method according to embodiments of the present disclosure.

[0022] Figure 8 A block diagram of a three-dimensional point cloud registration apparatus according to an embodiment of the present disclosure is illustrated schematically; and

[0023] Figure 9 A block diagram of an electronic device suitable for implementing a three-dimensional point cloud registration method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0024] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] In the embodiments disclosed herein, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0029] In the embodiments disclosed herein, user authorization or consent is obtained before acquiring or collecting user personal information.

[0030] Figure 1 An exemplary system architecture 100 according to embodiments of the present disclosure, to which a 3D point cloud registration method can be applied, is illustrated schematically. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0034] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0035] It should be noted that the 3D point cloud registration method provided in this embodiment can generally be executed by server 105. Correspondingly, the 3D point cloud registration device provided in this embodiment can generally be located in server 105. The 3D point cloud registration method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the 3D point cloud registration device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Alternatively, the 3D point cloud registration method provided in this embodiment can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the three-dimensional point cloud registration device provided in this embodiment can also be disposed in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or disposed in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0036] For example, the first and second 3D point clouds can be originally stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (e.g., the first terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into the first terminal device 101. Then, the first terminal device 101 can execute the 3D point cloud registration method provided in the embodiments of this disclosure locally, or send the first and second 3D point clouds to other terminal devices, servers, or server clusters, and have the other terminal devices, servers, or server clusters that receive the first and second 3D point clouds execute the 3D point cloud registration method provided in the embodiments of this disclosure.

[0037] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0038] Figure 2 A flowchart illustrating a three-dimensional point cloud registration method according to an embodiment of the present disclosure is shown.

[0039] like Figure 2 As shown, the method includes operations S201 to S206.

[0040] In operation S201, at least one first plane is extracted from the first three-dimensional point cloud.

[0041] In operation S202, at least one second plane is extracted from the second three-dimensional point cloud.

[0042] According to embodiments of this disclosure, the first 3D point cloud and the second 3D point cloud can represent two 3D point clouds sampled for the same object or different objects. The first 3D point cloud and the second 3D point cloud can include two overlapping 3D point clouds. For example, the first 3D point cloud and the second 3D point cloud can represent two 3D point clouds sampled for the same object from different viewpoints. The first 3D point cloud and the second 3D point cloud can also include two adjacent but non-overlapping 3D point clouds sampled from two shapes that need to be stitched together. For example, the first 3D point cloud and the second 3D point cloud can represent two 3D point clouds sampled for two objects that need to be stitched together. No limitation is made herein.

[0043] According to embodiments of this disclosure, after sampling to obtain a first three-dimensional point cloud and a second three-dimensional point cloud, a primitive detection algorithm based on RANSAC (Random Sample Consensus) can be used to extract planes from the two three-dimensional point clouds respectively to obtain a first plane and a second plane.

[0044] It should be noted that the above method for extracting a plane is only an exemplary embodiment, but is not limited thereto. It may also include other plane extraction or generation methods known in the art, as long as a four-dimensional plane can be extracted from a three-dimensional point cloud.

[0045] In operation S203, the first plane feature histogram of the first plane and the second plane feature histogram of the second plane are obtained.

[0046] According to embodiments of this disclosure, in order to obtain transformation parameters, a robust and compact planar feature descriptor PLFH (Plane Feature Histogram) can be introduced to establish the correspondence between planes.

[0047] For example, in an optional embodiment, a planar feature histogram of a plane can be obtained by observing the representation of feature points in the focused point cloud on the corresponding plane. For example, points in the first 3D point cloud can be mapped to a first plane first, and then features can be extracted from the first plane to obtain a first planar feature histogram.

[0048] It should be noted that the methods for determining the planar feature histogram are not limited to those described above.

[0049] In operation S204, the first plane and the second plane are matched according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets the preset conditions.

[0050] According to embodiments of this disclosure, for example, a RANSAC iterative process can be applied to find matching planes between a first 3D point cloud and a second 3D point cloud to obtain a plane matching pair.

[0051] According to embodiments of this disclosure, the preset conditions may include, but are not limited to, the similarity between the first planar feature histogram and the second planar feature histogram being greater than a preset threshold, which is applicable to the matching process of two planes extracted from two three-dimensional point clouds sampled from different angles for the same object.

[0052] In operation S205, transformation parameters are calculated in four-dimensional planar space based on the plane matching pair. The transformation parameters characterize the parameters related to registering the first plane and the second plane.

[0053] According to embodiments of this disclosure, the optimal transformation can be estimated in a four-dimensional planar space using two planes, a first plane and a second plane, as input. The core idea is to directly obtain the transformation parameters using planar parameters (A, B, C, D) through rigorous mathematical derivation.

[0054] According to embodiments of this disclosure, the transformation parameters may include at least one of translation parameters, rotation parameters, etc., and may not be limited thereto.

[0055] It should be noted that the method for calculating the transformation parameters in a four-dimensional planar space based on the planar matching pairs is not limited to the above-mentioned method. It may also include other calculation methods known in the art. For example, a calculation model can be trained to input the first and second planar feature histograms of the planar matching pairs into the training model and output the transformation parameters, etc., and it is not limited to this.

[0056] In operation S206, the first 3D point cloud and the second 3D point cloud are registered according to the conversion parameters.

[0057] According to embodiments of this disclosure, given two points respectively from a first three-dimensional point cloud Second and third-dimensional point clouds The first set of planes extracted respectively Second plane set First plane set It may include at least one first plane Second plane set It may include at least one second plane A plane with parameters (A, B, C, D) can be... ( Characterization or The expression is: Ax + By + Cz + D = 0, where ( (where the number is a real number), and It can make a plane The points in the 3D point cloud are represented by homogeneous coordinates (x, y, z, 1). To the second 3D point cloud The transformation process can be represented by formula (1).

[0058] Formula (1)

[0059] In formula (1), For the first three-dimensional point cloud The matrix representation, For the second three-dimensional point cloud The matrix representation, Let represent the transformation matrix, where Indicates rotation parameters, This represents the translation parameter.

[0060] Eliminating the homogeneous coordinates of the points from formula (1) yields formula (2), which represents the state from the first plane. Transform to the second plane The transformation equation.

[0061] Formula (2)

[0062] In formula (2), Represents the first plane Planar parameters, Represents the second plane Planar parameters.

[0063] Formulas (1) and (2) above show that the transformation relationship of point cloud is inverse to the transformation relationship of plane.

[0064] According to embodiments of this disclosure, in practical operation, the transformation matrix can be solved by defining plane parameters. For example, the target point cloud can be directly extracted into a plane set by combining formula (2). Transform to extract plane set from source point cloud The transformation parameters that can be used to register the first 3D point cloud and the second 3D point cloud are obtained by solving.

[0065] According to embodiments of this disclosure, after obtaining the transformation parameters, the second three-dimensional point cloud can be subjected to rotation and translation transformations based on the transformation parameters to obtain the transformed three-dimensional point cloud, so as to register the first three-dimensional point cloud and the transformed three-dimensional point cloud, thereby achieving the registration of the first three-dimensional point cloud and the second three-dimensional point cloud.

[0066] Through the above embodiments of this disclosure, a novel registration framework using planar structures is proposed for the registration and fusion of 3D point clouds. This method is universal and can solve the registration problem in 3D point space with different data sources, densities, and various transformations, and can achieve efficient registration.

[0067] The following describes specific embodiments. Figure 2 The method shown will be further explained.

[0068] Based on the aforementioned method, at least four pairs of matching planes are required to calculate the transformation parameters. In realizing this disclosed concept, the inventors discovered that directly applying the RANSAC iterative process to find matching planes between the source and target point clouds would require multiple iterations, and due to the stochastic nature of RANSAC, the matching results are uncertain.

[0069] To address this problem, this disclosure constructs a simple yet efficient descriptor based on planar relations, which is invariant to rigid transformations. This descriptor can be constructed based on statistical information from at least one of the distributions of angles and distances between the planes.

[0070] According to embodiments of this disclosure, there can be multiple first planes and multiple second planes. When the descriptor is constructed based on statistical information about angles between planes, the above operation S203 may include: for each selected first plane among the plurality of first planes, calculating the angle between the selected first plane and other first planes to form a first plane feature histogram of the selected first plane, wherein the other first planes represent other first planes among the plurality of first planes besides the selected first plane. For each selected second plane among the plurality of second planes, calculating the angle between the selected second plane and other second planes to form a second plane feature histogram of the selected second plane, wherein the other second planes represent other second planes among the plurality of second planes besides the selected second plane.

[0071] For example, for For each plane m in the cloud, its relationship with all other planes in the same point cloud can be calculated. A set of angles between . This represents the total number of planes (such as the first plane or the second plane) extracted from a 3D point cloud (such as the first 3D point cloud or the second 3D point cloud) minus 1. Represents the plane m and the first The angle between the planes.

[0072] According to embodiments of this disclosure, when the descriptor is constructed based on statistical information of distance distribution, the above operation S203 may include: obtaining a first fixed point determined for a first plane and a second fixed point corresponding to a second plane. The first fixed point and the second fixed point have the same position coordinates in the world coordinate system. Based on the position information of the first fixed point in the first coordinate system corresponding to the first 3D point cloud, a first relative distance between the first fixed point and a first intersection line is calculated, constituting a first plane feature histogram of the selected first plane. Other first planes represent other first planes among a plurality of first planes besides the selected first plane. The first intersection line represents the intersection line determined by the selected first plane and other first planes. Based on the position information of the second fixed point in the second coordinate system corresponding to the second 3D point cloud, a second relative distance between the second fixed point and a second intersection line is calculated, constituting a second plane feature histogram of the selected second plane. Other second planes represent other second planes among a plurality of second planes besides the selected second plane. The second intersection line represents the intersection line determined by the selected second plane and other second planes.

[0073] For example, for For each plane m in the equation, the distance from a fixed point P to m can be calculated. Distance between intersecting lines . Represents the relationship between a fixed point P and a plane m. The distance between the intersecting lines.

[0074] According to embodiments of this disclosure, the angle can also be calculated first. and distance Then, adjust the angle. and distance The data is compiled into two histograms. Specifically, the data can be divided into sub-partitions based on the range of values, and the number of features in each sub-partition can be calculated. Finally, a descriptor is returned by concatenating the two histograms together. Alternatively, a planar feature histogram of a plane in a given point cloud can be constructed.

[0075] According to embodiments of this disclosure, the selection of matching points can be achieved by randomly selecting a point in the world coordinate system to determine the first and second points. Alternatively, conventional point cloud feature descriptors (FPFH, FastPoint Feature Histograms) can be used to identify the best matching point pair to determine the first and second points. By selecting matching points, the distance consistency between the first and second 3D point clouds can be ensured when calculating the distance descriptor.

[0076] Figure 3 A schematic diagram illustrating a construction descriptor according to an embodiment of the present disclosure is shown.

[0077] like Figure 3 As shown, for example, a plane can be extracted from a 3D point cloud. . The plane The selected plane has been determined. The selected plane can be obtained through calculation. Other planes Each of their included angles .flat Other planes Intersecting lines can be formed By calculating the points P to the intersecting lines... The distance can be obtained from the distance. After that, the included angle can be... By analyzing the angle histogram, the distance can be... Statistics were compiled into a distance histogram.

[0078] Through the above embodiments of this disclosure, a simple yet efficient descriptor based on planar relationships is constructed, which statistically calculates the angles between planes and / or the distances from a fixed point to the intersection line of the planes, and possesses rotation and translation invariance. Compared with other registration methods that rely on combined planar feature descriptors, PLFH has significant advantages in terms of high efficiency and robustness. In terms of efficiency, due to the single traversal of each plane in the plane set, each plane only needs to be traversed once during the construction process, and all other planes extracted from the same point cloud will be directly used to calculate PLFH. The time complexity of PLFH construction is O(n^2). 2 Its construction process is extremely efficient. In terms of robustness, the descriptor is based on statistics of angles and distances, so it is more robust to noise and errors in plane extraction.

[0079] After obtaining the descriptor PLFH, the next step is to select a technique for calculating the transformation between the two point clouds, among which obtaining a reliable set of planar correspondences is particularly crucial. Although the number of planes is much smaller than the number of points, finding the correct planar correspondences through brute-force enumeration is still not easy, because the number of possible permutations grows exponentially.

[0080] This disclosure enables the robust RANSAC algorithm to be transferred to a four-dimensional planar space for efficient planar matching. Unlike previous methods that applied RANSAC to raw point clouds and thus converged slowly, the method disclosed in this disclosure significantly reduces the search space of RANSAC.

[0081] According to an embodiment of this disclosure, operation S204 may include: calculating a first plane distance between the first plane and the second plane based on a first plane feature histogram and a second plane feature histogram, to obtain at least one first plane distance. Based on the at least one first plane distance, determining a target second plane from at least one second plane that has the closest first plane distance to the target first plane. If the first plane determined from at least one first plane that has the closest first plane distance to the second plane is the target first plane, the target first plane and the target second plane are determined as a plane matching pair.

[0082] For example, in each iteration of RANSAC, four planes can be randomly selected from the first 3D point cloud as an initial set, and the closest corresponding plane in the target point cloud can be searched based on the similarity between their PLFH descriptors. When a pair of planes... A bilateral match is considered valid if it satisfies the closest distance represented by both formulas (3) and (4). This represents the first plane in the first three-dimensional point cloud. This represents the second plane in the second three-dimensional point cloud.

[0083] Formula (3)

[0084] Formula (4)

[0085] In formulas (3) and (4), A PLFH descriptor can be used to represent a plane m.

[0086] According to embodiments of this disclosure, a plane matching pair may include a target first plane and a target second plane. During the calculation of transformation parameters, a first plane normal matrix of the target first plane and a second plane normal matrix of the target second plane can be obtained first. This method may include: extracting the first plane normal matrix of the target first plane based on the first plane equation of the target first plane; and extracting the second plane normal matrix of the target second plane based on the second plane equation of the target second plane.

[0087] For example, you can use It represents the plane normal and can simplify formula (2) to obtain formula (5).

[0088] Formula (5)

[0089] According to embodiments of this disclosure, the concept of point cloud normal relocation technology can be used to eliminate the uncertainty of planar normals. Specifically, after obtaining the first and second planar normal matrices, a first angle can be calculated between the line connecting a first fixed point determined for the first plane and any target point located on the first plane, and the first plane normal of the first plane. In response to determining that the first angle is greater than 90°, the first plane normal is reversed to obtain the first plane normal matrix. A second angle is calculated between the line connecting a second fixed point determined for the second plane and any target point located on the second plane, and the second plane normal of the second plane. In response to determining that the second angle is greater than 90°, the second plane normal is reversed to obtain the second plane normal matrix.

[0090] For example, for the first plane any point on Its plane normal can be calculated first by determining the relationship between fixed point P and point [missing information]. The angle between the line connecting the two planes and the plane normal. ,if Therefore, the plane normal is reversed to eliminate the uncertainty of the plane normal. This is for the second plane extracted from the second 3D point cloud. The same redirection strategy can be used.

[0091] According to an embodiment of this disclosure, after eliminating the uncertainty of the plane normal, the solution equation as shown in formula (5) can be transformed again into the form of formula (6).

[0092] Formula (6)

[0093] In formula (6), It can represent the normal matrix of a first plane. It can represent the normal matrix of a second plane. A constant term parameter of the first plane can be represented, denoted as the first constant term. It can represent a constant term parameter of a second plane, denoted as the second constant term.

[0094] According to embodiments of this disclosure, the transformation parameters may further include rotation parameters. The above operation S205 may include: inputting the first plane parameter matrix and the second plane parameter matrix into a rotation matrix solver to obtain the rotation parameters.

[0095] For example, the Kabsch algorithm (an algorithm for calculating the optimal rotation matrix) can be used to solve for the rotation matrix R = By using SVD (singular value decomposition) to find U and V, we can obtain formula (7).

[0096] Formula (7)

[0097] According to the embodiments of this disclosure, a rotation matrix solver can be constructed by combining formula (7) to solve the rotation parameters.

[0098] According to embodiments of this disclosure, the transformation parameters may further include translation parameters. The above operation S205 may further include: extracting a first constant term for the target first plane based on the first plane equation; extracting a second constant term for the target second plane based on the second plane equation; and inputting the first plane normal matrix, the second plane normal matrix, the first constant term, and the second constant term into a translation parameter solver to obtain the translation parameters.

[0099] For example, used to solve translation vectors The translation parameter solution can be achieved by minimizing the energy function shown in formula (8), and by applying... Find the partial derivative and make The program shown in formula (9) is obtained, and it is not limited to this.

[0100] Formula (8)

[0101] Formula (9)

[0102] In formula (9), It can represent a total matrix composed of the normal matrices of all second planes. It can represent a matrix consisting of the first constant terms of all the first planes. It can represent the matrix consisting of the second constant terms of all the second planes.

[0103] Through the above embodiments of this disclosure, the classic RANSAC algorithm based on the hypothesis-test framework is introduced into the four-dimensional planar space, which reduces the negative impact of planar mismatch while reducing the data scale, and realizes a high-precision and high-efficiency solution for three-dimensional point cloud registration.

[0104] According to embodiments of this disclosure, there may be multiple plane matching pairs. Each plane matching pair may include a target first plane and a target second plane. The transformation parameters may include multiple candidate transformation parameters. The above operation S206 may include: performing rotation and translation transformations on the target second plane in each plane matching pair according to each candidate transformation parameter to obtain a transformed plane; calculating the second plane distance between the target first plane and the transformed plane in each plane matching pair; obtaining the number of plane matching pairs whose second plane distances determined based on different candidate transformation parameters are less than a given threshold, thereby obtaining multiple plane matching pair numbers; determining the target plane matching pair number with the largest value from the multiple plane matching pair numbers; determining the candidate transformation parameters related to obtaining the target plane matching pair number as target transformation parameters; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the target transformation parameters.

[0105] For example, after obtaining the plane matching relationship, candidate transformation parameters can be calculated using the methods shown in formulas (7) and (9) above. By performing efficient plane-based RANSAC, different candidate transformation parameters are compared in the iterations, and the best parameter that makes the most planes match is selected. For example, this may include: calculating the distance between matching plane pairs, counting the number of plane pairs whose distance is less than a given threshold, and finally selecting the candidate transformation parameter that makes the most planes match as the target transformation parameter output.

[0106] According to embodiments of this disclosure, the formula for measuring the distance in the second plane is shown in formula (10), but is not limited thereto.

[0107] Formula (10)

[0108] In formula (10), It can represent candidate transformation parameters.

[0109] Figure 4 The diagram illustrates the overall flowchart of a three-dimensional point cloud registration method according to an embodiment of the present disclosure.

[0110] like Figure 4As shown, a first 3D point cloud 410 and a second 3D point cloud 420 can be used as inputs. Then, plane extraction can be performed on the first 3D point cloud 410 to obtain a first plane 411, and plane extraction can be performed on the second 3D point cloud 420 to obtain a second plane 421. Next, a first PLFH 413 of the first plane 411 can be calculated in a first four-dimensional planar space 412 constructed based on a first coordinate system used to determine the first 3D point cloud 410, and a second PLFH 423 of the second plane 421 can be calculated in a second four-dimensional planar space 422 constructed based on a second coordinate system used to determine the second 3D point cloud 420. Then, by performing efficient plane-based RANSAC on the first PLFH 413 and the second PLFH 423, and comparing different candidate transformation parameters in the iterations, the target transformation parameters can be obtained. Finally, combining the target transformation parameters The first 3D point cloud 410 and the second 3D point cloud 420 can be registered to obtain the registered point cloud 430.

[0111] Through the above embodiments of this disclosure, an efficient plane-based transformation parameter solution process is achieved. Including the initial matching of PLFH, the total time complexity of plane-based RANSAC is O(n). The registration process combines the robustness of RANSAC with reduced computational costs.

[0112] According to embodiments of this disclosure, the above-described point cloud registration method can be tested using the ScanNet dataset (a three-dimensional laser point cloud obtained through an RGBD camera) and a general CAD dataset (a three-dimensional point cloud) called ShapeNet Part (point cloud segmentation).

[0113] First, preprocessing of the ScanNet and ShapeNet Part datasets can include normalizing the point clouds to ensure their size remains constant at 1. Then, random rotations and translations can be performed, including randomly rotating and translating the points along each coordinate axis [0, ...]. Three angles are randomly sampled within the range of [−5, 5] and a rotation matrix is ​​finally synthesized. Translation vectors are randomly generated within the range of [−5, 5].

[0114] For the ScanNet dataset, you can choose to transform the low-resolution point cloud and use the transformed point cloud as the target point cloud. For the ShapeNet Part dataset, you can directly choose to copy the transformation.

[0115] The accuracy of the 3D point cloud registration method provided in this disclosure is evaluated through experiments. The experimental platform can be a computer with an Intel 6226R central processing unit, 256GB RAM and a 64-bit Ubuntu 20.04 operating system, and the PyTorch open-source deep learning framework can be used on the platform.

[0116] Table 1 shows the comparative experimental results on the original ScanNet and the sampled ScanNet. The comparison methods include commonly used point-based methods such as FGR, RANSAC, Super4PCS, PLADE, DCP, and HRegNet. These algorithms are mostly integrated into well-known open-source libraries, such as open3d and Meshlab. The best experimental results are marked with circles, and the second-best results are marked with rectangles.

[0117]

[0118] Table 2 shows the comparative experimental results of partial overlap on ScanNet.

[0119]

[0120] The results fully demonstrate the superior robustness and accuracy of this disclosure compared to existing methods.

[0121] To fully demonstrate the robustness of plane-based registration, this disclosure also tested experiments with different point cloud densities under different types of models.

[0122] Figure 5 The illustration schematically shows visual comparison results of registration of two partial visibility scenes based on different methods according to embodiments of the present disclosure.

[0123] like Figure 5 As shown, due to the lack of sufficient point correspondences in small overlap scenarios, methods such as RANSAC, Super4PCS, FGR, HRegNet, DCP, and PLADE perform poorly.

[0124] Figure 6A The illustration schematically shows the visualization result of indoor scene point cloud registration implemented according to the three-dimensional point cloud registration method of the present disclosure.

[0125] Figure 6B The illustration schematically shows the visualization result of outdoor scene point cloud registration implemented according to the three-dimensional point cloud registration method of the present disclosure.

[0126] like Figure 6A and Figure 6BAs shown, the left column represents the input, and the right column represents the registered output. The results fully demonstrate the superior robustness and accuracy of the 3D point cloud registration method disclosed in this paper compared to existing methods.

[0127] Figure 7A The illustration shows recall curves for different overlap rates between points using a three-dimensional point cloud registration method according to an embodiment of the present disclosure.

[0128] Figure 7B The illustration shows recall curves for different planar overlap rates using a three-dimensional point cloud registration method according to embodiments of the present disclosure.

[0129] like Figure 7A and Figure 7B As shown, recall curves can also be displayed for different numbers of fragments and different rotation and displacement thresholds on the same dataset. The results indicate that the 3D point cloud registration method of this disclosure has the highest recall rate across the entire range, especially far exceeding point feature-based methods under low overlap conditions.

[0130] By comparing the experimental results of the three-dimensional point cloud registration method disclosed herein with those of PLADE, it is demonstrated that the three-dimensional point cloud registration method disclosed herein is more robust under different plane overlap rates.

[0131] Figure 8 A block diagram of a three-dimensional point cloud registration apparatus according to an embodiment of the present disclosure is shown schematically.

[0132] like Figure 8 As shown, the 3D point cloud registration device 800 includes a first plane extraction module 810, a second plane extraction module 820, a plane feature histogram acquisition module 830, a matching module 840, a transformation parameter calculation module 850, and a registration module 860.

[0133] The first plane extraction module 810 is used to extract at least one first plane from the first three-dimensional point cloud.

[0134] The second plane extraction module 820 is used to extract at least one second plane from the second three-dimensional point cloud.

[0135] The planar feature histogram acquisition module 830 is used to acquire the first planar feature histogram of the first plane and the second planar feature histogram of the second plane.

[0136] The matching module 840 is used to match the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets the preset conditions.

[0137] The transformation parameter calculation module 850 is used to calculate transformation parameters in a four-dimensional planar space based on the planar matching pair. The transformation parameters characterize the parameters related to the registration of the first plane and the second plane.

[0138] The registration module 860 is used to register the first 3D point cloud and the second 3D point cloud according to the conversion parameters.

[0139] According to embodiments of this disclosure, both the first plane and the second plane have multiple components. The plane feature histogram acquisition module includes a first plane angle calculation unit and a second plane angle calculation unit.

[0140] The first plane angle calculation unit is used to calculate the first plane angle between the selected first plane and other first planes for each selected first plane among a plurality of first planes, thereby forming a first plane feature histogram of the selected first plane, wherein the other first planes represent other first planes among the plurality of first planes besides the selected first plane.

[0141] The second plane angle calculation unit is used to calculate the second plane angle between the selected second plane and other second planes for each selected second plane among a plurality of second planes, forming a second plane feature histogram of the selected second plane, wherein the other second planes represent other second planes among the plurality of second planes besides the selected second plane.

[0142] According to embodiments of this disclosure, both the first plane and the second plane have multiple instances. The plane feature histogram acquisition module further includes a fixed-point acquisition unit, a first relative distance calculation unit, and a second relative distance calculation unit.

[0143] The fixed-point acquisition unit is used to acquire a first fixed point determined for the first plane and a second fixed point corresponding to the second plane, wherein the first fixed point and the second fixed point have the same position coordinates in the world coordinate system.

[0144] The first relative distance calculation unit is used to calculate the first relative distance between the first fixed point and the first intersection line based on the position information of the first fixed point in the first coordinate system corresponding to the first three-dimensional point cloud, thereby forming a first planar feature histogram of the selected first plane. The other first planes represent other first planes among a plurality of first planes besides the selected first plane, and the first intersection line represents the intersection line determined by the selected first plane and the other first planes.

[0145] The second relative distance calculation unit is used to calculate the second relative distance between the second fixed point and the second intersection line based on the position information of the second fixed point in the second coordinate system corresponding to the second three-dimensional point cloud, thereby forming a second plane feature histogram of the selected second plane. The other second planes represent other second planes among a plurality of second planes besides the selected second plane, and the second intersection line represents the intersection line determined by the selected second plane and the other second planes.

[0146] According to embodiments of this disclosure, the matching module includes a first planar distance calculation unit, a target second plane determination unit, and a planar matching pair determination unit.

[0147] The first planar distance calculation unit is used to calculate the first planar distance between the first plane and the second plane based on the first planar feature histogram and the second planar feature histogram, so as to obtain at least one first planar distance.

[0148] The target second plane determination unit is used to determine, based on at least one first plane distance, the target second plane that is closest to the target first plane from at least one second plane.

[0149] A plane matching pair determination unit is used to determine a target first plane and a target second plane as a plane matching pair when the first plane that is closest to the first plane of the second plane determined from at least one first plane is the target first plane.

[0150] According to embodiments of this disclosure, the planar matching pair includes a target first plane and a target second plane. The transformation parameters include rotation parameters. The transformation parameter calculation module includes a first plane normal matrix extraction unit, a second plane normal matrix extraction unit, and a rotation parameter solving unit.

[0151] The first plane normal matrix extraction unit is used to extract the first plane normal matrix of the target first plane according to the first plane equation of the target first plane.

[0152] The second plane normal matrix extraction unit is used to extract the second plane normal matrix of the target second plane according to the second plane equation of the target second plane.

[0153] The rotation parameter solving unit is used to input the normal matrix of the first plane and the normal matrix of the second plane into the rotation matrix solving program to obtain the rotation parameters.

[0154] According to embodiments of this disclosure, the transformation parameters further include translation parameters. The transformation parameter calculation module includes a first constant term extraction unit, a second constant term extraction unit, and a translation parameter solving unit.

[0155] The first constant term extraction unit is used to extract the first constant term of the target first plane according to the first plane equation.

[0156] The second constant term extraction unit is used to extract the second constant term of the target second plane based on the second plane equation.

[0157] The translation parameter solving unit is used to input the first plane normal matrix, the second plane normal matrix, the first constant term, and the second constant term into the translation parameter solving program to obtain the translation parameters.

[0158] According to embodiments of this disclosure, the three-dimensional point cloud registration device further includes a first included angle calculation module, a first plane normal reversal module, a second included angle calculation module, and a second plane normal reversal module.

[0159] The first included angle calculation module is used to calculate the first included angle between the line connecting the first fixed point determined for the first plane and any target point located on the first plane and the first plane normal of the first plane.

[0160] The first plane normal reversal module is used to reverse the first plane normal in response to determining that the first included angle is greater than 90°, so as to obtain the first plane normal matrix.

[0161] The second included angle calculation module is used to calculate the second included angle between the line connecting the second fixed point determined for the second plane and any target point located on the second plane and the normal of the second plane.

[0162] The second plane normal reversal module is used to reverse the second plane normal in response to determining that the second included angle is greater than 90°, so as to obtain the second plane normal matrix.

[0163] According to embodiments of this disclosure, there are multiple plane matching pairs, each plane matching pair including a target first plane and a target second plane, and the transformation parameters include multiple candidate transformation parameters. The registration module includes a transformation unit, a second plane distance calculation unit, a plane matching pair number acquisition unit, a target plane matching pair number determination unit, a target transformation parameter determination unit, and a registration unit.

[0164] The transformation unit is used to perform rotation and translation transformations on the target second plane in each plane matching pair according to each candidate transformation parameter to obtain the transformed plane.

[0165] The second planar distance calculation unit is used to calculate the second planar distance between the target first plane and the transformed plane in each planar matching pair.

[0166] The plane matching pair number acquisition unit is used to acquire the number of plane matching pairs whose second plane distance is less than a given threshold, determined based on different candidate transformation parameters, and thus obtain multiple plane matching pair numbers.

[0167] The target plane matching pair number determination unit is used to determine the target plane matching pair number with the largest value from multiple plane matching pair numbers.

[0168] The target transformation parameter determination unit is used to determine the candidate transformation parameters related to the number of matching pairs of the target plane as the target transformation parameters.

[0169] The registration unit is used to register the first 3D point cloud and the second 3D point cloud according to the target transformation parameters.

[0170] Any one or more of the modules or units according to embodiments of this disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules or units according to embodiments of this disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules or units according to embodiments of this disclosure can be at least partially implemented as hardware circuitry, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System-on-Chip, a System-on-Substrate, a System-on-Package, an Application-Specific Integrated Circuit (ASIC), or implemented by hardware or firmware in any other reasonable manner by integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, one or more of the modules or units according to embodiments of this disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0171] For example, any and more of the first plane extraction module 810, the second plane extraction module 820, the plane feature histogram acquisition module 830, the matching module 840, the transformation parameter calculation module 890, and the registration module 860 can be combined into one module / unit, or any one of these modules / units can be split into multiple modules / units. Alternatively, at least some of the functions of one or more of these modules / units can be combined with at least some of the functions of other modules / units and implemented in one module / unit. According to embodiments of this disclosure, at least one of the first plane extraction module 810, the second plane extraction module 820, the plane feature histogram acquisition module 830, the matching module 840, the conversion parameter calculation module 890, and the registration module 860 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuits, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the first plane extraction module 810, the second plane extraction module 820, the plane feature histogram acquisition module 830, the matching module 840, the conversion parameter calculation module 890, and the registration module 860 can be at least partially implemented as computer program modules, which can perform corresponding functions when the computer program module is run.

[0172] It should be noted that the three-dimensional point cloud registration device part in the embodiments of this disclosure corresponds to the three-dimensional point cloud registration method part in the embodiments of this disclosure. For a detailed description of the three-dimensional point cloud registration device part, please refer to the three-dimensional point cloud registration method part, which will not be repeated here.

[0173] Figure 9 A block diagram of an electronic device suitable for implementing a three-dimensional point cloud registration method according to an embodiment of the present disclosure is shown schematically. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0174] like Figure 9As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0175] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0176] According to embodiments of this disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0177] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by processor 901, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0178] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0179] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0180] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0181] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the three-dimensional point cloud registration method provided in the embodiments of this disclosure.

[0182] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0183] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0184] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0186] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for registering three-dimensional point clouds, comprising: extracting at least one first plane from a first three-dimensional point cloud; extracting at least one second plane from a second three-dimensional point cloud; obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; matching the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair satisfying a preset condition; calculating a conversion parameter in a four-dimensional plane space according to the plane matching pair, the conversion parameter representing a parameter related to registration of the first plane and the second plane; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the conversion parameter. The matching of the first plane and the second plane to obtain the plane matching pair satisfying the preset condition comprises: calculating a first plane distance between the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain at least one first plane distance; determining a target second plane having a first plane distance closest to a target first plane from the at least one second plane according to the at least one first plane distance; and determining the target first plane and the target second plane as the plane matching pair in a case that a first plane having a first plane distance closest to the second plane is determined from the at least one first plane as the target first plane. The first plane and the second plane each have a plurality; and the obtaining of the first plane feature histogram of the first plane and the second plane feature histogram of the second plane comprises:

2. The method of claim 1, wherein, calculating a first plane angle between each selected first plane and other first planes representing other first planes except the selected first plane from the plurality of first planes to form a first plane feature histogram of the selected first plane; and calculating a second plane angle between each selected second plane and other second planes representing other second planes except the selected second plane from the plurality of second planes to form a second plane feature histogram of the selected second plane. The first plane and the second plane each have a plurality; and the obtaining of the first plane feature histogram of the first plane and the second plane feature histogram of the second plane comprises:

3. The method of claim 1, wherein, obtaining a first fixed point determined for the first plane and a second fixed point corresponding to the second plane, wherein the first fixed point and the second fixed point have the same position coordinates in a world coordinate system; and ​ According to position information of the first point in a first coordinate system corresponding to the first three-dimensional point cloud, a first relative distance between the first point and a first intersection line is calculated to form a first plane feature histogram of a selected first plane, wherein the first intersection line represents an intersection line determined by the selected first plane and other first planes, and the other first planes represent other first planes except the selected first plane among a plurality of the first planes; and According to position information of the second point in a second coordinate system corresponding to the second three-dimensional point cloud, a second relative distance between the second point and a second intersection line is calculated to form a second plane feature histogram of the selected second plane, wherein the other second planes represent other second planes except the selected second plane among a plurality of the second planes, and the second intersection line represents an intersection line determined by the selected second plane and the other second planes.

4. The method of claim 1, wherein, The plane matching pair includes a target first plane and a target second plane, and the conversion parameter includes a rotation parameter; and the conversion parameter is calculated in a four-dimensional plane space according to the plane matching pair, including: a first plane normal matrix of the target first plane is extracted according to a first plane equation of the target first plane; a second plane normal matrix of the target second plane is extracted according to a second plane equation of the target second plane; and the first plane normal matrix and the second plane normal matrix are input into a rotation matrix solving program to obtain the rotation parameter.

5. The method of claim 4, wherein, The conversion parameter further includes a translation parameter; and the conversion parameter is calculated in a four-dimensional plane space according to the plane matching pair, including: a first constant term of the target first plane is extracted according to the first plane equation; a second constant term of the target second plane is extracted according to the second plane equation; and the first plane normal matrix, the second plane normal matrix, the first constant term and the second constant term are input into a translation parameter solving program to obtain the translation parameter.

6. The method of claim 4 or 5, further comprising: calculating a first included angle between a line connecting a first point determined for the first plane and any target point located on the first plane and a first plane normal of the first plane; in response to determining that the first included angle is greater than 90°, reversing the first plane normal to obtain the first plane normal matrix; calculating a second included angle between a line connecting a second point determined for the second plane and any target point located on the second plane and a second plane normal of the second plane; and in response to determining that the second included angle is greater than 90°, reversing the second plane normal to obtain the second plane normal matrix. The plane matching pair has a plurality of, each of the plane matching pairs includes a target first plane and a target second plane, and the conversion parameter includes a plurality of candidate conversion parameters; and the first three-dimensional point cloud and the second three-dimensional point cloud are registered according to the conversion parameter, including:

7. The method of claim 1, wherein, ​ According to each of the candidate conversion parameters, a rotation transformation and a translation transformation are performed on the target second plane in each of the plane matching pairs, to obtain a transformed plane; A second plane distance between the target first plane in each of the plane matching pairs and the transformed plane is calculated; A number of plane matching pairs in which the second plane distance determined based on different candidate conversion parameters is less than a given threshold value is obtained, to obtain a plurality of plane matching pair numbers; A target plane matching pair number with a maximum value is determined from the plurality of plane matching pair numbers; A candidate conversion parameter related to the target plane matching pair number is determined as a target conversion parameter; and According to the target conversion parameter, the first three-dimensional point cloud and the second three-dimensional point cloud are registered.

8. A three-dimensional point cloud registration apparatus, comprising: a first plane extraction module configured to extract at least one first plane from a first three-dimensional point cloud; a second plane extraction module configured to extract at least one second plane from a second three-dimensional point cloud; a plane feature histogram obtaining module configured to obtain a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; a matching module configured to match the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram, to obtain a plane matching pair satisfying a preset condition; a conversion parameter calculation module configured to calculate a conversion parameter in a four-dimensional plane space according to the plane matching pair, the conversion parameter representing a parameter related to registration of the first plane and the second plane; and a registration module configured to register the first three-dimensional point cloud and the second three-dimensional point cloud according to the conversion parameter; wherein the matching module comprises a first plane distance calculation unit, a target second plane determination unit, and a plane matching pair determination unit; the first plane distance calculation unit is configured to calculate a first plane distance between the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram, to obtain at least one first plane distance; the target second plane determination unit is configured to determine a target second plane closest to a target first plane from the at least one second plane according to at least one first plane distance; and the plane matching pair determination unit is configured to determine the target first plane and the target second plane as the plane matching pair in a case where a first plane closest to the second plane from the at least one first plane is the target first plane.

9. An electronic device, comprising: one or more processors; a memory configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

Images