Three-dimensional point cloud registration method and device and electronic equipment

Abstract

The invention provides a three-dimensional point cloud registration method and device and electronic equipment. The method comprises the following steps: extracting at least one first plane from a first three-dimensional point cloud; extracting at least one second plane from the second three-dimensional point cloud; acquiring a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; according to the first plane feature histogram and the second plane feature histogram, matching the first plane and the second plane to obtain a plane matching pair meeting a preset condition; according to the plane matching pairs, conversion parameters are obtained through calculation in the four-dimensional plane space, and the conversion parameters represent parameters related to registration of the first plane and the second plane; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the conversion parameters.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and more specifically, to a three-dimensional point cloud registration method, device and electronic device. Background Art

[0002] With the development of information technology and high-precision sensors, point cloud images have gradually become the preferred format for representing the three-dimensional world. However, due to the limitations of the sensor's viewing angle, a directly acquired point cloud image often only contains data from part of the scene. Therefore, in order to obtain a complete three-dimensional scene, point cloud registration technology is needed to register and fuse multiple acquired point clouds. Point cloud registration technology studies the transformation estimation problem between two or more point clouds. First, the transformation matrix is ​​obtained through feature matching, and then the transformation matrix is ​​used to merge partial point cloud data of the same scene into a complete scene. It has important applications in the fields of three-dimensional surface reconstruction, autonomous driving, industrial robots, and mixed reality.

[0003] Traditional 3D point cloud registration technology has achieved remarkable results. Most of them are based on local feature matching and then restore the rigid transformation parameters to minimize the geometric projection error between the two point clouds.

[0004] These traditional 3D point cloud registration methods extract key points and local features in the 3D point domain space. Although they have achieved good results on dense and clean point clouds, they still have great limitations. First, they are difficult to expand to large-scale scenes. The main reason is that it is very expensive to calculate the matching relationship of point pairs in large-scale scenes, and excessive point cloud sampling cannot guarantee the accuracy of the results. Secondly, these methods are very sensitive to noise, wild points, uneven density or missing data, and are prone to a large number of mismatches.

[0005] In the field of point cloud registration, deep learning-based point cloud registration algorithms can be divided into two categories: feature-based methods and end-to-end methods. The main purpose of the former is to establish a reliable deep feature matching relationship, and then obtain the transformation matrix based on the traditional pose optimization algorithm. The latter uses a deep neural network to implement the differentiable transformation matrix solution process, so that the transformation matrix can be directly restored.

[0006] Therefore, how to provide a method to solve the above problems (different data sources, density sizes, sampling and various transformations) is a problem that technicians in this field currently need to solve. Summary of the invention

[0007] In view of this, the present disclosure provides a three-dimensional point cloud registration method, device and electronic device.

[0008] One aspect of the present disclosure provides a three-dimensional point cloud registration method, including: extracting at least one first plane from a first three-dimensional point cloud; extracting at least one second plane from a second three-dimensional point cloud; obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; matching the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition; calculating conversion parameters in a four-dimensional plane space according to the plane matching pair, the conversion parameters representing parameters related to the registration of the first plane and the second plane; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the conversion parameters.

[0009] Another aspect of the present disclosure provides a three-dimensional point cloud registration device, including: a first plane extraction module, used to extract at least one first plane from a first three-dimensional point cloud; a second plane extraction module, used to extract at least one second plane from a second three-dimensional point cloud; a plane feature histogram acquisition module, used to acquire a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; a matching module, used to match the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition; a conversion parameter calculation module, used to calculate conversion parameters in a four-dimensional plane space according to the plane matching pair, the conversion parameters representing parameters related to the registration of the first plane and the second plane; and a registration module, used to register the first three-dimensional point cloud and the second three-dimensional point cloud based on the conversion parameters.

[0010] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional point cloud registration method of the present disclosure.

[0011] According to the embodiments of the present disclosure, at least one first plane is extracted from a first three-dimensional point cloud; at least one second plane is extracted from a second three-dimensional point cloud; a first plane feature histogram of the first plane and a second plane feature histogram of the second plane are obtained; according to the first plane feature histogram and the second plane feature histogram, the first plane and the second plane are matched to obtain a plane matching pair that meets preset conditions; according to the plane matching pair, conversion parameters are calculated in a four-dimensional plane space, and the conversion parameters represent parameters related to the registration of the first plane and the second plane; and according to the conversion parameters, a technical means for registering the first three-dimensional point cloud and the second three-dimensional point cloud is proposed. A method for registering and fusing three-dimensional point clouds using a new registration framework of a planar structure is proposed. The method is universal, can solve the registration problem in three-dimensional point space under different data sources, density sizes and various transformations, and can achieve efficient registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0013] Figure 1 An exemplary system architecture to which a three-dimensional point cloud registration method can be applied according to an embodiment of the present disclosure is schematically shown;

[0014] Figure 2 A flowchart of a three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown;

[0015] Figure 3 A schematic diagram of constructing a descriptor according to an embodiment of the present disclosure is schematically shown;

[0016] Figure 4 The overall flow chart of the three-dimensional point cloud registration method according to the embodiment of the present disclosure is schematically shown;

[0017] Figure 5 Schematically showing visual comparison results of registration of two partial visibility scenes based on different methods according to an embodiment of the present disclosure;

[0018] Fig. 6A The visualization result of indoor scene point cloud registration based on the three-dimensional point cloud registration method according to the embodiment of the present disclosure is schematically shown;

[0019] Figure 6B The visualization result of outdoor scene point cloud registration based on the three-dimensional point cloud registration method according to the embodiment of the present disclosure is schematically shown;

[0020] Fig. 7A Schematically showing a recall rate curve diagram under different overlap rates between points using a three-dimensional point cloud registration method according to an embodiment of the present disclosure;

[0021] Figure 7B Schematically showing a recall rate curve diagram under different plane overlap rates using a three-dimensional point cloud registration method according to an embodiment of the present disclosure;

[0022] Figure 8 A block diagram schematically shows a three-dimensional point cloud registration device according to an embodiment of the present disclosure; and

[0023] Fig. 9 A block diagram of an electronic device suitable for implementing a three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0029] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0030] Figure 1 The exemplary system architecture 100 to which the three-dimensional point cloud registration method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0035] It should be noted that the three-dimensional point cloud registration method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the three-dimensional point cloud registration device provided in the embodiment of the present disclosure can generally be set in the server 105. The three-dimensional point cloud registration method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the three-dimensional point cloud registration device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the three-dimensional point cloud registration method provided in the embodiment of the present disclosure can also be performed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the three-dimensional point cloud registration device provided in the embodiment of the present disclosure can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0036] For example, the first three-dimensional point cloud and the second three-dimensional point cloud may be originally stored in any one of the first terminal device 101, the second terminal device 102 or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored in an external storage device and may be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute the three-dimensional point cloud registration method provided in the embodiment of the present disclosure, or send the first three-dimensional point cloud and the second three-dimensional point cloud to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the first three-dimensional point cloud and the second three-dimensional point cloud may execute the three-dimensional point cloud registration method provided in the embodiment of the present disclosure.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] Figure 2 The flowchart of the three-dimensional point cloud registration method according to the embodiment of the present disclosure is schematically shown.

[0039] like Figure 2 As shown, the method includes operations S201 to S206.

[0040] In operation S201 , at least one first plane is extracted from a first three-dimensional point cloud.

[0041] In operation S202, at least one second plane is extracted from the second three-dimensional point cloud.

[0042] According to an embodiment of the present disclosure, the first three-dimensional point cloud and the second three-dimensional point cloud may represent two three-dimensional point clouds sampled from the same object or different objects. The first three-dimensional point cloud and the second three-dimensional point cloud may include two overlapping three-dimensional point clouds. For example, the first three-dimensional point cloud and the second three-dimensional point cloud may represent two three-dimensional point clouds sampled from the same object at different perspectives. The first three-dimensional point cloud and the second three-dimensional point cloud may also include two adjacent but non-overlapping three-dimensional point clouds sampled from two shapes that need to be spliced ​​together. For example, the first three-dimensional point cloud and the second three-dimensional point cloud may represent two three-dimensional point clouds sampled from two objects that need to be spliced ​​together. This is not limited here.

[0043] According to an embodiment of the present disclosure, after sampling to obtain a first three-dimensional point cloud and a second three-dimensional point cloud, a primitive detection algorithm based on RANSAC (Random Sample Consensus) can be used to extract planes from the two three-dimensional point clouds respectively to obtain a first plane and a second plane.

[0044] It should be noted that the above-mentioned method for extracting a plane is only an exemplary embodiment, but is not limited thereto, and may also include other plane extraction or generation methods known in the art, as long as a four-dimensional plane can be extracted from a three-dimensional point cloud.

[0045] In operation S203 , a first plane feature histogram of the first plane and a second plane feature histogram of the second plane are acquired.

[0046] According to an embodiment of the present disclosure, in order to obtain conversion parameters, a robust and compact plane feature descriptor PLFH (PLane Feature Histograms) may be introduced to establish a correspondence between planes.

[0047] For example, in an optional embodiment, the plane feature histogram of the plane can be obtained by focusing on the performance of the feature points in the point cloud on the corresponding plane. For example, the points in the first three-dimensional point cloud can be first mapped to the first plane, and then the features of the first plane can be extracted to obtain the first plane feature histogram.

[0048] It should be noted that the method for determining the plane feature histogram is not limited to the method described above.

[0049] In operation S204, the first plane and the second plane are matched according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition.

[0050] According to an embodiment of the present disclosure, for example, a RANSAC iteration process may be applied to find a matching plane between the first three-dimensional point cloud and the second three-dimensional point cloud to obtain a plane matching pair.

[0051] According to an embodiment of the present disclosure, the preset conditions may include but are not limited to: the similarity between the first plane feature histogram and the second plane feature histogram is greater than a preset threshold, which is applicable to the matching process of two planes extracted from two three-dimensional point clouds sampled from different angles for the same object.

[0052] In operation S205 , a conversion parameter is calculated in a four-dimensional plane space according to the plane matching pair, where the conversion parameter represents a parameter related to registering the first plane with the second plane.

[0053] According to the embodiments of the present disclosure, the first plane and the second plane can be used as input to estimate the best transformation in the four-dimensional plane space. The core idea is to directly use the plane parameters (A, B, C, D) to obtain the transformation parameters through rigorous mathematical derivation.

[0054] According to an embodiment of the present disclosure, the conversion parameter may include at least one of a translation parameter, a rotation parameter, etc., and may not be limited thereto.

[0055] It should be noted that the method for calculating the conversion parameters in the four-dimensional plane space based on the plane matching pair may not be limited to the above, and may also include other calculation methods known in the art. For example, the calculation model can be trained to input the first plane feature histogram and the second plane feature histogram of the plane matching pair into the training model, output the conversion parameters, etc., and it is not limited to this.

[0056] In operation S206 , the first three-dimensional point cloud and the second three-dimensional point cloud are registered according to the conversion parameters.

[0057] According to an embodiment of the present disclosure, given two points respectively obtained from a first three-dimensional point cloud and the second 3D point cloud The first plane set extracted separately and the second plane set , the first plane set may include at least one first plane , the second plane set may include at least one second plane . You can take a plane with parameters (A, B, C, D) ( Characterization or ) is expressed as: Ax+By+Cz+D = 0, where ( is a real number), and . You can put the plane The points in are represented as homogeneous coordinates (x, y, z, 1). Then from the first three-dimensional point cloud To the second 3D point cloud The transformation process can be expressed as formula (1).

[0058] Formula (1)

[0059] In formula (1), For the first three-dimensional point cloud The matrix representation of For the second 3D point cloud The matrix representation of represents the transformation matrix, where represents the rotation parameter, Represents the translation parameter.

[0060] Eliminating the homogeneous coordinates of the point in formula (1) yields formula (2), which represents the Transform to the second plane The transformation equation.

[0061] Formula (2)

[0062] In formula (2), Represents the first plane The plane parameters of Represents the second plane Plane parameters.

[0063] The above formulas (1) and (2) show that the transformation relationship of the point cloud is the inverse of the transformation relationship of the plane.

[0064] According to the embodiments of the present disclosure, in actual operation, the transformation matrix can be solved by defining plane parameters. For example, the target point cloud can be directly extracted from the plane set by combining formula (2). Transform to the source point cloud to extract the plane set , and solve for the conversion parameters that can register the first three-dimensional point cloud and the second three-dimensional point cloud.

[0065] According to an embodiment of the present disclosure, after obtaining the conversion parameters, the second three-dimensional point cloud can be subjected to rotation transformation and translation transformation according to the conversion parameters to obtain a transformed three-dimensional point cloud, so as to align the first three-dimensional point cloud and the transformed three-dimensional point cloud, thereby achieving alignment of the first three-dimensional point cloud and the second three-dimensional point cloud.

[0066] Through the above-mentioned embodiments of the present disclosure, a method for registering and fusing three-dimensional point clouds using a new registration framework using a planar structure is proposed. This method is universal and can solve registration problems in three-dimensional point spaces under different data sources, density sizes and various transformations, and can achieve efficient registration.

[0067] In conjunction with specific embodiments, Figure 2 The method shown is further explained.

[0068] Based on the above method, at least four pairs of matching planes are required to calculate the transformation parameters. In the process of implementing the concept of the present disclosure, the inventors found that if the RANSAC iteration process is directly applied to find matching planes between the source point cloud and the target point cloud, multiple iterations will be required, and due to the random nature of RANSAC, the matching results are uncertain.

[0069] To solve this problem, the present invention constructs a simple but efficient descriptor based on plane relationship, which is invariant to rigid transformation. The descriptor can be constructed based on statistical information of at least one of the angle and distance distribution between planes.

[0070] According to an embodiment of the present disclosure, there may be multiple first planes and second planes. In the case where the descriptor is constructed based on statistical information of angles between planes, the above operation S203 may include: for each selected first plane among the multiple first planes, calculating the first plane angle between the selected first plane and other first planes, forming a first plane feature histogram of the selected first plane, wherein the other first planes characterize the other first planes among the multiple first planes except the selected first plane. For each selected second plane among the multiple second planes, calculating the second plane angle between the selected second plane and other second planes, forming a second plane feature histogram of the selected second plane, wherein the other second planes characterize the other second planes among the multiple second planes except the selected second plane.

[0071] For example, for For each plane m in the point cloud, its distance from all other planes in the same point cloud can be calculated. A set of angles between . A value indicating the total number of planes (such as the first plane or the second plane) extracted from a 3D point cloud (such as the first 3D point cloud or the second 3D point cloud) minus 1. Represents the plane m and the The angle between the planes.

[0072] According to an embodiment of the present disclosure, in the case where the descriptor is constructed based on the statistical information of the distance distribution, the above operation S203 may include: obtaining a first fixed point determined for the first plane and a second fixed point corresponding to the second plane. The first fixed point and the second fixed point have the same position coordinates in the world coordinate system. According to the position information of the first fixed point in the first coordinate system corresponding to the first three-dimensional point cloud, the first relative distance between the first fixed point and the first intersection line is calculated to form a first plane feature histogram of the selected first plane. The other first planes represent the other first planes other than the selected first plane among the multiple first planes. The first intersection line represents the intersection line determined by the selected first plane and the other first planes. According to the position information of the second fixed point in the second coordinate system corresponding to the second three-dimensional point cloud, the second relative distance between the second fixed point and the second intersection line is calculated to form a second plane feature histogram of the selected second plane. The other second planes represent the other second planes other than the selected second plane among the multiple second planes. The second intersection line represents the intersection line determined by the selected second plane and the other second planes.

[0073] For example, for For each plane m in the image, we can calculate the distance from the fixed point P to m and The distance between the intersecting lines . Represents the fixed point P and the plane m and The distance between intersecting lines.

[0074] According to an embodiment of the present disclosure, the angle may also be calculated first and distance Then, change the angle and distance Specifically, we can divide the value range into some sub-partitions and calculate the number of features in each sub-partition, and finally return a descriptor by splicing the two histograms together, or we can construct a plane feature histogram of the plane in a given point cloud.

[0075] According to an embodiment of the present disclosure, for the selection of matching fixed points, the first fixed point and the second fixed point can be determined by randomly selecting a fixed point in the world coordinate system. The traditional point cloud feature descriptor FPFH (FastPoint Feature Histograms) can also be used to identify the best matching point pair to determine the first fixed point and the second fixed point. By selecting matching fixed points, the distance consistency between the first three-dimensional point cloud and the second three-dimensional point cloud can be ensured when the distance descriptor is calculated.

[0076] Figure 3 The schematic diagram of constructing a descriptor according to an embodiment of the present disclosure is schematically shown.

[0077] like Figure 3 As shown, for a 3D point cloud, for example, the plane . Determine the selected plane. The selected plane can be obtained by calculation With other planes Their respective angles .flat With other planes Can form intersecting lines . By calculating the fixed point P to the intersection line The distance can be obtained . After that, you can By counting the angle histogram, the distance Statistics are converted into distance histogram.

[0078] Through the above-mentioned embodiments of the present disclosure, a simple but efficient descriptor based on plane relationship is constructed, which counts the angle between planes and / or the distance from a fixed point to the plane intersection line, and has rotation and translation invariance. Compared with other registration methods that rely on combined plane feature descriptors, PLFH has significant advantages in terms of high efficiency and robustness. In terms of efficiency, due to the single traversal of each plane in the plane set, each plane only needs to be traversed once during the construction process, and all other planes extracted from the same point cloud will be directly used to calculate PLFH. The time complexity of PLFH construction is O(n 2 ), and its construction process is extremely efficient. In terms of robustness, the descriptor is based on statistics of angles and distances, so it is more robust to noise and errors in plane extraction.

[0079] After obtaining the above descriptor PLFH, we start to select the technical solution for calculating the transformation between two point clouds, among which obtaining a set of reliable plane correspondences is particularly critical. Although the number of planes is much smaller than the number of points, it is still not easy to find the correct plane correspondence by brute force enumeration because the number of possible permutations grows exponentially.

[0080] The present disclosure can migrate the robust RANSAC algorithm to a four-dimensional plane space for efficient plane matching. Unlike the previous method of applying RANSAC to the original point cloud and thus converging slowly, the method of the present disclosure significantly reduces the search space of RANSAC.

[0081] According to an embodiment of the present disclosure, the above operation S204 may include: calculating the first plane distance between the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain at least one first plane distance. According to the at least one first plane distance, determining the target second plane that is closest to the first plane distance between the target first plane from at least one second plane. In the case where the first plane that is closest to the first plane distance between the second plane determined from at least one first plane is the target first plane, determining the target first plane and the target second plane as a plane matching pair.

[0082] For example, in each iteration of RANSAC, four planes can be randomly selected from the first 3D point cloud as the initial set, and the closest corresponding planes in the target point cloud are searched based on the similarity between their PLFH descriptors. If the bilateral consistency of the closest distance represented by the following formula (3) and formula (4) is satisfied, it is considered a valid match. represents the first plane in the first three-dimensional point cloud, Represents a second plane in the second 3D point cloud.

[0083] Formula (3)

[0084] Formula (4)

[0085] In formula (3) and formula (4), The PLFH descriptor of plane m can be represented.

[0086] According to an embodiment of the present disclosure, a plane matching pair may include a target first plane and a target second plane. In the process of calculating the conversion parameters, a first plane normal matrix of the target first plane and a second plane normal matrix of the target second plane may be first obtained, and the method may include: extracting the first plane normal matrix of the target first plane according to the first plane equation of the target first plane. Extracting the second plane normal matrix of the target second plane according to the second plane equation of the target second plane.

[0087] For example, you can use represents the plane normal, and formula (2) can be simplified to obtain formula (5).

[0088] Formula (5)

[0089] According to an embodiment of the present disclosure, the idea of ​​point cloud normal redirection technology can be used to eliminate the uncertainty of the plane normal. Specifically, after obtaining the first plane normal matrix and the second plane normal matrix, the first angle between the line between the first fixed point determined for the first plane and any target point located on the first plane and the first plane normal of the first plane can be calculated. In response to determining that the first angle is greater than 90°, the first plane normal is reversed to obtain the first plane normal matrix. Calculate the second angle between the line between the second fixed point determined for the second plane and any target point located on the second plane and the second plane normal of the second plane. In response to determining that the second angle is greater than 90°, the second plane normal is reversed to obtain the second plane normal matrix.

[0090] For example, for the first plane Any point on , its plane normal can be calculated by first calculating the fixed point P and the point The angle between the line connecting the two planes and the plane normal ,if , then the plane normal is reversed to eliminate the uncertainty of the plane normal. The same redirection strategy can be used.

[0091] According to an embodiment of the present disclosure, after eliminating the uncertainty of the plane normal, the solution equation shown in formula (5) can be transformed again into the form of formula (6).

[0092] Formula (6)

[0093] In formula (6), It can represent the first plane normal matrix of a first plane, The second plane normal matrix of a second plane can be expressed as It can represent a constant term parameter of the first plane, recorded as the first constant term, It can represent a constant parameter of a second plane, recorded as the second constant term.

[0094] According to an embodiment of the present disclosure, the conversion parameters may also include rotation parameters. The above operation S205 may include: inputting the first plane parameter matrix and the second plane parameter matrix into a rotation matrix solving program to obtain the rotation parameters.

[0095] For example, the Kabsch algorithm (an algorithm for calculating the optimal rotation matrix) can be used to solve the rotation matrix R = , we can use SVD (singular value decomposition) to solve U and V, and get formula (7).

[0096] Formula (7)

[0097] According to the embodiments of the present disclosure, formula (7) can be combined to construct a rotation matrix solution program to solve the rotation parameters.

[0098] According to an embodiment of the present disclosure, the conversion parameters may also include translation parameters. The above operation S205 may also include: extracting a first constant term of the target first plane according to the first plane equation. Extracting a second constant term of the target second plane according to the second plane equation. Inputting the first plane normal matrix, the second plane normal matrix, the first constant term and the second constant term into a translation parameter solving program to obtain the translation parameters.

[0099] For example, to solve for the translation vector The translation parameter solution of can be to minimize the energy function shown in formula (8) and Find the partial derivative and make , the obtained formula is shown in formula (9), but it is not limited to this.

[0100] Formula (8)

[0101] Formula (9)

[0102] In formula (9), It can represent a total matrix composed of the second plane normal matrices of all second planes, It can represent the matrix composed of the first constant terms of all first planes, It can represent the matrix composed of the second constant terms of all second planes.

[0103] Through the above-mentioned embodiments of the present disclosure, the classic RANSAC algorithm based on the hypothesis-testing framework is introduced into the four-dimensional plane space, which reduces the negative impact of plane mismatching while reducing the data scale, and realizes a high-precision and high-efficiency three-dimensional point cloud registration solution.

[0104] According to an embodiment of the present disclosure, there may be multiple plane matching pairs. Each plane matching pair may include a target first plane and a target second plane. The conversion parameters may include multiple candidate conversion parameters. The above operation S206 may include: according to each candidate conversion parameter, the target second plane in each plane matching pair is subjected to a rotation transformation and a translation transformation to obtain a transformed plane. The second plane distance between the target first plane and the transformed plane in each plane matching pair is calculated. The number of plane matching pairs whose second plane distances determined based on different candidate conversion parameters are less than a given threshold is obtained to obtain multiple numbers of plane matching pairs. The number of target plane matching pairs with the largest value is determined from the multiple numbers of plane matching pairs. The candidate conversion parameters related to obtaining the target plane matching pair number are determined as target conversion parameters. According to the target conversion parameters, the first three-dimensional point cloud and the second three-dimensional point cloud are aligned.

[0105] For example, after obtaining the plane matching relationship, the candidate conversion parameters can be calculated by combining the methods shown in the above formulas (7) and (9). By performing efficient plane-based RANSAC, different candidate conversion parameters are compared in iterations, and the best parameters that match the most planes are selected. For example, it can include: calculating the distance between matching plane pairs, and counting the number of plane pairs whose distance is less than a given threshold, and finally selecting the candidate conversion parameters that match the most planes as the target conversion parameter output.

[0106] According to an embodiment of the present disclosure, the formula for measuring the second plane distance is shown in formula (10), but is not limited thereto.

[0107] Formula (10)

[0108] In formula (10), Can represent candidate transformation parameters.

[0109] Figure 4 The overall flow chart of the three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 4As shown, the first three-dimensional point cloud 410 and the second three-dimensional point cloud 420 may be first taken as input. Then, the first three-dimensional point cloud 410 may be subjected to plane extraction to obtain a first plane 411, and the second three-dimensional point cloud 420 may be subjected to plane extraction to obtain a second plane 421. Afterwards, a first PLFH 413 of the first plane 411 may be calculated in a first four-dimensional plane space 412 constructed based on a first coordinate system used to determine the first three-dimensional point cloud 410, and a second PLFH 423 of the second plane 421 may be calculated in a second four-dimensional plane space 422 constructed based on a second coordinate system used to determine the second three-dimensional point cloud 420. Next, by performing efficient plane-based RANSAC on the first PLFH 413 and the second PLFH 423, different candidate transformation parameters may be compared in iterations to obtain the target transformation parameters. Finally, combined with the target conversion parameters , the first three-dimensional point cloud 410 and the second three-dimensional point cloud 420 can be registered to obtain a registered point cloud 430.

[0111] Through the above embodiments of the present disclosure, an efficient plane-based conversion parameter solution process is realized. Including the initial matching of PLFH, the total time complexity of plane-based RANSAC is ,The registration process combines the robustness of RANSAC with reduced computational cost.

[0112] According to an embodiment of the present disclosure, the above point cloud registration method can be experimented with ScanNet (a three-dimensional laser point cloud obtained by an RGBD camera) dataset and a general CAD (a three-dimensional point cloud) dataset ShapeNet Part (point cloud segmentation).

[0113] First, the ScanNet and ShapeNet Part datasets are preprocessed, which may include: normalizing the point cloud so that the size of the point cloud is always kept at 1. Then, random rotation and translation are performed, which may include: randomly rotating and translating the point cloud in each coordinate axis [0, ] and randomly sample three angles in the range of [−5, 5] to synthesize the rotation matrix, and randomly generate a translation vector in the range of [−5, 5].

[0114] For the ScanNet dataset, you can choose to transform the low-resolution point cloud and use the transformed point cloud as the target point cloud. For the ShapeNet Part dataset, you can directly choose to copy the transformation.

[0115] The accuracy of the three-dimensional point cloud registration method provided by the present invention is evaluated through experiments. The experimental platform can be a computer with an Intel 6226R central processor, 256GB RAM and a 64-bit Ubuntu 20.04 operating system, and the pytorch open source deep learning framework can be used on this platform.

[0116] Table 1 shows the comparative experimental results on the original ScanNet and the sampled ScanNet. The comparative methods include: FGR, RANSAC, Super4PCS, PLADE, DCP, HRegNet and other common point-based methods. Most of these algorithms are integrated in some well-known open source libraries, such as open3d, Meshlab, etc. The best experimental result is marked with a circle, and the second best result is marked with a rectangular box.

[0117]

[0118] Table 2 shows the comparative experimental results on ScanNet with partial overlap.

[0119]

[0120] The results fully demonstrate the excellent robustness and accuracy of the present invention compared with existing methods.

[0121] In order to fully demonstrate the robustness of plane-based registration, the present disclosure also tests experiments under different categories of models and different point cloud densities.

[0122] Figure 5 The diagram schematically shows visual comparison results of registering two partial visibility scenes based on different methods according to an embodiment of the present disclosure.

[0123] like Figure 5 As shown in Figure 3, due to the lack of sufficient point correspondences in small overlapping scenes, methods such as RANSAC, Super4PCS, FGR, HRegNet, DCP, and PLADE perform poorly.

[0124] Fig. 6A The visualization result of indoor scene point cloud registration implemented based on the three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown.

[0125] Figure 6B The visualization result of outdoor scene point cloud registration implemented based on the three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown.

[0126] like Fig. 6A and Figure 6BAs shown, the left column is the input, and the right column is the output after registration. The results fully demonstrate the excellent robustness and accuracy of the three-dimensional point cloud registration method disclosed in the present invention compared with the existing methods.

[0127] Fig. 7A The recall rate curve diagram under different overlap rates between points using the three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown.

[0128] Figure 7B The recall rate curve diagram under different plane overlap rates using the three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown.

[0129] like Fig. 7A and Figure 7B As shown, the recall rate curves under different numbers of fragments and different rotation and displacement thresholds under the same data set can also be shown. The results show that the 3D point cloud registration method disclosed in the present invention has the highest recall rate in the entire range, especially at low overlap rates, far exceeding the point feature-based method.

[0130] By comparing the experimental results of the 3D point cloud registration method disclosed in the present invention with those of PLADE, it is proved that the 3D point cloud registration method disclosed in the present invention is more robust under different plane overlap rates.

[0131] Figure 8 A block diagram of a three-dimensional point cloud registration device according to an embodiment of the present disclosure is schematically shown.

[0132] like Figure 8 As shown, the three-dimensional point cloud registration device 800 includes a first plane extraction module 810, a second plane extraction module 820, a plane feature histogram acquisition module 830, a matching module 840, a conversion parameter calculation module 850 and a registration module 860.

[0133] The first plane extraction module 810 is used to extract at least one first plane from the first three-dimensional point cloud.

[0134] The second plane extraction module 820 is used to extract at least one second plane from the second three-dimensional point cloud.

[0135] The plane feature histogram acquisition module 830 is used to acquire a first plane feature histogram of the first plane and a second plane feature histogram of the second plane.

[0136] The matching module 840 is used to match the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition.

[0137] The conversion parameter calculation module 850 is used to calculate the conversion parameters in the four-dimensional plane space according to the plane matching pair, and the conversion parameters represent the parameters related to the registration of the first plane and the second plane.

[0138] The registration module 860 is used to register the first three-dimensional point cloud and the second three-dimensional point cloud according to the conversion parameters.

[0139] According to an embodiment of the present disclosure, there are multiple first planes and multiple second planes. The plane feature histogram acquisition module includes a first plane angle calculation unit and a second plane angle calculation unit.

[0140] The first plane angle calculation unit is used to calculate the first plane angle between each selected first plane among the multiple first planes and other first planes to form a first plane feature histogram of the selected first plane, wherein the other first planes represent other first planes among the multiple first planes except the selected first plane.

[0141] The second plane angle calculation unit is used to calculate the second plane angle between each selected second plane among the multiple second planes and other second planes to form a second plane feature histogram of the selected second plane, wherein the other second planes represent the other second planes among the multiple second planes except the selected second plane.

[0142] According to an embodiment of the present disclosure, there are a plurality of first planes and a plurality of second planes. The plane feature histogram acquisition module further includes a fixed point acquisition unit, a first relative distance calculation unit and a second relative distance calculation unit.

[0143] The fixed point acquisition unit is used to acquire a first fixed point determined for the first plane and a second fixed point corresponding to the second plane, wherein the first fixed point and the second fixed point have the same position coordinates in the world coordinate system.

[0144] A first relative distance calculation unit is used to calculate the first relative distance between the first fixed point and the first intersection line according to the position information of the first fixed point in the first coordinate system corresponding to the first three-dimensional point cloud, and form a first plane feature histogram of the selected first plane, wherein the other first planes represent the other first planes among the multiple first planes except the selected first plane, and the first intersection line represents the intersection line determined by the selected first plane and the other first planes.

[0145] A second relative distance calculation unit is used to calculate the second relative distance between the second fixed point and the second intersection line according to the position information of the second fixed point in the second coordinate system corresponding to the second three-dimensional point cloud, and form a second plane feature histogram of the selected second plane, wherein the other second planes represent the other second planes among the multiple second planes except the selected second plane, and the second intersection line represents the intersection line determined by the selected second plane and the other second planes.

[0146] According to an embodiment of the present disclosure, the matching module includes a first plane distance calculation unit, a target second plane determination unit and a plane matching pair determination unit.

[0147] The first plane distance calculation unit is used to calculate the first plane distance between the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain at least one first plane distance.

[0148] The target second plane determining unit is used to determine, from at least one second plane, a target second plane having a shortest first plane distance to the target first plane according to at least one first plane distance.

[0149] The plane matching pair determining unit is used to determine the target first plane and the target second plane as a plane matching pair when the first plane determined from at least one first plane and closest to the first plane of the second plane is the target first plane.

[0150] According to an embodiment of the present disclosure, the plane matching pair includes a target first plane and a target second plane. The conversion parameters include rotation parameters. The conversion parameter calculation module includes a first plane normal matrix extraction unit, a second plane normal matrix extraction unit and a rotation parameter solving unit.

[0151] The first plane normal matrix extraction unit is used to extract the first plane normal matrix of the target first plane according to the first plane equation of the target first plane.

[0152] The second plane normal matrix extraction unit is used to extract the second plane normal matrix of the target second plane according to the second plane equation of the target second plane.

[0153] The rotation parameter solving unit is used for inputting the first plane normal matrix and the second plane normal matrix into the rotation matrix solving program to obtain the rotation parameters.

[0154] According to an embodiment of the present disclosure, the conversion parameter also includes a translation parameter. The conversion parameter calculation module includes a first constant term extraction unit, a second constant term extraction unit and a translation parameter solving unit.

[0155] The first constant term extraction unit is used to extract the first constant term of the target first plane according to the first plane equation.

[0156] The second constant term extraction unit is used to extract the second constant term of the target second plane according to the second plane equation.

[0157] The translation parameter solving unit is used for inputting the first plane normal matrix, the second plane normal matrix, the first constant term and the second constant term into the translation parameter solving program to obtain the translation parameters.

[0158] According to an embodiment of the present disclosure, the three-dimensional point cloud registration device also includes a first angle calculation module, a first plane normal reversal module, a second angle calculation module and a second plane normal reversal module.

[0159] The first angle calculation module is used to calculate a first angle between a line between a first fixed point determined for the first plane and any target point located on the first plane and a first plane normal of the first plane.

[0160] The first plane normal reversal module is used to reverse the first plane normal in response to determining that the first angle is greater than 90 degrees to obtain a first plane normal matrix.

[0161] The second angle calculation module is used to calculate a second angle between a line between a second fixed point determined for the second plane and any target point on the second plane and a second plane normal of the second plane.

[0162] The second plane normal reversal module is used to reverse the second plane normal in response to determining that the second angle is greater than 90 degrees to obtain a second plane normal matrix.

[0163] According to an embodiment of the present disclosure, there are multiple plane matching pairs, each plane matching pair includes a target first plane and a target second plane, and the conversion parameters include multiple candidate conversion parameters. The registration module includes a transformation unit, a second plane distance calculation unit, a plane matching pair number acquisition unit, a target plane matching pair number determination unit, a target conversion parameter determination unit and a registration unit.

[0164] The transformation unit is used to perform rotation transformation and translation transformation on the target second plane in each plane matching pair according to each candidate transformation parameter to obtain a transformed plane.

[0165] The second plane distance calculation unit is used to calculate the second plane distance between the target first plane and the transformed plane in each plane matching pair.

[0166] The plane matching pair number acquisition unit is used to acquire the number of plane matching pairs whose second plane distances determined based on different candidate conversion parameters are less than a given threshold, and obtain multiple numbers of plane matching pairs.

[0167] The target plane matching pair number determining unit is used to determine the target plane matching pair number with the largest value from multiple plane matching pair numbers.

[0168] The target conversion parameter determination unit is used to determine the candidate conversion parameters related to obtaining the number of target plane matching pairs as target conversion parameters.

[0169] The registration unit is used to register the first three-dimensional point cloud and the second three-dimensional point cloud according to the target transformation parameter.

[0170] According to the embodiments of the present invention, any multiple of the modules and units, or at least part of the functions of any multiple of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules and units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules and units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules and units can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0171] For example, any multiple of the first plane extraction module 810, the second plane extraction module 820, the plane feature histogram acquisition module 830, the matching module 840, the conversion parameter calculation module 890 and the registration module 860 can be combined in one module / unit for implementation, or any one of the modules / units can be split into multiple modules / units. Alternatively, at least part of the functions of one or more of these modules / units can be combined with at least part of the functions of other modules / units and implemented in one module / unit. According to an embodiment of the present disclosure, at least one of the first plane extraction module 810, the second plane extraction module 820, the plane feature histogram acquisition module 830, the matching module 840, the conversion parameter calculation module 890 and the registration module 860 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first plane extraction module 810, the second plane extraction module 820, the plane feature histogram acquisition module 830, the matching module 840, the conversion parameter calculation module 890 and the registration module 860 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0172] It should be noted that the three-dimensional point cloud registration device part in the embodiment of the present disclosure corresponds to the three-dimensional point cloud registration method part in the embodiment of the present disclosure. The description of the three-dimensional point cloud registration device part specifically refers to the three-dimensional point cloud registration method part, which will not be repeated here.

[0173] Fig. 9 A block diagram of an electronic device suitable for implementing a three-dimensional point cloud registration method according to an embodiment of the present disclosure is schematically shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0174] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0175] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0176] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0177] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0178] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0179] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0180] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0181] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the three-dimensional point cloud registration method provided by the embodiment of the present disclosure.

[0182] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0183] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0184] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0186] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A three-dimensional point cloud registration method, comprising: extracting at least one first plane from the first three-dimensional point cloud; extracting at least one second plane from the second three-dimensional point cloud; Acquire a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; Matching the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition; According to the plane matching pair, a conversion parameter is calculated in a four-dimensional plane space, wherein the conversion parameter represents a parameter related to registering the first plane with the second plane; as well as The first three-dimensional point cloud and the second three-dimensional point cloud are registered according to the conversion parameters.

2. The method according to claim 1, wherein: The first plane and the second plane both have a plurality of planes; and the step of obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane comprises: For each selected first plane among the plurality of first planes, calculating a first plane angle between the selected first plane and other first planes to form a first plane feature histogram of the selected first plane, wherein the other first planes represent other first planes among the plurality of first planes except the selected first plane; and For each selected second plane among the multiple second planes, the second plane angle between the selected second plane and other second planes is calculated to form a second plane feature histogram of the selected second plane, wherein the other second planes characterize the other second planes among the multiple second planes except the selected second plane.

3. The method according to claim 1 or 2, wherein: The first plane and the second plane both have a plurality of planes; the step of obtaining a first plane feature histogram of the first plane and a second plane feature histogram of the second plane further includes: Acquire a first fixed point determined for the first plane and a second fixed point corresponding to the second plane, wherein the first fixed point and the second fixed point have the same position coordinates in a world coordinate system; Calculating a first relative distance between the first fixed point and a first intersection line according to position information of the first fixed point in a first coordinate system corresponding to the first three-dimensional point cloud, and forming a first plane feature histogram of the selected first plane, wherein the other first planes represent other first planes among a plurality of first planes except the selected first plane, and the first intersection line represents an intersection line determined by the selected first plane and the other first planes; and According to the position information of the second fixed point in the second coordinate system corresponding to the second three-dimensional point cloud, the second relative distance between the second fixed point and the second intersection line is calculated to form a second plane feature histogram of the selected second plane, wherein the other second planes represent the other second planes among the multiple second planes except the selected second plane, and the second intersection line represents the intersection line determined by the selected second plane and the other second planes.

4. The method according to claim 1, wherein: The matching of the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition includes: Calculating a first plane distance between the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain at least one first plane distance; According to at least one of the first plane distances, determining, from the at least one second plane, a target second plane that is closest to the first plane distance between the target first plane; and In a case where a first plane determined from the at least one first plane and having the shortest first plane distance to the second plane is the target first plane, the target first plane and the target second plane are determined as the plane matching pair.

5. The method according to claim 1, wherein: The plane matching pair includes a target first plane and a target second plane, and the conversion parameters include rotation parameters; the conversion parameters calculated in a four-dimensional plane space according to the plane matching pair include: Extracting a first plane normal matrix of the first target plane according to a first plane equation of the first target plane; Extracting a second plane normal matrix of the second target plane according to a second plane equation of the second target plane; and The first plane normal matrix and the second plane normal matrix are input into a rotation matrix solving program to obtain the rotation parameters.

6. The method according to claim 5, wherein: The conversion parameters also include translation parameters; the conversion parameters calculated in the four-dimensional plane space according to the plane matching pair include: Extracting a first constant term of the target first plane according to the first plane equation; Extracting a second constant term of the target second plane according to the second plane equation; and The first plane normal matrix, the second plane normal matrix, the first constant term and the second constant term are input into a translation parameter solving program to obtain the translation parameters.

7. The method according to claim 5 or 6, further comprising: Calculating a first angle between a line between a first fixed point determined for the first plane and any target point located on the first plane and a first plane normal of the first plane; In response to determining that the first angle is greater than 90°, reversing the first plane normal to obtain the first plane normal matrix; Calculating a second angle between a line between a second fixed point determined for the second plane and any target point located on the second plane and a second plane normal of the second plane; as well as In response to determining that the second angle is greater than 90°, the second plane normal is reversed to obtain the second plane normal matrix.

8. The method according to claim 1, wherein: There are a plurality of plane matching pairs, each of which includes a target first plane and a target second plane, and the conversion parameters include a plurality of candidate conversion parameters; and registering the first three-dimensional point cloud and the second three-dimensional point cloud according to the conversion parameters includes: According to each of the candidate transformation parameters, performing a rotation transformation and a translation transformation on the target second plane in each of the plane matching pairs to obtain a transformed plane; Calculating a second plane distance between the target first plane and the transformed plane in each of the plane matching pairs; Acquire the number of plane matching pairs whose second plane distances are less than a given threshold, respectively determined based on different candidate conversion parameters, to obtain multiple numbers of plane matching pairs; Determine a target plane matching pair number with the largest value from the plurality of plane matching pair numbers; determining candidate transformation parameters related to obtaining the number of target plane matching pairs as target transformation parameters; and The first three-dimensional point cloud and the second three-dimensional point cloud are registered according to the target conversion parameters.

9. A three-dimensional point cloud registration device, comprising: A first plane extraction module, used to extract at least one first plane from the first three-dimensional point cloud; A second plane extraction module, used to extract at least one second plane from the second three-dimensional point cloud; A plane feature histogram acquisition module, used to acquire a first plane feature histogram of the first plane and a second plane feature histogram of the second plane; A matching module, configured to match the first plane and the second plane according to the first plane feature histogram and the second plane feature histogram to obtain a plane matching pair that meets a preset condition; A conversion parameter calculation module, used to calculate conversion parameters in a four-dimensional plane space according to the plane matching pair, wherein the conversion parameters represent parameters related to registering the first plane and the second plane; as well as A registration module is used to register the first three-dimensional point cloud and the second three-dimensional point cloud based on the conversion parameters.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

Images