Point cloud rigid body registration method and device, electronic equipment and readable storage medium

Abstract

The invention provides a point cloud rigid body registration method and device, electronic equipment and a readable storage medium, and the method comprises the steps: photographing a plurality of point cloud images for each to-be-registered point cloud data set through a virtual camera at different poses, and inputting the point cloud images into a preset three-dimensional visual processing tool for pose estimation; based on a virtual camera pose of a virtual camera during shooting of the point cloud image corresponding to each to-be-registered point cloud data set and shooting pose estimation of the point cloud image corresponding to each to-be-registered point cloud data set, a conversion relation between a coordinate space of each to-be-registered point cloud data set and a target coordinate space is obtained; and the point cloud data in each point cloud data set to be registered can be easily converted into the same coordinate space based on the conversion relation, so that the registration among the point cloud data sets to be registered is realized. The scheme of the invention has the characteristics of small calculation amount, high registration efficiency and high registration precision.

Description

Technical Field

[0001] This application relates to the technical field of point cloud registration. Specifically, it relates to a point cloud rigid body registration method, device, electronic device, and readable storage medium. Background Art

[0002] The point cloud registration task is a common problem in the fields of computer vision and robotics. It involves aligning two or more point cloud data sets into the same coordinate space. This is usually for merging data from multiple perspectives, tracking and positioning, 3D reconstruction, or comparing scan results at different time points.

[0003] Point cloud rigid body registration belongs to a type of point cloud registration. During the registration process, it is assumed that the object does not undergo any deformation, that is, the shape and size of the object remain unchanged, only the position, orientation, or scale changes. Point cloud rigid body registration usually involves transformations such as translation, rotation, and scaling, and is usually applicable to scenarios where the shape and size of the object remain unchanged. For example, it is applicable to scenarios where, under machine vision, it is necessary to align object models from different perspectives for further analysis.

[0004] Common point cloud registration algorithms include the Iterative Closest Point (ICP) algorithm, coordinate descent method, feature matching method, and deep learning-based methods. Among them, the Iterative Closest Point algorithm has problems such as being prone to falling into local optima, being sensitive to noise and outliers, and having low accuracy of calculation results. The coordinate descent method requires a good initial estimate, has a large amount of calculation, and has a problem of low registration efficiency. The feature matching method needs to match significant feature points to improve the registration accuracy, has a strong dependence on the feature extraction algorithm. On the one hand, there is a problem of low registration efficiency, and on the other hand, the registration result depends on the accuracy of the feature extraction algorithm, and the feature extraction algorithm has a high risk of failure in feature sparse regions, thus affecting the registration accuracy. And deep learning-based methods require a large amount of training data and have high requirements for computing resources. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a point cloud rigid body registration method with simple calculation, high registration efficiency, and high registration accuracy, and to provide a device, electronic device, and readable storage medium corresponding to the point cloud rigid body registration method.

[0006] The embodiments of this application provide a point cloud rigid body registration method, including: obtaining each point cloud data set to be registered; wherein, different point cloud data sets to be registered are respectively located in different coordinate spaces, and there is the same point cloud data between each point cloud data set to be registered and at least one other point cloud data set except this point cloud data set to be registered.

[0007] For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, use a virtual camera to capture the point cloud dataset to be registered at different poses respectively, and obtain multiple point cloud images corresponding to the point cloud dataset to be registered;

[0008] Input all the point cloud images corresponding to all the point cloud datasets to be registered into a preset 3D vision processing tool for pose estimation, and obtain the pose estimation of the capture pose of each point cloud image in the target coordinate space;

[0009] For each point cloud dataset to be registered: Determine the conversion relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space according to the pose estimation of the capture pose of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered is captured;

[0010] According to the conversion relationship between the coordinate spaces of the point cloud datasets to be registered and the target coordinate space, convert the point cloud data in each point cloud dataset to the same coordinate space.

[0011] In the above implementation solution, after capturing multiple point cloud images of each point cloud dataset to be registered with a virtual camera at different poses and inputting them into a preset 3D vision processing tool for pose estimation, the pose estimation of each captured point cloud image is in the same target coordinate space. Then, based on the actual pose of the virtual camera (i.e., the virtual camera pose) when each point cloud image corresponding to each point cloud dataset to be registered is captured, and the pose estimation of the capture pose of each point cloud image corresponding to each point cloud dataset to be registered, it is easy to obtain the conversion relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space. Thus, based on this conversion relationship, it is easy to convert the point cloud data in each point cloud dataset to the same coordinate space, realizing the registration of each point cloud dataset to be registered. The entire solution uses the pose of the virtual camera to obtain the conversion relationship between the coordinate spaces of each point cloud dataset to be registered and the target coordinate space. The amount of calculation involved in the whole process is small, the registration efficiency is high. In addition, the conversion relationship obtained by this method is highly reliable, so the accuracy of the point cloud data registration result obtained by converting the point cloud data in each point cloud dataset to the same coordinate space is also relatively high (i.e., having a high registration accuracy). Compared with various methods introduced in the related art, the solution of this application is smaller than methods such as the coordinate descent method and the feature matching method, and has the characteristics of simple calculation and high registration efficiency. Compared with the iterative closest point algorithm, the calculation result is more accurate. Compared with the feature matching method, the registration result does not depend on the accuracy of the feature extraction algorithm, and the reliability of the registration result is also higher. Compared with the deep learning-based method, it also does not require a large amount of training data and has lower requirements for computing resources.

[0012] Further, for each point cloud dataset to be registered: in the coordinate space of the point cloud dataset to be registered, use a virtual camera to capture the point cloud dataset to be registered at different poses respectively, and obtain multiple point cloud images corresponding to the point cloud dataset to be registered, including:

[0013] For each point cloud dataset to be registered: in the coordinate space of the point cloud dataset to be registered, on a preset pose trajectory, use a virtual camera to capture the point cloud dataset to be registered at preset intervals in sequence, and obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0014] In the above implementation, by using a preset pose trajectory and using a virtual camera to capture the point cloud dataset to be registered at preset intervals in sequence, on the one hand, a large number of different point cloud images can be effectively obtained, and on the other hand, it is also convenient to record the pose of the virtual camera, so as to facilitate the subsequent determination of the conversion relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0015] Further, for each point cloud dataset to be registered: determine the conversion relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space according to the estimated capture pose of each point cloud image corresponding to the point cloud dataset to be registered and the pose of the virtual camera when each point cloud image corresponding to the point cloud dataset to be registered is captured, including:

[0016] For each point cloud dataset to be registered: substitute the estimated capture pose of each point cloud image corresponding to the point cloud dataset to be registered and the pose of the virtual camera when each point cloud image corresponding to the point cloud dataset to be registered is captured into a preset rigid body transformation formula, and obtain a coordinate system transformation matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space; wherein, the coordinate system transformation matrix is used to represent the conversion relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0017] In the above implementation, by substituting the estimated capture pose of each point cloud image corresponding to the point cloud dataset to be registered and the pose of the virtual camera when each point cloud image corresponding to the point cloud dataset to be registered is captured into a preset rigid body transformation formula, the coordinate system transformation matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space can be obtained, and thus the configuration of point cloud data can be easily realized based on the coordinate system transformation matrix. The scheme is simple and reliable to implement, has high registration efficiency, and high registration accuracy.

[0018] Furthermore, each of the point cloud datasets to be registered includes: a basic point cloud dataset to be registered and a target point cloud dataset to be registered other than the basic point cloud dataset to be registered; the coordinate space of the basic point cloud dataset to be registered is the basic coordinate space; according to the conversion relationship between the coordinate spaces of each point cloud dataset to be registered and the target coordinate space, converting the point cloud data in each point cloud dataset to be registered to the same coordinate space includes:

[0019] Obtain the basic transformation matrix when the target coordinate space is transformed to the basic coordinate space; wherein, the basic transformation matrix is the inverse matrix of the coordinate system transfer matrix between the basic coordinate space and the target coordinate space;

[0020] For each of the target point cloud datasets to be registered: according to the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space, and the basic transformation matrix, convert the point cloud data in the target point cloud dataset to be registered to the basic coordinate space.

[0021] Through the above implementation method, the point cloud data in each point cloud dataset to be registered can be unified into the coordinate space of a point cloud dataset to be registered, which has good applicability in some scenarios where it is specified that the coordinate space needs to be unified.

[0022] Furthermore, for each of the target point cloud datasets to be registered: according to the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space, and the basic transformation matrix, converting the point cloud data in the target point cloud dataset to be registered to the basic coordinate space includes:

[0023] For each of the target point cloud datasets to be registered:

[0024] Calculate the product of the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space and the basic transformation matrix to obtain the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space;

[0025] According to the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space, convert the point cloud data in the target point cloud dataset to be registered to the basic coordinate space.

[0026] In the above implementation, by calculating the product of the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space and the basic transformation matrix, the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space is obtained. Then, the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space can be used to convert the point cloud data in the target point cloud dataset to be registered into the basic coordinate space. The computational complexity of the whole process is very low, and the configuration process is simple and reliable.

[0027] Further, according to the conversion relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, converting the point cloud data in each point cloud dataset to be registered into the same coordinate space includes: respectively converting the point cloud data in each point cloud dataset to be registered into the target coordinate space according to the coordinate system transformation matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space.

[0028] In the above implementation, directly using the coordinate system transformation matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space to convert the point cloud data in each point cloud dataset to be registered into the target coordinate space, the whole registration process is simpler and the registration efficiency is higher.

[0029] Further, the 3D vision processing tool is colmap.

[0030] The embodiment of the present application further provides a point cloud rigid body registration device, including: an acquisition module, configured to acquire each point cloud data set to be registered; wherein, different point cloud data sets to be registered are respectively located in different coordinate spaces, and there is the same point cloud data between each point cloud data set to be registered and at least one other point cloud data set except this point cloud data set to be registered; a shooting module, configured to, for each point cloud data set to be registered: in the coordinate space of this point cloud data set to be registered, use a virtual camera to shoot the point cloud data set to be registered respectively with different poses, and obtain multiple point cloud images corresponding to this point cloud data set to be registered; a pose estimation module, configured to input all the point cloud images corresponding to all the point cloud data sets to be registered into a preset three-dimensional vision processing tool for pose estimation, and obtain the shooting pose estimation of each point cloud image in the target coordinate space; a conversion relationship determination module, configured to, for each point cloud data set to be registered: determine the conversion relationship between the coordinate space of this point cloud data set to be registered and the target coordinate space according to the shooting pose estimation of each point cloud image corresponding to this point cloud data set to be registered and the virtual camera pose when each point cloud image corresponding to this point cloud data set to be registered is shot; a coordinate conversion module, configured to convert the point cloud data in each point cloud data set to be registered to the same coordinate space according to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space.

[0031] The embodiment of the present application further provides an electronic device, including: a processor and a memory; the processor is configured to execute one or more programs stored in the memory to implement the point cloud rigid body registration method in any one of the above.

[0032] The embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the point cloud rigid body registration method in any one of the above. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0034] Figure 1 It is a schematic flowchart of the basic process of a point cloud rigid body registration method provided by an embodiment of the present application;

[0035] Figure 2 It is a schematic diagram of a virtual camera shooting method provided by an embodiment of the present application;

[0036] Figure 3 This is an example diagram of a point cloud image set obtained by a virtual camera provided in an embodiment of the present application;

[0037] Figure 4 This is a schematic structural diagram of a point cloud rigid body registration device provided in an embodiment of the present application;

[0038] Figure 5 This is a schematic structural diagram of an electronic device provided in an embodiment of the present application. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0040] To achieve point cloud rigid body registration, an embodiment of the present application provides a new point cloud rigid body registration method, which has the characteristics of simple calculation, high registration efficiency, and high registration accuracy. It can be seen in Figure 1 as shown in Figure 1 This is a schematic basic flow diagram of the point cloud rigid body registration method provided in an embodiment of the present application, including:

[0041] S101: Obtain each point cloud data set to be registered.

[0042] In an embodiment of the present application, the point cloud data set to be registered can be manually input by an engineer or a user, or can be transmitted by an upper-layer software (such as computer vision software, but not limited thereto) or a superior device (such as a camera, but not limited thereto).

[0043] In an embodiment of the present application, multiple point cloud data sets to be registered are obtained, and each point cloud data set to be registered has multiple point cloud data.

[0044] In an embodiment of the present application, different point cloud data sets to be registered are respectively located in different coordinate spaces, and there is the same point cloud data between each point cloud data set to be registered and at least one other point cloud data set except this point cloud data set to be registered.

[0045] Exemplarily, assume that there are three point cloud data sets to be registered, namely A, B, and C. Assume that there is a part of the same point cloud between the point cloud data set A to be registered and the point cloud data set B to be registered, and there is a part of the same point cloud between the point cloud data set B to be registered and the point cloud data set C to be registered. At this time, regardless of whether there is the same point cloud data between the point cloud data set A to be registered and the point cloud data set C to be registered, the point cloud data sets A, B, and C to be registered all satisfy that there is the same point cloud data between each point cloud data set to be registered and at least one other point cloud data set except this point cloud data set to be registered.

[0046] S102: For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, use a virtual camera to capture the point cloud dataset to be registered with different poses respectively, and obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0047] In an embodiment of the present application, the virtual camera can be a camera model implemented by program code, which can form an image by means of projection imaging and the like for three-dimensional points (i.e., point cloud data), and is called a point cloud image in the embodiment of the present application.

[0048] In an embodiment of the present application, after the virtual camera captures a point cloud image, it is necessary to record the pose of the virtual camera when each point cloud image is captured (denoted as the virtual camera pose in this article). The virtual camera pose includes the position and orientation of the virtual camera when capturing the point cloud image. Among them, the position refers to the position of the virtual camera pose in the coordinate space where the point cloud image is captured. In an embodiment of the present application, the virtual camera pose can be recorded by means of a rotation matrix and the like, but this is not a limitation. And how to record the pose using a rotation matrix belongs to well-known technology, so it will not be elaborated here.

[0049] In a feasible implementation manner of an embodiment of the present application, for each point cloud dataset to be registered: in the coordinate space of the point cloud dataset to be registered, on a preset pose trajectory, use the virtual camera to capture the point cloud dataset to be registered in sequence at a preset interval, and obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0050] In an embodiment of the present application, the pose trajectory may include two parts: a preset orbit and the camera orientation. Among them, the preset orbit is used to limit the position that the virtual camera can be set, and the camera orientation is used to limit the orientation of the virtual camera.

[0051] Optionally, for different point cloud datasets to be registered, different pose trajectories can be specified in advance according to the distribution size of the point cloud dataset to be registered in its coordinate space. For example, the pose trajectory can be set as: a circular orbit (i.e., the preset orbit) at a position set at a set distance (such as setting the distance to 10) from the point cloud dataset to be registered, and the virtual camera needs to always face the point cloud dataset to be registered (i.e., the camera orientation).

[0052] In an embodiment of the present application, the preset interval can be set according to the size of the pose trajectory. Exemplarily, for the aforementioned pose trajectory, the preset interval can be set as an angular interval, such as setting it to 7.2 degrees, that is, taking a shot every 7.2 degrees. In addition, the preset interval can also be set as the interval distance between different points on the orbit, that is, taking a shot every time the interval distance on the orbit is reached.

[0053] In this way, by using a virtual camera to capture the point cloud data set to be registered in the manner of preset pose trajectory and preset interval, a large number of different point cloud images can be effectively obtained, which is convenient for recording the pose of the virtual camera, and thus convenient for determining the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space subsequently.

[0054] In another feasible implementation manner of the embodiment of the present application, for each point cloud data set to be registered: the shooting position and orientation of the virtual camera can also be randomly or artificially set in the coordinate space of the point cloud data set to be registered, and then the point cloud data set to be registered is captured to obtain multiple point cloud images corresponding to the point cloud data set to be registered. At this time, each time the shooting position and orientation of the virtual camera are randomly generated or artificially set, the shooting position and orientation are recorded to generate the corresponding virtual camera pose.

[0055] S103: Input all the point cloud images corresponding to all the point cloud data sets to be registered into a preset three-dimensional vision processing tool for pose estimation to obtain the estimated shooting pose of each point cloud image in the target coordinate space.

[0056] In the embodiment of the present application, the preset three-dimensional vision processing tool can be implemented by using all software that can realize the shooting pose estimation based on images. For example, it can be but not limited to colmap, iTwin Capture, GodWork, etc. In this way, by using the capabilities of the preset three-dimensional vision processing tool, the shooting poses of all point cloud images can be estimated in the same coordinate space (hereinafter referred to as the target coordinate space used by the three-dimensional vision processing tool), and recorded as the estimated shooting poses.

[0057] In the embodiment of the present application, since there is the same point cloud data between each point cloud data set to be registered and at least one other point cloud data set except the point cloud data set to be registered, then among the multiple point cloud images captured for each point cloud data set to be registered, there will be at least some regions or contents in one or some point cloud images that match at least some regions or contents in one or some point cloud images among the multiple point cloud images captured for at least one other point cloud data set to be registered. Thus, the three-dimensional vision processing tool can estimate the shooting poses of all point cloud images in the target coordinate space based on these matching regions or contents, and the obtained estimated shooting poses can have high accuracy.

[0058] S104: For each point cloud data set to be registered: Determine the transformation relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space according to the estimated shooting pose of each point cloud image corresponding to the point cloud data set to be registered and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is captured.

[0059] In an alternative implementation manner of the embodiment of the present application, when performing the above step S104, for each point cloud data set to be registered: Substitute the estimated shooting pose of each point cloud image corresponding to the point cloud data set to be registered and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is captured into a preset rigid body transformation formula to obtain a coordinate system transformation matrix between the coordinate space of the point cloud data set to be registered and the target coordinate space; wherein, the coordinate system transformation matrix is used to represent the transformation relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space.

[0060] Among them, the rigid body transformation formula can be L i *M ic = C i . Among them, L i represents the virtual camera poses corresponding to the i-th point cloud data set to be registered, C i represents the estimated shooting poses of the point cloud images corresponding to the i-th point cloud data set to be registered, and M ic represents the coordinate system transformation matrix between the coordinate space of the i-th point cloud data set to be registered and the target coordinate space.

[0061] Exemplarily, assume there are two point cloud data sets A and B to be registered. The set of virtual camera poses corresponding to the point cloud data set A to be registered is denoted as La = {L a1 , L a2 ,...L an}, and the set of virtual camera poses corresponding to the point cloud data set B to be registered is denoted as Lb = {L b1 , L b2 ,...L bm}. Assume that the estimated shooting poses output by the 3D vision processing tool are C a1 , C a2 ,...C an , C b1 , C b2 ,...C bm , where C a1 , C a2 ,...C an correspond to L a1 , L a2 ,...L an respectively, and C b1 , Cb2 ,...C bm are respectively corresponding to L b1 , L b2 ,...L bm , then there is:

[0062] La * M ac = Ca, Lb * M bc = Cb; where: Ca = {C a1 , C a2 ,...C an}, Cb = {C b1 ,

[0063] C b2 ,...C bm}.

[0064] Thus, through the inverse operation, the coordinate system transformation matrix M ac from the coordinate space of the point cloud data set A to be registered to the target coordinate space can be calculated, and the coordinate system transformation matrix M bc from the coordinate space of the point cloud data set B to be registered to the target coordinate space can be calculated.

[0065] It can be understood that in the embodiment of the present application, the size of the coordinate system transformation matrix can be preset, for example, it can be a 4 * 4 matrix. At this time, when performing the inverse operation of matrix multiplication, since the number of data in La and Ca, and Lb and Cb may exceed the number of data required to calculate the element values of each element in the 4 * 4 matrix, an overdetermined problem may occur. When there is an overdetermined problem, in the embodiment of the present application, methods such as but not limited to the least squares method and the regularization method can be used to solve it, but it is not limited.

[0066] S105: According to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space, convert the point cloud data in each point cloud data set to the same coordinate space.

[0067] In an alternative embodiment of the embodiment of the present application, the point cloud data in each point cloud data set to be registered can be respectively converted to the target coordinate space according to the coordinate system transformation matrix between the coordinate space of each point cloud data set to be registered and the target coordinate space.

[0068] Exemplarily, taking the example of having two point cloud data sets A and B to be registered as shown in the previous text, the coordinate system transformation matrix from the coordinate space of the point cloud data set A to be registered to the target coordinate space is M ac , and the coordinate system transformation matrix from the coordinate space of the point cloud data set B to be registered to the target coordinate space is M bc . For example, assuming that the point cloud data in the point cloud data set A includes Pa = {P a1 , Pa2 ,...P an},The point cloud data in the point cloud data set B to be registered includes Pa = {P b1 , P b2 ,...P bm}. Then: Calculate Pa*M ac and Pb*M bc respectively. All the point cloud data calculated in this way are all transformed into the target coordinate space, realizing the unification of the coordinate systems of the point cloud data set A to be registered and the point cloud data set B to be registered, and realizing the point cloud registration between the point cloud data set A to be registered and the point cloud data set B to be registered.

[0069] The above optional implementation directly uses the coordinate system transformation matrix between the coordinate space of each point cloud data set to be registered and the target coordinate space to transform all the point cloud data in each point cloud data set to be registered into the target coordinate space. The entire registration process is simpler and the registration efficiency is higher.

[0070] In another optional implementation of the embodiment of the present application, each point cloud data set to be registered can be divided into a basic point cloud data set to be registered and a target point cloud data set to be registered other than the basic point cloud data set to be registered. Denote the coordinate space of the basic point cloud data set to be registered as the basic coordinate space. This optional implementation can transform the point cloud data in each target point cloud data set to be registered into this basic coordinate space, so as to realize the point cloud registration between each point cloud data set to be registered.

[0071] Exemplarily, the basic transformation matrix when the target coordinate space is transformed into the basic coordinate space can be obtained first. Among them, the basic transformation matrix is the inverse matrix of the coordinate system transformation matrix between the basic coordinate space and the target coordinate space.

[0072] Then, for each target point cloud data set to be registered: According to the coordinate system transformation matrix between the coordinate space of this target point cloud data set to be registered and the target coordinate space, and the basic transformation matrix, transform the point cloud data in this target point cloud data set to be registered into the basic coordinate space.

[0073] It can be understood that in some application scenarios, there is a situation where a coordinate space to be unified is specified. For example, in some application scenarios, it is necessary to uniformly transform all point cloud data into the coordinate space where the latest obtained point cloud data set to be registered is located. Then, based on the above optional implementation, the coordinate space where the latest obtained point cloud data set to be registered is located can be used as the basic coordinate space to realize the unified transformation of all point cloud data into the coordinate space where the latest obtained point cloud data set to be registered is located.

[0074] Exemplarily, still taking the two point cloud datasets A and B to be registered as shown in the previous text, the coordinate system transformation matrix from the coordinate space of the point cloud dataset A to the target coordinate space is M ac , and the coordinate system transformation matrix from the coordinate space of the point cloud dataset B to the target coordinate space is M bc as an example. Assume that the point cloud data in the point cloud dataset A to be registered includes Pa = {P a1 , P a2 ,... P an}. Assume that the point cloud dataset B is the basic point cloud dataset to be registered, then Pa * M ac * M bc -1 can be calculated, so as to transform each point cloud data in the point cloud dataset A to be registered into the coordinate space of the point cloud dataset B. Among them, M bc -1 is the inverse matrix of M bc , and is the basic transformation matrix in this example.

[0075] In a feasible embodiment of the above optional embodiment, for each target point cloud dataset to be registered: the product of the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space and the basic transformation matrix can be calculated first to obtain the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space. Taking the above example as an example, that is, M ac * M bc -1 can be calculated first to obtain the coordinate system transformation matrix from the coordinate space of the point cloud dataset A to be registered to the coordinate space of the point cloud dataset B.

[0076] Then, according to the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space, the point cloud data in the target point cloud dataset to be registered is transformed into the basic coordinate space.

[0077] In another feasible embodiment of the above optional embodiment, for each target point cloud dataset to be registered: it is also possible to first transform the point cloud data of the target point cloud dataset to be registered into the target coordinate space according to the coordinate system transformation matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space. Taking the above example as an example, that is, Pa * M ac can be calculated first to obtain the point cloud dataset Pac of the target point cloud dataset A in the target coordinate space.

[0078] Then, according to the basic transformation matrix, the point cloud data of the target point cloud dataset transformed into the target coordinate space is transformed into the basic coordinate space. Taking the above example as an example, that is, calculating Pac * M bc-1 .

[0079] Based on the point cloud rigid body registration method provided by the embodiments of the present application, after multiple point cloud images are taken of each point cloud data set to be registered with different poses using a virtual camera and input into a preset three-dimensional vision processing tool for pose estimation, the pose estimation of each captured point cloud image is in the same target coordinate space. Then, based on the actual pose of the virtual camera (i.e., the virtual camera pose) when the point cloud images corresponding to each point cloud data set to be registered are captured, and the pose estimation of the point cloud images corresponding to each point cloud data set to be registered, it is easy to obtain the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space. Thus, based on this conversion relationship, it is easy to convert the point cloud data in each point cloud data set to be registered into the same coordinate space, realizing the registration between each point cloud data set to be registered. The entire solution uses the pose of the virtual camera to obtain the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space. The amount of calculation involved in the entire process is small, the registration efficiency is high. In addition, the conversion relationship obtained by this method is highly reliable, and thus the accuracy of the point cloud data registration result obtained by converting the point cloud data in each point cloud data set to be registered into the same coordinate space is also relatively high. Compared with various methods introduced in the related art, the solution of the present application is smaller than methods such as the coordinate descent method and the feature matching method, and has the characteristics of simple calculation and high registration efficiency. Compared with the iterative closest point algorithm, the calculation result is more accurate. Compared with the feature matching method, the registration result does not depend on the accuracy of the feature extraction algorithm, and the reliability of the registration result is also higher. Compared with the method based on deep learning, it also does not require a large amount of training data and has lower requirements for computing resources.

[0080] To facilitate the understanding of the solution of the embodiments of the present application, the following takes a specific implementation manner as an example to further illustrate the present application.

[0081] Suppose the point cloud data set A to be registered and the point cloud data set B to be registered are obtained. Each point cloud data set to be registered has 50 pieces of point cloud data, which are respectively denoted as: Pa = {P a1 , P a2 ,...P a50}, Pb = {P b1 , P b2 ,...P b50}.

[0082] Set a pose trajectory for each point cloud dataset to be registered, including a pose trajectory that is always facing the point cloud dataset to be registered and is at a distance of 10 from the point cloud dataset to be registered. Control the virtual camera to capture a point cloud image every 7.2 degrees along this pose trajectory, and record the pose of the virtual camera at each capture point. Here, it is assumed that 50 images are captured respectively, and they are denoted as: La = {L a1 ,L a2 ,...L a50}, Lb = {L b1 ,L b2 ,...L b50}. As shown in Figure 2 and Figure 3 . Figure 2 The red part in Figure 3 is the example virtual camera, and the black dense part towards which the virtual camera is facing is the point cloud dataset to be registered. Figure 2 is the set of point cloud images captured by

[0083] Take the 100 obtained point cloud images as an image set and input them into colmap for estimating the capture poses. After colmap finishes the calculation, record the estimated capture poses of each point cloud image among the 100 point cloud images in a json file.

[0084] According to the correspondence between each point cloud image and the point cloud dataset to be registered and the correspondence with the virtual camera pose, extract from the json file the set of estimated capture poses corresponding to each virtual camera pose Ca = {C a1 ,C a2 ,...C a50}, Cb = {C b1 ,C b2 ,...C b50}.

[0085] Use the formula of rigid body transformation: La * M ac = Ca to calculate the coordinate system transformation matrix M ac from the coordinate space of the point cloud dataset A to be registered to the target coordinate space, and use Lb * M bc = Cb to calculate the coordinate system transformation matrix M bc from the coordinate space of the point cloud dataset B to be registered to the target coordinate space.

[0086] Calculate Pa * M ac * M bc -1 , and transform each point cloud data in the point cloud dataset A to the coordinate space of the point cloud dataset B to complete the point cloud registration.

[0087] The above scheme has the characteristics of simple calculation, high efficiency, high accuracy of point cloud registration algorithm, and can adapt to the registration tasks between various numbers of point cloud data sets to be registered.

[0088] Based on the same inventive concept, the present application also provides a point cloud rigid body registration device 400. Figure 4 As shown, Figure 4 Shows the use of Figure 1 The point cloud rigid body registration device of the method shown. It should be understood that the specific functions of the device 400 can be referred to the description above, and the detailed description is appropriately omitted here to avoid repetition. The device 400 includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system of the device 400. Specifically:

[0089] See also Figure 4 As shown, the device 400 includes: an acquisition module 401, a shooting module 402, a posture estimation module 403, a conversion relationship determination module 404 and a coordinate conversion module 405. Among them:

[0090] The acquisition module 401 is used to acquire each point cloud data set to be registered; wherein different point cloud data sets to be registered are located in different coordinate spaces, and each point cloud data set to be registered has the same point cloud data as at least one other point cloud data set to be registered except the point cloud data set to be registered;

[0091] The shooting module 402 is used for: for each point cloud data set to be registered, using a virtual camera to shoot the point cloud data set to be registered in different positions in the coordinate space of the point cloud data set to be registered, so as to obtain multiple point cloud images corresponding to the point cloud data set to be registered;

[0092] The pose estimation module 403 is used to input all point cloud images corresponding to all point cloud data sets to be registered into a preset 3D visual processing tool for pose estimation, and obtain the shooting pose estimation of each point cloud image in the target coordinate space;

[0093] The conversion relationship determination module 404 is used to determine, for each point cloud data set to be registered: according to the shooting pose estimation of each point cloud image corresponding to the point cloud data set to be registered and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is shot, the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space;

[0094] The coordinate conversion module 405 is used to convert the point cloud data in each of the point cloud data sets to be registered into the same coordinate space according to the conversion relationship between the coordinate space of each of the point cloud data sets to be registered and the target coordinate space.

[0095] In a feasible implementation manner of the embodiment of the present application, the shooting module 402 is specifically configured to: for each point cloud data set to be registered: in the coordinate space of the point cloud data set to be registered, on a preset pose trajectory, use a virtual camera to sequentially shoot the point cloud data set to be registered at preset intervals, and obtain multiple point cloud images corresponding to the point cloud data set to be registered.

[0096] In a feasible implementation manner of the embodiment of the present application, the conversion relationship determination module 404 is specifically configured to: for each point cloud data set to be registered: substitute the estimated shooting pose of each point cloud image corresponding to the point cloud data set to be registered and the pose of the virtual camera when each point cloud image corresponding to the point cloud data set to be registered is shot into a preset rigid body transformation formula, and obtain a coordinate system transformation matrix between the coordinate space of the point cloud data set to be registered and the target coordinate space; wherein, the coordinate system transformation matrix is used to represent the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space.

[0097] In an alternative embodiment of this feasible implementation manner, each of the point cloud data sets to be registered includes: a basic point cloud data set to be registered and a target point cloud data set to be registered other than the basic point cloud data set to be registered; the coordinate space of the basic point cloud data set to be registered is the basic coordinate space;

[0098] The conversion relationship determination module 404 is specifically configured to:

[0099] Obtain a basic transformation matrix when the target coordinate space is transformed into the basic coordinate space; wherein, the basic transformation matrix is the inverse matrix of the coordinate system transformation matrix between the basic coordinate space and the target coordinate space;

[0100] For each of the target point cloud data sets to be registered: according to the coordinate system transformation matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space, and the basic transformation matrix, transform the point cloud data in the target point cloud data set to be registered into the basic coordinate space.

[0101] In this alternative embodiment, the conversion relationship determination module 404 is specifically configured to: for each of the target point cloud data sets to be registered: calculate the product of the coordinate system transformation matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space and the basic transformation matrix, and obtain a coordinate system transformation matrix between the coordinate space of the target point cloud data set to be registered and the basic coordinate space; according to the coordinate system transformation matrix between the coordinate space of the target point cloud data set to be registered and the basic coordinate space, transform the point cloud data in the target point cloud data set to be registered into the basic coordinate space.

[0102] In another alternative embodiment of the feasible embodiment of the present invention, the conversion relationship determination module 404 is specifically configured to: respectively convert the point cloud data in each point cloud data set to be registered into the target coordinate space according to the coordinate system transformation matrix between the coordinate space of each point cloud data set to be registered and the target coordinate space.

[0103] In a feasible embodiment of the embodiment of the present application, the 3D vision processing tool is colmap.

[0104] It should be understood that for the sake of brevity of description, the content described in some method embodiments will not be repeated in the device embodiments.

[0105] Based on the same inventive concept, the embodiment of the present application provides an electronic device. Refer to Figure 5 As shown, it includes a processor 501 and a memory 502. Among them:

[0106] The processor 501 is configured to execute one or more programs stored in the memory 502 to implement the above-mentioned point cloud rigid body registration method.

[0107] The processor 501 may be a data processing core of a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), an AI (Artificial Intelligence) processor, an NPU (Neural network Processing Unit), an ISP (Image Signal Processor), a DPU (Display Processing Unit), a VPU (Video Processing Unit), a DSP (Digital Signal Processor), etc., or a processor chip applied to some scenarios of large-scale data operations. The memory 502 may be a RAM (Random Access Memory), a ROM (Read-Only Memory), or a flash memory. The above are only examples and should not limit the present application.

[0108] Figure 5 The structure shown is only schematic. The electronic device may further include more or fewer components than those Figure 5 shown, or have the same as Figure 5The different configurations shown. For example, it may also have an internal communication bus for implementing communication between the processor 501 and the memory 502; for another example, it may also have an external communication interface, such as a USB (Universal Serial Bus) interface, a CAN (Controller Area Network) bus interface, etc.; for another example, it may also have an information display component such as a display screen, etc., but this is not a limitation.

[0109] In the embodiments of the present application, the electronic device may be a device with data processing capabilities such as a server, a smart terminal (such as a host, a smart phone, a laptop computer, a desktop computer, a vehicle-mounted terminal, a smart operation platform, etc.), but this is not a limitation.

[0110] Based on the same inventive concept, this embodiment also provides a computer-readable storage medium, such as a floppy disk, an optical disc, a hard disk, a flash memory, a USB flash drive, an SD (Secure Digital Memory Card) card, an MMC (Multimedia Card) card, etc. One or more programs for implementing the above various steps are stored in the computer-readable storage medium, and these one or more programs can be executed by one or more processors to implement the above point cloud rigid body registration method. Details are not described herein again.

[0111] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be in an electrical, mechanical or other form.

[0112] In addition, the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] Furthermore, in each embodiment of the present application, the various functional modules may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0114] In this document, relational terms such as first and second are used solely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0115] In this document, "a plurality of" means two or more.

[0116] The above description is only for the embodiments of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, various modifications and changes can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A point cloud rigid body registration method, characterized in that: include: Acquire each point cloud data set to be registered; wherein different point cloud data sets to be registered are located in different coordinate spaces, and each point cloud data set to be registered has the same point cloud data as at least one other point cloud data set to be registered except the point cloud data set to be registered; For each point cloud data set to be registered: in the coordinate space of the point cloud data set to be registered, use a virtual camera to shoot the point cloud data set to be registered at different postures to obtain multiple point cloud images corresponding to the point cloud data set to be registered; Input all point cloud images corresponding to all point cloud data sets to be registered into the preset 3D visual processing tool for pose estimation, and obtain the shooting pose estimation of each point cloud image in the target coordinate space; For each point cloud data set to be registered: according to the shooting pose estimation of each point cloud image corresponding to the point cloud data set to be registered and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is shot, determine the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space; According to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space, the point cloud data in each point cloud data set to be registered are converted to the same coordinate space.

2. The point cloud rigid body registration method according to claim 1, characterized in that: For each point cloud data set to be registered: in the coordinate space of the point cloud data set to be registered, use a virtual camera to shoot the point cloud data set to be registered at different positions to obtain multiple point cloud images corresponding to the point cloud data set to be registered, including: For each point cloud data set to be registered: in the coordinate space of the point cloud data set to be registered, on a preset posture trajectory, use a virtual camera to photograph the point cloud data set to be registered in sequence at preset intervals to obtain multiple point cloud images corresponding to the point cloud data set to be registered.

3. The point cloud rigid body registration method according to claim 1, characterized in that: For each point cloud data set to be registered: according to the shooting pose estimation of each point cloud image corresponding to the point cloud data set to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is shot, the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space is determined, including: For each point cloud data set to be registered: the shooting pose estimation of each point cloud image corresponding to the point cloud data set to be registered and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered was shot are substituted into the preset rigid body transformation formula to obtain the coordinate system transfer matrix between the coordinate space of the point cloud data set to be registered and the target coordinate space; The coordinate system transfer matrix is ​​used to characterize the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space.

4. The point cloud rigid body registration method according to claim 3, characterized in that: Each of the point cloud data sets to be registered includes: a basic point cloud data set to be registered and a target point cloud data set to be registered other than the basic point cloud data set to be registered; the coordinate space of the basic point cloud data set to be registered is a basic coordinate space; According to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space, the point cloud data in each point cloud data set to be registered is converted to the same coordinate space, including: Acquire a basic transformation matrix when the target coordinate space is transformed into the basic coordinate space; wherein the basic transformation matrix is ​​an inverse matrix of a coordinate system transfer matrix between the basic coordinate space and the target coordinate space; For each of the target point cloud data sets to be registered: according to the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space, and the basic transformation matrix, the point cloud data in the target point cloud data set to be registered is converted to the basic coordinate space.

5. The point cloud rigid body registration method according to claim 4, characterized in that: For each of the target point cloud data sets to be registered: according to the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space, and the basic transformation matrix, the point cloud data in the target point cloud data set to be registered is converted to the basic coordinate space, including: For each of the target point cloud datasets to be registered: Calculate the product of the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space and the basic transformation matrix to obtain the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the basic coordinate space; According to the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the basic coordinate space, the point cloud data in the target point cloud data set to be registered is converted to the basic coordinate space.

6. The point cloud rigid body registration method according to claim 3, characterized in that: According to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space, the point cloud data in each point cloud data set to be registered is converted to the same coordinate space, including: According to the coordinate system transfer matrix between the coordinate space of each point cloud data set to be registered and the target coordinate space, the point cloud data in each point cloud data set to be registered are converted into the target coordinate space.

7. The point cloud rigid body registration method according to any one of claims 1 to 6, characterized in that: The three-dimensional visual processing tool is colmap.

8. A point cloud rigid body registration device, characterized in that: include: An acquisition module is used to acquire each point cloud data set to be registered; wherein different point cloud data sets to be registered are located in different coordinate spaces, and each point cloud data set to be registered has the same point cloud data as at least one other point cloud data set to be registered except the point cloud data set to be registered; A shooting module is used for shooting each point cloud data set to be registered using a virtual camera in different positions in the coordinate space of the point cloud data set to be registered, so as to obtain multiple point cloud images corresponding to the point cloud data set to be registered; The pose estimation module is used to input all point cloud images corresponding to all point cloud data sets to be registered into a preset 3D visual processing tool for pose estimation, and obtain the shooting pose estimation of each point cloud image in the target coordinate space; A conversion relationship determination module is used to determine, for each point cloud data set to be registered: according to the shooting pose estimation of each point cloud image corresponding to the point cloud data set to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is shot, the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space; The coordinate conversion module is used to convert the point cloud data in each point cloud data set to be registered into the same coordinate space according to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space.

9. An electronic device, characterized in that: include: Processor and memory; The processor is used to execute one or more programs stored in the memory to implement the point cloud rigid body registration method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the point cloud rigid body registration method as described in any one of claims 1-7.

Images