Point cloud rigid body registration method and device, electronic equipment and readable storage medium

Abstract

The application provides a point cloud rigid body registration method, device, electronic equipment and readable storage medium. The method comprises the following steps: using a virtual camera to shoot a plurality of point cloud images of each to-be-registered point cloud data set at different poses, and inputting the plurality of point cloud images into a preset three-dimensional vision processing tool for pose estimation; obtaining a conversion relationship between a coordinate space of each to-be-registered point cloud data set and a target coordinate space based on a virtual camera pose of the virtual camera when the point cloud images corresponding to the to-be-registered point cloud data set are shot, and a shooting pose estimation of the point cloud images; and converting point cloud data in each to-be-registered point cloud data set to the same coordinate space based on the conversion relationship, thereby realizing registration between the to-be-registered point cloud data sets. The scheme has the characteristics of small calculation amount, high registration efficiency and high registration accuracy.

Description

Technical Field

[0001] This application relates to the field of point cloud registration technology, and more specifically, to a point cloud rigid body registration method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Point cloud registration is a common task in computer vision and robotics, involving aligning two or more point cloud datasets into the same coordinate space. This is typically used for merging data from multiple perspectives, tracking and localization, 3D reconstruction, or comparing scan results from different time points.

[0003] Rigid body registration of point clouds is a type of point cloud registration. During the registration process, it is assumed that the object does not undergo any deformation; that is, the object's shape and size remain unchanged, only its position, orientation, or scale changes. Rigid body registration of point clouds typically involves transformations such as translation, rotation, and scaling. It is generally suitable for scenarios where the object's shape and size remain unchanged, such as in machine vision scenarios where it is necessary to align object models from different viewpoints for further analysis.

[0004] Common point cloud registration algorithms include Iterative Closest Point (ICP) algorithms, coordinate descent methods, feature matching methods, and deep learning-based methods. Among these, the ICP algorithm is prone to getting trapped in local optima, is sensitive to noise and outliers, and has relatively low accuracy. Coordinate descent requires good initial estimates, is computationally intensive, and suffers from low registration efficiency. Feature matching methods improve registration accuracy by matching salient feature points, making them highly dependent on feature extraction algorithms. This results in low registration efficiency, and the registration result depends on the accuracy of the feature extraction algorithm, which is prone to failure in sparse feature regions, thus affecting registration accuracy. Deep learning-based methods require large amounts of training data and have high computational resource requirements. Summary of the Invention

[0005] The purpose of this application is to provide a point cloud rigid body registration method that is computationally simple, has high registration efficiency, and can also have high registration accuracy, as well as to provide a corresponding device, electronic device, and readable storage medium for the point cloud rigid body registration method.

[0006] This application provides a point cloud rigid body registration method, including: acquiring each point cloud dataset to be registered; wherein, different point cloud datasets to be registered are located in different coordinate spaces, and each point cloud dataset to be registered has the same point cloud data with at least one other point cloud dataset to be registered.

[0007] For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, a virtual camera is used to take pictures of the point cloud dataset to be registered in different poses to obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0008] All point cloud images corresponding to all point cloud datasets to be registered are input into a preset 3D vision processing tool for pose estimation, so as to obtain the shooting pose estimate of each point cloud image in the target coordinate space.

[0009] For each point cloud dataset to be registered: based on the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot, the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space is determined.

[0010] Based on the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, the point cloud data in each point cloud dataset to be registered are transformed to the same coordinate space.

[0011] In the above implementation scheme, multiple point cloud images are captured by a virtual camera at different poses for each point cloud dataset to be registered, and then input into a preset 3D vision processing tool for pose estimation. In this way, the shooting pose estimation of each point cloud image is located in the same target coordinate space. Based on the actual pose of the virtual camera (i.e., virtual camera pose) of the point cloud images corresponding to each point cloud dataset to be registered during shooting, and the shooting pose estimation of the point cloud images corresponding to each point cloud dataset to be registered, the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space can be easily obtained. Based on this transformation relationship, the point cloud data in each point cloud dataset to be registered can be easily transformed to the same coordinate space, thereby achieving registration between the point cloud datasets to be registered. The entire scheme utilizes the pose of a virtual camera to obtain the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space. The computational load involved is relatively small, resulting in high registration efficiency. Furthermore, the transformation relationship obtained in this way is highly reliable, leading to high accuracy in the point cloud data registration results obtained by transforming the point cloud data in each dataset to the same coordinate space (i.e., high registration precision). Compared to various methods described in related technologies, the scheme in this application is smaller and features simpler computation and higher registration efficiency compared to coordinate descent and feature matching methods. Compared to the iterative nearest-point algorithm, the computational accuracy is higher. Compared to feature matching methods, the registration result does not depend on the accuracy of the feature extraction algorithm, resulting in higher reliability. Compared to deep learning-based methods, it does not require a large amount of training data and has lower computational resource requirements.

[0012] Furthermore, for each point cloud dataset to be registered: in the coordinate space of the point cloud dataset to be registered, a virtual camera is used to take pictures of the point cloud dataset to be registered in different poses, resulting in multiple point cloud images corresponding to the point cloud dataset to be registered, including:

[0013] For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, on the preset pose trajectory, the virtual camera is used to take pictures of the point cloud dataset to be registered at preset intervals to obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0014] In the above implementation, by presetting the pose trajectory and taking pictures of the point cloud dataset to be registered by using a virtual camera at preset intervals, a large number of different point cloud images can be obtained effectively. On the other hand, it is also convenient to record the pose of the virtual camera, which facilitates the subsequent determination of the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0015] Furthermore, for each point cloud dataset to be registered: based on the estimated shooting pose of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was captured, the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space is determined, including:

[0016] For each point cloud dataset to be registered: the estimated shooting pose of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was captured, are substituted into a preset rigid body transformation formula to obtain the coordinate system transition matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space; wherein, the coordinate system transition matrix is ​​used to characterize the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0017] In the above implementation, by substituting the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot into the preset rigid body transformation formula, the coordinate system transfer matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space can be obtained. Thus, the configuration of point cloud data can be easily realized based on the coordinate system transfer matrix. The solution is simple and reliable, with high registration efficiency and high registration accuracy.

[0018] Further, each of the point cloud datasets to be registered includes: a basic point cloud dataset to be registered and a target point cloud dataset to be registered other than the basic point cloud dataset to be registered; the coordinate space of the basic point cloud dataset to be registered is the basic coordinate space; according to the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, the point cloud data in each point cloud dataset to be registered are transformed to the same coordinate space, including:

[0019] Obtain the basic transformation matrix when the target coordinate space is transformed to the base coordinate space; wherein, the basic transformation matrix is ​​the inverse of the coordinate system transition matrix between the base coordinate space and the target coordinate space;

[0020] For each target point cloud dataset to be registered: based on the coordinate system transition matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space, and the basic transformation matrix, the point cloud data in the target point cloud dataset to be registered is transformed to the basic coordinate space.

[0021] The above implementation method can unify the point cloud data in each point cloud dataset to be registered into a coordinate space of a single point cloud dataset to be registered, which has good applicability in some scenarios where a unified coordinate space is required.

[0022] Further, for each target point cloud dataset to be registered: based on the coordinate system transition matrix between the coordinate space of the target point cloud dataset and the target coordinate space, and the basic transformation matrix, the point cloud data in the target point cloud dataset to be registered is transformed to the basic coordinate space, including:

[0023] For each of the target point cloud datasets to be registered:

[0024] Calculate the coordinate system transition matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space, and multiply it with the basic transformation matrix to obtain the coordinate system transition matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space.

[0025] Based on the coordinate system transition matrix between the coordinate space of the target point cloud dataset to be registered and the base coordinate space, the point cloud data in the target point cloud dataset to be registered is transformed to the base coordinate space.

[0026] In the above implementation, the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space is calculated, and the product of the coordinate system transfer matrix and the base transformation matrix is ​​obtained. Then, the point cloud data in the target point cloud dataset to be registered can be transformed to the base coordinate space using the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the base coordinate space. The entire process has a low computational load and the configuration process is simple and reliable.

[0027] Furthermore, based on the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, the point cloud data in each point cloud dataset to be registered is transformed to the same coordinate space, including: based on the coordinate system transfer matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space, the point cloud data in each point cloud dataset to be registered is transformed to the target coordinate space.

[0028] In the above implementation, the coordinate system transfer matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space is directly used to transform the point cloud data in each point cloud dataset to be registered into the target coordinate space. The whole registration process is simpler and the registration efficiency is higher.

[0029] Furthermore, the 3D vision processing tool is colmap.

[0030] This application embodiment also provides a point cloud rigid body registration device, including: an acquisition module, used to acquire each point cloud dataset to be registered; wherein, different point cloud datasets to be registered are located in different coordinate spaces, and each point cloud dataset to be registered has the same point cloud data with at least one other point cloud dataset to be registered besides itself; an image capture module, used for each point cloud dataset to be registered: using a virtual camera to capture images of the point cloud dataset to be registered in different poses in the coordinate space of the point cloud dataset to be registered, to obtain multiple point cloud images corresponding to the point cloud dataset to be registered; and a pose estimation module, used to estimate all point cloud images corresponding to all point cloud datasets to be registered. The data is input into a preset 3D vision processing tool for pose estimation to obtain the shooting pose estimate of each point cloud image in the target coordinate space. A transformation relationship determination module is used to determine the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space for each point cloud dataset to be registered, based on the shooting pose estimate of each point cloud image corresponding to the dataset and the virtual camera pose when each point cloud image was captured. A coordinate transformation module is used to transform the point cloud data in each point cloud dataset to be registered to the same coordinate space based on the transformation relationship between the coordinate spaces of each point cloud dataset to be registered and the target coordinate space.

[0031] This application also provides an electronic device, including a processor and a memory; the processor is used to execute one or more programs stored in the memory to implement any of the point cloud rigid body registration methods described above.

[0032] This application also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement any of the point cloud rigid body registration methods described above. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A basic flowchart of a point cloud rigid body registration method provided in this application embodiment;

[0035] Figure 2 This is a schematic diagram illustrating a virtual camera shooting method provided in an embodiment of this application;

[0036] Figure 3 An example diagram of a point cloud image set captured by a virtual camera, provided in an embodiment of this application;

[0037] Figure 4 A schematic diagram of a point cloud rigid body registration device provided in an embodiment of this application;

[0038] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0040] To achieve rigid body registration of point clouds, this application provides a novel method for rigid body registration of point clouds, which is characterized by simple calculation, high registration efficiency, and high registration accuracy. See also... Figure 1 As shown, Figure 1 This is a basic flowchart illustrating the point cloud rigid body registration method provided in the embodiments of this application, including:

[0041] S101: Obtain the cloud dataset of each point to be registered.

[0042] In the embodiments of this application, the point cloud dataset to be registered can be manually input by engineers or users, or it can be transmitted from upper-layer software (such as computer vision software, but not a limitation) or upper-level devices (such as cameras, but not a limitation).

[0043] In this embodiment of the application, multiple point cloud datasets to be registered are obtained, and each point cloud dataset contains multiple point cloud data.

[0044] In the embodiments of this application, different point cloud datasets to be registered are located in different coordinate spaces, and each point cloud dataset to be registered has the same point cloud data with at least one other point cloud dataset to be registered.

[0045] For example, suppose there are three point cloud datasets A, B, and C to be registered. Suppose that point cloud dataset A and point cloud dataset B have some point cloud data that are the same, and point cloud dataset B and point cloud dataset C have some point cloud data that are the same. In this case, regardless of whether there is any point cloud data that is the same between point cloud dataset A and point cloud dataset C, point cloud datasets A, B, and C all satisfy the condition that there is any point cloud data that is the same as at least one other point cloud dataset to be registered.

[0046] S102: For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, use a virtual camera to take pictures of the point cloud dataset to be registered in different poses to obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0047] In this embodiment, the virtual camera can be a camera model implemented by program code, which can form an image from three-dimensional points (i.e., point cloud data) through projection imaging and other methods, referred to as a point cloud image in this embodiment.

[0048] In this embodiment, after capturing point cloud images, the virtual camera needs to record the pose of the virtual camera at the time each point cloud image was captured (hereinafter referred to as the virtual camera pose). The virtual camera pose includes the position and orientation of the virtual camera when capturing the point cloud image. Here, position refers to the virtual camera pose's location in the coordinate space at the time the point cloud image was captured. In this embodiment, the virtual camera pose can be recorded using methods such as rotation matrices, but this is not a limitation. How to use rotation matrices to record poses is well-known technology and will not be elaborated upon here.

[0049] In one feasible implementation of this application, for each point cloud dataset to be registered: a virtual camera can be used to take pictures of the point cloud dataset to be registered sequentially at preset intervals on a preset pose trajectory in the coordinate space of the point cloud dataset to be registered, so as to obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0050] In this embodiment of the application, the pose trajectory may include two parts: a preset track and a camera orientation. The preset track is used to limit the position that the virtual camera can be set to, while the camera orientation is used to limit the orientation of the virtual camera.

[0051] Optionally, different pose trajectories can be pre-specified for different point cloud datasets to be registered, based on the distribution size of the point cloud datasets in their respective coordinate spaces. For example, the pose trajectory can be set as a circular track (i.e., a preset track) at a set distance (e.g., a distance of 10) from the point cloud dataset to be registered, and the virtual camera must always be facing the point cloud dataset to be registered (i.e., camera orientation).

[0052] In this embodiment, the preset interval can be set according to the size of the pose trajectory. For example, for the aforementioned pose trajectory, the preset interval can be set as an angular interval, such as 7.2 degrees, meaning that a shot is taken every 7.2 degrees. Alternatively, the preset interval can be set as the distance between different points on the track, meaning that a shot is taken every time the distance is reached on the track.

[0053] In this way, by using a virtual camera to capture images of the point cloud dataset to be registered through a preset pose trajectory and preset interval, it is possible to obtain a large number of different point cloud images and facilitate the recording of the virtual camera pose, thereby facilitating the subsequent determination of the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0054] In another feasible embodiment of this application, for each point cloud dataset to be registered: the shooting position and orientation of the virtual camera can be randomly or manually set in the coordinate space of the point cloud dataset to be registered, and then the point cloud dataset to be registered can be photographed to obtain multiple point cloud images corresponding to the point cloud dataset to be registered. At this time, each time the shooting position and orientation of the virtual camera are randomly generated or manually set, the shooting position and orientation are recorded to generate the corresponding virtual camera pose.

[0055] S103: Input all point cloud images corresponding to all point cloud datasets to be registered into the preset 3D vision processing tool for pose estimation to obtain the shooting pose estimate of each point cloud image in the target coordinate space.

[0056] In this embodiment, the preset 3D vision processing tool can be any software capable of image-based shooting pose estimation. For example, it can be, but is not limited to, colmap, iTwin Capture, GodWorks, etc. In this way, by utilizing the capabilities of the preset 3D vision processing tool, the shooting poses of all point cloud images can be estimated in the same coordinate space (hereinafter referred to as the target coordinate space) and recorded as shooting pose estimates.

[0057] In this embodiment, since each point cloud dataset to be registered has the same point cloud data with at least one other point cloud dataset to be registered, then among the multiple point cloud images captured for each point cloud dataset to be registered, at least some regions or contents in one or more point cloud images will match at least some regions or contents in one or more point cloud images captured for at least one other point cloud dataset to be registered. This allows the 3D vision processing tool to estimate the shooting pose of each point cloud image in the target coordinate space based on these matching regions or contents, and the obtained shooting pose estimation can have high accuracy.

[0058] S104: For each point cloud dataset to be registered: Based on the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot, determine the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0059] In an optional embodiment of this application, when performing step S104 above, for each point cloud dataset to be registered: the estimated shooting pose of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was captured, are substituted into a preset rigid body transformation formula to obtain the coordinate system transition matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space; wherein, the coordinate system transition matrix is ​​used to characterize the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0060] Among them, the rigid body transformation formula can be L i *M ic =C i L i C represents the poses of each virtual camera corresponding to the i-th point cloud dataset to be registered. i M represents the shooting pose estimation of each point cloud image corresponding to the i-th point cloud dataset to be registered. ic The coordinate system transition matrix represents the coordinate system transition between the coordinate space of the i-th point cloud dataset to be registered and the target coordinate space.

[0061] For example, suppose there are two point cloud datasets, A and B, to be registered. The set of virtual camera poses corresponding to the point cloud dataset A to be registered is denoted as La = {L}. a1 L a2 , ...L an Let Lb = {L} be the set of virtual camera poses corresponding to the point cloud dataset B to be registered. b1 L b2 , ...L bm Assume that the estimated shooting pose output by the 3D vision processing tool is C. a1 C a2 , ...C an C b1 C b2 , ...C bm , where C a1 C a2 , ...C an respectively with L a1 L a2 , ...L an Correspondingly, C b1 Cb2 , ...C bm respectively with L b1 L b2 , ...L bm Correspondingly, we have:

[0062] La*M ac =Ca, Lb*M bc =Cb; where: Ca = {C a1 C a2 , ...C an}, Cb={C b1 ,

[0063] C b2 , ...C bm}

[0064] Therefore, the coordinate system transition matrix M from the coordinate space of the point cloud dataset A to the target coordinate space can be calculated through inverse operation. ac Calculate the coordinate system transition matrix M from the coordinate space of the point cloud dataset B to the target coordinate space. bc .

[0065] It is understood that in this embodiment, the size of the coordinate system transition matrix can be preset, for example, it can be a 4*4 matrix. In this case, during the inverse operation of matrix multiplication, the number of data points in La and Ca, as well as Lb and Cb, may exceed the number of data points required to calculate the element values ​​of each element in the 4*4 matrix, which may lead to overdeterminism. When overdeterminism exists, this embodiment can employ methods such as, but is not limited to, least squares methods and regularization methods to solve the problem, but these are not considered limitations.

[0066] S105: Based on the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, transform the point cloud data in each point cloud dataset to be registered to the same coordinate space.

[0067] In one optional embodiment of this application, the point cloud data in each point cloud dataset to be registered can be transformed into the target coordinate space according to the coordinate system transfer matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space.

[0068] For example, as mentioned earlier, there are two point cloud datasets, A and B, to be registered. The coordinate system transition matrix from the coordinate space of the point cloud dataset A to the target coordinate space is M. ac The coordinate system transition matrix from the coordinate space of the point cloud dataset B to the target coordinate space is M. bc For example, suppose the point cloud data in the point cloud dataset A to be registered includes Pa = {P} a1 Pa2 ...P an The point cloud data in the point cloud dataset B to be registered includes Pa = {P} b1 P b2 ...P bm}, then: calculate Pa*M respectively. ac and Pb*M bc In this way, all the point cloud data obtained by calculation are transformed into the target coordinate space, realizing the unification of the coordinate systems of the point cloud dataset A and the point cloud dataset B to be registered, and realizing the point cloud registration between the point cloud dataset A and the point cloud dataset B to be registered.

[0069] The above optional implementation method directly utilizes the coordinate system transfer matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space to transform the point cloud data in each point cloud dataset to be registered into the target coordinate space. The entire registration process is simpler and the registration efficiency is higher.

[0070] In another optional embodiment of this application, each point cloud dataset to be registered can be divided into a basic point cloud dataset to be registered and target point cloud datasets to be registered other than the basic point cloud dataset to be registered. The coordinate space of the basic point cloud dataset to be registered is denoted as the basic coordinate space. This optional embodiment can transform the point cloud data of each target point cloud dataset to be registered into this basic coordinate space, thereby achieving point cloud registration between the various point cloud datasets to be registered.

[0071] For example, the fundamental transformation matrix when transforming from the target coordinate space to the base coordinate space can be obtained first. This fundamental transformation matrix is ​​the inverse of the coordinate system transition matrix between the base coordinate space and the target coordinate space.

[0072] Then, for each target point cloud dataset to be registered: based on the coordinate system transition matrix between the coordinate space of the target point cloud dataset and the target coordinate space, and the basic transformation matrix, the point cloud data in the target point cloud dataset to be registered is transformed to the basic coordinate space.

[0073] It is understandable that in some application scenarios, there are situations where a unified coordinate space is required. For example, in some application scenarios, it is necessary to uniformly transform all point cloud data into the coordinate space of the latest acquired point cloud dataset to be registered. In this case, based on the optional implementation method described above, the coordinate space of the latest acquired point cloud dataset to be registered can be used as the basic coordinate space to uniformly transform all point cloud data into the coordinate space of the latest acquired point cloud dataset to be registered.

[0074] For example, continuing from the previous example, we have two point cloud datasets, A and B, to be registered. The coordinate system transition matrix from the coordinate space of the point cloud dataset A to the target coordinate space is M. ac The coordinate system transition matrix from the coordinate space of the point cloud dataset B to the target coordinate space is M. bc For example, suppose the point cloud data in the point cloud dataset A to be registered includes Pa = {P} a1 P a2 ...P an Assuming that the point cloud dataset B to be registered is the base point cloud dataset to be registered, then Pa*M can be calculated. ac *M bc -1 This transforms the point cloud data in the point cloud dataset A to be registered into the coordinate space of the point cloud dataset B to be registered. Where M... bc -1 For M bc The inverse matrix is ​​the fundamental transformation matrix in this example.

[0075] In one feasible embodiment of the above optional implementation, for each target point cloud dataset to be registered: the coordinate system transition matrix between the coordinate space of the target point cloud dataset and the target coordinate space can be calculated first, and the product of this matrix and the basic transformation matrix can be obtained to obtain the coordinate system transition matrix between the coordinate space of the target point cloud dataset and the basic coordinate space. Taking the above example, M can be calculated first. ac *M bc -1 This yields the coordinate system transition matrix from the coordinate space of the point cloud dataset A to the coordinate space of the point cloud dataset B to be registered.

[0076] Then, based on the coordinate system transition matrix between the coordinate space of the target point cloud dataset to be registered and the base coordinate space, the point cloud data in the target point cloud dataset to be registered is transformed to the base coordinate space.

[0077] In another feasible embodiment of the above optional implementation, for each target point cloud dataset to be registered: the point cloud data of the target point cloud dataset can be first transformed into the target coordinate space according to the coordinate system transition matrix between the coordinate space of the target point cloud dataset and the target coordinate space. Taking the above example, Pa*M can be calculated first. ac This yields the point cloud dataset Pac of the target point cloud dataset A to be registered in the target coordinate space.

[0078] Then, based on the fundamental transformation matrix, the point cloud data of the target point cloud dataset to be registered is transformed back to the fundamental coordinate space. Taking the above example, this involves calculating Pac*M again. bc-1 .

[0079] Based on the rigid body registration method for point clouds provided in this application, multiple point cloud images are captured by a virtual camera at different poses for each point cloud dataset to be registered, and then input into a preset 3D vision processing tool for pose estimation. In this way, the captured pose estimation of each point cloud image is located in the same target coordinate space. Based on the actual pose of the virtual camera (i.e., virtual camera pose) of the point cloud images corresponding to each point cloud dataset to be registered during the capture, and the captured pose estimation of the point cloud images corresponding to each point cloud dataset to be registered, the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space can be easily obtained. Based on this transformation relationship, the point cloud data in each point cloud dataset to be registered can be easily transformed to the same coordinate space, thereby realizing the registration between the point cloud datasets to be registered. The entire scheme utilizes the pose of a virtual camera to obtain the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space. The computational load involved is relatively small, resulting in high registration efficiency. Furthermore, the transformation relationship obtained through this method is highly reliable, leading to high accuracy in the point cloud data registration results obtained by transforming the point cloud data from each dataset to the same coordinate space. Compared to various methods described in related technologies, the scheme in this application is smaller and features simpler computation and higher registration efficiency compared to coordinate descent and feature matching methods. Compared to the iterative nearest-point algorithm, the computational accuracy is higher. Compared to feature matching methods, the registration result does not depend on the accuracy of the feature extraction algorithm, resulting in higher reliability. Compared to deep learning-based methods, it does not require a large amount of training data and has lower computational resource requirements.

[0080] To facilitate understanding of the solutions in the embodiments of this application, the following example illustrates one implementation method.

[0081] Suppose we have obtained point cloud datasets A and B to be registered, each containing 50 point cloud data points, denoted as: Pa = {P} a1 P a2 ...P a50}, Pb={P b1 P b2 ...P b50}

[0082] For each point cloud dataset to be registered, a pose trajectory is set, including the pose trajectory that is always directly facing the dataset and is 10 units away from it. A virtual camera is controlled to capture a point cloud image every 7.2 degrees along this pose trajectory, and the virtual camera pose at each capture point is recorded. Assuming 50 images are captured, they are denoted as: La={L a1 L a2 , ...L a50}, Lb={L b1 L b2 , ...L b50}.like Figure 2 and Figure 3 As shown. Figure 2 The red part represents the example virtual camera, and the dense black part facing the virtual camera represents the point cloud dataset to be registered. Figure 3 for Figure 2 A collection of point cloud images captured.

[0083] The 100 point cloud images obtained are input as an image set into colmap for shooting pose estimation. After colmap calculation is completed, the shooting pose estimation of each of the 100 point cloud images is recorded in a JSON file.

[0084] Based on the correspondence between each point cloud image and the point cloud dataset to be registered, as well as the correspondence between each point cloud image and the virtual camera pose, the estimated shooting pose set Ca = {C} corresponding to each virtual camera pose is extracted from the JSON file. a1 C a2 , ...C a50}, Cb={C b1 C b2 , ...C b50}

[0085] Using the formula for rigid body transformation: La*M ac =Ca calculates the coordinate system transition matrix M from the coordinate space of the point cloud dataset A to the target coordinate space. ac and using Lb*M bc =Cb calculates the coordinate system transition matrix M from the coordinate space of the point cloud dataset B to the target coordinate space. bc .

[0086] Calculate Pa*M ac *M bc -1 The point cloud data in the point cloud dataset A to be registered is transformed into the coordinate space of the point cloud dataset B to be registered, thus completing the point cloud registration.

[0087] The above scheme has the advantages of simple calculation, high efficiency, high accuracy of point cloud registration algorithm, and adaptability to registration tasks between a variety of point cloud datasets to be registered.

[0088] Based on the same inventive concept, this application also provides a point cloud rigid body registration device 400. Please refer to... Figure 4 As shown, Figure 4 It shows the use of Figure 1 The method shown is a point cloud rigid body registration device. It should be understood that the specific functions of device 400 can be found in the description above; to avoid repetition, detailed descriptions are appropriately omitted here. Device 400 includes at least one software function module that can be stored in memory or embedded in the operating system of device 400 in the form of software or firmware. Specifically:

[0089] See Figure 4 As shown, the device 400 includes: an acquisition module 401, an image capture module 402, a pose estimation module 403, a transformation relationship determination module 404, and a coordinate transformation module 405. Wherein:

[0090] The acquisition module 401 is used to acquire each point cloud dataset to be registered; wherein, the different point cloud datasets to be registered are located in different coordinate spaces, and each point cloud dataset to be registered has the same point cloud data with at least one other point cloud dataset to be registered.

[0091] The shooting module 402 is used to take pictures of each point cloud dataset to be registered using a virtual camera in different poses in the coordinate space of the point cloud dataset to be registered, thereby obtaining multiple point cloud images corresponding to the point cloud dataset to be registered.

[0092] The pose estimation module 403 is used to input all point cloud images corresponding to all point cloud datasets to be registered into a preset 3D vision processing tool for pose estimation, so as to obtain the shooting pose estimation of each point cloud image in the target coordinate space.

[0093] The transformation relationship determination module 404 is used to determine the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space for each point cloud dataset to be registered, based on the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot.

[0094] The coordinate transformation module 405 is used to transform the point cloud data in each point cloud dataset to be registered to the same coordinate space according to the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space.

[0095] In one feasible embodiment of this application, the shooting module 402 is specifically used to: for each point cloud dataset to be registered, in the coordinate space of the point cloud dataset to be registered, on a preset pose trajectory, sequentially use a virtual camera to shoot the point cloud dataset to be registered at preset intervals, so as to obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

[0096] In one feasible embodiment of this application, the transformation relationship determination module 404 is specifically used to: for each point cloud dataset to be registered: substitute the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot into a preset rigid body transformation formula to obtain the coordinate system transition matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space; wherein, the coordinate system transition matrix is ​​used to characterize the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

[0097] In one optional embodiment of this feasible implementation, each of the point cloud datasets to be registered includes: a basic point cloud dataset to be registered and a target point cloud dataset to be registered other than the basic point cloud dataset to be registered; the coordinate space of the basic point cloud dataset to be registered is the basic coordinate space;

[0098] The conversion relationship determination module 404 is specifically used for:

[0099] Obtain the basic transformation matrix when the target coordinate space is transformed to the base coordinate space; wherein, the basic transformation matrix is ​​the inverse of the coordinate system transition matrix between the base coordinate space and the target coordinate space;

[0100] For each target point cloud dataset to be registered: based on the coordinate system transition matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space, and the basic transformation matrix, the point cloud data in the target point cloud dataset to be registered is transformed to the basic coordinate space.

[0101] In this optional embodiment, the transformation relationship determination module 404 is specifically used to: for each target point cloud dataset to be registered: calculate the product of the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the target coordinate space, and the basic transformation matrix, to obtain the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space; and transform the point cloud data in the target point cloud dataset to be registered to the basic coordinate space according to the coordinate system transfer matrix between the coordinate space of the target point cloud dataset to be registered and the basic coordinate space.

[0102] In another optional embodiment of this feasible implementation, the transformation relationship determination module 404 is specifically used to: transform the point cloud data in each of the point cloud datasets to be registered into the target coordinate space according to the coordinate system transfer matrix between the coordinate space of each point cloud dataset to be registered and the target coordinate space.

[0103] In one feasible embodiment of this application, the three-dimensional vision processing tool is colmap.

[0104] It should be understood that, for the sake of brevity, some of the content described in the method embodiments will not be repeated in the device embodiment section.

[0105] Based on the same inventive concept, this application provides an electronic device, see [link to relevant documentation]. Figure 5 As shown, it includes a processor 501 and a memory 502. Wherein:

[0106] The processor 501 is used to execute one or more programs stored in the memory 502 to implement the point cloud rigid body registration method described above.

[0107] The processor 501 can be a GPU (Graphics Processing Unit), CPU (Central Processing Unit), AI (Artificial Intelligence) processor, NPU (Neural Network Processing Unit), ISP (Image Signal Processor), DPU (Display Processing Unit), VPU (Video Processing Unit), DSP (Digital Signal Processor) data processing core, or a processor chip used in scenarios involving large-scale data computation. The memory 502 can be RAM (Random Access Memory), ROM (Read-Only Memory), or flash memory. The above are merely examples and should not be construed as limiting this application.

[0108] Figure 5 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5Different configurations are shown. For example, it may also have an internal communication bus for communication between the processor 501 and the memory 502; or it may have an external communication interface, such as a USB (Universal Serial Bus) interface, a CAN (Controller Area Network) bus interface, etc.; or it may have an information display component such as a display screen, but this is not a limitation.

[0109] In the embodiments of this application, the electronic device may be a server, a smart terminal (such as a host, a smartphone, a laptop, a desktop computer, a vehicle terminal, a smart operating platform, etc.) or other device with data processing capabilities, but this is not a limitation.

[0110] Based on the same inventive concept, this embodiment also provides a computer-readable storage medium, such as a floppy disk, optical disk, hard disk, flash memory, USB flash drive, SD (Secure Digital Memory Card), MMC (Multimedia Card), etc., in which one or more programs implementing the above steps are stored. These one or more programs can be executed by one or more processors to implement the above point cloud rigid body registration method. Further details will not be elaborated here.

[0111] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0112] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0114] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0115] In this article, "multiple" refers to two or more.

[0116] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of point cloud rigid registration, the method comprising: include: Obtain each point cloud dataset to be registered; wherein, different point cloud datasets to be registered are located in different coordinate spaces, and each point cloud dataset to be registered has the same point cloud data with at least one other point cloud dataset to be registered. For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, a virtual camera is used to take pictures of the point cloud dataset to be registered in different poses to obtain multiple point cloud images corresponding to the point cloud dataset to be registered. All point cloud images corresponding to all point cloud datasets to be registered are input into a preset 3D vision processing tool for pose estimation, so as to obtain the shooting pose estimate of each point cloud image in the target coordinate space. For each point cloud dataset to be registered: based on the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot, the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space is determined. Based on the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, the point cloud data in each point cloud dataset to be registered are transformed to the same coordinate space.

2. The point cloud rigid registration method of claim 1, wherein, For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, a virtual camera is used to take pictures of the point cloud dataset at different poses, resulting in multiple point cloud images corresponding to the dataset, including: For each point cloud dataset to be registered: In the coordinate space of the point cloud dataset to be registered, on the preset pose trajectory, the virtual camera is used to take pictures of the point cloud dataset to be registered at preset intervals to obtain multiple point cloud images corresponding to the point cloud dataset to be registered.

3. The point cloud rigid registration method of claim 1, wherein, For each point cloud dataset to be registered: based on the estimated shooting pose of each point cloud image corresponding to the point cloud dataset, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset was captured, the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space is determined, including: For each point cloud dataset to be registered: the shooting pose estimation of each point cloud image corresponding to the point cloud dataset to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud dataset to be registered was shot are substituted into the preset rigid body transformation formula to obtain the coordinate system transition matrix between the coordinate space of the point cloud dataset to be registered and the target coordinate space. The coordinate system transition matrix is ​​used to characterize the transformation relationship between the coordinate space of the point cloud dataset to be registered and the target coordinate space.

4. The point cloud rigid registration method of claim 3, wherein, Each of the point cloud datasets to be registered includes: a basic point cloud dataset to be registered and a target point cloud dataset to be registered other than the basic point cloud dataset to be registered; the coordinate space of the basic point cloud dataset to be registered is the basic coordinate space; Based on the transformation relationship between the coordinate space of each point cloud dataset to be registered and the target coordinate space, the point cloud data in each point cloud dataset to be registered are transformed to the same coordinate space, including: obtain a basic conversion matrix when the target coordinate space is converted to the basic coordinate space; wherein the basic conversion matrix is an inverse matrix of a coordinate system transfer matrix between the basic coordinate space and the target coordinate space; For each of the target point cloud data sets to be registered: according to the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space, and the basic conversion matrix, the point cloud data in the target point cloud data set to be registered is converted to the basic coordinate space.

5. The point cloud rigid registration method of claim 4, wherein, For each of the target point cloud data sets to be registered: according to the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space, and the basic conversion matrix, the point cloud data in the target point cloud data set to be registered is converted to the basic coordinate space, comprising: For each of the target point cloud data sets to be registered: Calculate the product of the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the target coordinate space, and the basic conversion matrix, to obtain the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the basic coordinate space; According to the coordinate system transfer matrix between the coordinate space of the target point cloud data set to be registered and the basic coordinate space, the point cloud data in the target point cloud data set to be registered is converted to the basic coordinate space.

6. The point cloud rigid registration method of claim 3, wherein, According to the conversion relationship between the coordinate space of each point cloud data set to be registered and the target coordinate space, the point cloud data in each point cloud data set to be registered is converted to the same coordinate space, comprising: According to the coordinate system transfer matrix between the coordinate space of each point cloud data set to be registered and the target coordinate space, the point cloud data in each of the point cloud data sets to be registered is converted to the target coordinate space.

7. The point cloud rigid registration method of any one of claims 1-6, wherein, The three-dimensional visual processing tool is colmap.

8. A point cloud rigid registration apparatus, characterized by, Comprising: An acquisition module is configured to acquire each point cloud data set to be registered; wherein different point cloud data sets to be registered are located in different coordinate spaces, and each point cloud data set to be registered has the same point cloud data as at least one other point cloud data set to be registered except for the point cloud data set to be registered; A shooting module is configured to, for each point cloud data set to be registered: in the coordinate space of the point cloud data set to be registered, use a virtual camera to shoot the point cloud data set to be registered at different poses to obtain multiple point cloud images corresponding to the point cloud data set to be registered; A pose estimation module is configured to input all point cloud images corresponding to all point cloud data sets to be registered into a preset three-dimensional visual processing tool to perform pose estimation, to obtain a shooting pose estimation of each point cloud image in the target coordinate space; A conversion relationship determination module is configured to, for each point cloud data set to be registered: according to the shooting pose estimation of each point cloud image corresponding to the point cloud data set to be registered, and the virtual camera pose when each point cloud image corresponding to the point cloud data set to be registered is shot, determine the conversion relationship between the coordinate space of the point cloud data set to be registered and the target coordinate space. The coordinate conversion module is configured to convert the point cloud data in each of the point cloud data sets to be registered to the same coordinate space according to a conversion relationship between a coordinate space of each of the point cloud data sets to be registered and the target coordinate space.

9. An electronic device, comprising: The method comprises: a processor and a memory; the processor is configured to execute one or more programs stored in the memory to implement the point cloud rigid registration method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs, which can be executed by one or more processors to implement the point cloud rigid registration method according to any one of claims 1-7.

Images