Three-dimensional point cloud registration method, device, system, and storage medium

Abstract

The application provides a three-dimensional point cloud registration method, device, system and storage medium. The point cloud registration method provided by the application removes the repeated part of the subsequent texture map and the previous texture map to obtain a de-duplicated texture map, and selects the point cloud of the de-duplicated texture map as the incomplete point cloud of the subsequent texture map according to the texture point cloud mapping relationship of the subsequent texture map in the point cloud of the subsequent texture map, so as to reduce the number of points participating in the point cloud registration calculation in the point cloud registration process, and effectively reduce the calculation amount in the point cloud registration process. Meanwhile, in the process of registering multiple point cloud data, the point cloud after each registration is mapped into the corresponding incomplete point cloud of the next subsequent texture map until all the texture maps are processed to obtain the point cloud of the measured object. Compared with the mode of mapping all the subsequent point cloud data into the first point cloud, the calculation amount is effectively reduced, and the calculation accuracy is higher.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling, specifically a 3D point cloud registration method, apparatus, system, and storage medium. Background Technology

[0002] Currently, most point cloud registration methods for indoor scenes are based on traditional registration algorithms, such as ICP (Intermediate Point Proximity), NTD (Normal Distribution), SAC (Sample Consistency Algorithm), and FPFH (Fast Point Feature Histogram). Traditional point cloud registration algorithms are based on processing the obtained 3D point cloud data, so they have high requirements for point cloud acquisition. When the obtained point cloud contains a lot of noise points, the registration often fails.

[0003] Meanwhile, for the removal of redundant points in the stitched point cloud, the traditional algorithm uses point cloud filtering. As the number of point clouds involved in the stitching increases, the number of filtering operations required also increases. Filtering non-repeating point clouds will undoubtedly increase the algorithm's time consumption and memory usage. Summary of the Invention

[0004] Based on the above situation, the main objective of this invention is to provide a three-dimensional point cloud registration method, apparatus, system, and storage medium to improve registration efficiency.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A 3D point cloud registration method includes the following steps:

[0007] S100: Acquire multiple texture maps and corresponding depth maps of the object under test, wherein every two adjacent texture maps have overlapping parts;

[0008] S200 processes each texture map and its corresponding depth map to obtain the point cloud of each texture map, and establishes a texture point cloud mapping relationship between the two-dimensional coordinates of each texture map and the three-dimensional coordinates of the corresponding point cloud.

[0009] S300, Establish a registration mapping relationship. The registration mapping relationship is based on the three-dimensional coordinate system of the point cloud of the later texture map in two adjacent texture maps. It is a mapping relationship formed when the point cloud of the earlier texture map is mapped to the point cloud of the later texture map.

[0010] S400, remove the duplicate parts of the later texture map and the earlier texture map in two adjacent texture maps to obtain a deduplicated texture map, and select the point cloud of the later texture map and the point cloud of the deduplicated texture map from the point cloud of the later texture map to generate the residual point cloud of the later texture map.

[0011] S500, according to the registration mapping relationship, the point cloud of the first texture map is mapped to the residual point cloud of the subsequent texture map to obtain the registered point cloud, and each registered point cloud is mapped to the residual point cloud corresponding to the next subsequent texture map, until all texture maps are processed to obtain the point cloud of the object under test.

[0012] Preferably, the texture point cloud mapping relationship is established through the following steps:

[0013] The coordinates of each pixel in each depth map are multiplied with the intrinsic parameter matrix of the depth camera to obtain the three-dimensional coordinates of each pixel in the depth camera coordinate system, wherein the coordinate system of the depth camera is set as the world coordinate system.

[0014] The depth camera and texture camera are calibrated to establish a 3D transformation matrix for depth and texture.

[0015] The 3D coordinates of each pixel in the depth camera coordinate system are multiplied by the depth texture 3D transformation matrix to obtain the 3D coordinates of each pixel in the texture camera coordinate system.

[0016] The three-dimensional coordinates of each pixel in the texture camera coordinate system are multiplied with the intrinsic parameter matrix of the texture camera to obtain the two-dimensional coordinates of each pixel in the texture map.

[0017] The two-dimensional coordinates of each pixel in the texture map are associated with the three-dimensional coordinates of each pixel in the depth camera coordinate system, thereby establishing the texture point cloud mapping relationship.

[0018] Preferably, the two-dimensional coordinates of each pixel in the texture map that establishes the association are stored in a lookup table corresponding to the three-dimensional coordinates of each pixel in the depth camera coordinate system. The step of generating the residual point cloud according to the texture point cloud mapping relationship includes: querying the lookup table for each coordinate in the deduplicated texture map to obtain the corresponding three-dimensional coordinates of each pixel in the depth camera coordinate system. The three-dimensional coordinates of each pixel in the depth camera coordinate system are the three-dimensional coordinates of the residual point cloud in the subsequent texture map.

[0019] Preferably, the registration mapping relationship is obtained in the following manner:

[0020] Multiple feature points are extracted from the prior texture map and the subsequent texture map respectively, and paired in pairs to obtain multiple successfully paired two-dimensional feature point pairs. The feature points in the prior texture map are the first feature points, and the feature points in the subsequent texture map are the second feature points.

[0021] The three-dimensional coordinates of the first feature point in the prior point cloud are found by looking up the lookup table corresponding to the prior texture map, and the three-dimensional coordinates of the second feature point in the subsequent point cloud are found by looking up the lookup table corresponding to the subsequent texture map.

[0022] Based on the three-dimensional coordinates of the prior point cloud and the subsequent point cloud, and using the three-dimensional coordinate system of the point cloud in the subsequent texture map as a basis, the registration mapping relationship is established.

[0023] Preferably, removing the overlapping portions between the later and earlier texture maps in two adjacent texture maps to obtain a deduplicated texture map includes the following steps:

[0024] Multiple feature points are extracted from the prior texture map and the subsequent texture map respectively, and then paired in pairs to obtain multiple successfully paired two-dimensional feature point pairs.

[0025] Find at least four pairs of the two-dimensional feature points, and calculate a first transformation matrix for transforming the subsequent texture map to the preceding texture map, and a second transformation matrix for transforming the preceding texture map to the subsequent texture map based on them;

[0026] In the two-dimensional coordinate system of the subsequent texture map, a first template image is generated that is consistent with the two-dimensional coordinates of each pixel in the subsequent texture map, and the pixel value of each pixel in the first template image is set to 1.

[0027] The coordinates and pixel value of each pixel in the first template image are multiplied with the first transformation matrix to obtain the coordinates and pixel value of each pixel in the prior texture map coordinate system in one mapping of the first template image.

[0028] The coordinates and pixel value of each pixel after the first mapping are multiplied by the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping.

[0029] The pixel value of each pixel after the secondary mapping is multiplied by the pixel value of the pixel with the same coordinates in the subsequent texture map. In the resulting calculation, the coordinates of pixels with a value of 0 represent the non-repeating parts of adjacent texture maps, and those with a non-zero value represent the repeating parts of adjacent texture maps.

[0030] The texture image after deduplication is obtained by subtracting the calculated duplicate portion from the texture image after deduplication.

[0031] Preferably, the SuperGlue neural network model is used to extract multiple feature points in the prior texture map and the subsequent texture map respectively, and pair them up to obtain multiple successfully paired two-dimensional feature point pairs.

[0032] Preferably, the proportion of the repeated portion between the later texture map and the earlier texture map is less than a preset value.

[0033] A three-dimensional point cloud registration device, comprising:

[0034] The image acquisition module acquires multiple texture maps and corresponding depth maps of the object under test, with each pair of adjacent texture maps having overlapping parts;

[0035] The texture point cloud mapping relationship establishment module processes each texture map and its corresponding depth map to obtain the point cloud of each texture map, and at the same time establishes the texture point cloud mapping relationship between the two-dimensional coordinates of each texture map and the three-dimensional coordinates of the corresponding point cloud.

[0036] The registration mapping relationship establishment module establishes a registration mapping relationship. The registration mapping relationship is based on the three-dimensional coordinate system of the point cloud of the later texture map in two adjacent texture maps. It is a mapping relationship formed when the point cloud of the earlier texture map is mapped to the point cloud of the later texture map.

[0037] The residual point cloud generation module removes the duplicate parts of the later texture map and the earlier texture map in two adjacent texture maps to obtain a deduplicated texture map. Based on the texture point cloud mapping relationship of the later texture map, it selects the point cloud of the later texture map and the point cloud of the deduplicated texture map to generate the residual point cloud of the later texture map.

[0038] The registration module maps the point cloud of the preceding texture map to the residual point cloud of the subsequent texture map according to the registration mapping relationship to obtain the registered point cloud. It then maps each registered point cloud to the corresponding residual point cloud of the next subsequent texture map until all texture maps have been processed to obtain the point cloud of the object under test.

[0039] Preferably, the texture point cloud mapping relationship is established in the following manner:

[0040] The coordinates of each pixel in each depth map are multiplied with the intrinsic parameter matrix of the depth camera to obtain the three-dimensional coordinates of each pixel in the depth camera coordinate system, wherein the coordinate system of the depth camera is set as the world coordinate system.

[0041] The depth camera and texture camera are calibrated to establish a 3D transformation matrix for depth and texture.

[0042] The 3D coordinates of each pixel in the depth camera coordinate system are multiplied by the depth texture 3D transformation matrix to obtain the 3D coordinates of each pixel in the texture camera coordinate system.

[0043] The three-dimensional coordinates of each pixel in the texture camera coordinate system are multiplied with the intrinsic parameter matrix of the texture camera to obtain the two-dimensional coordinates of each pixel in the texture map.

[0044] The two-dimensional coordinates of each pixel in the texture map are associated with the three-dimensional coordinates of each pixel in the depth camera coordinate system, thereby establishing the texture point cloud mapping relationship.

[0045] Preferably, the two-dimensional coordinates of each pixel in the texture map that establishes the association are stored in a lookup table corresponding to the three-dimensional coordinates of each pixel in the depth camera coordinate system. The process of generating the residual point cloud according to the texture point cloud mapping relationship includes: querying the lookup table for each coordinate in the deduplicated texture map to obtain the corresponding three-dimensional coordinates of each pixel in the depth camera coordinate system. The three-dimensional coordinates of each pixel in the depth camera coordinate system are the three-dimensional coordinates of the residual point cloud in the subsequent texture map.

[0046] Preferably, the registration mapping relationship is obtained in the following manner:

[0047] Multiple feature points are extracted from the prior texture map and the subsequent texture map respectively, and paired in pairs to obtain multiple successfully paired two-dimensional feature point pairs. The feature points in the prior texture map are the first feature points, and the feature points in the subsequent texture map are the second feature points.

[0048] The three-dimensional coordinates of the first feature point in the prior point cloud are found by looking up the lookup table corresponding to the prior texture map, and the three-dimensional coordinates of the second feature point in the subsequent point cloud are found by looking up the lookup table corresponding to the subsequent texture map.

[0049] Based on the three-dimensional coordinates of the prior point cloud and the subsequent point cloud, and using the three-dimensional coordinate system of the point cloud in the subsequent texture map as a basis, the registration mapping relationship is established.

[0050] Preferably, removing the overlapping portions between the later and earlier texture maps in two adjacent texture maps to obtain a deduplicated texture map includes:

[0051] Multiple feature points are extracted from the prior texture map and the subsequent texture map respectively, and then paired in pairs to obtain multiple successfully paired two-dimensional feature point pairs.

[0052] Find at least four pairs of the two-dimensional feature points, and calculate a first transformation matrix for transforming the subsequent texture map to the preceding texture map, and a second transformation matrix for transforming the preceding texture map to the subsequent texture map based on them;

[0053] In the two-dimensional coordinate system of the subsequent texture map, a first template image is generated that is consistent with the two-dimensional coordinates of each pixel in the subsequent texture map, and the pixel value of each pixel in the first template image is set to 1.

[0054] The coordinates and pixel value of each pixel in the first template image are multiplied with the first transformation matrix to obtain the coordinates and pixel value of each pixel in the prior texture map coordinate system in one mapping of the first template image.

[0055] The coordinates and pixel value of each pixel after the first mapping are multiplied by the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping.

[0056] The pixel value of each pixel after the secondary mapping is multiplied by the pixel value of the pixel with the same coordinates in the subsequent texture map. In the resulting calculation, the coordinates of pixels with a value of 0 represent the non-repeating parts of adjacent texture maps, and those with a non-zero value represent the repeating parts of adjacent texture maps.

[0057] The texture image after deduplication is obtained by subtracting the calculated duplicate portion from the texture image after deduplication.

[0058] Preferably, the SuperGlue neural network model is used to extract multiple feature points in the prior texture map and the subsequent texture map respectively, and pair them up to obtain multiple successfully paired two-dimensional feature point pairs.

[0059] Preferably, the proportion of the repeated portion between the later texture map and the earlier texture map is less than a preset value.

[0060] A three-dimensional point cloud registration system, which performs three-dimensional point cloud registration using the method described above, or includes the three-dimensional point cloud registration device described above.

[0061] A computer-readable storage medium having a computer program stored thereon, which, when executed, enables the implementation of the method.

[0062] The point cloud registration method provided by this invention removes the duplicate portions between the subsequent texture image and the preceding texture image to obtain a deduplicated texture image. Based on the texture point cloud mapping relationship of the subsequent texture image, it selects point clouds from the point cloud of the subsequent texture image that correspond to the deduplicated texture image as residual point clouds in the subsequent texture image. This reduces the number of points involved in the point cloud registration calculation, effectively lowering the computational load. Furthermore, in the registration process of multiple point cloud data, each registered point cloud is mapped to the corresponding residual point cloud of the next subsequent texture image until all texture images are processed to obtain the point cloud of the measured object. Compared to the method of mapping all subsequent point cloud data to the first point cloud, this method effectively reduces the computational load and achieves higher computational accuracy.

[0063] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0064] The preferred embodiments of the three-dimensional point cloud registration method of the present invention will be described below with reference to the accompanying drawings. In the drawings:

[0065] Figure 1 A flowchart of a preferred embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the texture map shooting position according to a preferred embodiment of the present invention;

[0067] Figure 3 This is the first texture image according to a preferred embodiment of the present invention;

[0068] Figure 4 This is a second texture image according to a preferred embodiment of the present invention;

[0069] Figure 5 This is an image obtained by removing duplicate portions from the second texture image according to a preferred embodiment of the present invention;

[0070] Figure 6 This is a schematic diagram of three-dimensional point cloud registration according to a preferred embodiment of the present invention;

[0071] Figure 7 A schematic diagram of 3D point cloud registration for existing technologies;

[0072] Figure 8 This is a schematic diagram of the calibration of a depth map camera and a texture map camera according to a preferred embodiment of the present invention;

[0073] Figure 9 This is a schematic diagram of a two-dimensional feature point pair according to a preferred embodiment of the present invention;

[0074] Figure 10 A schematic diagram of the first texture map in the registration of 3D point clouds in the prior art;

[0075] Figure 11 A schematic diagram of the second texture map in the registration of 3D point clouds in the prior art;

[0076] Figure 12 A schematic diagram of the image obtained by multiplying the second texture map with the first transformation matrix in the existing 3D point cloud registration process;

[0077] Figure 13A schematic diagram of the image obtained by multiplying the coordinates and pixel value of each pixel after one mapping in the existing 3D point cloud registration with the second transformation matrix;

[0078] Figure 14 A schematic diagram of the non-repeating portion of the image obtained in the registration of 3D point clouds using existing technologies;

[0079] Figure 15 This is the first texture image;

[0080] Figure 16 This is the second texture image;

[0081] Figure 17 The image obtained by multiplying the second texture map with the first transformation matrix in the existing 3D point cloud registration process;

[0082] Figure 18 The image obtained by multiplying the coordinates and pixel value of each pixel after the first mapping in the existing 3D point cloud registration with the second transformation matrix;

[0083] Figure 19 Non-repeating portion images obtained in existing 3D point cloud registration techniques;

[0084] Figure 20 This is a schematic diagram of a first template image according to a preferred embodiment of the present invention;

[0085] Figure 21 This is a first template image according to a preferred embodiment of the present invention;

[0086] Figure 22 This is a schematic diagram of an image obtained by multiplying the coordinates and pixel value of each pixel in the first template image with the first transformation matrix according to a preferred embodiment of the present invention.

[0087] Figure 23 This is a schematic diagram of an image obtained by multiplying the coordinates and pixel values ​​of each pixel after a first mapping of the first template image with the second transformation matrix, according to a preferred embodiment of the present invention.

[0088] Figure 24 This is an image obtained by removing duplicate portions from a second texture map according to a preferred embodiment of the present invention;

[0089] Figure 25 A front view of a three-dimensional point cloud registration effect diagram according to a preferred embodiment of the present invention;

[0090] Figure 26 This is a top view of a three-dimensional point cloud registration effect diagram according to a preferred embodiment of the present invention;

[0091] Figure 27 This is a rendering of a three-dimensional point cloud registration according to a preferred embodiment of the present invention. Detailed Implementation

[0092] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail, but well-known methods, processes, procedures, and elements are not described in detail in order to avoid obscuring the essence of the present invention.

[0093] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0094] Unless the context explicitly requires it, the words "comprising," "including," and similar terms throughout the specification and claims should be interpreted as encompassing rather than being exclusive or exhaustive; that is, meaning "including but not limited to."

[0095] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0096] To obtain a complete 3D data model of the object being measured, a suitable coordinate transformation needs to be determined to aggregate the point clouds obtained from various perspectives into a unified coordinate system, forming a complete point cloud. Then, various visualization operations can be easily performed on the complete point cloud of the object being measured. This is the point cloud registration.

[0097] like Figure 1 As shown in the figure, this application provides a point cloud registration method, including the following steps:

[0098] S100: Acquire multiple texture maps and corresponding depth maps of the object under test, where every two adjacent texture maps have overlapping portions. Specifically, as shown... Figure 2 As shown, the object being measured is 100. The first texture map is the first texture map, and the second texture map is the second texture map. Both the first texture map and the corresponding first depth map were taken at point a. The first texture map is shown below. Figure 3 As shown. Both the second texture map and the corresponding second depth map were taken at point b. The second texture map is shown below. Figure 4 As shown, the second texture map has overlapping parts with the first texture map.

[0099] S200 processes each texture map and its corresponding depth map to obtain the point cloud of each texture map, and simultaneously builds the point cloud of each texture map. Figure 2 The mapping relationship between 3D coordinates and the corresponding 3D coordinate system of the point cloud;

[0100] S400, remove the duplicate parts of the later texture map and the earlier texture map in two adjacent texture maps to obtain a deduplicated texture map, and select the point cloud of the later texture map that is the same as the deduplicated texture map as the residual point cloud of the later texture map according to the texture point cloud mapping relationship of the later texture map.

[0101] S500: Based on the registration mapping relationship, the point cloud of the first texture map is mapped to the residual point cloud of the subsequent texture map to obtain the registered point cloud. Each registered point cloud is then mapped to the corresponding residual point cloud of the next subsequent texture map until all texture maps have been processed to obtain the point cloud of the object under test.

[0102] Specifically, such as Figure 5 As shown, Figure 5 The texture image after deduplication of the second texture image. Figure 5 The black areas in the image represent the parts in the second texture that overlap with the first texture. Based on the texture point cloud mapping relationship of the second texture, the point cloud of the second texture is selected from the point cloud of the deduplicated texture (excluding duplicate parts) as the residual point cloud (second residual point cloud B0) in the subsequent texture. Figure 6 As shown, after calculating the first point cloud A0 and the second residual point cloud B0, according to the first registration mapping relationship, the first point cloud A0 is mapped to the coordinate system of the second residual point cloud B0 to obtain the first point cloud A1, and the first point cloud A1 is added to the second residual point cloud B0 to obtain the second stitched point cloud (B0+A1).

[0103] Obtain the third texture map and the corresponding third depth map of the object under test to obtain the third point cloud, and build the third texture. Figure 2 The mapping relationship between the 3D coordinates of the third texture map and the 3D coordinates of the third point cloud is established; the overlapping parts between the third texture map and the second texture map are removed to generate the third residual point cloud; according to the second registration mapping relationship, the second stitched point cloud is mapped to the third residual point cloud to obtain the third stitched point cloud. The processing of the third texture map and the third depth map is the same as that of the second texture map and the second depth map, and will not be repeated here.

[0104] The shooting positions of the (i-1)th texture image and the i-th texture image are as follows: Figure 2As shown. For the i-th texture map, perform the following steps: acquire the i-th texture map and the corresponding i-th depth map of the object under test, obtain the i-th point cloud, and establish the i-th texture point cloud mapping relationship between the pixel coordinates of the i-th texture map and the 3D coordinates of the i-th point cloud; the (i-1)-th texture map has a duplicate part with the i-th texture map, remove the duplicate part of the i-th texture map and the (i-1)-th texture map; generate the i-th residual point cloud; according to the (i-1)-th registration mapping relationship, map the (i-1)-th stitched point cloud to the i-th residual point cloud to obtain the i-th stitched point cloud, where the object under test corresponds to N texture maps and N depth maps, i is greater than or equal to 4 and less than or equal to N, and the (i-1)-th texture map has a common part with the i-th texture map.

[0105] refer to Figure 6 In this embodiment, to complete the registration of N point clouds, a total of (N-1) steps are required to map the previous point cloud to the next residual point cloud, where 1 < i < N.

[0106] Please refer to Figure 7 Traditional point cloud registration methods typically map the i-th (N≥i≥2, where N is the number of point clouds) point cloud to the coordinate system of the first point cloud using a mapping relationship, and then accumulate these mappings until the final point cloud is generated. For example, the third point cloud C0 is first mapped to the second point cloud coordinate system to generate the third point cloud C1, the third point cloud C1 is then mapped to the first point cloud coordinate system to generate the third point cloud data C2, and so on. The Nth point cloud N0 is first mapped to the (N-1)-th point cloud coordinate system, then to the (N-2)-th point cloud coordinate system, and finally to the first point cloud coordinate system. It can be seen that simply adding the third point cloud data C0 to the final point cloud data requires one more mapping transformation calculation step than the method in this embodiment. To obtain the final complete point cloud, the traditional point cloud registration method requires M mapping transformations, where: M=1+2+…+(N-1)=N(N-1) / 2, where N is the number of point clouds. Since the value of N is greater than 2 in actual point cloud registration, N(N-1) / 2 is greater than (N-1). Therefore, the amount of computation for mapping changes in this embodiment is smaller and the operation speed is faster.

[0107] Furthermore, traditional methods for removing duplicate points in adjacent point clouds involve traversing the 3D coordinates of the (i-1)th point cloud within the i-th point cloud, identifying any mapping points as duplicates. This approach is computationally intensive, time-consuming, and its effectiveness depends heavily on parameter settings. In contrast, the method in this embodiment removes duplicate points by comparing the 2D coordinates of the overlapping portions of the (i-1)th texture image with those of the i-th texture image. This method is computationally less complex and highly accurate. It also avoids registration operations on the overlapping parts of the point cloud, further reducing the amount of data required for point cloud registration, increasing registration speed, enabling real-time acquisition and display of the registration data, and achieving high registration accuracy.

[0108] In this embodiment, the 3D point cloud registration method provides a coarse registration, namely an initial pose estimate, for point cloud registration. Subsequently, it is combined with fine registration such as ICP (Iterative Closest Point) to achieve fast and high-precision 3D point cloud registration.

[0109] Taking the establishment of the mapping relationship between the pixel coordinates of the first texture map and the 3D coordinates of the first point cloud as an example, the specific process of the texture point cloud mapping relationship is explained, including the following steps:

[0110] Step 1: Multiply the coordinates of each pixel in the first depth map with the intrinsic parameter matrix of the depth camera to obtain the 3D coordinates of each pixel in the depth camera coordinate system. The world coordinate system of the first point cloud is set to the depth camera coordinate system; therefore, the 3D coordinates in the depth camera coordinate system are equal to the 3D coordinates in the world coordinate system. Specifically, substitute the pixel coordinates (ud, vd, Z) of the depth map into Equation 1 to obtain the corresponding 3D coordinates (Xd, Yd, Zd) of the first point cloud.

[0111]

[0112] In the formula, ud0 and vd0 are the origin coordinates in the depth map, Xd, Yd, and Zd are the coordinates in the world coordinate system corresponding to the depth map camera, fdx is the focal length of the depth camera in the x-direction, and fdy is the focal length of the depth map camera in the y-direction. This is the intrinsic parameter matrix of the depth camera.

[0113] Step 2: Calibrate the depth camera and texture camera, such as... Figure 8 As shown, the texture map camera is calibrated with the calibration board, and the transformation relationship between the pixel coordinates of the texture map captured by the texture map camera and the coordinates of the calibration board is calculated. Then, the depth map camera is calibrated with the calibration board 200, and the transformation relationship between the pixel coordinates of the depth map captured by the depth map camera and the coordinates of the calibration board is calculated, thereby establishing a three-dimensional transformation matrix for the depth texture.

[0114] Step 3: Multiply the 3D coordinates of each pixel in the depth camera coordinate system with the 3D transformation matrix of the depth texture to obtain the 3D coordinates of each pixel in the texture camera coordinate system. Specifically, substitute the 3D coordinates (Xd, Yd, Zd) of the first point cloud into Equation 2 to obtain the corresponding 3D coordinates (Xc, Yc, Zc) in the texture camera coordinate system.

[0115]

[0116] In the formula, [R|T] is the 3D transformation matrix of the depth texture, which is determined by camera calibration, where R is the rotation matrix and T is the translation matrix.

[0117] Step 4: Multiply the 3D coordinates of each pixel in the texture camera coordinate system with the intrinsic parameter matrix of the texture camera to obtain the 2D coordinates of each pixel in the texture map. Specifically, substitute the 3D coordinates (Xc, Yc, Zc) in the texture camera coordinate system into Equation 3 to obtain the pixel coordinates (uc, vc) in the texture map pixel coordinate system.

[0118]

[0119] In the formula, fcx is the focal length of the depth camera in the x-direction, and fcy is the focal length of the depth camera in the y-direction. This is the intrinsic parameter matrix of the texture camera.

[0120] By associating the two-dimensional coordinates of each pixel in the texture map with the three-dimensional coordinates of each pixel in the depth camera coordinate system, a texture point cloud mapping relationship is established.

[0121] For ease of subsequent calculations, the two-dimensional coordinates of each pixel in the associated texture map and the three-dimensional coordinates of each pixel in the depth camera coordinate system are stored in a lookup table. The step of generating the residual point cloud based on the texture point cloud mapping relationship includes: querying the lookup table for each coordinate in the deduplicated texture map to obtain the corresponding three-dimensional coordinates of each pixel in the depth camera coordinate system. The three-dimensional coordinates of each pixel in the depth camera coordinate system are the three-dimensional coordinates of the residual point cloud in the subsequent texture map.

[0122] In some embodiments, the registration mapping relationship is obtained in the following manner, taking the registration mapping relationship between the first point cloud and the second point cloud as an example:

[0123] Step 1: As Figure 9 As shown, Figure 9 The first texture image is on the left. Figure 9 The right side is the second texture map. Feature points are extracted from the first texture map and the second texture map respectively and paired to obtain the first set of two-dimensional feature point pairs. In the first set of two-dimensional feature point pairs, the feature points of the first texture map are the first feature points, and the feature points of the second texture map are the second feature points. The number of feature point pairs is greater than or equal to 4.

[0124] Step 2: In the lookup table corresponding to the first texture map, find the 3D coordinates of the first point cloud corresponding to the first feature point. In the lookup table corresponding to the second texture map, find the 3D coordinates of the second point cloud corresponding to the second feature point.

[0125] Step 3: Based on the 3D coordinates of the paired first and second point clouds, and using the 3D coordinate system of the point cloud in the second texture map as a foundation, establish a registration mapping relationship. Specifically, substitute the 3D coordinates of the first point cloud data into Equation 4 (x... A y A z A Substitute the three-dimensional coordinates of the second point cloud data into Equation 4 (x B y B z B Substituting the four sets of feature points into Equation 4 yields a set of equations. Using SVD (Singular Value Decomposition), the final least squares solution can be obtained, and the first registration mapping relationship between the first point cloud and the second point cloud can be determined.

[0126]

[0127] In some embodiments, the overlapping portions of the later texture image and the earlier texture image in two adjacent texture images are removed to obtain deduplicated texture images. Taking the removal of the overlapping portions between the second texture image and the first texture image as an example, the steps include the following:

[0128] Step 1: Extract multiple feature points from the first and second texture maps respectively and pair them up to obtain multiple successfully paired two-dimensional feature point pairs; find at least four two-dimensional feature point pairs, and use the perspective transformation principle to calculate the first transformation matrix from the second texture map to the first texture map, and the second transformation matrix from the first texture map to the second texture map.

[0129] The principle for calculating the first and second transformation matrices is similar to the registration mapping relationship between the first and second point cloud data in the above embodiment. The four sets of feature point pairs are transformed into corresponding homogeneous coordinates and substituted into Equation 5 to solve for the results. The various parameters in it.

[0130]

[0131] Where (uvw) corresponds to the homogeneous coordinates of the first feature point, and [x' y' z'] corresponds to the homogeneous coordinates of the second feature point, the solution is obtained.

[0132] Will Substituting the parameters into Equation 6, we obtain the second transformation matrix from the first texture map to the second texture map. The principle of the first transformation matrix is ​​similar and will not be repeated here.

[0133]

[0134] Step 2: Based on the first transformation matrix, multiply the second texture image with the first transformation matrix to obtain the coordinates and pixel value of each pixel in the coordinate system of the first template image mapped to the coordinate system of the first texture image. Specifically, a schematic diagram of the first texture image is shown below. Figure 10 As shown, in the first texture image, part A1 is the portion that does not overlap with the second texture image, and part B1 is the portion that overlaps with the second texture image. A schematic diagram of the second texture image is shown below. Figure 11 As shown, part b1 is the portion that overlaps with the first texture image, and part c1 is the portion that does not overlap with the first texture image. Multiplying the second texture image with the first transformation matrix yields the image shown below. Figure 12 As shown, the second texture map b1 corresponds to B1' in the first texture map. Since the second texture map does not have coordinates corresponding to the first texture map A1, the pixel values ​​that do not have corresponding coordinates are filled with 0.

[0135] Step 3: Multiply the coordinates and pixel value of each pixel after the first mapping with the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping. The resulting image is shown below. Figure 13 As shown, b1' is the image of the repeated portion.

[0136] Step 4: As shown in Equation 7, subtract the coordinates and pixel values ​​of each pixel after the second texture image from the obtained quadratic mapping to obtain the non-repeating portion of the image, as shown below. Figure 14 As shown.

[0137] Non-repeating portion of the image = second texture image - coordinates and pixel value of each pixel after secondary mapping (Equation 7)

[0138] Because pixel coordinate values ​​drift during the transformation calculations in steps 2 and 3, image distortion occurs. The first texture image is shown below. Figure 15 As shown, the second texture image is as follows: Figure 16 As shown, the overlapping area between the first and second texture images in the first texture image is approximately region B1, and the non-overlapping area is approximately region A1. In the second texture image, the overlapping area with the first texture image is approximately region b1, and the non-overlapping area is region c1. In step 2, based on the first transformation matrix, the second texture image is multiplied by the first transformation matrix, resulting in the image shown below. Figure 17 As shown, region B1' is the area that overlaps with the second texture image; pixels without corresponding values ​​are filled with 0. Figure 17 The black area in the image. In step 3, the coordinates and pixel value of each pixel after the first mapping are multiplied by the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping, such as... Figure 18As shown, region b1' corresponds to the area in the second texture image that overlaps with the first texture image. The second texture image ( Figure 16 )and Figure 18 Subtracting corresponding pixels will result in pixel coordinate shifts during the mapping transformation calculation between the first and second texture images, leading to distortion. Figure 19 As shown.

[0139] To address the aforementioned problem, preferably, the steps of removing the overlapping portions between the later and earlier texture images in two adjacent texture images to obtain deduplicated texture images include the following, taking the removal of the overlapping portions between the second and first texture images as an example for specific explanation, including the following steps:

[0140] Step 1: Extract multiple feature points from the first and second texture images respectively and pair them up to obtain multiple successfully paired 2D feature point pairs; find at least four 2D feature point pairs, and use the perspective transformation principle to calculate the first transformation matrix from the second texture image to the first texture image, and the second transformation matrix from the first texture image to the second texture image. Step 1 is the same as in the above embodiment, and will not be repeated here. The first texture image is as follows: Figure 10 As shown, the second texture map is shown in Figure 11.

[0141] Step 2: In the two-dimensional coordinate system of the second texture image, generate a first template image that matches the two-dimensional coordinates of each pixel in the subsequent texture image. The pixel value of each pixel in the first template image is set to 1. Figure 20 As shown.

[0142] Step 3: Multiply the coordinates and pixel value of each pixel in the first template image with the first transformation matrix to obtain the coordinates and pixel value of each pixel in the coordinate system of the first texture image mapped from the first template image, such as... Figure 21 As shown.

[0143] Step 4: Multiply the coordinates and pixel value of each pixel after the first mapping with the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping, such as... Figure 22 As shown.

[0144] Step 5: Multiply the pixel value of each pixel after the second mapping by the pixel value of the pixel with the same coordinates in the second texture map. In the calculation result, the coordinates of the pixel with a value of 0 are the non-repeating parts of the adjacent texture map, and the coordinates of the non-zero parts are the repetitive parts of the adjacent texture map.

[0145] Step 6: Subtract the calculated duplicate portion from the second texture image to obtain the deduplicated texture image, as shown below. Figure 23 As shown.

[0146] Since each pixel value after the second mapping is either 1 or 0, the pixel value of each pixel in the second texture image with the same coordinates is multiplied by the pixel value of the second texture image, and then subtracted from the second texture image. The subtraction operation involves the second texture image and the second texture image multiplied by the pixel value after the second mapping. Therefore, it is not affected by pixel coordinate drift, i.e., no distortion occurs. Using the method in this embodiment, processing... Figure 15 and Figure 16 The duplicate parts are obtained by deduplicating the image as shown below. Figure 24 As shown.

[0147] In some embodiments, feature points are extracted from the preceding and subsequent texture maps using a SuperGlue neural network model and then paired to obtain two-dimensional feature point pairs. Using the SuperGlue neural network for feature point computation provides better robustness and stronger resistance to interference, significantly reducing the occurrence of misregistration compared to traditional 3D point cloud registration methods.

[0148] In some embodiments, to reduce the computational load of point cloud registration, the proportion of coordinates of the overlapping portion of the image between the subsequent texture image and the preceding texture image is less than a preset value. If the overlapping portion between the subsequent texture image and the preceding texture image is greater than 50%, the subsequent texture image and its corresponding depth map are discarded, and the next texture image and depth map are read. If there is a large amount of overlap between the subsequent texture images, more texture images and depth maps, as well as more registration steps, are required to complete the registration of the entire point cloud. Furthermore, a large amount of overlap between the subsequent texture image and the preceding texture image results in a larger number of corresponding feature point pairs, which increases the time required to find the corresponding 3D coordinates of the point cloud using 2D feature point pairs, thus increasing the time consumption.

[0149] Compared to traditional point cloud redundancy removal algorithms, this embodiment of the invention first calculates the duplicate points between texture maps, and then calculates the three-dimensional coordinates in the point cloud corresponding to the duplicate points between texture maps through perspective transformation, thereby removing the duplicate parts in the point cloud. This method is faster, more accurate, and unaffected by the number of point clouds.

[0150] The effect diagram of 3D point cloud registration in the embodiment of the present invention is shown below. Figures 25-27 As shown, Figure 25 This is a front view. Figure 26 This is a top view. Figure 27This embodiment demonstrates that the final point cloud is obtained by registering point clouds 300, 400, and 500. The depth camera used in this embodiment is a speckle depth camera. The scene for 3D point cloud registration modeling is an office. Because the office contains many objects with black surfaces, the speckle-based depth camera is easily affected by environmental interference. When the object being photographed has black or reflective surfaces, its depth value cannot be obtained, resulting in a significant loss in the point cloud captured by the depth camera. Using a different depth camera in subsequent iterations would further improve the registration effect. This embodiment employs CPU multi-threading acceleration based on OpenMP (a multi-threaded concurrent programming API supporting cross-platform shared memory) to achieve real-time acquisition and display of registration results. Subsequent replacement of the depth camera and the use of GPU (graphics processing unit) acceleration could achieve even better results.

[0151] A three-dimensional point cloud registration device, comprising:

[0152] The image acquisition module acquires multiple texture maps and corresponding depth maps of the object under test, with each pair of adjacent texture maps having overlapping parts;

[0153] The texture point cloud mapping relationship establishment module processes each texture map and its corresponding depth map to obtain the point cloud of each texture map, and at the same time establishes the texture point cloud mapping relationship between the two-dimensional coordinates of each texture map and the three-dimensional coordinates of the corresponding point cloud.

[0154] The registration mapping relationship establishment module establishes a registration mapping relationship. The registration mapping relationship is based on the three-dimensional coordinate system of the point cloud of the later texture map in two adjacent texture maps. It is a mapping relationship formed when the point cloud of the earlier texture map is mapped to the point cloud of the later texture map.

[0155] The residual point cloud generation module removes the duplicate parts of the later texture map and the earlier texture map in two adjacent texture maps to obtain a deduplicated texture map. Based on the texture point cloud mapping relationship of the later texture map, it selects the point cloud of the later texture map and the point cloud of the deduplicated texture map to generate the residual point cloud of the later texture map.

[0156] The registration module maps the point cloud of the preceding texture map to the residual point cloud of the subsequent texture map according to the registration mapping relationship to obtain the registered point cloud. It then maps each registered point cloud to the corresponding residual point cloud of the next subsequent texture map until all texture maps have been processed to obtain the point cloud of the object under test.

[0157] In addition, the present invention provides a computer-readable storage medium for three-dimensional point cloud registration, such as a chip, an optical disc, etc., wherein an executable program is stored on the computer-readable storage medium, and when the executable program is executed, it implements the control method as described in any of the above claims.

[0158] It should be noted that the computer-readable storage medium described in the embodiments of this disclosure is not limited to the embodiments given above. For example, it can also be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the embodiments of this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0159] It will be understood by those skilled in the art that the above-described preferred solutions can be freely combined and superimposed without conflict. The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings; for example, two consecutively indicated blocks may actually be executed substantially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. The numbering of each step in this document is for ease of explanation and reference only and is not intended to limit the order of execution. The specific execution order is determined by the technology itself, and those skilled in the art can determine various permissible and reasonable orders based on the technology itself.

[0160] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in this invention is merely for the purpose of convenience and brevity in description, and is by no means intended to restrict the order of these method steps. Those skilled in the art will understand that the order of the relevant method steps should be determined by the technology itself and should not be unduly restricted by the existence of step numbers. Those skilled in the art can determine various permissible and reasonable orderings of steps based on the technology itself.

[0161] Those skilled in the art will understand that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0162] It should be understood that the above embodiments are merely exemplary and not restrictive. Various obvious or equivalent modifications or substitutions that can be made by those skilled in the art regarding the above details without departing from the basic principles of the present invention will be included within the scope of the claims of the present invention.

Claims

1. A three-dimensional point cloud registration method, characterized in that, Including the following steps: S100: Acquire multiple texture maps and corresponding depth maps of the object under test, wherein every two adjacent texture maps have overlapping parts; S200 processes each texture map and its corresponding depth map to obtain the point cloud of each texture map, and establishes a texture point cloud mapping relationship between the two-dimensional coordinates of each texture map and the three-dimensional coordinates of the corresponding point cloud. S300, Establish a registration mapping relationship. The registration mapping relationship is based on the three-dimensional coordinate system of the point cloud of the later texture map in two adjacent texture maps. It is a mapping relationship formed when the point cloud of the earlier texture map is mapped to the point cloud of the later texture map. S400, remove the duplicate parts of the later texture map and the earlier texture map in two adjacent texture maps to obtain a deduplicated texture map, and select the point cloud of the later texture map and the point cloud of the deduplicated texture map from the point cloud of the later texture map to generate the residual point cloud of the later texture map. S500, according to the registration mapping relationship, the point cloud of the first texture map is mapped to the residual point cloud of the subsequent texture map to obtain the registered point cloud, and each registered point cloud is mapped to the corresponding residual point cloud of the next subsequent texture map until all texture maps are processed to obtain the point cloud of the object under test. The step of removing the overlapping portions of the later texture image and the earlier texture image in two adjacent texture images to obtain a deduplicated texture image includes the following steps: extracting multiple feature points in the earlier texture image and the later texture image respectively and pairing them in pairs to obtain multiple successfully paired two-dimensional feature point pairs; finding at least four of the two-dimensional feature point pairs and calculating a first transformation matrix and a second transformation matrix of the earlier texture image to the later texture image based on them; generating a first template image in the two-dimensional coordinate system of the later texture image that is consistent with the two-dimensional coordinates of each pixel in the later texture image, wherein the pixel value of each pixel in the first template image is set to 1; multiplying the coordinates and pixel value of each pixel in the first template image with the first transformation matrix to obtain the coordinates and pixel value of each pixel in the coordinate system of the earlier texture image mapped from the first template image. The coordinates and pixel value of each pixel after the first mapping are multiplied by the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping. The pixel value of each pixel after the second mapping is multiplied by the pixel value of the pixel with the same coordinates in the subsequent texture map. In the calculation result, the coordinates of the pixel with a value of 0 are the non-repeating parts of the adjacent texture map, and the coordinates of the non-zero parts are the repetitive parts of the adjacent texture map. The subsequent texture map is subtracted from the calculated repetitive parts to obtain the deduplicated texture map. The two-dimensional coordinates of each pixel in the texture map that establishes the association are stored in a lookup table corresponding to the three-dimensional coordinates of each pixel in the depth camera coordinate system. The generation step of the residual cloud includes: querying the lookup table for each coordinate in the deduplicated texture map to obtain the corresponding three-dimensional coordinates of each pixel in the depth camera coordinate system. The three-dimensional coordinates of each pixel in the depth camera coordinate system are the three-dimensional coordinates of the residual cloud in the subsequent texture map. The registration mapping relationship is obtained as follows: multiple feature points are extracted from the prior texture map and the subsequent texture map respectively, and paired in pairs to obtain multiple successfully paired two-dimensional feature point pairs, wherein the feature point in the prior texture map is the first feature point, and the feature point in the subsequent texture map is the second feature point; the three-dimensional coordinates of the prior point cloud corresponding to the first feature point are found in the lookup table corresponding to the prior texture map, and the three-dimensional coordinates of the subsequent point cloud corresponding to the second feature point are found in the lookup table corresponding to the subsequent texture map; based on the three-dimensional coordinates of the prior point cloud and the subsequent point cloud, the registration mapping relationship is established using the three-dimensional coordinate system of the point cloud in the subsequent texture map as the basis.

2. The method according to claim 1, characterized in that, The texture point cloud mapping relationship is established through the following steps: The coordinates of each pixel in each depth map are multiplied with the intrinsic parameter matrix of the depth camera to obtain the three-dimensional coordinates of each pixel in the depth camera coordinate system, wherein the coordinate system of the depth camera is set as the world coordinate system. The depth camera and texture camera are calibrated to establish a 3D transformation matrix for depth and texture. The 3D coordinates of each pixel in the depth camera coordinate system are multiplied by the depth texture 3D transformation matrix to obtain the 3D coordinates of each pixel in the texture camera coordinate system. The three-dimensional coordinates of each pixel in the texture camera coordinate system are multiplied with the intrinsic parameter matrix of the texture camera to obtain the two-dimensional coordinates of each pixel in the texture map. The two-dimensional coordinates of each pixel in the texture map are associated with the three-dimensional coordinates of each pixel in the depth camera coordinate system, thereby establishing the texture point cloud mapping relationship.

3. The method according to claim 1, characterized in that, The SuperGlue neural network model is used to extract multiple feature points from the prior texture map and the subsequent texture map, and then pair them up to obtain multiple successfully paired two-dimensional feature point pairs.

4. The method according to claim 1, characterized in that, The proportion of the overlapping portion between the later texture map and the earlier texture map is less than a preset value.

5. A three-dimensional point cloud registration device, characterized in that, include: The image acquisition module acquires multiple texture maps and corresponding depth maps of the object under test, with each pair of adjacent texture maps having overlapping parts; The texture point cloud mapping relationship establishment module processes each texture map and its corresponding depth map to obtain the point cloud of each texture map, and at the same time establishes the texture point cloud mapping relationship between the two-dimensional coordinates of each texture map and the three-dimensional coordinates of the corresponding point cloud. The registration mapping relationship establishment module establishes a registration mapping relationship. The registration mapping relationship is based on the three-dimensional coordinate system of the point cloud of the later texture map in two adjacent texture maps. It is a mapping relationship formed when the point cloud of the earlier texture map is mapped to the point cloud of the later texture map. The residual point cloud generation module removes the overlapping portions between the later and earlier texture maps in two adjacent texture maps to obtain deduplicated texture maps. Based on the texture point cloud mapping relationship of the later texture map, it selects points from the point cloud of the later texture map and points from the point cloud of the deduplicated texture map to generate residual point clouds for the later texture map. The step of removing the overlapping portions between the later and earlier texture maps in two adjacent texture maps to obtain deduplicated texture maps includes the following steps: extracting multiple feature points from both the earlier and later texture maps and pairing them in pairs to obtain multiple successfully paired two-dimensional feature point pairs; identifying at least four... The system calculates a first transformation matrix for transforming the subsequent texture map to the preceding texture map and a second transformation matrix for transforming the preceding texture map to the subsequent texture map based on the given two-dimensional feature point pairs. In the two-dimensional coordinate system of the subsequent texture map, a first template image is generated that corresponds to the two-dimensional coordinates of each pixel in the subsequent texture map, with the pixel value of each pixel in the first template image set to 1. The coordinates and pixel values ​​of each pixel in the first template image are multiplied by the first transformation matrix to obtain the coordinates and pixel values ​​of each pixel in the coordinate system of the preceding texture map mapped from the first template image. The coordinates and pixel value of each pixel after the first mapping are multiplied by the second transformation matrix to obtain the coordinates and pixel value of each pixel after the second mapping. The pixel value of each pixel after the second mapping is multiplied by the pixel value of the pixel with the same coordinates in the subsequent texture map. In the calculation result, the coordinates of the pixel with a value of 0 are the non-repeating parts of the adjacent texture map, and the coordinates of the non-zero parts are the repetitive parts of the adjacent texture map. The subsequent texture map is subtracted from the calculated repetitive parts to obtain the deduplicated texture map. The two-dimensional coordinates of each pixel in the texture map that establishes the association are stored in a lookup table corresponding to the three-dimensional coordinates of each pixel in the depth camera coordinate system. The generation step of the residual cloud includes: querying the lookup table for each coordinate in the deduplicated texture map to obtain the corresponding three-dimensional coordinates of each pixel in the depth camera coordinate system. The three-dimensional coordinates of each pixel in the depth camera coordinate system are the three-dimensional coordinates of the residual cloud in the subsequent texture map. The registration module maps the point cloud of the preceding texture map to the residual point cloud of the subsequent texture map according to the registration mapping relationship to obtain the registered point cloud. It then maps each registered point cloud to the corresponding residual point cloud of the next subsequent texture map until all texture maps are processed to obtain the point cloud of the measured object. The registration mapping relationship is obtained as follows: multiple feature points are extracted from both the preceding and subsequent texture maps and paired in pairs to obtain multiple successfully paired two-dimensional feature point pairs. The feature point in the preceding texture map is the first feature point, and the feature point in the subsequent texture map is the second feature point. The three-dimensional coordinates of the preceding point cloud corresponding to the first feature point are found in a lookup table corresponding to the preceding texture map, and the three-dimensional coordinates of the subsequent point cloud corresponding to the second feature point are found in a lookup table corresponding to the subsequent texture map. Based on the three-dimensional coordinates of the preceding and subsequent point clouds, and using the three-dimensional coordinate system of the point cloud in the subsequent texture map as a foundation, the registration mapping relationship is established.

6. The apparatus according to claim 5, characterized in that, The texture point cloud mapping relationship is established in the following way: The coordinates of each pixel in each depth map are multiplied with the intrinsic parameter matrix of the depth camera to obtain the three-dimensional coordinates of each pixel in the depth camera coordinate system, wherein the coordinate system of the depth camera is set as the world coordinate system. The depth camera and texture camera are calibrated to establish a 3D transformation matrix for depth and texture. The 3D coordinates of each pixel in the depth camera coordinate system are multiplied by the depth texture 3D transformation matrix to obtain the 3D coordinates of each pixel in the texture camera coordinate system. The three-dimensional coordinates of each pixel in the texture camera coordinate system are multiplied with the intrinsic parameter matrix of the texture camera to obtain the two-dimensional coordinates of each pixel in the texture map. The two-dimensional coordinates of each pixel in the texture map are associated with the three-dimensional coordinates of each pixel in the depth camera coordinate system, thereby establishing the texture point cloud mapping relationship.

7. The apparatus according to claim 5, characterized in that, The SuperGlue neural network model is used to extract multiple feature points from the prior texture map and the subsequent texture map, and then pair them up to obtain multiple successfully paired two-dimensional feature point pairs.

8. The apparatus according to claim 5, characterized in that, The proportion of the overlapping portion between the later texture map and the earlier texture map is less than a preset value.

9. A three-dimensional point cloud registration system, characterized in that, The method described in any one of claims 1-4 is used for three-dimensional point cloud registration, or the method includes the three-dimensional point cloud registration device described in any one of claims 5-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it can perform the method as described in any one of claims 1-4.

Images