Point cloud registration method, system, device and storage medium based on multi-source fusion

Abstract

The application discloses a point cloud registration method based on multi-source fusion, which comprises the following steps: fixing a camera and a laser radar and jointly calibrating the camera and the laser radar to obtain camera parameters; obtaining two-dimensional images of a target object through the camera, obtaining three-dimensional point cloud data of the target object through the laser radar, obtaining corresponding points of the three-dimensional point cloud in the two-dimensional images according to the camera parameters, and assigning color information of the corresponding points to the three-dimensional point cloud; extracting feature points in two two-dimensional images to be spliced by using a feature matching algorithm, matching the extracted feature points, mapping the matched feature points to a three-dimensional space respectively, and performing point cloud coarse registration on source point cloud and target point cloud; and performing point cloud fine registration on coarse registration point cloud after the point cloud coarse registration and the target point cloud. The application further discloses a point cloud registration system based on multi-source fusion, corresponding equipment and a storage medium. The application is good for registration effect of a scene with unobvious change of a target three-dimensional structure feature.

Description

Technical Field

[0001] This application relates to the field of point cloud registration technology, and more specifically, to a point cloud registration method, system, device and storage medium based on multi-source fusion. Background Technology

[0002] With the development of sensor technology, 3D structure acquisition devices such as lidar and sonar are increasingly widely used in various fields. These devices can record the 3D surface structure of a target object in the form of point clouds, which can then be used for reverse engineering to reconstruct the object's digital model. However, in practical engineering, due to the limited field of view of the acquisition device or the occlusion of spatial structures, the acquired single point cloud image is often not the complete 3D structure of the target. Therefore, to obtain the complete 3D structure of the target, it is necessary to stitch together multiple acquired point clouds. This requires point cloud registration to register multiple local point clouds into a complete point cloud.

[0003] Point cloud registration typically involves two steps. The first step is coarse registration, which refers to the process of bringing the source and target point clouds as close as possible to the correct transformation when the initial pose of the point clouds is completely unknown. This is usually achieved by extracting and matching the 3D structural features of the point clouds. The second step is fine registration, which is a further calibration based on the coarse registration. Iterative calibration is performed on the details to obtain more accurate pose information.

[0004] In real-world scenarios, building complexes, walls, and other civil structures often exhibit similar 3D features. These large scenes often cannot be captured in a single acquisition and require point cloud registration to match multiple point clouds together. Currently, commonly used point cloud registration methods rely entirely on the 3D structural features of the point clouds, especially coarse registration, which is prone to failure in such scenarios. However, coarse registration is crucial in the entire process; if there are significant deviations in coarse registration, fine registration struggles to accurately align the point clouds, easily leading to registration errors. For scenes where the target's 3D structural features do not change significantly, it is difficult to register two point clouds together using general registration methods. Summary of the Invention

[0005] To address at least one deficiency or improvement requirement of the existing technology, this invention provides a point cloud registration method, system, device, and storage medium based on multi-source fusion, which has a better registration effect for scenarios where the target's three-dimensional structural features do not change significantly.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a point cloud registration method based on multi-source fusion is provided, the method comprising:

[0007] The camera and lidar are fixed and jointly calibrated to obtain camera parameters;

[0008] Several two-dimensional images of the target object are acquired by a camera, and three-dimensional point cloud data of the target object is acquired by a lidar. The corresponding points of the three-dimensional point cloud in the two-dimensional images are obtained according to the camera parameters, and the color information of the corresponding points is assigned to the three-dimensional point cloud.

[0009] Feature matching algorithm is used to extract feature points from two two-dimensional images to be stitched together. Then, the extracted feature points are matched and the matched feature points are mapped to three-dimensional space respectively to perform coarse registration of source point cloud and target point cloud.

[0010] The coarsely registered point cloud is then finely registered with the target point cloud.

[0011] Furthermore, in the above-mentioned point cloud registration method based on multi-source fusion, the step of obtaining the corresponding points of the 3D point cloud in the 2D image according to the camera parameters and assigning the color information of the corresponding points to the 3D point cloud specifically includes:

[0012] Construct a world coordinate system to obtain the coordinate information of feature points in a 3D point cloud;

[0013] Based on the transformation relationships between the world coordinate system, camera coordinate system, planar coordinate system and pixel coordinate system, and combined with the camera's intrinsic and extrinsic parameters, the coordinates of feature points in the 3D point cloud are converted into the coordinates of feature points in their corresponding 2D images.

[0014] Obtain the pixel information of feature points in a 2D image and assign the RGB color values ​​of the feature points to a 3D point cloud.

[0015] Furthermore, in the above-mentioned point cloud registration method based on multi-source fusion, the step of extracting feature points from the two two-dimensional images to be stitched using a feature matching algorithm, matching the extracted feature points, and mapping the matched feature points to three-dimensional space specifically includes:

[0016] Feature points of the two images to be stitched together are extracted using a feature matching algorithm.

[0017] Feature points are matched using the fast nearest neighbor algorithm, and several pairs of best matching feature points are found.

[0018] The best matching feature point pairs are mapped to three-dimensional space to obtain two sets of three-dimensional feature points corresponding to the best matching feature point pairs, and the spatial transformation matrix between the feature points in the two-dimensional image and their corresponding three-dimensional feature points is obtained.

[0019] Furthermore, in the above-mentioned point cloud registration method based on multi-source fusion, the step of obtaining the spatial transformation matrix between feature points in a two-dimensional image and their corresponding three-dimensional feature points specifically includes:

[0020] Obtain the k nearest neighbors {p1, p2, ..., pk} from the search point set P of any feature point in the best matching feature point pair. k At this point, the depth value of the 3D feature point corresponding to this feature point is:

[0021]

[0022] Where, depth(p i ) represents a two-dimensional feature point p i The depth value, dist(p) i ) represents the two-dimensional feature point p. i The distance;

[0023] The coordinates of the 3D feature point corresponding to the feature point are calculated using depth information and the intrinsic and extrinsic parameters of the camera, as shown by the following formula:

[0024]

[0025] in, This is the intrinsic parameter matrix. Let (u,v) be the extrinsic parameter matrix, (x,y,z) be the coordinates of the feature point, and (x,y,z) be the coordinates of the corresponding 3D feature point.

[0026] Furthermore, in the above-mentioned point cloud registration method based on multi-source fusion, the coarse registration of the source point cloud and the target point cloud specifically includes:

[0027] Based on the spatial transformation matrix between the feature points in the two-dimensional image and their corresponding three-dimensional feature points, the coordinate information of the two sets of three-dimensional feature points is obtained respectively.

[0028] The spatial transformation matrix of two sets of three-dimensional feature points is obtained by using the singular value decomposition method, and the source point cloud and the target point cloud are aligned according to the spatial transformation matrix.

[0029] Furthermore, in the above-mentioned point cloud registration method based on multi-source fusion, the step of performing fine point cloud registration between the coarsely registered point cloud and the target point cloud specifically includes:

[0030] After coarse registration of the point cloud, a coarsely registered point cloud is obtained. The corresponding nearest points of the coarsely registered point cloud in the target point cloud set are calculated, and the translation and rotation parameters between the feature points in the coarsely registered point cloud and their corresponding nearest points are calculated.

[0031] A new set of transform points is calculated based on the frequency shift parameters and rotation parameters. If the average distance between the new set of transform points and the coarsely registered point cloud is greater than a preset threshold, the calculation continues iteratively. If the average distance between the new set of transform points and the coarsely registered point cloud is less than or equal to the preset threshold, the fine registration of the point cloud ends.

[0032] According to a second aspect of the present invention, a point cloud registration system based on multi-source fusion is also provided, the system comprising an image acquisition module, a three-dimensional point cloud data acquisition module, a point cloud coarse registration module, and a point cloud fine registration module.

[0033] The image acquisition module includes a camera, and the image acquisition module is used to acquire several two-dimensional images of the target object through the camera;

[0034] The 3D point cloud data acquisition module is connected to the image acquisition module. The 3D point cloud data acquisition module includes a lidar, which is used to acquire 3D point cloud data of the target object through the lidar, and to obtain the corresponding points of the 3D point cloud in the 2D image according to the camera parameters, and to assign the color information of the corresponding points to the 3D point cloud.

[0035] The point cloud coarse registration module is connected to the image acquisition module and the three-dimensional point cloud data acquisition module respectively. The point cloud coarse registration module is used to extract feature points from the two two-dimensional images to be stitched, then match the extracted feature points, and map the matched feature points to three-dimensional space respectively to perform point cloud coarse registration between the source point cloud and the target point cloud.

[0036] The point cloud fine registration module is used to perform fine registration of the coarsely registered point cloud and the target point cloud.

[0037] According to a third aspect of the present invention, a point cloud registration device based on multi-source fusion is also provided, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit performs the steps of any of the methods described above.

[0038] According to a fourth aspect of the invention, a storage medium is also provided, which stores a computer program executable by a multi-source fusion-based point cloud registration device, wherein when the computer program is run on the multi-source fusion-based point cloud registration device, the multi-source fusion-based point cloud registration device performs the steps of any of the methods described above.

[0039] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0040] (1) The point cloud registration method based on multi-source fusion provided by the present invention coarsely registers the source point cloud and the target point cloud by solving the spatial relationship matrix between the two-dimensional feature points and their corresponding three-dimensional feature points. This method has a better registration effect for scenes where the three-dimensional structural features of the target do not change significantly.

[0041] (2) The point cloud registration method based on multi-source fusion provided by the present invention integrates the color information of the target object into the point cloud to obtain color, which can enhance the effect of point cloud registration. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a point cloud registration method based on multi-source fusion provided in an embodiment of this application;

[0044] Figure 2 Two point cloud diagrams before splicing are provided for embodiments of this application;

[0045] Figure 3 This is a schematic diagram of the feature point alignment results provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of coarse registration of point clouds provided in an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of point cloud fine registration provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0050] On the one hand, this application provides a point cloud registration method based on multi-source fusion. Figure 1 A flowchart illustrating a point cloud registration method based on multi-source fusion provided in this application embodiment is available for reference. Figure 1 The method includes the following steps:

[0051] (1) Fix the camera and lidar together and perform joint calibration to obtain camera parameters;

[0052] Specifically, the camera and the LiDAR are fixed to the same mounting device, thereby fixing their relative poses. In one specific embodiment, the mounting device is a tripod or other mounting device, the camera is a ZED2 camera, but other types of cameras can also be used, without specific limitations. The LiDAR is a Livox LiDAR, but other types of LiDAR can also be used, without specific limitations.

[0053] Furthermore, a checkerboard calibration map is created to jointly calibrate the camera and LiDAR. The PNP problem is calculated by the relationship between the positions of the corner points of the checkerboard in the camera's 2D image and their positions in the LiDAR's 3D point cloud to obtain the camera's intrinsic and extrinsic parameters.

[0054] (2) Acquire several two-dimensional images of the target object through a camera, acquire three-dimensional point cloud data of the target object through a lidar, obtain the corresponding points of the three-dimensional point cloud in the two-dimensional images according to the camera parameters, and assign the color information of the corresponding points to the three-dimensional point cloud.

[0055] Specifically, a fixed camera and a lidar are used to collect two-dimensional image data and three-dimensional point cloud data of the target object in the target scene, respectively. In a specific embodiment, in order to verify the registration effect in a scene where the three-dimensional features do not change significantly, a wall is selected as the target scene.

[0056] First, a world coordinate system is constructed to obtain the coordinate information of feature points in the 3D point cloud. Then, based on the transformation relationship between the world coordinate system, camera coordinate system, planar coordinate system, and pixel coordinate system, and combined with the camera's intrinsic and extrinsic parameters, the coordinates of the feature points in the 3D point cloud are converted into the coordinates of the feature points in the corresponding 2D image. Based on the coordinate information, the corresponding 2D image feature points are found, the pixel information of the feature points in the 2D image is obtained, and the RGB color values ​​of the feature points are assigned to the 3D point cloud.

[0057] (3) The feature matching algorithm is used to extract the feature points in the two two-dimensional images to be stitched together, and then the extracted feature points are matched. The matched feature points are mapped to three-dimensional space respectively, and the source point cloud and the target point cloud are coarsely registered.

[0058] Figure 2 These are two point cloud diagrams provided in this application embodiment before stitching. Figure 3 This is a schematic diagram of the feature point alignment results provided in an embodiment of this application. Figure 4 This is a schematic diagram of coarse registration of point clouds provided in an embodiment of this application. Feature points are extracted from the two 2D images to be stitched together using a feature matching algorithm (SURF algorithm); the feature points are matched using a fast nearest neighbor algorithm (FLANN algorithm), and several pairs of optimal matching feature points are found; the optimal matching feature point pairs are mapped to 3D space to obtain two sets of 3D feature points corresponding to the optimal matching feature point pairs, and the spatial transformation matrix between the feature points in the 2D image and their corresponding 3D feature points is obtained.

[0059] Specifically, obtain the k nearest neighbors {p1, p2, ..., pk} in the search point set P of any feature point in the best matching feature point pair. k At this point, the depth value of the 3D feature point corresponding to this feature point is:

[0060]

[0061] Where, depth(p i ) represents a two-dimensional feature point p i The depth value, dist(p) i ) represents the two-dimensional feature point p. i The distance;

[0062] The coordinates of the 3D feature point corresponding to the feature point are calculated using depth information and the intrinsic and extrinsic parameters of the camera, as shown by the following formula:

[0063]

[0064] in, This is the intrinsic parameter matrix. Let (u,v) be the extrinsic parameter matrix, (x,y,z) be the coordinates of the feature point, and (x,y,z) be the coordinates of the corresponding 3D feature point.

[0065] Based on the spatial transformation matrix between the feature points in the two-dimensional image and their corresponding three-dimensional feature points in the above method, the coordinate information of the two sets of three-dimensional feature points is obtained respectively.

[0066] The spatial transformation matrices of two sets of three-dimensional feature points are obtained by using the singular value decomposition (SVD) method, and the source point cloud and the target point cloud are aligned according to the spatial transformation matrices.

[0067] (4) Perform fine registration of the coarsely registered point cloud with the target point cloud.

[0068] Figure 5 The diagram below illustrates the fine registration of point clouds provided in this embodiment of the application. The process of fine registration of point clouds using the Iterative Closest Point Algorithm (ICP algorithm) with color is as follows:

[0069] After coarse registration of the point cloud, a coarsely registered point cloud is obtained. The corresponding nearest points of the coarsely registered point cloud in the target point cloud set are calculated, and the translation and rotation parameters between the feature points in the coarsely registered point cloud and their corresponding nearest points are calculated.

[0070] A new set of transform points is calculated based on the frequency shift parameters and rotation parameters. If the average distance between the new set of transform points and the coarsely registered point cloud is greater than a preset threshold, the calculation continues iteratively. If the average distance between the new set of transform points and the coarsely registered point cloud is less than or equal to the preset threshold, the fine registration of the point cloud ends.

[0071] Furthermore, more datasets were used to verify the effectiveness of the proposed method in different scenarios. The experimental data used three sets of data—box, indoor, and wall—as different target scenarios. The general method (SAC-IA+ICP) was used for comparison, and the final root mean square error (RMSE) was calculated. The results are shown in Table 1.

[0072] Table 1 shows the comparison results with the general method.

[0073]

[0074]

[0075] The smaller the RMSE, the better the registration result. The results show that the method in this paper has a better registration effect than the general method in scenarios where the 3D features do not change significantly, while the registration effect in general scenarios is comparable to the general method.

[0076] On the other hand, this application also provides a point cloud registration system based on multi-source fusion, which includes an image acquisition module, a three-dimensional point cloud data acquisition module, a point cloud coarse registration module, and a point cloud fine registration module.

[0077] The image acquisition module includes a camera, which is used to acquire several two-dimensional images of the target object through the camera;

[0078] The 3D point cloud data acquisition module is connected to the image acquisition module. The 3D point cloud data acquisition module includes a lidar, which is used to acquire the 3D point cloud data of the target object through the lidar, and to obtain the corresponding points of the 3D point cloud in the 2D image according to the camera parameters, and to assign the color information of the corresponding points to the 3D point cloud.

[0079] The point cloud coarse registration module is connected to the image acquisition module and the 3D point cloud data acquisition module respectively. The point cloud coarse registration module is used to extract feature points from the two 2D images to be stitched, then match the extracted feature points, and map the matched feature points to 3D space respectively to perform point cloud coarse registration between the source point cloud and the target point cloud.

[0080] The point cloud fine registration module performs fine registration between the coarsely registered point cloud and the target point cloud.

[0081] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0082] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0083] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0087] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0088] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0089] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud registration method based on multi-source fusion, characterized in that, include: The camera and lidar are fixed and jointly calibrated to obtain camera parameters; Several two-dimensional images of the target object are acquired by a camera, and three-dimensional point cloud data of the target object is acquired by a lidar. The corresponding points of the three-dimensional point cloud in the two-dimensional images are obtained according to the camera parameters, and the color information of the corresponding points is assigned to the three-dimensional point cloud. Feature matching algorithms are used to extract feature points from two 2D images to be stitched together. These extracted feature points are then matched, and the matched feature points are mapped to 3D space to perform coarse registration of the source and target point clouds. Specifically, coarse registration of the source and target point clouds includes: obtaining the coordinate information of two sets of 3D feature points based on the spatial transformation matrix between the feature points in the 2D images and their corresponding 3D feature points; obtaining the spatial transformation matrix of the two sets of 3D feature points using singular value decomposition; and aligning the source and target point clouds based on the spatial transformation matrix. The coarsely registered point cloud is then finely registered with the target point cloud.

2. The point cloud registration method based on multi-source fusion as described in claim 1, characterized in that, The step of obtaining the corresponding points of the 3D point cloud in the 2D image based on the camera parameters, and assigning the color information of the corresponding points to the 3D point cloud, specifically includes: Construct a world coordinate system to obtain the coordinate information of feature points in a 3D point cloud; Based on the transformation relationships between the world coordinate system, camera coordinate system, planar coordinate system and pixel coordinate system, and combined with the camera's intrinsic and extrinsic parameters, the coordinates of feature points in the 3D point cloud are converted into the coordinates of feature points in their corresponding 2D images. Obtain the pixel information of feature points in a 2D image and assign the RGB color values ​​of the feature points to a 3D point cloud.

3. The point cloud registration method based on multi-source fusion as described in claim 1, characterized in that, The process involves using a feature matching algorithm to extract feature points from the two two-dimensional images to be stitched together, then matching the extracted feature points, and finally mapping the matched feature points to three-dimensional space. Specifically, this includes: Feature points of the two images to be stitched together are extracted using a feature matching algorithm. Feature points are matched using the fast nearest neighbor algorithm, and several pairs of best matching feature points are found. The best matching feature point pairs are mapped to three-dimensional space to obtain two sets of three-dimensional feature points corresponding to the best matching feature point pairs, and the spatial transformation matrix between the feature points in the two-dimensional image and their corresponding three-dimensional feature points is obtained.

4. The point cloud registration method based on multi-source fusion as described in claim 3, characterized in that, The process of obtaining the spatial transformation matrix between feature points in a two-dimensional image and their corresponding three-dimensional feature points specifically includes: Obtain any feature point search point set P from the best matching feature point pair Neighboring points At this point, the depth value of the 3D feature point corresponding to this feature point is: ； in, Two-dimensional feature points The depth value, For this feature point to a two-dimensional feature point The distance; The coordinates of the 3D feature point corresponding to the feature point are calculated using depth information and the intrinsic and extrinsic parameters of the camera, as shown by the following formula: ； in, This is the intrinsic parameter matrix. It is an extrinsic parameter matrix. The coordinates of the feature point in the two-dimensional image of the feature point. The coordinates of the feature point in the 3D point cloud corresponding to this feature point.

5. The point cloud registration method based on multi-source fusion as described in claim 1, characterized in that, The step of performing fine registration of the coarsely registered point cloud with the target point cloud specifically includes: After coarse registration of the point cloud, a coarsely registered point cloud is obtained. The corresponding nearest points of the coarsely registered point cloud in the target point cloud set are calculated, and the translation and rotation parameters between the feature points in the coarsely registered point cloud and their corresponding nearest points are calculated. A new set of transform points is calculated based on the frequency shift parameters and rotation parameters. If the average distance between the new set of transform points and the coarsely registered point cloud is greater than a preset threshold, the calculation continues iteratively. If the average distance between the new set of transform points and the coarsely registered point cloud is less than or equal to the preset threshold, the fine registration of the point cloud ends.

6. A point cloud registration system based on multi-source fusion, characterized in that, The system includes an image acquisition module, a 3D point cloud data acquisition module, a point cloud coarse registration module, and a point cloud fine registration module to implement the steps of the method as described in any one of claims 1 to 5; The image acquisition module includes a camera, and the image acquisition module is used to acquire several two-dimensional images of the target object through the camera; The 3D point cloud data acquisition module is connected to the image acquisition module. The 3D point cloud data acquisition module includes a lidar, which is used to acquire 3D point cloud data of the target object through the lidar, and to obtain the corresponding points of the 3D point cloud in the 2D image according to the camera parameters, and to assign the color information of the corresponding points to the 3D point cloud. The point cloud coarse registration module is connected to the image acquisition module and the three-dimensional point cloud data acquisition module respectively. The point cloud coarse registration module is used to extract feature points from the two two-dimensional images to be stitched, then match the extracted feature points, and map the matched feature points to three-dimensional space respectively to perform point cloud coarse registration between the source point cloud and the target point cloud. The point cloud fine registration module is used to perform fine registration of the coarsely registered point cloud and the target point cloud.

7. A point cloud registration device based on multi-source fusion, characterized in that, It includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of the method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, It stores a computer program that can be executed by a point cloud registration device based on multi-source fusion. When the computer program is run on the point cloud registration device based on multi-source fusion, it causes the point cloud registration device based on multi-source fusion to perform the steps of the method according to any one of claims 1 to 5.

Images