Point cloud fusion method based on image and laser and power grid model modeling method thereof

Abstract

The invention discloses an image and laser-based point cloud fusion method and a power grid model modeling method thereof, and the method comprises the steps: S1, obtaining an image point cloud and a laser point cloud, and respectively extracting point cloud curvature feature points on the surfaces of the point clouds; s2, adopting a point cloud feature histogram algorithm, and performing coarse registration according to the point cloud curvature feature points of the point cloud surfaces of the image point cloud and the laser point cloud extracted in S1 to obtain an initial transformation relation between the image point cloud and the laser point cloud; s3, performing downsampling on the image point cloud and the laser point cloud obtained in the step S1, and performing fine registration on the downsampled image point cloud and laser point cloud by adopting an ICP registration algorithm based on point-to-surface according to the initial transformation relation obtained in the step S2; and S4, converting the image point cloud and the laser point cloud to be under the same reference according to the fine registration in the S3, and fusing the image point cloud and the laser point cloud. According to the method, the technical advantages of ground three-dimensional laser scanning and oblique photogrammetry are effectively combined, and the point cloud cavity problem caused by the technical characteristics of the oblique photogrammetry and the three-dimensional laser scanning is eliminated.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image and laser-based point cloud fusion method and a power grid modeling method thereof. Background Art

[0002] Using drone oblique photogrammetry technology to build a digital twin city geospatial data base has the advantages of low cost, high quality, and strong timeliness. However, when using only oblique photogrammetry technology to model complex structural buildings, there are problems such as unclear details, model smears, and holes.

[0003] In recent years, many scholars have conducted a lot of research on solving the problem of hollow patterns in tilted 3D measurement models. Currently, there are two better solutions: one is to use close-up photogrammetry to obtain images close to the modeling object, which has the disadvantages of high difficulty, high risk, and rapid expansion of data volume; the other is to use 3D model repair software to perform post-repair and adjustment on the model, which requires skilled operators and rich experience. At the same time, the rapid development of 3D laser scanning technology provides new means for measurement and modeling. Bracket-type 3D laser scanners have won the favor of technicians with their high efficiency, speed, and comprehensiveness. They can obtain high-precision point cloud 3D models, but because of their fixed scanning position, there are also scanning blind spots.

[0004] Recently, the use of laser scanning combined with drones for integrated air-ground fusion measurement has become a trend, and has achieved good results. However, in the technical route of air-ground data fusion, surveyors usually use control points to rotate the spatial coordinate system of drone image data and three-dimensional laser point cloud data, and achieve the purpose of registration by unifying the spatial coordinates. However, this approach has obvious technical defects, because the control points are usually only arranged on the ground. When there is an error between the image point cloud and the plane determined by the control points of the three-dimensional laser scanning point cloud, the gap between the fused data increases with the increase of the point cloud height. At the same time, the iterative closest point (ICP) search algorithm is directly used to match the point cloud data, which has the problems of large data volume and insufficient computing space. How to integrate the advantages of the two technologies and make up for each other's shortcomings has become a hot issue in the current point cloud modeling research. Summary of the invention

[0005] Purpose of the invention: In view of the above-mentioned deficiencies, the present invention provides a point cloud fusion method based on images and lasers and a power grid modeling method thereof, which can effectively integrate point cloud data from different sources, eliminate the point cloud hole problem caused by the technical characteristics of oblique photogrammetry and three-dimensional laser scanning, and optimize the geometry and color attributes of the data.

[0006] Technical solution:

[0007] The present invention provides a point cloud fusion method based on images and lasers, comprising:

[0008] S1, obtaining an image point cloud and a laser point cloud, and extracting point cloud curvature feature points of the point cloud surfaces of the image point cloud and the laser point cloud respectively;

[0009] S2, using the point cloud feature histogram algorithm, roughly align the point cloud curvature feature points of the point cloud surface of the image point cloud and the laser point cloud extracted by S1, and obtain the initial transformation relationship between the image point cloud and the laser point cloud;

[0010] S3, downsampling the image point cloud and laser point cloud obtained in S1, and accurately registering the downsampled image point cloud and laser point cloud using the point-to-plane based ICP registration algorithm according to the initial transformation relationship obtained in S2;

[0011] S4. According to the precise registration in S3, the image point cloud and the laser point cloud are transformed to the same reference, and the image point cloud and the laser point cloud are fused accordingly.

[0012] Specifically, in S3, the initial transformation relationship obtained in S2 is used to precisely align the downsampled image point cloud and the laser point cloud using an ICP registration algorithm based on point to surface, specifically:

[0013] S31, using one of the downsampled image point cloud and the laser point cloud as a reference point cloud and the other as a point cloud to be registered, triangulating the reference point cloud, and calculating a normal vector of each triangular face;

[0014] S32, transforming the point cloud to be registered according to the initial transformation relationship obtained in S2, and calculating the distance between each point in the transformed point cloud to be registered and the closest triangular face in the reference point cloud, thereby calculating the matching error;

[0015] S33, calculating and obtaining the transformation relationship between the reference point cloud and the point cloud to be registered by minimizing the matching error;

[0016] S34, applying the transformation relationship obtained in S33, calculating the matching error according to S32, and judging whether the matching error satisfies the set condition. If so, stop the iteration and obtain the final transformation relationship. Otherwise, return to S33 until the set condition or the number of iterations is met.

[0017] More specifically, the matching error adopts the average value or variance of the distance between each point in the transformed point cloud to be registered and the closest triangular face in the reference point cloud.

[0018] Specifically, in S3, the image point cloud and laser point cloud obtained in S1 are downsampled using a Kd tree.

[0019] Specifically, in S4, the fused image point cloud and laser point cloud include:

[0020] S41, after transforming the image point cloud and the laser point cloud to the same reference, for each point in the image point cloud, calculate the distance between it and its nearest neighbor point in the laser point cloud, and when the distance is less than a set threshold, delete any point; when the distance is greater than the set threshold, retain the point in the image point cloud;

[0021] S42, obtaining the color value of each point in the plurality of groups of coincident points obtained in S41, setting weights for the points belonging to the image point cloud and the laser point cloud respectively, and obtaining the color value of the position of the coincident point by weighted average calculation; wherein the weight corresponding to the image point cloud is greater than the weight corresponding to the laser point cloud;

[0022] S43, traversing the points in the image point cloud, obtaining the points except the coincident points, and retaining their color values;

[0023] S44, merging the points obtained in S42 and the points obtained in S43 and their corresponding color values ​​to obtain a final fused point cloud.

[0024] Specifically, the image point cloud is obtained by collecting image data through oblique photography, or is obtained by preprocessing a multi-source point cloud obtained by combining image data collected by oblique photography and image data collected by close-up photography.

[0025] More specifically, the preprocessing of the multi-source point cloud is as follows:

[0026] The feature points in the image data collected by oblique photography and the image data collected by close photography are identified respectively, and the image data collected by close photography are imported into the image data collected by oblique photography. After that, aerial triangulation is performed, and the spatial coordinates of each point in the image are obtained by solving the mathematical model. Finally, the pre-processed multi-source point cloud is obtained by combining the direction and posture of the UAV.

[0027] Specifically, in S1, after the laser point cloud is obtained, a step of filling holes in the laser point cloud is also included.

[0028] More specifically, before filling holes in the laser point cloud, the method further includes: filtering the laser point cloud to remove isolated points, points whose distance to adjacent points is greater than a first set value, or discrete points whose distance to the point cloud fitting plane is greater than a second set value.

[0029] The present invention also provides a method for modeling a power grid model, which obtains image point clouds and laser point clouds of the power grid respectively, uses the aforementioned point cloud fusion method to obtain fused point clouds, and then models the power grid model based on the fused point clouds.

[0030] Beneficial effects: The present invention effectively combines the advantages of ground-based three-dimensional laser scanning and oblique photogrammetry technology, eliminating the point cloud hole problem caused by the technical characteristics of oblique photogrammetry and three-dimensional laser scanning, which not only improves modeling efficiency but also has good visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are merely embodiments of the present invention, and ordinary technicians in this field can obtain other drawings based on these drawings without creative work.

[0032] Figure 1 The figure is a flow chart of the point cloud fusion method based on image and laser of the present invention. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0034] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0035] The process of the point cloud fusion method based on image and laser of the present invention is as follows: Figure 1 As shown, the steps include:

[0036] S1, obtaining an image point cloud and a laser point cloud, and extracting point cloud curvature feature points of the point cloud surfaces of the image point cloud and the laser point cloud respectively; wherein the point cloud curvature feature points of the point cloud surface are the curvature extreme value points in the point cloud surface;

[0037] In the present invention, the image point cloud can be obtained by collecting image data through oblique photography, or by preprocessing a multi-source point cloud obtained by combining image data collected through oblique photography and image data collected through close-up photography.

[0038] Among them, the oblique photography of the drone can be used to collect image data, and the close-up photography of the drone can also be used. The use of drones to collect image data can shoot the target from different angles to obtain image data from multiple perspectives.

[0039] In a specific embodiment of the present invention, after the image data is collected by oblique photography of a drone, the collected image data can be subjected to processing such as de-distortion and radiation correction, and the direction and posture of the drone when collecting the image data can also be determined.

[0040] In a specific embodiment of the present invention, the aforementioned image point cloud is obtained by preprocessing the multi-source point cloud, wherein the preprocessing is as follows:

[0041] The characteristic points in the image data collected by oblique photography and the image data collected by close photography are respectively identified, and the image data collected by close photography is imported into the image data collected by oblique photography. Then, aerial triangulation is performed, and the spatial coordinates of each point in the image are obtained by solving the mathematical model. Finally, the pre-processed multi-source point cloud is obtained by combining the direction and attitude of the drone. In this process, the present invention uses characteristic points for error compensation and precision correction, which can effectively ensure the accuracy of the measurement results.

[0042] In a specific embodiment of the present invention, after the image point cloud is obtained, it can also be filtered, denoised and other processes to improve the data quality.

[0043] In a specific embodiment of the present invention, after obtaining the image point cloud, the image point cloud can also be converted into other formats, such as digital elevation model (DEM), three-dimensional model, etc., according to actual application.

[0044] In the present invention, although there is no streaking and deformation in the laser point cloud, due to the special data collection method of the laser point cloud, there are obvious holes in the obtained laser point cloud. Therefore, after obtaining the laser point cloud, a step of filling the holes in the laser point cloud is also included. The hole filling can use a point cloud surface reconstruction algorithm based on planar structure features, a repair algorithm based on radial basis function (RBF) or a deep neural network algorithm to estimate and fill these missing parts to obtain a more complete point cloud model.

[0045] In a specific embodiment of the present invention, before filling holes in the laser point cloud, the laser point cloud is also filtered to remove noise and irrelevant features in the laser point cloud, including removing isolated points, removing points whose distance to adjacent points is greater than a first set value, or removing discrete points whose distance to the point cloud fitting plane is greater than a second set value.

[0046] S2, using the point cloud feature histogram algorithm, roughly aligning the point cloud curvature feature points of the point cloud surface of the image point cloud and the laser point cloud extracted by S1, and then obtaining the rotation matrix and translation matrix between the image point cloud and the laser point cloud, and then obtaining the initial transformation relationship between the image point cloud and the laser point cloud;

[0047] S3, downsampling the image point cloud and laser point cloud obtained in S1, and performing fine registration on the downsampled image point cloud and laser point cloud using the point-to-plane based ICP registration algorithm according to the rough registration in S2;

[0048] In the present invention, a Kd tree (kdimensional tree, Kd tree) can be used to downsample the image point cloud and the laser point cloud obtained by S1. In an embodiment of the present invention, the data volume of the point cloud can be reduced to 10% of the original data volume by downsampling.

[0049] Among them, according to the rough registration of S2, the ICP registration algorithm based on point-to-plane is used to perform fine registration on the downsampled image point cloud and laser point cloud, as follows:

[0050] S31, using one of the downsampled image point cloud and the laser point cloud as a reference point cloud and the other as a point cloud to be registered, triangulating the reference point cloud, and calculating a normal vector of each triangular face;

[0051] S32, transforming the initial transformation relationship of the point cloud to be registered based on the rough registration in S2, and calculating the distance between each point in the point cloud to be registered after the transformation and the closest triangular face in the reference point cloud, thereby calculating the matching error;

[0052] In the present invention, the matching error can be the average value or variance of the distance between each point in the point cloud to be registered after transformation and the closest triangular face in the reference point cloud;

[0053] S33, calculating and obtaining the transformation relationship between the reference point cloud and the point cloud to be registered by minimizing the matching error;

[0054] S34, applying the transformation relationship obtained in S33, calculating the matching error according to S32, and judging whether the matching error satisfies the set condition. If so, stop the iteration and obtain the final transformation relationship. Otherwise, return to S33 until the set condition or the number of iterations is met.

[0055] S4, according to the precise registration in S3, transform the image point cloud and the laser point cloud to the same reference, and fuse the image point cloud and the laser point cloud accordingly;

[0056] After the rough registration of S2 and the fine registration of S3, the image point cloud and the laser point cloud are registered under the same reference, but there are two problems at this time: one is that the data volume of the point cloud is too large and there are too many redundant data points; the other is that there is obvious color difference between the laser point cloud and the image point cloud; based on this, the present invention fuses the image point cloud and the laser point cloud, as follows:

[0057] S41, transforming the image point cloud and the laser point cloud to the same reference according to the precise registration of S3, calculating the distance between each point in the image point cloud and its nearest neighbor in the laser point cloud, and when the distance is less than a set threshold, the two corresponding points are considered to be coincident, and any one of the points is deleted to eliminate redundancy; when the distance is greater than the set threshold, the point in the image point cloud is retained;

[0058] In a specific embodiment of the present invention, the threshold value may be set to 3δ, where δ may be set based on the accuracy of the point cloud data or specific application requirements.

[0059] S42, performing color weighted averaging on the coincident points to eliminate color difference;

[0060] Obtain the color value of each point in the plurality of groups of coincident points obtained in S41, set weights w1 and w2 for the points belonging to the image point cloud and the laser point cloud, respectively, and obtain the color value of the position of the coincident point by weighted average calculation; wherein w1>w2;

[0061] In the present invention, the image point cloud has advantages in photometry and color restoration, so the weight w1 set for the image point cloud is greater than the weight w2 set for the laser point cloud. In a specific embodiment of the present invention, w1 can be set to 0.8 and w2 can be set to 0.2.

[0062] In the present invention, the color value of a point is usually expressed in RGB format, and the weighted average calculation is to recalculate the three channels of R, G, and B respectively;

[0063] S43, traversing the points in the image point cloud, obtaining the points except the coincident points, and retaining their color values;

[0064] S44, merging the points of S42 and S43 and their corresponding color values ​​to obtain a final fused point cloud.

[0065] In the present invention, the holes in the fused point cloud are completely repaired, and the colors of the point cloud are unified and coordinated.

[0066] The present invention can effectively integrate point cloud data from different sources. In point cloud data processing, laser point clouds usually provide high-precision three-dimensional spatial data, while image point clouds may come from photogrammetry or remote sensing technology, which can provide rich color and texture information. The present invention can effectively combine the advantages of ground three-dimensional laser scanning and oblique photogrammetry technology, make full use of their respective advantages, eliminate the point cloud hole problem caused by the technical characteristics of oblique photogrammetry and three-dimensional laser scanning, and optimize the geometric and color attributes of the data, providing a high-quality data foundation for subsequent three-dimensional modeling, visualization and analysis, which can not only improve modeling efficiency, but also obtain a more comprehensive and accurate three-dimensional model.

[0067] The present invention also provides a power grid modeling method based on the aforementioned point cloud fusion method, including: respectively acquiring the image point cloud and laser point cloud of the power grid, using the aforementioned point cloud fusion method to obtain the fused point cloud, and modeling the power grid model based on this, wherein the power grid modeling adopts any method in the prior art, which will not be repeated here. The power grid model constructed by the present invention has natural colors, smooth lines, and obvious edges and corners.

[0068] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0069] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A point cloud fusion method based on image and laser, characterized in that: include: S1, obtaining an image point cloud and a laser point cloud, and extracting point cloud curvature feature points of the point cloud surfaces of the image point cloud and the laser point cloud respectively; S2, using the point cloud feature histogram algorithm, roughly align the point cloud curvature feature points of the point cloud surface of the image point cloud and the laser point cloud extracted by S1, and obtain the initial transformation relationship between the image point cloud and the laser point cloud; S3, downsampling the image point cloud and laser point cloud obtained in S1, and accurately registering the downsampled image point cloud and laser point cloud using the point-to-plane based ICP registration algorithm according to the initial transformation relationship obtained in S2; S4. According to the precise registration in S3, the image point cloud and the laser point cloud are transformed to the same reference, and the image point cloud and the laser point cloud are fused accordingly.

2. The point cloud fusion method based on image and laser according to claim 1, characterized in that: In S3, the initial transformation relationship obtained in S2 is used to precisely align the downsampled image point cloud and the laser point cloud using an ICP registration algorithm based on point to surface, specifically: S31, using one of the downsampled image point cloud and the laser point cloud as a reference point cloud and the other as a point cloud to be registered, triangulating the reference point cloud, and calculating a normal vector of each triangular face; S32, transforming the point cloud to be registered according to the initial transformation relationship obtained in S2, and calculating the distance between each point in the transformed point cloud to be registered and the closest triangular face in the reference point cloud, thereby calculating the matching error; S33, calculating and obtaining the transformation relationship between the reference point cloud and the point cloud to be registered by minimizing the matching error; S34, applying the transformation relationship obtained in S33, calculating the matching error according to S32, and judging whether the matching error satisfies the set condition. If so, stop the iteration and obtain the final transformation relationship. Otherwise, return to S33 until the set condition or the number of iterations is met.

3. The point cloud fusion method based on image and laser according to claim 2, characterized in that: The matching error adopts the average value or variance of the distance between each point in the point cloud to be registered after the transformation and the closest triangular face in the reference point cloud.

4. The point cloud fusion method based on image and laser according to claim 1, characterized in that: In S3, the image point cloud and laser point cloud obtained in S1 are downsampled using a Kd tree.

5. The point cloud fusion method based on image and laser according to claim 1, characterized in that: In S4, the fused image point cloud and laser point cloud include: S41, after transforming the image point cloud and the laser point cloud to the same reference, for each point in the image point cloud, calculate the distance between it and its nearest neighbor point in the laser point cloud, and when the distance is less than a set threshold, delete any point; when the distance is greater than the set threshold, retain the point in the image point cloud; S42, obtaining the color value of each point in the plurality of groups of coincident points obtained in S41, setting weights for the points belonging to the image point cloud and the laser point cloud respectively, and obtaining the color value of the position of the coincident point by weighted average calculation; wherein the weight corresponding to the image point cloud is greater than the weight corresponding to the laser point cloud; S43, traversing the points in the image point cloud, obtaining the points except the coincident points, and retaining their color values; S44, merging the points obtained in S42 and the points obtained in S43 and their corresponding color values ​​to obtain a final fused point cloud.

6. The image and laser based point cloud fusion method according to claim 1, characterized in that: The image point cloud is obtained by collecting image data through oblique photography, or is obtained by preprocessing a multi-source point cloud obtained by combining image data collected by oblique photography and image data collected by close-up photography.

7. The image and laser based point cloud fusion method according to claim 6, characterized in that: The preprocessing of the multi-source point cloud is specifically as follows: The feature points in the image data collected by oblique photography and the image data collected by close photography are identified respectively, and the image data collected by close photography are imported into the image data collected by oblique photography. After that, aerial triangulation is performed, and the spatial coordinates of each point in the image are obtained by solving the mathematical model. Finally, the pre-processed multi-source point cloud is obtained by combining the direction and posture of the UAV.

8. The point cloud fusion method based on image and laser according to claim 1, characterized in that: In the above S1, after the laser point cloud is obtained, the step of filling holes in the laser point cloud is also included.

9. The method for point cloud fusion based on image and laser according to claim 8, characterized in that: Before the holes of the laser point cloud are filled, the method further includes filtering the laser point cloud to remove isolated points, points whose distances to adjacent points are greater than a first set value, or discrete points whose distances to the point cloud fitting plane are greater than a second set value.

10. A method for modeling a power grid model, characterized in that: The image point cloud and laser point cloud of the power grid are obtained respectively, and the fused point cloud is obtained by using the point cloud fusion method described in any one of claims 1 to 9, and the power grid model is built based on the point cloud.

Images