A method for constructing a real-scene 3D model based on the fusion of multi-source heterogeneous ground point clouds

Abstract

The present invention belongs to the technical field of real-scene three-dimensional model construction, and specifically relates to a real-scene three-dimensional model construction method based on the fusion of ground multi-source heterogeneous point clouds. The method includes drone image acquisition and three-dimensional modeling, ground multi-source heterogeneous point cloud data collection, ground multi-source heterogeneous point cloud registration, ground fusion point cloud processing, drone image three-dimensional model and ground fusion point cloud spatial registration and fusion modeling. The present invention solves the problem of missing details in large-scale scene modeling by fusing drone images with ground multi-source heterogeneous point clouds. The combination of the high precision of ground point clouds and the wide coverage of drone images significantly improves the integrity and refinement of the model. In addition, the point cloud coloring technology enhances the realism of the model and provides efficient and reliable technical support for smart cities, cultural heritage protection and other fields.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-scene three-dimensional model construction, and specifically relates to a real-scene three-dimensional model construction method based on ground multi-source heterogeneous point cloud fusion. Background Art

[0002] Real-life 3D modeling technology is one of the important technologies in the fields of digital cities, smart cities and digital twins. Its application scenarios cover urban planning, architectural surveying and mapping, cultural heritage protection, disaster assessment and other fields. At present, aerial drone photogrammetry technology is widely used in 3D modeling of large-scale scenes due to its high efficiency and flexibility. However, relying solely on image data obtained by drones, ground details (such as the bottom of buildings, under bridges, and areas blocked by vegetation) are prone to missing data or insufficient accuracy, resulting in incomplete model results. On the other hand, ground-side point cloud acquisition technology (such as stand-mounted laser scanners, handheld / mobile backpack scanners) has significant advantages in refined modeling, but its data collection range is limited, and it is difficult to obtain point clouds in areas such as the top of buildings. In addition, the data collected by different types of scanning equipment are heterogeneous, and there are differences in point cloud coordinate systems and resolutions, making direct fusion difficult to achieve. Summary of the invention

[0003] In order to solve the above problems, the present invention proposes a real-scene 3D modeling method that integrates multi-view images of drones and multi-source heterogeneous point clouds on the ground. By optimizing point cloud registration, filtering denoising and point cloud image fusion strategies, the integrity and details of large-scale real-scene 3D modeling are achieved while ensuring model accuracy.

[0004] The technical solution provided by the present invention is: a method for constructing a real-scene three-dimensional model based on the fusion of multi-source heterogeneous point clouds on the ground, comprising the following steps:

[0005] (1) UAV image acquisition and 3D modeling: Obtain multi-view UAV image data of the modeling area according to the predetermined route, obtain POS data and camera parameters; arrange image control points according to the terrain conditions of the survey area, and collect the spatial 3D coordinates of the image control points; perform feature point matching, sparse reconstruction and dense point cloud generation on the image, and introduce the image control point coordinate information to construct the UAV image 3D model;

[0006] (2) Acquisition of multi-source heterogeneous point cloud data on the ground: Use a stand-mounted 3D laser scanner combined with a handheld or mobile backpack scanner to obtain ground point cloud data; place several markers in the overlapping area of ​​the scanning range of the two devices;

[0007] (3) Registration of multi-source heterogeneous point clouds on the ground: Based on the ground landmark or spatial feature point matching algorithm, the point cloud scanned by the standing machine is preliminarily registered with the point cloud scanned by the handheld or mobile backpack. The ICP algorithm is used to fine-tune the preliminary registration results and unify the coordinate systems of the point cloud scanned by the standing machine and the point cloud scanned by the handheld or mobile backpack.

[0008] (4) Ground fusion point cloud processing: The radius filtering algorithm is used to remove outliers and noise points in the point cloud. For point cloud data with high local density or overlapping areas, the voxel downsampling method is used to optimize the point cloud density.

[0009] (5) Spatial registration between the UAV image 3D model and the ground fused point cloud: extract a number of spatial feature points with significant features from the constructed UAV image 3D model, and the ground fused point cloud also contains these feature points. Based on the 3D coordinates of these feature points, transform and match the coordinate system of the ground fused point cloud to the absolute coordinate system used by the UAV image 3D model.

[0010] (6) Fusion modeling: The registered UAV image 3D model and the ground fusion point cloud are imported into the 3D modeling software for fusion modeling to construct a high-precision, panoramic real-scene 3D model.

[0011] Preferably, in step (1), a GNSS receiver is used to collect the spatial three-dimensional coordinates of the image control points in an RTK or PPK manner.

[0012] Preferably, in step (2), a stand-type 3D laser scanner is used to scan key areas, and a handheld or mobile backpack scanner is used to scan complex scenes that are blind spots for the stand-type 3D laser scanner.

[0013] Preferably, in step (5), the number of selected spatial feature points is not less than four, and they are evenly distributed in space.

[0014] Preferably, in step (6), the strategy of fusion modeling is: in the area covered by the point cloud, the spatial structure uses the point cloud as the skeleton, and the drone image is used to texture the point cloud; in the area covered by the drone image but without the point cloud, the grid model generated by the drone image is used.

[0015] Compared with the prior art, the present invention has the following beneficial effects: By fusing drone images with multi-source heterogeneous point clouds on the ground, the present invention solves the problem of missing details in large-scale scene modeling. The high precision of the ground point cloud and the wide coverage of drone images combine to significantly improve the integrity and precision of the model. In addition, the point cloud coloring technology enhances the realism of the model, providing efficient and reliable technical support for smart cities, cultural heritage protection and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention provides a flowchart of the steps of a method for constructing a real-scene three-dimensional model based on the fusion of multi-source heterogeneous point clouds on the ground provided in the implementation of the present invention. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with specific embodiments and the accompanying drawings. More details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention is obviously capable of being implemented in a variety of other ways different from the description herein. For those skilled in the art, any substitution, improvement or change made to the embodiments of the present invention is within the protection scope of the present invention, and the protection scope of the present invention should not be limited by the content of this specific embodiment.

[0018] The method for constructing a real-scene 3D model based on the fusion of multi-source heterogeneous point clouds on the ground provided by the present invention has the following process: Figure 1 As shown, the specific process is as follows:

[0019] 1. UAV image acquisition and 3D modeling

[0020] Use drones equipped with high-resolution cameras to collect high-overlap multi-view images of the modeling area. The flight route is reasonably set according to the specific conditions of the survey area (such as the size of the modeling area, the complexity of the terrain, the density of buildings, etc.) and the expected resolution, and obtain POS data and camera parameters at the same time. Deploy image control points based on the terrain conditions of the survey area, and use GNSS receivers to collect the spatial three-dimensional coordinates of the image control points using RTK or PPK.

[0021] ContextCapture software is used to perform aerial triangulation and modeling of drone aerial images according to the aerial photogrammetry process. First, the images are automatically matched and free-net aerial triangulation is performed. According to the beam method equation, it is expressed as follows:

[0022] (1);

[0023] Formula (1) represents the perspective projection relationship between image points and ground points, which is the basis of free network adjustment. are the plane coordinates of the image point, are the coordinates of the principal point of the image, is the focal length of the camera, is the spatial coordinate of the ground point, are the coordinates of the photography center, is the rotation matrix, defined as:

[0024] (2);

[0025] In formula (2), the rotation matrix B is the rotation angle of the exterior orientation element function.

[0026] After the free network aerial triangulation meets the requirements, the image control point coordinate information is introduced to perform regional network aerial triangulation. The goal at this time is to constrain the measurement results of the entire regional network to the coordinate system of the image control points to achieve the absolute orientation of the exterior orientation elements and ground point coordinates. After the image control points are introduced, the mathematical model of the bundle adjustment is:

[0027] (3);

[0028] In formula (3), Residual vector representing the observed value (image point coordinates), is a coefficient matrix containing the geometric relationship between image points and ground points. are the parameters to be determined, including the external orientation elements and the ground point coordinates, is the observation vector, which contains the measured value of the image point and the coordinates of the image control point, and is solved by the least squares adjustment , we can get the exterior orientation elements of the image and the spatial coordinates of the ground points. Finally, we can reconstruct the 3D model and perform texture mapping based on the aerial triangulation results that meet the accuracy requirements.

[0029] 2. Ground multi-source heterogeneous point cloud data collection

[0030] Deploy a stand-type 3D laser scanner in key areas to obtain a high-precision point cloud data set P. For complex scenes, use a handheld or mobile backpack scanner to scan and fill in the blind spots of the stand-type 3D laser scanner. The point cloud data set obtained by the handheld or mobile backpack scanner is Q. To ensure that the point cloud of the stand-type scanner and the point cloud of the mobile scan (handheld or mobile backpack) can match, a certain number of markers, such as targets, are deployed in the overlapping area of ​​the scanning range of the two devices.

[0031] 3. Ground multi-source heterogeneous point cloud registration

[0032] Based on the ground markers (such as targets) or spatial feature point matching algorithm, the stand-type scanning point cloud and the handheld / backpack scanning point cloud are preliminarily registered, so that the targets or spatial feature points used for matching have a common The coordinates of the stand-type scanning point cloud dataset P and the handheld / backpack scanning point cloud dataset Q are:

[0033] (4);

[0034] (5);

[0035] Use the least squares method to calculate the optimal rotation matrix from P to Q and translation vectors :

[0036] (6);

[0037] In formula (6), , They are respectively the first elements.

[0038] Then use the ICP algorithm (iterative closest point algorithm) to iteratively adjust the preliminary registration results to minimize the difference in the same-name points between the two sets of point clouds, that is:

[0039] (7).

[0040] By fine-tuning the preliminary registration results, the coordinate systems of the ground-mounted scanning point cloud and the handheld / backpack scanning point cloud are unified.

[0041] 4. Ground fusion point cloud processing

[0042] Apply the radius filtering algorithm to remove outliers and noise points in the point cloud. For each point in the ground fusion point cloud , calculation point With other points in the point cloud The Euclidean distance is:

[0043] (8) .

[0044] Define a search radius Indicate point The neighborhood range of The number of neighboring points , if the number of neighborhood points Less than the set threshold , then it is considered that point If it is noise, remove it; otherwise, keep it.

[0045] For point cloud data with high local density or overlapping areas, the voxel downsampling method is used to optimize the point cloud density. The three-dimensional space where the point cloud is located is divided into uniform cubic grids (voxels). The side length of each voxel is , each voxel may contain multiple points or be empty. For each non-empty voxel, the centroid of the points in the voxel is used to replace all the points in the voxel:

[0046] (9);

[0047] In formula (9), is the number of points within a voxel, The voxel The coordinates of the points, It is the representative point of the voxel, and the set of representative points of all voxels is output as downsampled point cloud data.

[0048] 5. Spatial registration of drone image 3D model and ground fusion point cloud

[0049] Since drone aerial photography can obtain 3D model results in an absolute coordinate system through image control points, and the multi-source point cloud on the ground lacks spatial positioning information, it is necessary to unify their coordinate systems. Extracting spatial feature points with significant features from the 3D model results of drone aerial photography images Such as the corners of houses, the corners of ground markings, etc., the selected feature points are no less than 4, that is, , and the spatial distribution is uniform and reasonable, and the ground fusion point cloud also contains this feature points, based on The three-dimensional coordinates of the spatial feature points are used to transform and match the coordinate system of the ground fusion point cloud to the absolute coordinate system used by the drone aerial photography model. The mathematical model is the same as step 3.

[0050] 6. Fusion Modeling

[0051] Import the registered drone image 3D model and the ground fusion point cloud into RealityCapture 3D modeling software for fusion modeling to build a high-precision, panoramic real-scene 3D model. The fusion modeling strategy is to use the point cloud as the skeleton of the spatial structure in the area covered by the point cloud, and use the drone image to texture the point cloud. In the area covered by the drone image but without the point cloud, the grid model generated by the drone image is used. The specific implementation method is as follows: For the point set in the ground fusion point cloud, , the Delaunay triangulation algorithm is used to construct a triangular mesh that meets the following conditions:

[0052] (1) The internal angle of the triangle should be as close to 60 degrees as possible, and long triangles should be avoided to ensure mesh quality;

[0053] (2) For any triangle, its circumcircle contains no other points.

[0054] For the generated mesh, texture mapping is performed using drone imagery. Texture mapping calculates the pixel coordinates of the mesh vertices in the image and assigns the corresponding drone imagery texture to the mesh. The pixel coordinates of are calculated by the projection formula from point cloud to drone image:

[0055] (10);

[0056] In formula (10), is the pixel coordinate of the mesh vertex on the drone image. The texture value of the pixel is assigned to the mesh vertex , are the spatial 3D coordinates of the mesh vertices, is the camera intrinsic parameter matrix, defined as:

[0057] (11);

[0058] In formula (11), is the focal length, It is the coordinate of the main point of the image. In the area covered by the point cloud, a high-precision mesh model generated by the point cloud is used, and texture mapping is performed. In the area covered by the drone image but without point cloud, the image is directly used to generate the mesh model.

[0059] Finally, all meshes and textures are integrated through 3D modeling software (such as RealityCapture) to output a complete real-life 3D model.

Claims

1. A method for constructing a real-scene three-dimensional model based on the fusion of multi-source heterogeneous point clouds on the ground, characterized in that: The following steps are involved: (1) UAV image acquisition and 3D modeling: Obtain multi-view UAV image data of the modeling area according to the predetermined route, obtain POS data and camera parameters; arrange image control points according to the terrain conditions of the survey area, and collect the spatial 3D coordinates of the image control points; perform feature point matching, sparse reconstruction and dense point cloud generation on the image, and introduce the image control point coordinate information to construct the UAV image 3D model; (2) Acquisition of multi-source heterogeneous point cloud data on the ground: Use a stand-mounted 3D laser scanner combined with a handheld or mobile backpack scanner to obtain ground point cloud data; place several markers in the overlapping area of ​​the scanning range of the two devices; (3) Registration of multi-source heterogeneous point clouds on the ground: Based on the ground landmark or spatial feature point matching algorithm, the point cloud of the standing scanning is preliminarily registered with the point cloud of the handheld or mobile backpack scanning. The ICP algorithm is used to fine-tune the preliminary registration results and unify the coordinate systems of the standing scanning point cloud and the handheld or mobile backpack scanning point cloud. (4) Ground fusion point cloud processing: The radius filtering algorithm is used to remove outliers and noise points in the point cloud. For point cloud data with high local density or overlapping areas, the voxel downsampling method is used to optimize the point cloud density. (5) Spatial registration between the UAV image 3D model and the ground fused point cloud: extract a number of spatial feature points with significant features from the constructed UAV image 3D model, and the ground fused point cloud also contains these feature points. Based on the 3D coordinates of these feature points, transform and match the coordinate system of the ground fused point cloud to the absolute coordinate system used by the UAV image 3D model. (6) Fusion modeling: The registered UAV image 3D model and the ground fusion point cloud are imported into the 3D modeling software for fusion modeling to construct a high-precision, panoramic real-scene 3D model.

2. The method for constructing a real-scene 3D model based on ground multi-source heterogeneous point cloud fusion according to claim 1, characterized in that: In the step (1), a GNSS receiver is used to collect the spatial three-dimensional coordinates of the image control points in an RTK or PPK manner.

3. The method for constructing a real-scene 3D model based on ground multi-source heterogeneous point cloud fusion according to claim 1, characterized in that: In the step (2), a stand-mounted 3D laser scanner is used to scan key areas, and a handheld or mobile backpack scanner is used to scan complex scenes that are blind spots for the stand-mounted 3D laser scanner.

4. The method for constructing a real-scene 3D model based on ground multi-source heterogeneous point cloud fusion according to claim 1, characterized in that: In the step (5), the number of selected spatial feature points is no less than four, and they are evenly distributed in the space.

5. The method for constructing a real-scene 3D model based on fusion of multi-source heterogeneous ground point clouds according to claim 1, characterized in that: In step (6), the strategy of fusion modeling is as follows: in the area covered by the point cloud, the spatial structure uses the point cloud as the skeleton, and the drone image is used to texture the point cloud; in the area covered by the drone image but without the point cloud, the grid model generated by the drone image is used.

Images