An unmanned aerial vehicle oblique photography three-dimensional model and unmanned vehicle image combined modeling method

Abstract

The present application belongs to the technical field of unmanned platform, and particularly relates to a method for jointly modeling a three-dimensional model of unmanned aerial vehicle oblique photography and an image of unmanned vehicle. The present application jointly models the three-dimensional model of unmanned aerial vehicle oblique photography and the image of unmanned vehicle, and can automatically and efficiently construct a complete three-dimensional model of ground objects. Based on a SIFT operator, the present application solves the problem of low matching point quantity and precision of air-ground image jointing caused by the mismatch of scale and shooting angle of the three-dimensional model of unmanned aerial vehicle oblique photography and the image of unmanned vehicle. Through least square method, the normal vector and adjacent points of the plane point cloud are optimized, the regularization of the boundary features of ground objects in the air-ground fusion point cloud is realized, and the clear boundary of the ground objects is obtained.

Description

Technical Field

[0001] This invention belongs to the field of unmanned platform technology, specifically relating to a method for joint modeling of UAV oblique photogrammetry 3D models and unmanned vehicle images. Background Technology

[0002] Based on oblique photogrammetric imagery data collected by UAVs, large-scale 3D models of ground features can be constructed through steps such as automatic aerial triangulation, dense point cloud generation, mesh construction, and automatic texture mapping. Based on ground imagery data collected by unmanned vehicles, relatively accurate ground point cloud maps can be created through front-end matching, localization, and triangulation, back-end loop closure detection, and local mapping using SLAM (Simultaneous Localization and Mapping) technology.

[0003] However, due to the limited field of view of oblique photogrammetric imagery data collected by drones, traditional drone oblique photogrammetric 3D models suffer from missing local information due to occlusion. Manual model repair in later stages would be time-consuming, labor-intensive, and inefficient. Furthermore, the map scale created by unmanned vehicles is relatively limited due to their limited field of view. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] The technical problem to be solved by this invention is: how to provide a method for joint modeling of UAV oblique photogrammetry 3D models and unmanned vehicle images.

[0006] (II) Technical Solution

[0007] To address the aforementioned technical problems, this invention provides a method for jointly modeling a 3D model using UAV oblique photogrammetry and an unmanned vehicle image. The modeling method includes the following steps:

[0008] Step 1: Using the reference plane of the UAV oblique photogrammetry 3D model as a reference, and combining the initial orientation information of the UAV, project the UAV image onto an angle that is approximately orthogonal to the reference plane of the UAV oblique photogrammetry 3D model, and correct the resolution of the UAV image; then use the SIFT operator to perform feature extraction, feature matching, and gross error removal to obtain matching feature points.

[0009] Step 2: Calculate the rigid body transformation parameters between the UAV oblique photogrammetry 3D model and the UAV image using matching feature points, incorporate all matching feature points into the air-to-ground observation network for adjustment, and use the rigid body transformation parameters as weighted observation constraints. While taking into account the internal consistency between images, improve the geometric fusion effect of matching feature points on the air-to-ground observation network to obtain optimized matching feature points.

[0010] Step 3: The 4PCS (4-point equal set registration) algorithm is used to achieve the initial registration between the point cloud of the UAV oblique photogrammetry 3D model and the point cloud generated by the UAV image. Combined with the optimized matching feature points and ICP (closest point iteration) algorithm, the two point clouds after the initial registration are finely registered to obtain the fused point cloud.

[0011] Step 4: Extract planar point clouds from the fused point cloud based on the RANSAC (Random Sample Consensus) algorithm, and group planar point clouds whose normal vectors approximate each other according to their normal vectors; extract two-dimensional boundaries for each planar point cloud, and calculate the adjacent points and initial normal vectors of each two-dimensional boundary point;

[0012] Based on this, the normal vectors of the two-dimensional boundary points are first optimized by adjacency relationship and initial normal vector. Then, the positions of the adjacent points are fine-tuned by least squares method according to the optimized normal vector to obtain piecewise smooth line segments. After the above processing is performed on each planar point cloud in the planar point cloud group, global regularization processing is performed to obtain globally regularized ground feature boundaries.

[0013] Step 4 includes:

[0014] Step 41: Abstract the straight line segments in the planar point cloud group into center points and their direction angles, and merge all direction angles in the same group and the included angles between different planar groups to generate a direction angle pool;

[0015] Step 42: Generate a series of hypothetical line segments based on the direction angle in the direction angle pool and the center point of each line segment, ensuring that the regularized line segments are a combination of these hypothetical line segments;

[0016] Step 43: Describe the data fitting penalty and smoothness penalty for each hypothetical line segment using the energy equation. The higher the degree of fit of the hypothetical line segment, the smaller the data fitting penalty, and vice versa.

[0017] Step 44: Obtain the optimal solution through the graph cut method to obtain the globally regularized ground feature boundary.

[0018] In step 3, the 4PCS algorithm is a 4-point congruent set registration algorithm.

[0019] In step 3, the ICP algorithm is the nearest point iteration algorithm.

[0020] In step 4, the RANSAC algorithm is a random sampling consensus algorithm.

[0021] (III) Beneficial Effects

[0022] This invention uses ground image data collected by unmanned vehicles to repair missing local information in UAV oblique photogrammetry 3D models, efficiently constructing complete and large-scale 3D models of ground features.

[0023] Compared with the prior art, the key point of this invention is:

[0024] This invention combines UAV oblique photogrammetry 3D models and unmanned vehicle imagery for 3D modeling, enabling the automatic and efficient construction of complete 3D models of ground features. It addresses the issue of low matching point quantity and accuracy in air-to-ground image fusion due to scale and shooting angle mismatches between the UAV oblique photogrammetry 3D model and the unmanned vehicle imagery by using the SIFT operator. Furthermore, it optimizes the normal vectors and adjacent points of the planar point cloud using the least squares method, thereby regularizing the boundary features of ground features in the air-to-ground fused point cloud and obtaining clear boundaries for ground features.

[0025] Compared with the prior art, the advantages of the present invention are as follows:

[0026] Using the method of this invention to jointly model a UAV oblique photogrammetry 3D model with unmanned vehicle imagery, it is possible to effectively restore the details lost in the UAV oblique photogrammetry 3D model, such as... Figures 3a to 3d As shown in the comparison between circles 1 and 2, the air-ground joint modeling models a and b have restored the missing details in the UAV oblique photogrammetry 3D models a and b very well. Attached Figure Description

[0027] Figure 1 This is a flowchart of the technical solution of the present invention.

[0028] Figure 2 This is a schematic diagram of the joint air-to-ground orientation matching results.

[0029] Figures 3a to 3d This is a schematic diagram comparing a UAV oblique photogrammetry 3D model with an air-to-ground joint modeling model. Among them, Figure 3a For the 3D model a, which is a result of oblique photography by a UAV; Figure 3b Model a for joint air-ground modeling; Figure 3c b is a 3D model created by oblique photography of a UAV; Figure 3d Model b is the joint air-ground model. Detailed Implementation

[0030] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0031] To address the aforementioned technical problems, this invention provides a method for jointly modeling a 3D model using UAV oblique photogrammetry and UAV vehicle imagery, such as... Figure 1 As shown, the modeling method includes the following steps:

[0032] Step 1: Using the reference plane of the UAV oblique photogrammetry 3D model as a reference, and combining the initial orientation information of the UAV, project the UAV image onto an angle that is approximately orthogonal to the reference plane of the UAV oblique photogrammetry 3D model, and correct the resolution of the UAV image; then use the SIFT operator to perform feature extraction, feature matching, and gross error removal to obtain matching feature points.

[0033] Step 2: Calculate the rigid body transformation parameters between the UAV oblique photogrammetry 3D model and the UAV image using matched feature points. Incorporate all matched feature points into the air-to-ground observation network for adjustment, and use the rigid body transformation parameters as weighted observation constraints. While considering the internal consistency between images, this improves the geometric fusion effect of the matched feature points on the air-to-ground observation network, resulting in optimized matched feature points; for example... Figure 2 The image shown is the result of the air-to-ground image matching.

[0034] Step 3: The 4PCS (4-point equal set registration) algorithm is used to achieve the initial registration between the point cloud of the UAV oblique photogrammetry 3D model and the point cloud generated by the UAV image. Combined with the optimized matching feature points and ICP (closest point iteration) algorithm, the two point clouds after the initial registration are finely registered to obtain the fused point cloud.

[0035] Step 4: Extract planar point clouds from the fused point cloud based on the RANSAC (Random Sample Consensus) algorithm, and group planar point clouds whose normal vectors approximate each other according to their normal vectors; extract two-dimensional boundaries for each planar point cloud, and calculate the adjacent points and initial normal vectors of each two-dimensional boundary point;

[0036] Based on this, the normal vectors of the two-dimensional boundary points are first optimized by adjacency relationship and initial normal vector. Then, the positions of the adjacent points are fine-tuned by least squares method according to the optimized normal vector to obtain piecewise smooth line segments. After the above processing is performed on each planar point cloud in the planar point cloud group, global regularization processing is performed to obtain globally regularized ground feature boundaries.

[0037] Step 4 includes:

[0038] Step 41: Abstract the straight line segments in the planar point cloud group into center points and their direction angles, and merge all direction angles in the same group and the included angles between different planar groups to generate a direction angle pool.

[0039] Step 42: Generate a series of hypothetical line segments based on the direction angle in the direction angle pool and the center point of each line segment, ensuring that the regularized line segments are a combination of these hypothetical line segments;

[0040] Step 43: Describe the data fitting penalty and zero smoothness penalty for each hypothetical line segment using the energy equation. The higher the degree of fit of the hypothetical line segment, the smaller the data fitting penalty, and vice versa.

[0041] Step 44: Obtain the optimal solution through the graph cut method to obtain the globally regularized ground feature boundary.

[0042] In step 3, the 4PCS algorithm is a 4-point congruent set registration algorithm.

[0043] In step 3, the ICP algorithm is the nearest point iteration algorithm.

[0044] 5. In step 4, the RANSAC algorithm is a random sampling consensus algorithm.

[0045] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for joint modeling of UAV oblique photogrammetry 3D models and unmanned vehicle images, characterized in that, The modeling method includes the following steps: Step 1: Using the reference plane of the UAV oblique photogrammetry 3D model as a reference, and combining the initial orientation information of the UAV, project the UAV image onto an angle that is approximately orthogonal to the reference plane of the UAV oblique photogrammetry 3D model, and correct the resolution of the UAV image; then use the SIFT operator to perform feature extraction, feature matching, and gross error removal to obtain matching feature points. Step 2: Calculate the rigid body transformation parameters between the UAV oblique photogrammetry 3D model and the UAV image using matching feature points, incorporate all matching feature points into the air-to-ground observation network for adjustment, and use the rigid body transformation parameters as weighted observation constraints. While taking into account the internal consistency between images, improve the geometric fusion effect of matching feature points on the air-to-ground observation network to obtain optimized matching feature points. Step 3: The 4PCS algorithm is used to achieve the initial registration between the point cloud of the UAV oblique photogrammetry 3D model and the point cloud generated by the UAV image. Combined with the optimized matching feature points and ICP algorithm, the two point clouds after the initial registration are finely registered to obtain the fused point cloud. Step 4: Extract planar point clouds from the fused point cloud based on the RANSAC algorithm, and group planar point clouds with approximate normal vectors according to their normal vectors; extract two-dimensional boundaries for each planar point cloud, and calculate the adjacent points and initial normal vectors of each two-dimensional boundary point; Based on this, the normal vectors of the two-dimensional boundary points are first optimized by adjacency relationship and initial normal vector. Then, the positions of the adjacent points are fine-tuned based on the least squares method according to the optimized normal vector to obtain piecewise smooth line segments. After the above processing is performed on each planar point cloud in the planar point cloud group, global regularization processing is performed to obtain globally regularized ground feature boundaries. Step 4 includes: Step 41: Abstract the straight line segments in the planar point cloud group into center points and their direction angles, and merge all direction angles in the same group and the included angles between different planar groups to generate a direction angle pool; Step 42: Generate a series of hypothetical line segments based on the direction angle in the direction angle pool and the center point of each line segment, ensuring that the regularized line segments are a combination of these hypothetical line segments; Step 43: Describe the data fitting penalty and smoothness penalty for each hypothetical line segment using the energy equation. The higher the degree of fit of the hypothetical line segment, the smaller the data fitting penalty, and vice versa. Step 44: Obtain the optimal solution through the graph cut method to obtain the globally regularized ground feature boundary.

Images