A Feature Preservation-Driven Drone Oblique Photography Scene Restoration Method

Abstract

The present invention relates to a method for repairing an unmanned aerial vehicle (UAV) oblique photography scene driven by feature retention. A point cloud model is reconstructed from a set of low-altitude remote sensing images continuously captured from multiple perspectives on a UAV platform, and then it is converted into a triangular mesh model through the Poisson reconstruction algorithm. By defining a variational model to solve the set of feature regions and simultaneously optimizing the surface normal. By extending the dilation and erosion operations in computer graphics, the selected candidate feature regions are processed to exclude discrete points and connect adjacent regions, obtaining an optimized candidate feature region. By continuously solving the variational model and optimizing the candidate feature region, the smooth regions are filtered and the range of the candidate feature region is narrowed. When the iteration meets the termination condition, the repaired scene model is output. The present invention accurately locates the scene model constructed by UAV oblique photography, solving the practical application problem that the UAV oblique photography cannot accurately construct a three-dimensional scene model due to noise interference.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, image and video processing, and relates to a feature-preserving driven unmanned aerial vehicle oblique photography scene restoration method, and specifically to a scene restoration method having feature-preserving properties for a triangular mesh model constructed by unmanned aerial vehicle oblique photography. Background Art

[0002] Oblique photography is a new type of aerial remote sensing 3D reconstruction technology that has emerged in recent years. It uses drones to take photos at low altitudes to obtain high-resolution full-element geographic information of the survey area. It can quickly and efficiently obtain information on the side and top surfaces of the target object, and combine real-time positioning technology with 3D reconstruction technology to generate a 3D model of the real scene. However, factors such as external environmental interference and imaging equipment accuracy will bring certain noise to the final generated 3D scene. In order to better use the collected 3D scene, the scene restoration task must exclude various noises while retaining the real data of the 3D scene as much as possible, that is, the model feature information.

[0003] Among the many ways to represent three-dimensional scenes, the triangular mesh model has certain advantages over all types of data structures due to its powerful ability to represent three-dimensional objects with any complex topological structure. It has now become a common and effective means of representing three-dimensional geometric data. Triangular mesh surface repair has now become a classic problem in digital geometry processing. Both academia and industry are committed to removing model surface noise while maintaining its original feature information.

[0004] As a data structure with spatial topological information, triangular mesh surfaces contain not only geometric information of vertex coordinates, but also topological information describing the relationship between points, lines, and surfaces. In the analysis of triangular mesh surface data, features exist in both geometric information and topological information. General denoising algorithms will cause feature information loss in triangular mesh surfaces because noise carries feature information and is of similar scale. Preserving features when denoising triangular mesh surfaces is a challenging task and is the focus of many current studies.

[0005] Therefore, a method is provided to improve the accuracy of the acquired three-dimensional scene by repairing the triangular mesh model established by drone oblique photography, while retaining the edge and surface texture features in the model, thereby solving the distortion problem caused by noise in the scene model established directly by oblique photography and the damage to features caused by general denoising methods. Summary of the invention

[0006] Technical issues to be solved

[0007] To avoid the deficiencies of the prior art, the present invention proposes a feature-preserving-driven method for repairing UAV oblique photography scenes, which mainly solves the problems of noise interference and information loss in 3D reconstruction by the UAV oblique photography method, removes noise from the obtained triangular mesh model and preserves feature information, so as to achieve the effect of repairing the scene model. Specifically, the object of the present invention is to improve the following aspects:

[0008] 1. The reconstruction results of UAV oblique photography at the present stage are greatly affected by noise, and the accuracy of its 3D scene is low;

[0009] 2. Establish a feature-preserving-driven variational model to repair the scene model reconstructed by oblique photography;

[0010] 3. While removing noise, it can preserve the fine feature information of the triangular mesh model, and reduce information loss during the repair process to the greatest extent.

[0011] Technical Solution

[0012] A feature-preserving-driven method for repairing UAV oblique photography scenes, characterized in that the steps are as follows:

[0013] Step 1: Convert UAV oblique photography into a triangular mesh model: Reconstruct a point cloud model from a set of low-altitude remote sensing images taken continuously from multiple perspectives on the UAV platform, and then convert it into a triangular mesh model through the Poisson reconstruction algorithm: Where represents the set of vertices, ε = {e i ∈ V × V, i = 1, 2,..., E} represents the set of edges, represents the set of faces;

[0014] Step 2: Define the feature region: Divide the triangular mesh scene model into a feature region and a non-feature region according to the face normal gradient, where the face normal gradient of the feature region is greater than that of the non-feature region;

[0015] The segmentation of the two regions optimizes and solves the variational model to obtain a candidate feature region and a preliminarily denoised triangular mesh model, where the noisy triangular mesh and the set of all face normals of it, the variational model is:

[0016]

[0017]

[0018] Where: represents a connected subset of the candidate feature region, and Γ represents the set of feature edges composed of Γ l constituted, represents the optimized face normal; Γ l refers to the set of edges that satisfy |Γ l |≥t and are connected to each other. The parameter p∈(0,1) represents the upper limit of the proportion of characteristic edges among all edges, and the parameter λ represents the influence degree of the face normal gradient in the optimization objective, that is, the degree of filtering, Γ c = ε\Γ represents the complement of Γ;

[0019] Step 3: Set all features. Although the tiny features have a small scale, they still have connectivity. Use erosion and dilation operators to solve the variational model:

[0020] For a connected subset of the candidate feature region Define the erosion operator and the dilation operator:

[0021]

[0022]

[0023] where represents the shortest path from Γ l to and the number of vertices passed by this path should be less than t - 1;

[0024] In the optimization iteration, the candidate feature region is processed as follows to obtain an optimized candidate feature region:

[0025] Γ′ = D(E(Γ))

[0026] Step 4: Continuously optimize the variational model in Step 3 to filter the smooth region and narrow the range of the feature region. When the iteration meets the termination condition, output the final denoising model, use the candidate feature region as the feature region, and output the repaired scene model expressed in the form of a triangular mesh.

[0027] The operation in Step 3 satisfies the distributive law:

[0028]

[0029]

[0030] The iteration termination condition: where ε represents the threshold of the change in the face normal required for each iteration. If it is less than this threshold, it is considered that the repair has approached convergence.

[0031] Beneficial effects

[0032] A method for repairing an unmanned aerial vehicle (UAV) oblique photography scene driven by feature retention is proposed in the present invention. The UAV oblique photography obtains a scene model represented by a triangular mesh. A point cloud model is reconstructed from a set of low-altitude remote sensing images taken continuously from multiple perspectives on the UAV platform, and then it is converted into a triangular mesh model through the Poisson reconstruction algorithm. By defining a variational model to solve the set of feature regions and simultaneously optimize the surface normal. By extending the dilation and erosion operations in computer graphics, the candidate feature regions selected in step 2 are optimized, discrete points are excluded, and adjacent regions are connected to obtain an optimized candidate feature region. By continuously solving the variational model and optimizing the candidate feature regions, the smooth regions are filtered and the range of the candidate feature regions is reduced. When the iteration meets the termination condition, the repaired scene model is output.

[0033] Generally speaking, the present invention is a method for repairing an unmanned aerial vehicle (UAV) oblique photography scene driven by feature retention, which can accurately locate the feature distribution of the scene model constructed by UAV oblique photography, remove the random noise in the non-feature regions, greatly reduce the influence of acquisition noise on the scene model, and retain the refined scene features, thereby solving the practical application problem that the UAV oblique photography at the present stage cannot accurately construct a three-dimensional scene model due to noise influence. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of the present invention, which briefly summarizes the above algorithm steps and the input / output form of the system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] The present invention will be further described in combination with the embodiments and the drawings:

[0036] The present invention proposes a method for repairing an unmanned aerial vehicle (UAV) oblique photography scene driven by feature retention to repair the scene of the three-dimensional reconstruction model of UAV oblique photography. Through a variational model, the feature regions and smooth regions are finely iteratively partitioned, and the smooth regions are gradually filtered, so as to completely retain the feature information while denoising the scene model. The technical solution is as follows:

[0037] Step 1: The UAV oblique photography obtains a scene model represented by a triangular mesh. A point cloud model is reconstructed from a set of low-altitude remote sensing images taken continuously from multiple perspectives on the UAV platform, and then it is converted into a triangular mesh model through the Poisson reconstruction algorithm. The above are existing algorithms that are already mature and widely used.

[0038] Step 2: Define the feature regions and optimize the surface normal. By defining a variational model to solve the set of feature regions and simultaneously complete the optimization of the surface normal.

[0039] Step 3: Optimize the candidate feature regions. By expanding the dilation and erosion operations in computer graphics, optimize the candidate feature regions selected in Step 2, exclude discrete points, and connect adjacent regions to obtain an optimized candidate feature region.

[0040] Step 4: Iteratively repeat the process of Steps 2 - 3. By continuously solving the variational model and optimizing the candidate feature regions, filter the smooth regions and narrow down the scope of the candidate feature regions. When the iteration meets the termination condition, output the repaired scene model.

[0041] First, briefly introduce the basic operations of triangular meshes. A triangular mesh model can be defined as

[0042]

[0043] where denotes the set of vertices, ε = {e i ∈ V × V, i = 1, 2, …, E} denotes the set of edges, denotes the set of faces.

[0044] The set of all vertices that are connected to vertex v i by an edge is called the 1 - ring neighborhood of v i and is denoted as Similarly, we can define the n - ring neighborhood of v i The normal vector of face f i is denoted as the face normal n i , and the set of all face normals is denoted as and assume that for any Here, for any vector let be its l2 norm.

[0045] For two triangular mesh models and with the same topological structure, we define

[0046]

[0047]

[0048] For any edge e ∈ ε and the two adjacent faces f i and f j , let

[0049]

[0050] Similarly, for a given subset of edges let

[0051]

[0052] In addition, we make the following explanations for other symbols used. For the matrix and the index set Ω = {1, 2, …, m}, let M Ω denote the submatrix of M whose row indices are included in Ω. For a finite set Σ, let |Σ| denote the number of its elements. For a vector let let denote the number of its non-zero elements.

[0053] Referring to Figure 1 , the implementation steps of the present invention are as follows:

[0054] Step 1: The UAV conducts oblique photography to obtain a scene model represented by a triangular mesh. The main idea is to combine oblique photography with the Poisson reconstruction method. The above algorithms have been well studied and are not related to the core essence of the present invention, so they will not be elaborated here.

[0055] Step 2: Define the feature region and optimize the face normal. For the triangular mesh scene model obtained in Step 1, we define the local area with a relatively large face normal gradient as the feature region, and at the same time consider that the face normal gradient in the non-feature region should be relatively small. Therefore, for a noisy triangular mesh and the set of all face normals therein, we propose the following variational model

[0056]

[0057]

[0058] where represents a connected subset of the candidate feature region, Γ represents the set of feature edges composed of Γ l , represents the optimized face normal. Γ l refers to the set of edges that satisfy |Γ l | ≥ t and are connected to each other. The parameter p ∈ (0, 1) represents the upper limit of the proportion of feature edges among all edges, and the parameter λ represents the influence degree of the face normal gradient in the optimization objective, that is, the filtering degree. Γ c = ε\Γ represents the complement of Γ.

[0059] This optimization model actually solves a set of feature regions and target face normals that minimize the face normal gradient in the non-feature region under the premise of the smallest comprehensive change in all face normals, obtaining the candidate feature region and the triangular mesh model after preliminary denoising.

[0060] Step 3: Optimize the candidate feature regions. We propose two operators, erosion and dilation, to optimize the candidate feature regions, making this set more precisely encompass geometric features and exclude random noise. The premise of this operation is that we assume that although tiny features have a small scale, they still have connectivity.

[0061] For a connected subset of the candidate feature regions Define the erosion operator and the dilation operator as follows

[0062]

[0063]

[0064] where represents the shortest path from Γ l to and the number of vertices passed by this path should be less than t - 1. In addition, the above operations satisfy the following distributive law

[0065]

[0066]

[0067] Therefore, we process the candidate feature regions as follows to exclude random noise and retain tiny geometric features, obtaining an optimized candidate feature region.

[0068] Γ′ = D(E(Γ))

[0069] Step 4: Iteratively repeat the process of Steps 2 - 3. By continuously solving the variational model and optimizing the candidate feature regions, filter the smooth regions and narrow down the scope of the candidate feature regions. When the iteration meets the following termination condition, we consider that the candidate feature region is approximately equivalent to the real feature region, and output the repaired scene model expressed in the form of a triangular mesh.

[0070]

[0071] where ε represents the threshold of the change in surface normal required for each iteration. If it is less than this threshold, it is considered that the repair has approached convergence.

Claims

1. A feature-preserving driven method for repairing UAV oblique photography scenes, characterized in that The steps are as follows: Step 1. Convert the drone oblique photography into a triangular mesh model: Reconstruct a point cloud model from a set of low-altitude remote sensing images taken from multiple consecutive perspectives on the drone platform, and then convert it into a triangular mesh model through the Poisson reconstruction algorithm: where V = {v i , i = 1, 2, …, V} represents the set of vertices, E = {e i ∈ V × V, i = 1, 2, …, E} represents the set of edges, and F = {f i ∈ V × V × V, i = 1, 2, …, F} represents the set of faces; Step 2: Define the feature region: Divide the triangular mesh scene model into a feature region and a non-feature region according to the surface normal gradient, where the surface normal gradient of the feature region is greater than that of the non-feature region; The variational model is optimized and solved by dividing the two regions, and the candidate feature region and the triangular mesh model after preliminary denoising are obtained, including the noisy triangular mesh and the set of all face normals therein The variational model is as follows: Wherein: represents a connected subset of the candidate feature regions, Γ represents the set of feature edges composed of Γ l N represents the optimized face normal; Γ l refers to the set of edges that satisfy |Γ l |≥t and are connected to each other. The parameter p∈(0,1) represents the upper limit of the proportion of feature edges among all edges, and the parameter λ represents the influence degree of the face normal gradient in the optimization objective, that is, the degree of filtering. Γ c = E\Γ represents the complement set of Γ; Step 3: Set all features, including tiny features. Although they have a small scale, they still have connectivity. Use two operators, erosion and dilation, to solve the variational model: For a connected subset of the candidate feature regions Define the erosion operator and the dilation operator: wherein represents the shortest path from Γ l to and the number of vertices passed by this path should be less than t - 1; In the optimization iteration, process the candidate feature region as follows to obtain an optimized candidate feature region: Γ′ = D(E(Γ)) Step 4: Continuously optimize the variational model through Step 3, filter the smooth region and narrow the range of the feature region. When the iteration meets the termination condition, output the final denoising model, use the candidate feature region as the feature region, and output the repaired scene model expressed in the form of a triangular mesh.

2. The method for repairing an oblique photography scene of a feature-preserving driven drone according to claim 1, wherein: The operations in Step 3 satisfy the distributive law:

3. The method for repairing an oblique photography scene of a feature-preserving driven drone according to claim 1, wherein: The iteration termination condition: where ε represents the threshold of the change in the surface normal required for each iteration. If it is less than this threshold, it is considered that the repair has approached convergence.

Images