Unmanned aerial vehicle-based image data automatic generation method and system, electronic equipment and computer readable storage medium

Through the automatic generation method of drone image data, including distortion correction, depth map generation and visual classification model recognition, combined with the RBF neural network to fit the elevation surface, the problems of low efficiency and poor accuracy of drone image data generation are solved, and the problem of low efficiency and poor accuracy of drone image data generation is realized. High-precision automated DEM data generation.

CN119068134AInactive Publication Date: 2024-12-03BEIJING DATA INTELLIGENCE INFORMATION TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411563146.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-05
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing drone image data generation methods are inefficient and have poor accuracy when generating high-precision digital surface model (DSM), digital orthograph image map (DOM) and digital elevation model (DEM), especially in low-altitude photogrammetry. It is difficult to completely filter out non-ground factors, affecting the accuracy of the DEM.

Method used

The automatic generation method based on drone image data is adopted, including distortion correction processing, depth map generation, texture mapping and grid processing, combined with the pre-trained visual classification model to identify non-ground elements, and fit the elevation surface through the RBF neural network. Implement automated generation of DEM.

Benefits of technology

It realizes automated production of high-precision DSM and DOM data, and automatically recognizes non-ground elements, significantly improving the rate and accuracy of the filling area of ​​DEM data, reducing manual intervention, and achieving full automated production from DSM to DEM.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068134A_ABST
    Figure CN119068134A_ABST
Patent Text Reader

Abstract

The invention provides a method for automatically generating image data based on an unmanned aerial vehicle. The method comprises the following steps: S1, acquiring a primary image data set; s2, acquiring an advanced sequence image set and an advanced camera pose data set; s3, acquiring aerial triangulation result data and performing densification processing to obtain a depth map; s4, sequentially converting the depth map into three-dimensional point cloud data, and performing texture mapping and gridding processing to obtain three-dimensional grid data; s5, generating DSM data and DOM data; s6, identifying non-ground element sub-regions in the DOM data by using a pre-trained visual classification model to form a filling region; searching an advanced correlation elevation data point set of the filling area; and S7, carrying out fitting calculation on an elevation curved surface of the filling area based on the advanced correlation elevation data point set, and generating final DEM data. The invention develops a method and a system for automatically generating high-precision DSM, DOM and DEM data. The technical problems of low efficiency, poor precision and the like of automatic production of DSM, DOM and DEM data are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and specifically relates to a method, system, electronic device and computer-readable storage medium for automatically generating image data based on unmanned aerial vehicles. Background Art

[0002] In recent years, due to the need for rapid and precise mapping, drones can quickly acquire images in a very short time thanks to their advantages of low-altitude and low-speed flight, good reliability, high safety, strong maneuverability and low cost. These are features that traditional aerospace remote sensing platforms cannot possess. They have become an effective supplement to satellite remote sensing and traditional aerial photogrammetry, and have been valued and widely used in various fields.

[0003] The digital surface model (DSM) is a unified spatial information of natural or artificial objects on and above the surface of the earth. It is an important information source for making digital orthophotos (DOM), updating geographic information, extracting and reconstructing buildings, etc. It plays an important role in land management, urban change detection, and mineral resource survey. For the DOM generation problem, the most representative method is based on computer vision, that is, the motion recovery structure technology is used to solve the camera pose and the sparse point cloud in three-dimensional space, and then the multi-view stereo vision technology is used to generate a three-dimensional model to obtain a DOM with geographic information. Although the existing DOM generation method is fast, the effect of DOM is greatly affected by the accuracy of the model. When the accuracy of the result of the three-dimensional model reconstruction is not high, the generated DOM will have problems such as distortion, dislocation, and ghosting. The digital elevation model (DEM) is one of the important national basic geographic information data. It simulates the ground terrain by digitalizing limited terrain elevation data, and then describes the ground elevation information. It is widely used in surveying and mapping, hydrology, earth science, physical geography and other fields. How to quickly generate high-precision and high-accuracy DEM has always been one of the hot topics in geographic information science research. In the process of low-altitude photogrammetry DEM generation, the commonly used DEM interpolation method or simple filtering method is difficult to completely filter out non-ground factors such as houses and vegetation, which in turn affects the accuracy of DEM, and also brings great difficulties to the automatic generation of DEM. The commonly used methods to solve this kind of problem mainly include manual post-processing and DSM filtering methods. However, the degree of automation of manual post-processing is low and the accuracy is poor.

[0004] Therefore, there is an urgent need to develop a method and system for automatically generating high-precision DSM, DOM and DEM data based on UAV image data. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for automatically generating DSM, DOM and DEM data based on UAV image data, which solves the technical problems of low efficiency and poor accuracy in the automated production of DSM, DOM and DEM data.

[0006] The first object of the present invention is to provide a method for automatically generating image data based on drones, the method comprising:

[0007] S1. Acquire a primary image dataset according to the aerial survey range, wherein the primary image dataset includes a primary sequence image set and a corresponding primary camera pose dataset;

[0008] S2 performs distortion correction processing on the primary sequence image set to obtain an advanced sequence image set; optimizes the primary camera pose data set to obtain an advanced camera pose data set;

[0009] S3 obtains the aerial triangulation result data of each advanced sequence image in the advanced sequence image set; based on the aerial triangulation result data and the advanced camera pose data set, densely matches each advanced sequence image in turn to obtain a depth map of each advanced sequence image;

[0010] S4 sequentially converts the depth map of each advanced sequence image into three-dimensional point cloud data, and performs texture mapping and gridding on the three-dimensional point cloud data to obtain three-dimensional grid data;

[0011] S5 uses an interpolation algorithm in a two-dimensional coordinate system to perform grid interpolation on the three-dimensional grid data to generate DSM data; and calculates the range of each vertex in the three-dimensional grid to generate DOM data;

[0012] S6 uses the pre-trained visual classification model to identify non-ground element sub-areas and ground element sub-areas in DOM data; merges the identified non-ground element sub-areas to form a filling area; regards the ground element sub-area as a non-filling area; and searches for the primary relevant elevation data point set in the filling area; removes the gross error points in the primary relevant elevation data point set to obtain the advanced relevant elevation data point set;

[0013] S7 uses RBF neural network to fit the high-level related elevation data point set to calculate the elevation surface of the filled area, and obtain the DEM data of the filled area; generates the DEM data of the non-filled area through smoothing filtering; the DEM data of the filled area and the DEM data of the non-filled area generate the final DEM data.

[0014] Specifically, step S6 further includes:

[0015] S61 uses a classification network algorithm to segment the DOM data into multiple sub-areas, and inputs each sub-area obtained after segmentation into a pre-trained visual classification model for classification and recognition, thereby obtaining multiple non-ground element sub-areas and ground element sub-areas; the ground element sub-areas are used as non-filled areas;

[0016] S62: randomly select a non-ground element sub-area and search outwards according to the radius R, and the sub-areas within the radius R are regarded as the related sub-areas of the non-ground element sub-area;

[0017] S63: Based on the relevant sub-region, search outward for other relevant sub-regions again until the search is completed, merge all relevant sub-regions, and construct a circumscribed polygon to form a filling region;

[0018] S64 obtains the primary relevant elevation data point set of the filling area; and uses the DoG extreme value detection method to detect gross error points, removes gross error points in the primary relevant elevation data point set, and uses the least squares moving surface fitting method to perform elevation interpolation on the gross error points to obtain the advanced relevant elevation data point set.

[0019] Specifically, the specific method of obtaining the primary relevant elevation data point set of the filling area in step S64 includes:

[0020] The height difference capability attenuation function is used to perform iterative search to determine the outward extension distance of the filling area, and the primary relevant elevation data point set of the filling area is obtained according to the outward extension distance of the filling area;

[0021] The height difference capability attenuation function expression is:

[0022]

[0023] in The outward extension range of the filling area during the nth iteration search, is the number of related elevation data points in the filled area during the nth iteration search, is the difference between the i-th elevation point and the average elevation value in the filled area at the n-th iteration;

[0024] During the n+1th iteration search, the filling area extends outward When searching for the nth iteration The relationship is:

[0025] ;

[0026] in is the sensitivity, is the preset outward search distance step;

[0027] when When , the iterative search ends, and the corresponding filling area is extended outward All the pixels within are taken as the primary relevant elevation data point set of the filling area; is the preset threshold.

[0028] Specifically, the pre-trained visual classification model in step S6 includes an image encoding network, a prompt encoder and a decoding network; the prompt encoder is used to obtain the position encoding and image block encoding of DOM data; the image encoding network is used to obtain the invisible features and encoding features of DOM data; the decoding network realizes the identification of non-ground element sub-areas in DOM data based on the invisible features and encoding features.

[0029] Specifically, the image encoding network in step S6 includes a multi-layer Transformer encoder block; the decoding network includes a fusion module, a linear layer and a prediction head.

[0030] Specifically, the specific method of identifying the non-ground element sub-area in step S6 includes:

[0031] The DOM data is input into the prompt encoder to obtain the image block code and the position code; the image block code and the position code are summed and input into the multi-layer Transformer encoder block coding network of the image encoder to obtain a coding feature and twelve hidden features; the coding feature and all the hidden features are input into the decoding network; the twelve hidden features are fused into the summed feature and the dimension is adjusted; the summed feature is upsampled three times to obtain three upsampled features; the coding feature and the three upsampled features are interpolated to obtain four features a, feature b, feature c and feature d of the same dimension; feature a, feature b, feature c and feature d are feature fused, and then input into the linear layer for semantic segmentation to identify non-ground feature sub-areas and ground feature sub-areas.

[0032] Specifically, the RBF neural network in step S7 includes an input layer, a hidden layer and an output layer; wherein the hidden layer is a nonlinear RBF activation function.

[0033] The second object of the present invention is to provide a system for automatically generating drone image data, comprising:

[0034] A data acquisition module is configured to acquire a primary image data set according to the aerial survey range, wherein the primary image data set includes a primary sequence image set and a corresponding primary camera pose data set;

[0035] The distortion correction processing module is configured to perform distortion correction processing on the primary sequence image set to obtain an advanced sequence image set; and optimize the primary camera pose data set to obtain an advanced camera pose data set;

[0036] The depth map generation module is configured to obtain the aerial triangulation result data of each advanced sequence image in the advanced sequence image set; the aerial triangulation result data includes image internal and external parameters and sparse point clouds; based on the aerial triangulation result data, each advanced sequence image is densified in turn to obtain a depth map of each advanced sequence image;

[0037] The grid data generation module is configured to sequentially convert the depth map of each advanced sequence image into three-dimensional point cloud data, and perform texture mapping and gridding processing on the three-dimensional point cloud data to obtain three-dimensional grid data;

[0038] The DOM and DSM generation module is configured to perform grid interpolation on the three-dimensional grid data using an interpolation algorithm in a two-dimensional coordinate system to generate DSM data; and calculate the range of each vertex in the three-dimensional grid to generate DOM data;

[0039] The infill area generation module is configured to use the pre-trained visual classification model to identify non-ground element sub-areas and ground element sub-areas in the DOM data; merge the identified non-ground element sub-areas to form infill areas; use the ground element sub-areas as non-infill areas; and search for a primary relevant elevation data point set in the infill area; remove gross error points in the primary relevant elevation data point set to obtain a high-level relevant elevation data point set;

[0040] The DEM generation module is configured to fit the high-level related elevation data point set through the RBF neural network to calculate the elevation surface of the filled area, and obtain the DEM data of the filled area; generate the DEM data of the non-filled area through smoothing filtering; and generate the final DEM data from the DEM data of the filled area and the DEM data of the non-filled area.

[0041] The third object of the present invention is to provide an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0042] A third object of the present invention is to provide a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0043] The beneficial effects of the present invention are:

[0044] (1) The present invention realizes the automatic production of DSM and DOM data, and combines the automatic interpretation function of forest land, buildings and other ground objects, automatically identifies the features of non-ground objects, extracts DEM data, reduces manual intervention, and realizes the complete automation of the conversion from DSM to DEM;

[0045] (2) The present invention constructs a visual classification model to identify DOM data and extract non-ground elements, which significantly improves the speed and accuracy of obtaining DEM data infill areas. It also uses a height difference energy attenuation function to iteratively search for relevant elevation data point sets in the DEM data infill area, and uses an elevation surface fitting method to achieve elevation surface fitting of the infill area, thereby realizing automated production of high-precision DEM data. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 Schematic diagram of the process of the method for automatically generating drone image data according to an embodiment of the present invention;

[0048] Figure 2 A flow chart of a method for automatically generating DEM data in an embodiment of the present invention;

[0049] Figure 3 This is an overall structural diagram of a visual classification model in an embodiment of the present invention;

[0050] Figure 4 Schematic diagram of the structure of the automatic generation system based on drone image data in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0052] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0053] Please refer to Figure 1 , Figure 1The first object of the present invention is to provide a method for automatically generating image data based on drones, the method comprising:

[0054] S1. Acquire a primary image dataset according to the aerial survey range, wherein the primary image dataset includes a primary sequence image set and a corresponding primary camera pose dataset.

[0055] S2 performs distortion correction processing on the primary sequence image set to obtain an advanced sequence image set; and optimizes the primary camera pose data set to obtain an advanced camera pose data set.

[0056] In an embodiment of the present invention, when the drone is in flight, the primary sequence image set obtained usually carries a matching primary camera pose data set. The primary camera pose data set includes a location data set (GPS) and a POS data set, which makes it easier to process images. GPS data is generally represented by x, y, and z, representing the geographic location information of the drone at the exposure point during flight. The POS data set mainly includes heading angle, pitch angle, and roll angle, which are represented by φ, ω, and k respectively. In an embodiment of the present invention, the primary sequence image set is subjected to distortion correction processing, including uniform light and color and distortion correction processing. In an embodiment of the present invention, the primary camera pose data set is optimized to obtain an advanced camera pose data set. In an embodiment of the present invention, image control points are selected on the spot, image control point coordinate information is obtained, and the plane coordinates and elevation values ​​in the POS data set are converted and optimized to obtain an advanced camera pose data set and an advanced POS data set therein.

[0057] S3 obtains the aerial triangulation result data of each advanced sequence image in the advanced sequence image set; based on the aerial triangulation result data and the advanced camera pose data set, densely matches each advanced sequence image in turn to obtain a depth map of each advanced sequence image.

[0058] S4 converts the depth map of each advanced sequence image into three-dimensional point cloud data in turn, and performs texture mapping and gridding on the three-dimensional point cloud data to obtain three-dimensional grid data.

[0059] In an embodiment of the present invention, aerial triangulation is performed on each advanced sequence image in the advanced sequence image set through the control point coordinate information and the advanced POS data set in the advanced camera pose data set to obtain aerial triangulation result data; the aerial triangulation result data includes image internal and external parameters and sparse point clouds; based on the aerial triangulation result data and the advanced camera pose data set, dense matching is performed on each advanced sequence image in turn to obtain a depth map of each advanced sequence image. The depth map of each advanced sequence image is fused and converted into three-dimensional point cloud data; a Delaunay tetrahedron set is constructed based on the three-dimensional point cloud data, and the Delaunay tetrahedron set is divided into multiple surface triangles using the graph cut method, and texture mapping is performed on it to obtain three-dimensional grid data.

[0060] S5 uses an interpolation algorithm in a two-dimensional coordinate system to perform grid interpolation on the three-dimensional grid data to generate DSM data; and calculates the range of each vertex in the three-dimensional grid to generate DOM data.

[0061] In an embodiment of the present invention, the three-dimensional grid data is projected into a plane coordinate system, and an interpolation algorithm is used in the plane coordinate system to perform grid interpolation on the three-dimensional grid data to generate DSM data; and then the range of each vertex in the three-dimensional grid data is calculated in the plane coordinate system to generate DOM data. The interpolation algorithm can be any one of a moving least squares algorithm, an inverse distance weighted algorithm, and a bilinear interpolation algorithm.

[0062] S6 uses a pre-trained visual classification model to identify non-ground feature sub-areas and ground feature sub-areas in DOM data; merges the identified non-ground feature sub-areas to form a filling area; uses the ground feature sub-areas as non-filling areas; and searches for the primary relevant elevation data point set in the filling area; removes gross error points in the primary relevant elevation data point set to obtain the advanced relevant elevation data point set.

[0063] Please refer to Figure 2 , Figure 2 Flow chart of the method for automatically generating DEM data in an embodiment of the present invention. In an embodiment of the present invention, step S6 further includes:

[0064] S61 uses a classification network algorithm to segment the DOM data into multiple sub-areas, and inputs each sub-area obtained after segmentation into a pre-trained visual classification model for classification and recognition to obtain multiple non-ground element sub-areas and ground element sub-areas; the ground element sub-areas are used as non-filled areas.

[0065] In the embodiment of the present invention, in order to improve efficiency and reduce computer burden, the DOM data is divided into fixed-size grids, and the Fractal Network Algorithm (FNEA) is used to segment the DOM into multiple sub-areas. The basic idea of ​​FNEA is: first, a pixel is randomly selected as an object in the area to be segmented, and then all pixels in the neighborhood of the pixel that meet the set heterogeneity threshold are merged together, and then the merged area is taken as the object, and the pixels in its neighborhood that meet the set heterogeneity threshold are continued to be searched, and merged again, and this is repeated until the heterogeneity between the objects does not meet the set threshold. The set threshold is called a scale parameter. When the threshold is larger, the area of ​​the segmented object is larger and the number is smaller; when the threshold is smaller, the area of ​​the segmented object is smaller and the number is larger. Different scale parameters can be set to change the segmentation result. The multiple sub-areas obtained by segmentation are input into the pre-trained visual classification model for classification and recognition, and the areas belonging to houses, vegetation, etc. in the classification results are regarded as multiple non-ground element sub-areas (filled sub-areas), and other ground element sub-areas are regarded as non-filled areas. For non-filled areas, DEM data is generated by smoothing filtering, while for multiple filled sub-areas, further processing is required.

[0066] In the embodiment of the present invention, the multiple non-ground element sub-regions obtained by classification and identification according to the pre-trained visual classification model often cannot construct a continuous associated area, so it is necessary to merge the multiple non-ground element sub-regions to construct the final filling area.

[0067] S62: randomly select a non-ground element sub-area and search outwards according to the radius R, and the sub-areas within the radius R are regarded as the related sub-areas of the non-ground element sub-area;

[0068] S63: Based on the relevant sub-region, search outward for other relevant sub-regions again until the search is completed, merge all relevant sub-regions, and construct a circumscribed polygon to form a filling region;

[0069] S64 obtains the primary relevant elevation data point set of the filling area; and uses the DoG extreme value detection method to detect gross error points, removes gross error points in the primary relevant elevation data point set, and uses the least squares moving surface fitting method to perform elevation interpolation on the gross error points to obtain the advanced relevant elevation data point set.

[0070] In the embodiment of the present invention, the specific method of obtaining the primary relevant elevation data point set of the filling area in step S64 includes:

[0071] The height difference capability attenuation function is used to perform iterative search to determine the outward extension distance of the filling area, and the primary relevant elevation data point set of the filling area is obtained according to the outward extension distance of the filling area;

[0072] The height difference capability attenuation function expression is:

[0073]

[0074] in The outward extension range of the filling area during the nth iteration search, is the number of related elevation data points in the filled area during the nth iteration search, is the difference between the i-th elevation point and the average elevation value in the filled area at the n-th iteration;

[0075] During the n+1th iteration search, the filling area extends outward When searching for the nth iteration The relationship is:

[0076] ;

[0077] in is the sensitivity, is the preset outward search distance step;

[0078] when When , the iterative search ends, and the corresponding filling area is extended outward All the pixels within are taken as the primary relevant elevation data point set of the filling area; is the preset threshold.

[0079] Please refer to Figure 3 , Figure 3 Schematic diagram of the overall structure of the visual classification model in an embodiment of the present invention.

[0080] In an embodiment of the present invention, the pre-trained visual classification model in step S6 includes an image encoding network, a prompt encoder and a decoding network; the prompt encoder is used to obtain the position encoding and image block encoding of DOM data; the image encoding network is used to obtain the invisible features and encoding features of DOM data; the decoding network realizes the non-ground feature sub-area identified in the DOM data based on the invisible features and encoding features. The image encoding network includes a multi-layer Transformer encoder block; the decoding network includes a fusion module, a linear layer and a prediction head.

[0081] In an embodiment of the present invention, the specific method for identifying the non-ground element sub-area in step S6 includes: inputting DOM data into a prompt encoder to obtain an image block code and a position code; summing the image block code and the position code, and inputting them into the multi-layer Transformer encoder block coding network of the image encoder to obtain a coding feature and twelve hidden layer features; then inputting the coding feature and all the hidden features into a decoding network; fusing the twelve hidden layer features into a summed feature and performing dimensionality adjustment; and upsampling the summed feature three times to obtain three upsampled features; interpolating the coding feature and the three upsampled features to obtain four features a, feature b, feature c and feature d of the same dimension; performing feature fusion on feature a, feature b, feature c and feature d, and then inputting them into the linear layer for semantic segmentation to identify the non-ground element sub-area and the ground element sub-area.

[0082] S7 uses RBF neural network to fit the high-level related elevation data point set to calculate the elevation surface of the filled area, and obtain the DEM data of the filled area; generates the DEM data of the non-filled area through smoothing filtering; the DEM data of the filled area and the DEM data of the non-filled area generate the final DEM data.

[0083] In the embodiment of the present invention, the RBF neural network (robust radial neural network) in step S7 includes an input layer, a hidden layer and an output layer; wherein the hidden layer is a nonlinear RBF activation function.

[0084] The input high-level related elevation data point set is represented as a vector X, and the corresponding output layer output is a scalar function of X. Then, in the interpolation using n elevation data points, each input vector Mapping to output The function expression is , where p=1,2,...,n, in RBF, the output It is a weighted linear combination of a series of radial basis functions and can be expressed as:

[0085]

[0086] in, is the weight of each high-level related elevation data point, is the high-level related elevation data point x to p high-level related elevation data points The Euclidean distance of It is a linear function, and each high-level related elevation data point corresponds to n basis functions . Then the difference of the qth point can be expressed as:

[0087]

[0088] Let vector , weight matrix , , then ,like is a reversible matrix, then the weight matrix solution formula is:

[0089]

[0090] At this point, the continuous fitting surface can be used It means that using complex quadratic function as radial basis function in RBF fitting can obtain better fitting effect, and its expression is:

[0091]

[0092] Where 0≤ <1, is the Euclidean distance from the high-level related elevation data point x to p high-level related elevation data points, is a custom smoothing coefficient. When it is irreversible, the gradient descent method is used to solve the weight. The weight objective function is defined as:

[0093]

[0094] in, is the error when inputting the qth high-level related elevation data point, assuming is the expected output of the qth high-level related elevation data point, then:

[0095]

[0096] The weight adjustment formula is:

[0097]

[0098] in, is the learning rate, ∈(0,1), through iteration until the weight objective function E converges, the weight is obtained and a continuous fitting surface is obtained .

[0099] The RBF neural network is used to fit the high-level related elevation data point set to obtain the elevation surface of the filled area with higher accuracy.

[0100] Please refer to Figure 4 , Figure 4 The second object of the present invention is a system 10 for automatically generating image data based on drones, comprising:

[0101] The data acquisition module 110 is configured to acquire a primary image data set according to the aerial survey range, wherein the primary image data set includes a primary sequence image set and a corresponding primary camera pose data set;

[0102] The distortion correction processing module 120 is configured to perform distortion correction processing on the primary sequence image set to obtain an advanced sequence image set; and optimize the primary camera pose data set to obtain an advanced camera pose data set;

[0103] The depth map generation module 130 is configured to obtain the aerial triangulation result data of each advanced sequence image in the advanced sequence image set; the aerial triangulation result data includes image internal and external parameters and sparse point clouds; based on the aerial triangulation result data, densification processing is performed on each advanced sequence image in turn to obtain a depth map of each advanced sequence image;

[0104] The grid data generation module 140 is configured to sequentially convert the depth map of each advanced sequence image into three-dimensional point cloud data, and perform texture mapping and grid processing on the three-dimensional point cloud data to obtain three-dimensional grid data;

[0105] The DOM and DSM generation module 150 is configured to perform grid interpolation on the three-dimensional grid data using an interpolation algorithm in a two-dimensional coordinate system to generate DSM data; and calculate the range of each vertex in the three-dimensional grid to generate DOM data;

[0106] The infill area generation module 160 is configured to use the pre-trained visual classification model to identify non-ground element sub-areas and ground element sub-areas in the DOM data; merge the identified non-ground element sub-areas to form infill areas; use the ground element sub-areas as non-infill areas; and search for a primary relevant elevation data point set in the infill area; remove gross error points in the primary relevant elevation data point set to obtain a high-level relevant elevation data point set;

[0107] The DEM generation module 170 is configured to fit the high-level related elevation data point set through the RBF neural network to calculate the elevation surface of the filled area to obtain the DEM data of the filled area; generate the DEM data of the non-filled area through smoothing filtering; and generate the final DEM data from the DEM data of the filled area and the DEM data of the non-filled area.

[0108] The third object of the present invention is to provide an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0109] A third object of the present invention is to provide a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0110] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for automatically generating drone image data. It is characterized in that The method comprises: S1. Acquire a primary image dataset according to the aerial survey range, wherein the primary image dataset includes a primary sequence image set and a corresponding primary camera pose dataset; S2 performs distortion correction processing on the primary sequence image set to obtain an advanced sequence image set; optimizes the primary camera pose data set to obtain an advanced camera pose data set; S3 obtains the aerial triangulation result data of each advanced sequence image in the advanced sequence image set; based on the aerial triangulation result data and the advanced camera pose data set, densely matches each advanced sequence image in turn to obtain a depth map of each advanced sequence image; S4 sequentially converts the depth map of each advanced sequence image into three-dimensional point cloud data, and performs texture mapping and gridding on the three-dimensional point cloud data to obtain three-dimensional grid data; S5 uses an interpolation algorithm in a two-dimensional coordinate system to perform grid interpolation on the three-dimensional grid data to generate DSM data; and calculates the range of each vertex in the three-dimensional grid to generate DOM data; S6 uses the pre-trained visual classification model to identify non-ground element sub-areas and ground element sub-areas in DOM data; merges the identified non-ground element sub-areas to form a filling area; regards the ground element sub-area as a non-filling area; and searches for the primary relevant elevation data point set in the filling area; removes the gross error points in the primary relevant elevation data point set to obtain the advanced relevant elevation data point set; S7 uses RBF neural network to fit the high-level related elevation data point set to calculate the elevation surface of the filled area, and obtain the DEM data of the filled area; generates the DEM data of the non-filled area through smoothing filtering; the DEM data of the filled area and the DEM data of the non-filled area generate the final DEM data.

2. The method according to claim 1, It is characterized in that Step S6 specifically includes: S61 uses a classification network algorithm to segment the DOM data into multiple sub-areas, and inputs each sub-area obtained after segmentation into a pre-trained visual classification model for classification and recognition, thereby obtaining multiple non-ground element sub-areas and ground element sub-areas; the ground element sub-areas are used as non-filled areas; S62: randomly select a non-ground element sub-area and search outwards according to the radius R, and the sub-areas within the radius R are regarded as the related sub-areas of the non-ground element sub-area; S63: Based on the relevant sub-region, search outward for other relevant sub-regions again until the search is completed, merge all relevant sub-regions, and construct a circumscribed polygon to form a filling region; S64 obtains the primary relevant elevation data point set of the filling area; and uses the DoG extreme value detection method to detect gross error points, removes gross error points in the primary relevant elevation data point set, and uses the least squares moving surface fitting method to perform elevation interpolation on the gross error points to obtain the advanced relevant elevation data point set.

3. The method according to claim 1, It is characterized in that The specific method of obtaining the primary relevant elevation data point set of the filling area in step S64 includes: The height difference capability attenuation function is used to perform iterative search to determine the outward extension distance of the filling area, and the primary relevant elevation data point set of the filling area is obtained according to the outward extension distance of the filling area; The height difference capability attenuation function expression is: in The outward extension range of the filling area during the nth iteration search, is the number of related elevation data points in the filled area during the nth iteration search, is the difference between the i-th elevation point and the average elevation value in the filled area at the n-th iteration; During the n+1th iteration search, the filling area extends outward When searching for the nth iteration The relationship is: ; in is the sensitivity, is the preset outward search distance step; when When , the iterative search ends, and the corresponding filling area is extended outward All the pixels within are taken as the primary relevant elevation data point set of the filling area; is the preset threshold.

4. The method according to claim 1, It is characterized in that The pre-trained visual classification model in step S6 includes an image encoding network, a prompt encoder and a decoding network; the prompt encoder is used to obtain the position encoding and image block encoding of DOM data; the image encoding network is used to obtain the invisible features and encoding features of DOM data; the decoding network realizes the identification of non-ground element sub-areas in DOM data based on the invisible features and encoding features.

5. The method according to claim 4, It is characterized in that The image encoding network in step S6 includes a multi-layer Transformer encoder block; the decoding network includes a fusion module, a linear layer and a prediction head.

6. The method according to claim 5, It is characterized in that The specific method of identifying the non-ground element sub-area in step S6 includes: Input the DOM data into the prompt encoder to obtain the image block code and position code; The image block code and position code are added together and input into the multi-layer Transformer encoder block coding network of the image encoder to obtain a coding feature and twelve hidden layer features; Then the coding features and all the hidden features are input into the decoding network; the twelve hidden layer features are fused into the sum feature and the dimension is adjusted; the sum feature is upsampled three times to obtain three upsampled features; the coding features and the three upsampled features are interpolated to obtain four features a, feature b, feature c and feature d of the same dimension; Features a, b, c, and d are fused and then input into the linear layer for semantic segmentation to identify non-ground feature sub-areas and ground feature sub-areas.

7. The method according to claim 1, It is characterized in that The RBF neural network in step S7 includes an input layer, a hidden layer and an output layer; wherein the hidden layer is a nonlinear RBF activation function.

8. An automatic generation system based on drone image data, It is characterized in that The system comprises: A data acquisition module is configured to acquire a primary image data set according to the aerial survey range, wherein the primary image data set includes a primary sequence image set and a corresponding primary camera pose data set; The distortion correction processing module is configured to perform distortion correction processing on the primary sequence image set to obtain an advanced sequence image set; and optimize the primary camera pose data set to obtain an advanced camera pose data set; The depth map generation module is configured to obtain the aerial triangulation result data of each advanced sequence image in the advanced sequence image set; the aerial triangulation result data includes image internal and external parameters and sparse point clouds; based on the aerial triangulation result data, each advanced sequence image is densified in turn to obtain a depth map of each advanced sequence image; The grid data generation module is configured to sequentially convert the depth map of each advanced sequence image into three-dimensional point cloud data, and perform texture mapping and gridding processing on the three-dimensional point cloud data to obtain three-dimensional grid data; The DOM and DSM generation module is configured to perform grid interpolation on the three-dimensional grid data using an interpolation algorithm in a two-dimensional coordinate system to generate DSM data; and calculate the range of each vertex in the three-dimensional grid to generate DOM data; The infill area generation module is configured to use the pre-trained visual classification model to identify non-ground element sub-areas and ground element sub-areas in the DOM data; merge the identified non-ground element sub-areas to form infill areas; use the ground element sub-areas as non-infill areas; and search for a primary relevant elevation data point set in the infill area; remove gross error points in the primary relevant elevation data point set to obtain a high-level relevant elevation data point set; The DEM generation module is configured to fit the high-level related elevation data point set through the RBF neural network to calculate the elevation surface of the filled area, and obtain the DEM data of the filled area; generate the DEM data of the non-filled area through smoothing filtering; and generate the final DEM data from the DEM data of the filled area and the DEM data of the non-filled area.

9. An electronic device, It is characterized in that include: one or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Photogrammetry point cloud filtering method fusing image information

    CN112561981A

  • DSM calculation method and device, computer equipment and storage medium

    CN113496550A

  • Digital elevation model extraction method based on target context convolutional neural network

    CN116385892A

  • Photogrammetry method and system suitable for low-overlapping-degree unmanned aerial vehicle images

    CN118089666A

  • Method and device for detecting traffic area in unstructured scene based on visual large model

    CN118379499A