Image data processing method and system based on unmanned aerial vehicle, electronic equipment and storage medium

Abstract

The invention relates to an image data processing method and system based on an unmanned aerial vehicle, electronic equipment and a storage medium. The method comprises the following steps: S1, obtaining a qualified aerial survey data set; s2, acquiring a pre-processed aerial survey data set, wherein the pre-processed aerial survey data set includes correcting an unmanned aerial vehicle image; s3, acquiring aerial triangulation result data; the aerial triangulation result data comprises camera internal and external parameters and sparse point cloud; s4, performing block processing on the corrected unmanned aerial vehicle image to obtain a plurality of region sub-blocks; s5, obtaining an associated image cluster of each region sub-block; s6, performing independent dense matching on each corrected unmanned aerial vehicle image of each region sub-block in sequence to obtain a depth map; the depth maps are fused to obtain dense point clouds; s7, constructing a three-dimensional triangular mesh based on the dense point cloud; s8, obtaining a triangular mesh with textures; and S9, based on the triangular mesh with the texture, generating DSM data, DOM data and DEM data in sequence. According to the invention, a block dense matching method is adopted, so that the dense matching efficiency is improved; and meanwhile, high-precision DSM, DOM and DEM data can be automatically generated.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to an image data processing method, system, electronic device and storage medium based on unmanned aerial vehicles. Background Art

[0002] In recent years, due to the need for rapid and precise mapping, drones can quickly acquire images in a very short time with the advantages of low-altitude and low-speed flight, good reliability, high safety, strong maneuverability and low cost. These are all advantages that traditional aerospace remote sensing platforms cannot have. They have become an effective supplement to satellite remote sensing and traditional aerial photogrammetry, and have been valued and widely used in various fields. In particular, they have achieved remarkable results in the acquisition of small-area images and high-resolution digital mapping in difficult areas. The research on the use of drones to obtain aerial images for processing and mapping has played an important role in promoting national emergency relief, geographic national conditions surveys and digital city construction. UAV oblique photogrammetry technology uses drones equipped with oblique photography devices to quickly acquire oblique three-dimensional images and then use computer automatic graphics processing technology for aerial triangulation. Through technical means such as image matching and surface texture mapping, the surface of the earth is truly restored.

[0003] However, in the process of using drones for remote sensing, it was found that the data images collected by drones have problems such as a large number of images, large image inclination, high and irregular overlap, and uneven color. In the prior art, there have been some studies on improving the accuracy of DSM and DOM. The main idea is to improve the accuracy of DOM. The key work is to edit and refine DEM. Although this method improves the accuracy and quality of DOM, the degree of automation is not high, there is a lot of manual participation and the steps are cumbersome, and the production efficiency needs to be improved. Therefore, how to efficiently process the data collected by drones and quickly generate panoramic images of the survey area, and how to automatically produce high-precision DSM and DOM in the case of large amounts of data have become urgent problems to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a drone-based image data processing method, system, electronic device and storage medium to solve the technical problems in the prior art that dense matching is inefficient when the data volume is large and high-precision DSM and DOM cannot be automatically generated.

[0005] The first object of the present invention is to provide a method for processing image data based on a drone, the method comprising:

[0006] S1 designs a route for a target aerial survey area to obtain a target route, and obtains a qualified aerial survey data set based on the target route; the qualified aerial survey data set includes an original UAV image set, a position data set, and a flight attitude data set;

[0007] S2 preprocesses the qualified aerial survey data set to obtain a preprocessed aerial survey data set; the preprocessed aerial survey data set includes a corrected drone image set;

[0008] S3 solves the camera internal and external parameters and sparse point cloud of each corrected drone image in the corrected drone image set according to the pre-processed aerial survey data set, obtains the aerial triangulation result data of each corrected drone image, and forms an aerial triangulation result data set;

[0009] S4 divides the target aerial survey area into blocks according to a preset size to obtain a plurality of regional sub-blocks; sequentially searches for a dense matching effective range for each regional sub-block, wherein a plurality of corrected drone images within the dense matching effective range constitute an initial image cluster of the regional sub-block;

[0010] S5 searches for associated images of each of the multiple corrected drone images in sequence based on the multiple corrected drone images in the initial image cluster of each regional sub-block, and forms an associated image cluster of each regional sub-block with the multiple corrected drone images in the initial image cluster of each regional sub-block and the associated images corresponding thereto;

[0011] S6 performs independent dense matching on each corrected UAV image in the associated image cluster of each regional sub-block according to the data information in the aerial triangulation result data set, the position data set, and the flight attitude data set, and obtains a depth map of each corrected UAV image; and fuses the depth maps of each corrected UAV image to obtain a final dense point cloud of the target aerial survey area;

[0012] S7 constructs a Delaunay tetrahedron set based on the final dense point cloud of the target aerial survey area, and uses the graph cut method to split the Delaunay tetrahedron set into multiple surface triangles to obtain a three-dimensional triangular mesh;

[0013] S8 automatically maps textures to the 3D triangular mesh based on multiple corrected UAV images in the corrected UAV image set to obtain a textured triangular mesh;

[0014] S9 generates DSM data, DOM data and DEM data in sequence based on textured triangular meshes.

[0015] Specifically, step S9 further includes:

[0016] S91 projects the textured triangular mesh into a plane coordinate system, and uses an interpolation algorithm in the plane coordinate system to perform grid interpolation on the triangular mesh to generate DSM data;

[0017] S92 calculates the range of each vertex in the triangular mesh in the plane coordinate system and generates DOM data;

[0018] S93 identifies non-ground object features in DOM data and obtains non-ground masks; then removes non-ground point cloud data corresponding to the non-ground masks in DSM data to obtain ground point cloud data; and then uses hierarchical interpolation method to fill holes in the ground point cloud data to generate DEM data.

[0019] Specifically, step S93 further includes:

[0020] S931 uses a semantic segmentation model to identify non-ground object features in the DOM data, and creates a non-ground mask based on the non-ground object features; the non-ground object features include buildings, trees, and bridges;

[0021] S932 searches for non-ground point cloud data corresponding to the non-ground mask in the DSM data, removes the non-ground point cloud data, and obtains ground point cloud data;

[0022] S933 uses hierarchical interpolation method to fill the gaps in ground point cloud data, thus generating complete ground point cloud data, i.e. DEM data.

[0023] Specifically, the interpolation algorithm in step S91 is any one of a moving least squares algorithm, an inverse distance weighted algorithm and a bilinear interpolation algorithm.

[0024] Specifically, the specific method of sequentially searching for the associated images of each of the multiple corrected drone images in step S5 is:

[0025] S51 uses an image association algorithm to sequentially obtain multiple candidate images corresponding to each corrected drone image in the regional sub-block;

[0026] S52 scores the multiple candidate images corresponding to each corrected drone image in sequence according to the view angle, the overlapping area, and the scale factor to obtain a score for each candidate image;

[0027] S53 uses the candidate images whose scores exceed the preset correlation degree as the associated images corresponding to the corrected drone image.

[0028] Specifically, step S6 further includes:

[0029] S61 performs independent dense matching on each corrected UAV image in the associated image cluster of each regional sub-block in turn according to the data information in the aerial triangulation result data set, the position data set, and the flight attitude data set, to obtain a depth map of each corrected UAV image in each regional sub-block;

[0030] S62 fuses the depth maps of all corrected drone images in all sub-block areas to obtain an initial dense point cloud of the target aerial survey area;

[0031] S63 filters and screens each point cloud in the initial dense point cloud, and performs point cloud downsampling processing on the screened initial dense point cloud by using voxel filtering to obtain a final dense point cloud of the target aerial survey area.

[0032] Specifically, the preprocessing in step S2 includes light and color uniformity and distortion correction processing.

[0033] The second object of the present invention is to provide an image data processing system based on a drone, comprising:

[0034] The data acquisition module is configured to pre-process the qualified aerial survey data set to obtain a pre-processed aerial survey data set; the pre-processed aerial survey data set includes corrected drone images;

[0035] The aerial triangulation result generation module is configured to solve the camera internal and external parameters and sparse point cloud of the corrected UAV image according to the pre-processed aerial survey data set, and obtain the aerial triangulation result data of the corrected UAV image;

[0036] A scene segmentation module is configured to segment the distortion-corrected drone image into blocks according to a preset size to obtain a plurality of regional sub-blocks;

[0037] The module for obtaining associated image clusters is configured to sequentially search for a dense matching effective range of each regional sub-block, wherein a plurality of corrected drone images within the dense matching effective range constitute an initial image cluster of the regional sub-block; and then sequentially search for associated images of a plurality of corrected drone images within the initial image cluster of each regional sub-block, wherein a plurality of corrected drone images within the initial image cluster of each regional sub-block and the associated images corresponding thereto constitute an associated image cluster of each regional sub-block;

[0038] The dense matching module is configured to independently and densely match each corrected UAV image in the target image cluster of each regional sub-block according to the data information in the aerial triangulation result data set, the position data set and the flight attitude data set, and obtain the depth map of each corrected UAV image; and fuse the depth maps of each corrected UAV image to obtain the final dense point cloud of the target aerial survey area;

[0039] The point cloud meshing module is configured to construct a Delaunay,tetrahedron set based on the dense point cloud, and use the graph cut method to split the Delaunay,tetrahedron set into multiple surface triangles to obtain a three-dimensional triangular mesh;

[0040] A texture mapping module is configured to automatically map textures to the three-dimensional triangular mesh based on the corrected drone image to obtain a textured triangular mesh;

[0041] The data product generation module is configured to generate DSM data, DOM data and DEM data in sequence based on the textured triangular mesh.

[0042] The third object of the present invention is to provide an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0043] A third object of the present invention is to provide a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0044] The beneficial effects of the present invention are:

[0045] (1) The present invention divides the entire scene into blocks using a spatial stereo automatic clustering method to obtain multiple regional sub-blocks; and sequentially searches for the associated images of multiple corrected drone images in each regional sub-block to form a target image cluster for each regional sub-block; and based on the target image cluster, extracts the matching pairs required for dense matching of each corrected drone image in the sub-block to obtain a depth map of each corrected drone image; and finally uses the PMVS algorithm to estimate, filter and fuse the depth map to generate a dense point cloud, thereby improving the efficiency of dense matching in large scenes and the accuracy of subsequent acquisition of three-dimensional data;

[0046] (2) The present invention obtains a sparse point cloud of a two-dimensional image sequence from the two-dimensional image sequence data of the drone, densifies the sparse point cloud by dense matching, and obtains a dense point cloud; and constructs a triangular mesh based on the dense point cloud, attaches corresponding texture information, and sequentially generates DSM, DOM, and DEM, thereby solving the problem of DOM distortion and dislocation caused by insufficient three-dimensional reconstruction accuracy, and improving the rate and accuracy of automatically generating three-dimensional data in the entire scene;

[0047] (3) The automatic interpretation function of the combined forest land, buildings and other ground objects of the present invention automatically identifies the features of non-ground objects, extracts DEM data, reduces manual intervention, and realizes full automation of the conversion from DSM to DEM. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 Schematic diagram of the process of the image data processing method based on the drone in the embodiment of the present invention;

[0050] Figure 2 This is a diagram of image data processing based on a drone in an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of an image data processing method based on a drone in an embodiment of the present invention;

[0052] Figure 4 Schematic diagram of an image data processing system based on a drone in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0054] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0055] The first object of the present invention is to provide a method for processing image data based on a drone, the method comprising:

[0056] S1 designs a route for a target aerial survey area, obtains a target route, and obtains a qualified aerial survey data set based on the target route; the qualified aerial survey data set includes an original UAV image set, a position data set, and a flight attitude data set.

[0057] In the embodiment of the present invention, when the drone is in flight, the drone image obtained usually carries a matching location data set (GPS) and a flight attitude data set, so that it is more convenient to process the image. GPS data is generally represented by x, y, and z, representing the geographic location information of the drone at the exposure point during flight. The flight attitude information mainly includes: heading angle, pitch angle, and roll angle, which are represented by φ, ω, and k respectively.

[0058] In an embodiment of the present invention, the oblique images in the aerial survey data set are detected based on quality items, and the availability of the oblique images is evaluated and the evaluation results are output. The quality items include cloud detection, shadow detection, null value detection and histogram detection. In an embodiment of the present invention, the availability of the oblique images is evaluated based on multiple quality items to achieve a comprehensive detection of remote sensing image data. There can be multiple detection methods for each quality item. For example, the detection method of cloud detection can be a physical threshold method, a full probability Bayesian method, or other detection methods. The result of cloud detection is the cloud content of the oblique image. When the cloud content is less than a preset threshold, it is considered that the corresponding oblique image is evaluated as qualified based on the availability of the oblique image detected by cloud detection. Based on multiple quality items, the availability of the oblique images is evaluated in turn, and the detection results of the availability evaluation of each quality item are automatically normalized and summarized to obtain the final availability evaluation result of the oblique image. In an embodiment of the present invention, when the availability evaluation results of each quality item are qualified, the availability evaluation result of the oblique image is qualified, otherwise it is unqualified. The original UAV images, position data and flight attitude data corresponding to the qualified oblique images are regarded as qualified aerial survey data to form a qualified aerial survey data set.

[0059] S2 preprocesses the qualified aerial survey data set to obtain a preprocessed aerial survey data set; the preprocessed aerial survey data set includes a corrected drone image set;

[0060] In the embodiment of the present invention, the preprocessing in step S2 includes light and color uniformity and distortion correction processing.

[0061] S3 solves the camera internal and external parameters and sparse point clouds of each corrected drone image in the corrected drone image set according to the preprocessed aerial survey data set, obtains the aerial triangulation result data of each corrected drone image, and forms an aerial triangulation result data set.

[0062] In the embodiment of the present invention, aerial triangulation is the basis for a series of subsequent photogrammetric processing and applications. The present invention performs aerial triangulation on the pre-processed aerial survey data set by using DPGrid software, including free network adjustment, regional network adjustment and other operations, to obtain an aerial triangulation result data set.

[0063] S4 divides the target aerial survey area into blocks according to a preset size to obtain multiple regional sub-blocks; searches for the effective range of dense matching for each regional sub-block in turn, and the multiple corrected drone images within the effective range of dense matching constitute the initial image cluster of the regional sub-block. In an embodiment of the present invention, the target aerial survey area is divided into blocks according to a preset size, and the row and column numbers, i.e., the spatial range, of each regional sub-block are recorded; each regional sub-block is traversed in turn, and the spatial distance of the regional sub-block is expanded to the surroundings according to the overlapping distance to obtain a spatial range with a certain degree of overlap, and the spatial range is used as the effective range of dense matching for each regional sub-block.

[0064] The projection calculation of the image center point is used to obtain all the corrected drone images within the spatial range of the regional sub-block, forming the initial image cluster of the regional sub-block; the projection calculation formula of the image center point is:

[0065]

[0066] Among them, X, Y, Z are the world coordinates corresponding to the center of image projection, K is the camera internal parameter, R, t are the camera external parameters, Z depth is the depth value of the projection center, u and v are the coordinates of the image projection center.

[0067] S5 searches for the associated images of each of the multiple corrected drone images in sequence based on the multiple corrected drone images in the initial image cluster of each regional sub-block, and combines the multiple corrected drone images in the initial image cluster of each regional sub-block and the corresponding associated images to form an associated image cluster of each regional sub-block.

[0068] Specifically, the specific method of sequentially searching for the associated images of each of the multiple corrected drone images in step S5 is:

[0069] S51 uses an image association algorithm to sequentially obtain multiple candidate images corresponding to each corrected drone image in the regional sub-block;

[0070] S52 scores the multiple candidate images corresponding to each corrected drone image in sequence according to the view angle, the overlapping area, and the scale factor to obtain a score for each candidate image;

[0071] S53 uses the candidate images whose scores exceed the preset correlation degree as the associated images corresponding to the corrected drone image.

[0072] In the embodiment of the present invention, the parameters such as the view angle, the overlapping area and the scale factor are used to score the multiple candidate images corresponding to each corrected drone image in turn, and the scores of each candidate image are obtained. The average value of the product of the overlapping area ratio, the camera angle and the camera scale is used as the score of the candidate image. According to the projection direction and the camera position, the candidate images are divided into four directions: front, back, left, right and right, and sorted according to the scores. The scoring formula is as follows:

[0073]

[0074] in, It represents the overlapping area of ​​the corrected drone image and its corresponding candidate image projected onto the fitting plane. represents the area of ​​the corrected drone image projected onto the fitting plane; N represents the number of sparse points; w scale (p) is the scale factor; It means that a single pixel in the two images represents the length ratio at the projection center point (p); represents the shared area; w angel (p) represents the angle factor.

[0075] Finally, the candidate images with scores exceeding 90 are used as the associated images of the corrected drone image. The multiple corrected drone images and the corresponding associated images in the initial image cluster of each regional sub-block are combined to form an associated image cluster of each regional sub-block.

[0076] S6 performs independent dense matching on each corrected UAV image in the associated image cluster of each regional sub-block according to the data information in the aerial triangulation results dataset, the position dataset, and the flight attitude dataset to obtain a depth map of each corrected UAV image; and fuses the depth maps of each corrected UAV image to obtain the final dense point cloud of the target aerial survey area.

[0077] S61 performs independent dense matching on each corrected UAV image in the associated image cluster of each regional sub-block in turn according to the data information in the aerial triangulation result data set, the position data set, and the flight attitude data set, to obtain a depth map of each corrected UAV image in each regional sub-block;

[0078] S62 fuses the depth maps of all the corrected drone images in all the sub-block areas to obtain an initial dense point cloud of the target aerial survey area;

[0079] S63 filters and screens each point cloud in the initial dense point cloud, and performs point cloud downsampling processing on the screened initial dense point cloud by using voxel filtering to obtain a final dense point cloud of the target aerial survey area.

[0080] S7 constructs a Delaunay tetrahedron set based on the final dense point cloud of the target aerial survey area, and uses the graph cut method to split the Delaunay tetrahedron set into multiple surface triangles to obtain a three-dimensional triangular mesh.

[0081] In an embodiment of the present invention, a directed graph is constructed with a set of Delaunay tetrahedrons as graph nodes and directed triangles between adjacent tetrahedrons as directed edges, and a graph cut method is used to segment multiple surface triangles to form a three-dimensional triangular mesh.

[0082] S8 automatically maps textures to the 3D triangular mesh based on multiple corrected UAV images in the corrected UAV image set to obtain a textured triangular mesh;

[0083] Texture mapping is done by mapping texture pixels in texture space to screen space. First, the texture object is defined, then the glBlindTexture function is used to select the texture object and load the corresponding texture when drawing the 3D real scene, and finally the glBlindTexture function is used to delete the texture object before outputting the 3D reconstruction result. Among them, glBindTexture is a function in the OpenGL core function library that allows the creation of a named texture bound to the target texture.

[0084] S9 generates DSM data, DOM data and DEM data in sequence based on textured triangular meshes.

[0085] Spatial data include digital elevation model (DEM), digital surface model (DSM), and digital orthophoto (DOM).

[0086] The Digital Surface Model (DSM) contains the ground elevation model of the height of surface buildings, bridges, trees, etc. DSM contains the contour and elevation information of the objects, so DEM can be obtained from DSM through segmentation and interpolation.

[0087] Digital orthophoto (DOM) is a digital map generated by processing satellite remote sensing image data. It has high-precision spatial information, rich texture details and intuitive visual effects, and is an important data source in the fields of digital earth, smart city, environmental monitoring, etc.

[0088] Digital elevation model (DEM) is a digital simulation of ground terrain (i.e., a digital expression of terrain surface morphology) achieved through limited terrain elevation data.

[0089] In this embodiment of the present invention, step S9 further includes:

[0090] S91 projects the textured triangular mesh into a plane coordinate system, and uses an interpolation algorithm in the plane coordinate system to perform grid interpolation on the triangular mesh to generate DSM data.

[0091] In the embodiment of the present invention, the interpolation algorithm in step S91 is any one of a moving least squares algorithm, an inverse distance weighted algorithm and a bilinear interpolation algorithm.

[0092] S92 calculates the range of each vertex in the triangular mesh in the plane coordinate system and generates DOM data.

[0093] In an embodiment of the present invention, a textured triangular mesh is projected onto an XY plane, and the XY ranges of vertices in the textured triangular mesh are calculated, thereby determining the length and width of the DOM data.

[0094] S93 identifies non-ground object features in DOM data and obtains non-ground masks; then removes non-ground point cloud data corresponding to the non-ground masks in DSM data to obtain ground point cloud data; and then uses hierarchical interpolation method to fill holes in the ground point cloud data to generate DEM data.

[0095] In the embodiment of the present invention, step S93 further includes:

[0096] S931 uses a semantic segmentation model to identify non-ground object features in the DOM data, and creates a non-ground mask based on the non-ground object features; the non-ground object features include buildings, trees, and bridges;

[0097] S932 searches for non-ground point cloud data corresponding to the non-ground mask in the DSM data, removes the non-ground point cloud data, and obtains ground point cloud data;

[0098] S933 uses hierarchical interpolation method to fill the gaps in ground point cloud data, thus generating complete ground point cloud data, i.e. DEM data.

[0099] In an embodiment of the present invention, the semantic segmentation model is any one of FCN, U-Net, SegNet and DeepLab.

[0100] The second object of the present invention is to provide an image data processing system 10 based on a drone, comprising:

[0101] The data acquisition module 110 is configured to pre-process the qualified aerial survey data set to obtain a pre-processed aerial survey data set; the pre-processed aerial survey data set includes a corrected drone image;

[0102] The aerial triangulation result generation module 120 is configured to solve the camera internal and external parameters and the sparse point cloud of the corrected UAV image according to the preprocessed aerial survey data set to obtain the aerial triangulation result data of the corrected UAV image;

[0103] The scene segmentation module 130 is configured to segment the distortion-corrected drone according to a preset size.

[0104] The image is processed into blocks to obtain multiple regional sub-blocks;

[0105] The module 140 for obtaining associated image clusters is configured to sequentially search for a dense matching effective range of each regional sub-block, wherein a plurality of corrected drone images within the dense matching effective range constitute an initial image cluster of the regional sub-block; and then sequentially search for associated images of a plurality of corrected drone images within the initial image cluster of each regional sub-block, wherein a plurality of corrected drone images within the initial image cluster of each regional sub-block and the associated images corresponding thereto constitute an associated image cluster of each regional sub-block;

[0106] The dense matching module 150 is configured to independently and densely match each corrected drone image in the target image cluster of each regional sub-block according to the data information in the aerial triangulation result data set, the position data set and the flight attitude data set, and obtain a depth map of each corrected drone image; and fuse the depth maps of each corrected drone image to obtain a final dense point cloud of the target aerial survey area;

[0107] The point cloud meshing module 160 is configured to construct a Delaunay,tetrahedron set based on the dense point cloud, and use a graph cut method to split the Delaunay,tetrahedron set into a plurality of surface triangles to obtain a three-dimensional triangular mesh;

[0108] The texture mapping module 170 is configured to automatically map textures to the three-dimensional triangular mesh based on the corrected drone image to obtain a textured triangular mesh;

[0109] The data product generation module 180 is configured to sequentially generate DSM data, DOM data and DEM data based on the textured triangular mesh.

[0110] The third object of the present invention is to provide an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0111] A third object of the present invention is to provide a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0112] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for processing image data based on drone, characterized in that: The method comprises: S1 designs a route for a target aerial survey area to obtain a target route, and obtains a qualified aerial survey data set based on the target route; the qualified aerial survey data set includes an original UAV image set, a position data set, and a flight attitude data set; S2 preprocesses the qualified aerial survey data set to obtain a preprocessed aerial survey data set; the preprocessed aerial survey data set includes a corrected drone image set; S3 solves the camera internal and external parameters and sparse point cloud of each corrected drone image in the corrected drone image set according to the pre-processed aerial survey data set, obtains the aerial triangulation result data of each corrected drone image, and forms an aerial triangulation result data set; S4 divides the target aerial survey area into blocks according to a preset size to obtain a plurality of regional sub-blocks; sequentially searches for a dense matching effective range for each regional sub-block, wherein a plurality of corrected drone images within the dense matching effective range constitute an initial image cluster of the regional sub-block; S5 searches for associated images of each of the multiple corrected drone images in sequence based on the multiple corrected drone images in the initial image cluster of each regional sub-block, and forms an associated image cluster of each regional sub-block with the multiple corrected drone images in the initial image cluster of each regional sub-block and the associated images corresponding thereto; S6 performs independent dense matching on each corrected UAV image in the associated image cluster of each regional sub-block according to the data information in the aerial triangulation result data set, the position data set, and the flight attitude data set, and obtains a depth map of each corrected UAV image; and fuses the depth maps of each corrected UAV image to obtain a final dense point cloud of the target aerial survey area; S7 constructs a Delaunay tetrahedron set based on the final dense point cloud of the target aerial survey area, and uses the graph cut method to split the Delaunay tetrahedron set into multiple surface triangles to obtain a three-dimensional triangular mesh; S8 automatically maps textures to the 3D triangular mesh based on multiple corrected UAV images in the corrected UAV image set to obtain a textured triangular mesh; S9 generates DSM data, DOM data and DEM data in sequence based on textured triangular meshes.

2. The method according to claim 1, characterized in that Step S9 further comprises: S91 projects the textured triangular mesh into a plane coordinate system, and uses an interpolation algorithm in the plane coordinate system to perform grid interpolation on the triangular mesh to generate DSM data; S92 calculates the range of each vertex in the triangular mesh in the plane coordinate system and generates DOM data; S93 identifies non-ground object features in DOM data and obtains non-ground masks; then removes non-ground point cloud data corresponding to the non-ground masks in DSM data to obtain ground point cloud data; and then uses hierarchical interpolation method to fill holes in the ground point cloud data to generate DEM data.

3. The method according to claim 2, characterized in that Step S93 further includes: S931 uses a semantic segmentation model to identify non-ground object features in the DOM data, and creates a non-ground mask based on the non-ground object features; the non-ground object features include buildings, trees, and bridges; S932 searches for non-ground point cloud data corresponding to the non-ground mask in the DSM data, removes the non-ground point cloud data, and obtains ground point cloud data; S933 uses hierarchical interpolation method to fill the gaps in ground point cloud data, thus generating complete ground point cloud data, i.e. DEM data.

4. The method according to claim 2, characterized in that: The interpolation algorithm in step S91 is any one of a moving least squares algorithm, an inverse distance weighted algorithm and a bilinear interpolation algorithm.

5. The method according to claim 1, characterized in that The specific method of sequentially searching for the associated images of each of the multiple corrected drone images in step S5 is: S51 uses an image association algorithm to sequentially obtain multiple candidate images corresponding to each corrected drone image in the regional sub-block; S52 scores the multiple candidate images corresponding to each corrected drone image in sequence according to the view angle, the overlapping area, and the scale factor to obtain a score for each candidate image; S53 uses the candidate images whose scores exceed the preset correlation degree as the associated images corresponding to the corrected drone image.

6. The method according to claim 1, characterized in that Step S6 further comprises: S61 performs independent dense matching on each corrected UAV image in the associated image cluster of each regional sub-block in turn according to the data information in the aerial triangulation result data set, the position data set, and the flight attitude data set, to obtain a depth map of each corrected UAV image in each regional sub-block; S62 fuses the depth maps of all corrected drone images in all sub-block areas to obtain an initial dense point cloud of the target aerial survey area; S63 filters and screens each point cloud in the initial dense point cloud, and performs point cloud downsampling processing on the screened initial dense point cloud by using voxel filtering to obtain a final dense point cloud of the target aerial survey area.

7. The method according to claim 1, characterized in that The preprocessing in step S2 includes light and color uniformity and distortion correction processing.

8. An image data processing system based on drone, characterized in that: include: The data acquisition module is configured to pre-process the qualified aerial survey data set to obtain the pre-processed aerial survey data set; The preprocessed aerial survey dataset includes rectifying the drone imagery; The aerial triangulation result generation module is configured to solve the camera internal and external parameters and sparse point cloud of the corrected UAV image according to the pre-processed aerial survey data set, and obtain the aerial triangulation result data of the corrected UAV image; The scene segmentation module is configured to segment the distortion-corrected drone according to a preset size. The image is processed into blocks to obtain multiple regional sub-blocks; The module for obtaining associated image clusters is configured to sequentially search for a dense matching effective range of each regional sub-block, wherein a plurality of corrected drone images within the dense matching effective range constitute an initial image cluster of the regional sub-block; and then sequentially search for associated images of a plurality of corrected drone images within the initial image cluster of each regional sub-block, wherein a plurality of corrected drone images within the initial image cluster of each regional sub-block and the associated images corresponding thereto constitute an associated image cluster of each regional sub-block; The dense matching module is configured to independently and densely match each corrected UAV image in the target image cluster of each regional sub-block in turn according to the data information in the aerial triangulation result data set, the position data set and the flight attitude data set to obtain the depth map of each corrected UAV image; and fuse the depth maps of each corrected UAV image to obtain the final dense point cloud of the target aerial survey area; The point cloud meshing module is configured to construct a Delaunay ,tetrahedron set based on the dense point cloud, and use the graph cut method to ,split the Delaunay ,tetrahedron set into multiple surface triangles to obtain a 3D triangular mesh; A texture mapping module is configured to automatically map textures to the three-dimensional triangular mesh based on the corrected drone image to obtain a textured triangular mesh; The data product generation module is configured to generate DSM data, DOM data and DEM data in sequence based on the textured triangular mesh.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method according to claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the method according to claims 1 to 7.

Images