Water conservancy project unmanned aerial vehicle oblique photography and three-dimensional modeling method

Abstract

The invention discloses a hydraulic engineering unmanned aerial vehicle oblique photography and three-dimensional modeling method, and the method comprises the steps: enabling an unmanned aerial vehicle to fly at a first height through setting the first height, and obtaining a first image set in a selected aerial photography region; and at the first height, the unmanned aerial vehicle laterally flies on the outer side of the obstacle in the first image set, the lateral height is set to be a second height, and the optimal aerial photography path of the unmanned aerial vehicle is selected from the second height. The unmanned aerial vehicle collects multilateral radiation landforms in the optimal aerial photography path for multiple times in the optimal aerial photography path, obtains a second image set, and builds a landform model by adopting a neural network according to the second image set, thereby solving the problem that when the existing unmanned aerial vehicle oblique photography technology is used for obtaining the landform structure information of the region, the landform structure information cannot be obtained due to the existence of numerous irregular, complex and changeable elements. Therefore, the problems of difficulty in accurate matching and difficulty in feature recognition of a large amount of acquired image data are solved.

Description

Technical Field

[0001] The invention relates to the technical field of unmanned aerial vehicle photography and modeling, and in particular to a method for unmanned aerial vehicle oblique photography and three-dimensional modeling of a water conservancy project. Background Art

[0002] Drone oblique photography is a surveying and mapping method. It uses a multi-lens camera mounted on a drone to collect image data from vertical and multiple oblique angles simultaneously. Compared with traditional photography, it can obtain more comprehensive and detailed texture information of objects, and the constructed three-dimensional model has high precision and good restoration, which can truly show the terrain and building appearance. The core of this technology lies in the principle of multi-view photogrammetry, that is, obtaining sufficient redundant data through a certain degree of overlap and multi-angle coverage to ensure the accuracy and integrity of three-dimensional reconstruction.

[0003] The landforms and texture structures in the areas related to water conservancy projects are extremely complex. For example, there are often irregular bank slopes around reservoirs, river channels with varying widths and complex undulations on the riverbed. When using drone oblique photography technology to obtain geomorphic structure information in such areas, due to the presence of many irregular, complex and changeable elements, the collected image data has a lot of problems that are difficult to accurately match and difficult to identify features, which greatly increases the difficulty of subsequent three-dimensional modeling, and is prone to model distortion, making it difficult to meet the demand for high-quality geomorphic models for accurate planning, construction and maintenance of water conservancy projects. Summary of the invention

[0004] The purpose of the present invention is to provide a method for oblique photography and three-dimensional modeling of a water conservancy project by unmanned aerial vehicle to solve the problems described in the background technology.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A method for oblique photography and three-dimensional modeling of a water conservancy project by using an unmanned aerial vehicle, comprising:

[0007] Step S1, setting a first height, the drone selects an aerial photography area according to the first height, and acquires a first image set;

[0008] Step S2, the UAV performs sideways flight according to the first height combined with obstacles in the first image set, sets a second height according to the sideways height, and selects the best aerial photography path of the UAV at the second height;

[0009] Step S3: the drone collects multiple images of the multi-radiation landforms in the optimal aerial photography path in the optimal aerial photography path, and obtains a second image set;

[0010] Step S4: constructing a landform model using a neural network according to the second image set.

[0011] A further technical solution is that step S1 specifically includes:

[0012] Step 101: According to an on-site survey of the measurement address, the drone is raised to a height higher than the highest obstacle at the measurement address, and the height higher than the highest obstacle is marked as the first height;

[0013] Step 102: The drone acquires the first image set above the measurement address based on the first altitude, wherein the first image set includes obstacle images and overhead images of the measurement address.

[0014] A further technical solution is that step 102 includes:

[0015] The distances between the multiple obstacles in the obstacle map are determined by using a distance measurement method to determine the two obstacles that are closest to each other.

[0016] A further technical solution is that step S2 specifically includes:

[0017] Step 201: in the first image set acquired in the aerial photography area, the heights of a plurality of obstacles in the first image set are set as a set H, wherein the set H includes {h, h1, h2, h3...hn};

[0018] Step 202: Flying at the side of the aerial photography area using the first height and the set H, the height of which is lower than the heights of all obstacles in the set H, is calibrated as a second height;

[0019] Step 203: Determine a target sub-area in the aerial photography area using an artificial bee colony algorithm according to the first height, the second height and the obstacle.

[0020] A further technical solution is that step 203 specifically includes:

[0021] Initializing a bee colony in the aerial photography area;

[0022] Among the plurality of paths, a first swarm of bees is used to calculate a first evaluation value of each path, and a first weight is assigned to the first evaluation value;

[0023] Calculating a second evaluation value using a second swarm of bees based on the first evaluation value obtained by the first swarm of bees, and assigning a second weight to the second evaluation value;

[0024] The optimum flight altitude is calculated using the first weight and the second weight.

[0025] A further technical solution is to further include:

[0026] The first weight and the second weight are calculated through iterative calculation to obtain an optimum flight altitude, and a target area in the aerial photography area is determined according to the optimum flight altitude.

[0027] A further technical solution is to further include:

[0028] After the first and second swarms of bees identify the path, the third swarm of bees continues to search for an alternative path in the aerial photography area;

[0029] An optimum flight altitude is calculated based on the alternative path and the first weight.

[0030] A further technical solution is that step S3 specifically includes:

[0031] Step 301: The drone acquires a plurality of topographic images of the optimal aerial photography path from above and from the side in the optimal aerial photography path;

[0032] Step 302: Generate the second image set according to the landform picture;

[0033] Step 303: Determine the modeling contour in the second image set.

[0034] A further technical solution is to involve a Transformer model, wherein step S4 specifically includes:

[0035] Step 401: receiving the second image set input coordinate element;

[0036] Step 402: Capturing the spatial relationship between different coordinate elements according to the coordinate elements;

[0037] Step 403: Outputting landform-related features according to the plurality of spatial relationships.

[0038] A further technical solution involves a GAN model, specifically including:

[0039] Step 404: receiving the landform-related features;

[0040] Step 405: training according to the landform-related features received multiple times;

[0041] Step 406: judging the accuracy of the training results of the landform-related features after training;

[0042] Step 407: Output the judgment training result, and model the landform-related features through a three-dimensional reconstruction model.

[0043] The beneficial effects of the present invention are:

[0044] 1. The present invention sets a first altitude, flies the drone at the first altitude, and obtains a first image set in the selected aerial photography area. At the first altitude, the drone flies sideways outside the obstacle in the first image set, sets the side altitude to a second altitude, and selects the best aerial photography path for the drone at the second altitude. In the best aerial photography path, the drone collects multilateral radiating landforms in the best aerial photography path for multiple times, and obtains a second image set. Based on the second image set, a landform model is built using a neural network, thereby solving the problem that the existing drone oblique photography technology is difficult to accurately match and difficult to identify features in the collected image data due to the presence of many irregular, complex and changeable elements when obtaining landform structure information in such areas.

[0045] 2. The present invention utilizes an artificial bee colony algorithm to select the best aerial photography path from multiple aerial photography paths, so that the UAV can quickly obtain image data of irregular, complex and changeable landform features, facilitate subsequent modeling with high precision, and assist in the maintenance of water conservancy projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0047] Figure 2 This is a schematic diagram of the method flow of step S1 of the present invention;

[0048] Figure 3 This is a schematic diagram of the method flow of step S2 of the present invention;

[0049] Figure 4 This is a schematic diagram of the method flow of step S3 of the present invention;

[0050] Figure 5 Schematic diagram of the method flow of step S4 of the present invention. DETAILED DESCRIPTION

[0051] In order to better understand the technical content of the present invention, specific embodiments are provided below, and the present invention is further described in conjunction with the accompanying drawings.

[0052] See also Figures 1 to 5A method for oblique photography and three-dimensional modeling of a water conservancy project by using an unmanned aerial vehicle, wherein a first height is set, the unmanned aerial vehicle is flown at the first height, and a first image set is obtained in a selected aerial photography area. At the first height, the unmanned aerial vehicle flies sideways outside the obstacle in the first image set, and the side height is set to a second height, and the best aerial photography path of the unmanned aerial vehicle is selected at the second height. In the best aerial photography path, the unmanned aerial vehicle collects the multi-sided radiating landforms in the best aerial photography path for multiple times, and obtains the second image set. According to the second image set, a landform model is built using a neural network, thereby solving the problem that the existing unmanned aerial vehicle oblique photography technology is difficult to accurately match and difficult to identify features in the collected image data due to the existence of many irregular, complex and changeable elements when obtaining landform structure information in such areas, thereby reducing the difficulty of subsequent three-dimensional modeling, improving the real state of the model, and meeting the demand for high-quality landform models for accurate planning, construction and maintenance of water conservancy projects.

[0053] The method of the present invention comprises: step S1, setting a first altitude, the unmanned aerial vehicle selecting an aerial photography area according to the first altitude, and acquiring a first image set.

[0054] The first image set refers to obtaining the overall picture of the target area that needs construction in the water conservancy project within the range of manual visual inspection. The overall picture area makes it easy to understand the approximate appearance of the construction area and provides a good feature soil sampling basis for the subsequent near-ground landforms.

[0055] In this embodiment, step S1 specifically includes:

[0056] Step 101: According to the on-site survey of the measurement address, the drone is raised to a height higher than the highest obstacle at the measurement address, and the height higher than the highest obstacle is marked as a first height;

[0057] Step 102: The drone acquires a first image set above the measurement address at a first altitude, wherein the first image set includes obstacle images and bird's-eye view images of the measurement address.

[0058] Select the aerial photography area as the measurement address. Before the field survey, lift the drone and select the highest obstacle in the measurement address, such as a concrete building, a tree or a signal tower. Then set the obstacle at the height of the drone higher than the highest obstacle as the first height. When the drone flies at the first height, avoid hitting the highest obstacle, which may cause the drone to be damaged. When flying at the first height, it is easy to obtain the comprehensive topography of the measurement address, and obtain the distance between multiple obstacles at the measurement address, as well as the distance between the obstacle and the location of the irregular, complex and changeable topography that needs to be obtained.

[0059] Preferably, step 102 includes:

[0060] The distance between multiple obstacles in the obstacle map is determined by the distance measurement method to determine the two obstacles closest to each other.

[0061] When using the distance measurement method, the ratio between the image of the measurement address and the actual distance is calculated by the height of the drone. The actual distance between multiple obstacles and the actual distance between obstacles and the location of irregular, complex and changeable terrain that needs to be obtained are calculated by the distance measurement method.

[0062] The method of the present invention comprises step S2, the UAV performs sideways flight according to the first height combined with obstacles in the first image set, sets the second height according to the side height, and selects the best aerial photography path of the UAV at the second height.

[0063] Side flight means that the UAV flies around the measurement address outside multiple obstacles in the measurement address to measure the lateral distance between obstacles, so as to facilitate the subsequent acquisition of the best aerial photography path in combination with the vertical distance obtained between the obstacles, and to facilitate the UAV to quickly and accurately obtain image data of irregular, complex and changeable terrain according to the best aerial photography path.

[0064] In this embodiment, step S2 specifically includes:

[0065] Step 201: In a first image set acquired in an aerial photography area, the heights of multiple obstacles in the first image set are set as a set H, where the set H includes {h, h1, h2, h3...h n};

[0066] Step 202: Flying sideways in the aerial photography area through the first altitude and the set H, the altitude being lower than the heights of all obstacles in the set H, is calibrated as the second altitude;

[0067] Step 203: According to the first height, the second height and obstacles, an artificial bee colony algorithm is used to determine a target sub-area in the aerial photography area.

[0068] The set H includes the heights of concrete buildings, trees or signal towers. Since the heights of multiple obstacles are different, multiple height elements are swung between multiple obstacles. Taking h, h1, h2, and h3 as examples, it is convenient to clearly know the heights of each obstacle. When the drone flies at the first height, the first image set collected is a bird's-eye view, and the picture obtained in the measurement address is a bird's-eye view orthographic projection. When the drone flies at the second height, it flies around the measurement address to obtain a side view of the measurement address, and preliminarily constructs a three-dimensional space path, which is convenient for the drone to select the optimal aerial photography path in the space path. According to the paths obtained at the first height and the second height and the obstacle height, the artificial bee colony algorithm is used for global search to improve the screening of the optimal path, thereby improving the drone's rapid travel through multiple obstacles and finding the target sub-area in the aerial photography area, which is convenient for subsequent landform modeling.

[0069] Preferably, step 203 specifically includes:

[0070] Initialize the swarm in the aerial photography area;

[0071] Among the multiple paths, a first swarm of bees is used to calculate a first evaluation value of each path, and a first weight is assigned to the first evaluation value;

[0072] Calculate a second evaluation value using the second swarm of bees based on the first evaluation value obtained by the first swarm of bees, and assign a second weight to the second evaluation value;

[0073] The optimum flight altitude is calculated using the first weight and the second weight.

[0074] In the path formed by multiple obstacles, irregular, complex and changeable landforms are usually between multiple obstacles. Therefore, the first swarm of bees screens out the path close to the irregular, complex and changeable landforms through the multiple paths obtained by the first height, the second height and the obstacles. For example, in the irregular, complex and changeable landforms near the concrete building, the first swarm of bees finds the shortest path and the space suitable for the oblique shooting of the drone, which is set as the first evaluation value in the process of searching the path, and the search result obtained is set as the first weight; then, the second swarm of bees searches for another shortest path and the space suitable for the oblique shooting of the drone, which is set as the second evaluation value in the process of searching for another path, and the search result of the other path obtained is set as the second weight; by comparing the first weight with the second weight, if the first weight is better than the second weight, the search result of the first weight is selected as the best aerial photography path, and the drone passes through multiple obstacles through the path and flies to the top of the irregular, complex and changeable landforms to obtain oblique shooting images, which is convenient for subsequent modeling.

[0075] Preferably, it also includes:

[0076] The first weight and the second weight are calculated through iterative calculation to obtain the optimum flight altitude, and the target area in the aerial photography area is determined according to the optimum flight altitude.

[0077] In the process of searching the path by the first swarm and the second swarm, the first weight and the second weight are calculated through multiple iterations, and the optimal weight is selected. If the first weight is the optimal path after calculating the first weight and the second weight, then on this basis, the optimal flight altitude of the drone is set, and the drone flies through the target sub-area at the optimal flight altitude, wherein the target sub-area is a location of irregular, complex and changeable landforms that is convenient for tilted shooting.

[0078] Preferably, it also includes:

[0079] After the first and second swarms of bees have identified the path, the third swarm of bees continues to search for alternative paths in the aerial photography area;

[0080] The optimum flight altitude is calculated based on the alternative paths and the first weight.

[0081] While the first and second swarms are selecting the best aerial photography path, the third swarm will search for another best aerial photography path based on the best aerial photography path, combined with humidity, luminosity and temperature. Therefore, the third swarm uses the iterative experience of the first and second swarms to shorten the search time, thereby improving the third swarm's search for the best aerial photography path, greatly shortening the time it takes for the drone to tilt and shoot, while reducing the drone's energy consumption.

[0082] In addition, when searching for the best aerial photography path by combining humidity, luminosity and temperature, the drone's camera can avoid the appropriate environment when passing through obstacles, preventing the camera from being fogged and unable to obtain clear terrain images.

[0083] The method of the present invention comprises: step S3, the unmanned aerial vehicle collects multiple images of the multilateral radiation landforms in the best aerial photography path in the best aerial photography path, and obtains a second image set.

[0084] The second image set refers to the pictures taken by the drone in irregular, complex and changeable terrain, where the irregular, complex and changeable terrain includes multiple target sub-areas in the aerial photography area.

[0085] Multilateral radiation landform refers to the extension of one irregular, complex and changeable landform to other irregular, complex and changeable landforms in multiple target sub-areas.

[0086] In this embodiment, step S3 specifically includes:

[0087] Step 301: The drone obtains multiple topographic images of the optimal aerial photography path from above and from the side in the optimal aerial photography path;

[0088] Step 302: Generate a second image set according to the landform picture;

[0089] Step 303: Determine a modeling contour in the second image set.

[0090] After the drone passes through the optimal aerial photography path above the irregular, complex and changeable landform, the drone forms an oblique camera from above and from the side to obtain an oblique image of the irregular, complex and changeable landform. After obtaining the image, all images are formed into a second image set for easy summary and management. In obtaining each irregular, complex and changeable landform image, it is necessary to collect an image with a clear outline, so that in the subsequent modeling, the use of images with clear outlines can improve the modeling efficiency. At the same time, the second image set is input into the neural network model to improve the training speed of the model.

[0091] The method of the present invention comprises: step S4, using a neural network to build a landform model according to the second image set.

[0092] The neural network model is used to output the trained second image set to improve the efficiency of modeling irregular, complex and changeable landforms.

[0093] In this embodiment, a Transformer model is involved, and step S4 specifically includes:

[0094] Step 401: receiving input coordinate elements of a second image set;

[0095] Step 402: Capturing the spatial relationship between different coordinate elements according to the coordinate elements;

[0096] Step 403: Outputting landform-related features according to multiple spatial relationships.

[0097] The Transformer model is involved in the neural network model. When the second image set is input into the Transformer model, since the drone has coordinate elements in the image collection process, multiple coordinate elements of the irregular, complex and changeable landforms in the second image set are expressed by (x1, y1, z1). For example, the coordinates of the first point are (1, 1, 1), another point is (1, 1, 2), and another point is (1, 2, 1). Through multiple coordinate elements, the 3D spatial relationship formed by different coordinate elements is captured, and the landform-related features of the preliminary outline of the 3D spatial relationship are output to facilitate subsequent rendering and modeling.

[0098] In this embodiment, the GAN model is involved, specifically including:

[0099] Step 404: receiving landform-related features;

[0100] Step 405: training according to the geomorphic related features received multiple times;

[0101] Step 406: judging the accuracy of the training results of the trained landform-related features;

[0102] Step 407: Output the judgment training results and model the landform-related features through a three-dimensional reconstruction model.

[0103] The neural network model still involves the GAN model. The generator is used to receive the landform-related features output by the Transformer model. After a large number of iterative training, the discriminator is input. After the landform-related features are identified with the standard landform-related features by the discriminator, the best landform-related features are output through multiple iterative optimizations, and then the landform-related features are rendered and modeled through the three-dimensional reconstruction model. After the modeling is completed, the irregular, complex and changeable landforms are fully, truly and clearly displayed in the water conservancy project map, improving the real situation of the model and meeting the demand for high-quality landform models for accurate planning, construction and maintenance of water conservancy projects.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for oblique photography and three-dimensional modeling of a water conservancy project by using an unmanned aerial vehicle, characterized in that: include: Step S1, setting a first height, the drone selects an aerial photography area according to the first height, and acquires a first image set; Step S2, the UAV performs sideways flight according to the first height combined with obstacles in the first image set, sets a second height according to the sideways height, and selects the best aerial photography path of the UAV at the second height; Step S3, the drone collects multiple radiation landforms in the optimal aerial photography path in the optimal aerial photography path, and obtains a second image set; Step S4: constructing a landform model using a neural network according to the second image set.

2. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 1, characterized in that: The step S1 specifically includes: Step 101: According to an on-site survey of the measurement address, the drone is raised to a height higher than the highest obstacle at the measurement address, and the height higher than the highest obstacle is marked as the first height; Step 102: The drone acquires the first image set above the measurement address based on the first altitude, wherein the first image set includes obstacle images and overhead images of the measurement address.

3. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 2, characterized in that: The step 102 includes: The distances between the multiple obstacles in the obstacle map are determined by using a distance measurement method to determine the two obstacles that are closest to each other.

4. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 1, characterized in that: The step S2 specifically includes: Step 201: In the first image set acquired in the aerial photography area, the heights of multiple obstacles in the first image set are set as a set H, wherein the set H includes {h, h1, h2, h3...h n }; Step 202: Flying at the side of the aerial photography area using the first height and the set H, the height of which is lower than the heights of all obstacles in the set H, is calibrated as a second height; Step 203: Determine a target sub-area in the aerial photography area using an artificial bee colony algorithm according to the first height, the second height and the obstacle.

5. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 4, characterized in that: The step 203 specifically includes: Initializing a bee colony in the aerial photography area; Among the plurality of paths, a first swarm of bees is used to calculate a first evaluation value of each path, and a first weight is assigned to the first evaluation value; Calculating a second evaluation value using a second swarm of bees based on the first evaluation value obtained by the first swarm of bees, and assigning a second weight to the second evaluation value; The optimum flight altitude is calculated using the first weight and the second weight.

6. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 5, characterized in that: Also includes: The first weight and the second weight are calculated through iterative calculation to obtain an optimum flight altitude, and a target area in the aerial photography area is determined according to the optimum flight altitude.

7. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 5, characterized in that: Also includes: After the first and second swarms of bees identify the path, the third swarm of bees continues to search for an alternative path in the aerial photography area; An optimum flight altitude is calculated based on the alternative path and the first weight.

8. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 1, characterized in that: The step S3 specifically includes: Step 301: The drone acquires a plurality of topographic images of the optimal aerial photography path from above and from the side in the optimal aerial photography path; Step 302: Generate the second image set according to the landform picture; Step 303: Determine the modeling contour in the second image set.

9. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 1, characterized in that: Involving the Transformer model, step S4 specifically includes: Step 401: receiving the second image set input coordinate element; Step 402: Capturing the spatial relationship between different coordinate elements according to the coordinate elements; Step 403: Outputting landform-related features according to the plurality of spatial relationships.

10. The method for oblique photography and three-dimensional modeling of a hydraulic engineering project using an unmanned aerial vehicle according to claim 9, characterized in that: Involving GAN models, specifically including: Step 404: receiving the landform-related features; Step 405: training according to the landform-related features received multiple times; Step 406: judging the accuracy of the training results of the landform-related features after training; Step 407: Output the judgment training result, and model the landform-related features through a three-dimensional reconstruction model.

Images