Complex terrain surveying and mapping method and system, computer equipment and medium thereof

Abstract

The invention relates to a complex terrain surveying and mapping method, a complex terrain surveying and mapping system, computer equipment and a computer readable storage medium. The method comprises the following steps: controlling an unmanned aerial vehicle to execute an aerial photography task to obtain a plurality of aerial photography images; a pre-trained target detection algorithm model deployed on the unmanned aerial vehicle is utilized to identify complex terrain content from the aerial image, and a complex terrain area corresponding to the complex terrain content is marked; obtaining an aerial image, importing the aerial image into terminal equipment, and carrying out conventional modeling on the aerial image by utilizing three-dimensional modeling software to obtain a main body three-dimensional model; acquiring a complex terrain area correspondingly marked by each aerial image, and performing refined modeling on complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model; generating an overall three-dimensional model diagram of the target area according to the main body three-dimensional model and the complex terrain three-dimensional model; according to the technical scheme, the surveying and mapping complexity specialization degree requirements including complex terrains are reduced, and popularization and application of the unmanned aerial vehicle surveying and mapping technology are promoted.

Description

Technical Field

[0001] The present application relates to the field of UAV surveying and mapping technology, and in particular to a complex terrain surveying and mapping method, a complex terrain surveying and mapping system, a computer device, and a computer-readable storage medium. Background Art

[0002] With the development of drone technology, surveying and mapping technology based on drone aerial photography has also developed rapidly. Digital aerial photography technology based on drone platforms has shown its unique advantages and has broad prospects in basic surveying and mapping, land resource investigation and monitoring, dynamic monitoring of land use, digital city construction, and emergency disaster relief surveying and mapping data acquisition.

[0003] In drone aerial surveys, local complex terrain is often encountered. The data processing volume of complex terrain is several times that of ordinary plains. In order to ensure that the data error is within a certain range, although powerful and advanced equipment is used to complete the surveying and mapping needs of complex terrain in most scenarios; however, these technical solutions have complex data processing for aerial images, large data volumes, high processing costs, and high professional requirements, which affects the promotion and use of drone surveying and mapping technology. Summary of the Invention

[0004] In order to solve one of the above-mentioned defects, the present application provides a complex terrain surveying and mapping method, a complex terrain surveying and mapping system, a computer device and a computer-readable storage medium, so as to realize the surveying and mapping of complex terrain at a relatively low cost.

[0005] A method for surveying and mapping complex terrain, comprising:

[0006] Control the UAV to perform aerial photography missions and obtain multiple aerial images;

[0007] Using a pre-trained target detection algorithm model deployed on a drone to identify complex terrain content from aerial images, and marking the complex terrain areas corresponding to the complex terrain content;

[0008] Acquire the aerial image and import it into a terminal device, and perform conventional modeling on the aerial image using 3D modeling software to obtain a main body 3D model; wherein the main body 3D model includes the main body except for the complex terrain area;

[0009] Obtaining the complex terrain areas marked in the corresponding aerial images, and performing detailed modeling on the complex terrain content of the complex terrain areas to obtain a three-dimensional model of the complex terrain;

[0010] An overall three-dimensional model map of the target area is generated based on the main three-dimensional model and the complex terrain three-dimensional model.

[0011] In one embodiment, conventional modeling is performed on the aerial image using 3D modeling software to obtain a main body 3D model; wherein the main body 3D model includes the main body except the complex terrain area, including:

[0012] Importing the aerial image into a 3D modeling software for conventional modeling to obtain a basic 3D model;

[0013] Generate an occlusion layer according to the marked complex terrain area; wherein the complex terrain area is the occlusion area, and the remaining area is the transparent area;

[0014] The occlusion layer is superimposed on the basic three-dimensional model for rendering to obtain a main three-dimensional model.

[0015] In one embodiment, performing detailed modeling of the complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model includes:

[0016] intercepting a local image from the aerial image according to the marked complex terrain area;

[0017] Calculating position parameters of each of the local images in the aerial image;

[0018] Dividing the local image into a plurality of local image sets corresponding to complex terrain areas according to the location parameters;

[0019] A three-dimensional model is generated based on the local image set to obtain a three-dimensional model of the complex terrain corresponding to each complex terrain area.

[0020] In one embodiment, generating an overall three-dimensional model map of the target area based on the main three-dimensional model and the complex terrain three-dimensional model includes:

[0021] Obtain a first extreme point on the edge line of the complex terrain three-dimensional model; wherein the first extreme point includes the uppermost point, the lowermost point, the leftmost point, and the rightmost point in four directions;

[0022] Obtaining second extreme points of the main body three-dimensional model along the edge line of the occluded area and the transparent area; wherein the second extreme points include the uppermost point, the lowermost point, the leftmost point, and the rightmost point in four directions;

[0023] The main body three-dimensional model and the complex terrain three-dimensional model are aligned and merged according to the first extreme point and the second extreme point to obtain an overall three-dimensional model map of the target area.

[0024] In one embodiment, before controlling the drone to perform an aerial photography mission to obtain a plurality of aerial images, the method further includes:

[0025] Use a drone to capture several real-life images of the target area at a preset location;

[0026] Inputting the real scene image into a pre-trained deep learning model to obtain an initial topographic map containing the target area;

[0027] Obtain device parameters of the drone, obtain aerial photography parameters of the drone based on the device parameters and the initial terrain map, and generate an aerial photography task based on the aerial photography parameters and configure it in the drone.

[0028] In one embodiment, after inputting the real scene image into a pre-trained deep learning model to obtain an initial topographic map containing the target area, the method further includes:

[0029] In response to an editing operation performed by a user, the target area is determined on the initial topographic map and topographic information of the initial topographic map is edited.

[0030] In one embodiment, determining the target area on the initial topographic map and editing the topographic information of the initial topographic map include:

[0031] Selecting an aerial photography range of a target area on the initial topographic map according to a range selection operation by a user;

[0032] Dividing the initial topographic map within the aerial photography range into blocks and setting each block to an editable state;

[0033] The terrain information of the blocks is edited according to the user's editing operation.

[0034] A complex terrain surveying and mapping system, comprising: a drone and a terminal device;

[0035] The drone is used to perform aerial photography tasks to obtain multiple aerial images, identify complex terrain content from the aerial images using a deployed pre-trained target detection algorithm model, and mark the complex terrain areas corresponding to the complex terrain content;

[0036] The terminal device is used to import the aerial image, and use the three-dimensional modeling software to perform conventional modeling on the aerial image to obtain a main three-dimensional model; wherein, the main three-dimensional model includes the main part except the complex terrain area; obtain the complex terrain area correspondingly marked in each aerial image, and perform detailed modeling on the complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model; generate an overall three-dimensional model map of the target area based on the main three-dimensional model and the complex terrain three-dimensional model.

[0037] A computer device comprising:

[0038] one or more processors;

[0039] Memory;

[0040] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the steps of the above-mentioned method for surveying and mapping complex terrain.

[0041] A computer-readable storage medium stores at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, at least one program, code set or instruction set is loaded by the processor and executes the steps of the above-mentioned complex terrain surveying and mapping method.

[0042] The technical solution of the above embodiment is to control the drone to perform aerial photography tasks and obtain multiple aerial images; use the pre-trained target detection algorithm model deployed on the drone to identify complex terrain content from the aerial images, and mark the complex terrain areas corresponding to the complex terrain content; then obtain the aerial images and import them into the terminal device, use the 3D modeling software on the terminal device to perform 3D modeling on the aerial images to obtain a main 3D model; based on the marked complex terrain areas, the complex terrain content of the complex terrain areas is refined to obtain a complex terrain 3D model; finally, based on the main 3D model and the complex terrain 3D model, an overall 3D model map of the target area is generated; this technical solution, when using the drone for aerial photography, first pre-marks the complex terrain area, and during 3D modeling, combines the main 3D model and the refined modeling of the complex terrain area to obtain an overall 3D model map of the target area, thereby reducing the complexity and professional requirements of surveying and mapping involving complex terrain, so that resource consumption can be reduced during modeling and rendering, thereby adapting to low-performance equipment and promoting the promotion and use of drone surveying and mapping technology.

[0043] Furthermore, by using real-life images and deep learning models to generate an initial topographic map, users do not need to perform specialized drone route planning and other configuration operations. Instead, they can edit terrain information on the initial topographic map to configure drone aerial photography parameters. This allows users to respond to surveying and mapping needs based on low-cost drone equipment, reduces the difficulty of parameter configuration for drone aerial photography and mapping technology, and improves the usability of drone surveying and mapping technology.

[0044] Additional aspects and advantages of the present application will be given in part in the following description, which will become apparent from the following description, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 is a flow chart of a method for surveying and mapping complex terrain according to an embodiment;

[0047] Figure 2 This is a modeling flow chart of a main body three-dimensional model of an example;

[0048] Figure 3 This is an example of a complex terrain 3D modeling flow chart;

[0049] Figure 4 This is an example of a complex terrain content annotation diagram;

[0050] Figure 5 This is an example of a flow chart for modeling the overall three-dimensional model diagram;

[0051] Figure 6 This is a schematic diagram of the initial topographic map of an example;

[0052] Figure 7 This is a schematic diagram of the structure of a complex terrain surveying and mapping system;

[0053] Figure 8 is a block diagram of an example computer device. DETAILED DESCRIPTION

[0054] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0055] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a," "an," "the," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of the stated features, integers, steps, and operations, but does not preclude the presence or addition of one or more other features, integers, steps, and operations.

[0056] The technical solution of the present application provides a surveying and mapping solution for complex terrain, which can realize the surveying and mapping of complex terrain using low-performance drone equipment; the initial terrain map can be generated using real-life images and deep learning models, and the user does not need to perform professional drone route planning and other configuration operations. Instead, the user edits the terrain information on the initial terrain map to realize the configuration of drone aerial photography parameters, responds to surveying and mapping needs based on low-cost drone equipment, reduces the difficulty of parameter configuration of drone aerial photography and mapping technology, and improves the usability of drone surveying and mapping technology; when using drone aerial photography, the complex terrain area is first pre-marked, and when three-dimensional modeling is performed, the main three-dimensional model and the refined modeling of the complex terrain area are combined to obtain the overall three-dimensional model of the target area, which reduces the data processing volume and the degree of specialization required for the complexity of surveying and mapping involving complex terrain, so that resource consumption can be reduced during modeling and rendering, thereby adapting to low-performance equipment, promoting the promotion and use of drone surveying and mapping technology, and making it more conducive to the application of drone surveying and mapping technology in various scenarios.

[0057] refer to Figure 1 As shown, Figure 1 The flowchart of a method for surveying and mapping complex terrain according to an embodiment mainly includes the following steps:

[0058] S10, controlling the UAV to perform an aerial photography mission to obtain multiple aerial images.

[0059] In this step, the UAV is controlled to perform an aerial photography mission to obtain multiple aerial images. The UAV can perform the aerial photography mission, fly along the flight path, and capture aerial images according to the set camera parameters.

[0060] S20, using a pre-trained target detection algorithm model deployed on the UAV to identify complex terrain content from the aerial image, and marking the complex terrain area corresponding to the complex terrain content.

[0061] In this step, a pre-trained target detection algorithm model deployed on the UAV is used to identify complex terrain content from the aerial image, and the complex terrain area corresponding to the complex terrain content is marked.

[0062] Exemplarily, the target detection algorithm model can be a YOLO-Fastest model. The corresponding YOLO-Fastest model can be pre-trained according to the terrain type, and the YOLO-Fastest model can be deployed in a drone to obtain aerial images taken by the camera in real time. The YOLO-Fastest model is used to detect each aerial image, identify complex terrain content contained in the aerial image, and then locally annotate it, usually using a minimum circumscribed rectangular box for annotation.

[0063] In the above solution, by deploying the target detection algorithm model on the drone, complex terrain content can be directly identified and quickly labeled from the aerial image, which can ensure the accuracy of labeling according to the complex terrain conditions of different needs. It also facilitates the subsequent 3D modeling, and can accurately identify complex terrain areas, thereby improving modeling accuracy.

[0064] S30, acquiring the aerial image and importing it into a terminal device, and performing conventional modeling on the aerial image using 3D modeling software to obtain a main body 3D model; wherein the main body 3D model includes the main body part except the complex terrain area.

[0065] In this step, all aerial images captured by the drone are extracted and then imported into a terminal device for processing. For example, the terminal device here can be a computer device, etc., and the 3D modeling software can be Pix4D, Blender, Maya, or 3dsMax.

[0066] Generally speaking, 3D modeling levels are divided into white model, target element outline modeling, conventional modeling, refined modeling, and highly refined modeling. Because each level differs significantly in detail, resource consumption, and application scenarios, the equipment cost, performance, and processing efficiency requirements of the surveying and mapping system are different. 3D conventional modeling is performed using 3D modeling software invoked by aerial imagery to obtain a main 3D model of the target area, excluding areas with complex terrain. This main 3D model is the basic model describing the target area and is processed using conventional modeling methods to reduce data processing complexity, processing costs, and the required level of specialization.

[0067] In one embodiment, reference Figure 2 As shown, Figure 2 This is an example flow chart for modeling a main body three-dimensional model. Step S30 of performing conventional modeling on the aerial image using three-dimensional modeling software to obtain the main body three-dimensional model may include:

[0068] S301: Import the aerial image into 3D modeling software to perform conventional modeling to obtain a basic 3D model. Specifically, the basic 3D model can be obtained by performing conventional modeling using 3D modeling rendering and production software.

[0069] S302 : generating an occlusion layer according to the marked complex terrain area; wherein the complex terrain area is an occlusion area, and the remaining areas are transparent areas.

[0070] Specifically, a layer consistent with the aerial image is generated in the marked complex terrain area as an occlusion layer, the complex terrain area is set as the occlusion area, and the remaining areas except the occlusion area are set as transparent areas.

[0071] S303: superimpose the occlusion layer on the basic 3D model and render to obtain a main 3D model. Specifically, the main 3D model is rendered by superimposing and fusing the occlusion layer and the basic 3D model.

[0072] As in the solution of the above embodiment, based on the fact that some aerial images contain complex terrain content, these complex terrain contents are not rendered in detail when the main three-dimensional model is obtained through 3D modeling. Since the main three-dimensional model is the main part except the complex terrain area, this part usually does not involve complex terrain. By performing conventional modeling and rendering during 3D modeling, it is possible to quickly model with lower resource consumption on the basis of lower performance equipment, thereby improving modeling efficiency.

[0073] S40 , obtaining the complex terrain areas marked in the corresponding aerial images, and performing refined modeling on the complex terrain contents of the complex terrain areas to obtain a complex terrain three-dimensional model.

[0074] In this step, since the complex terrain areas on each aerial image containing complex terrain content have been marked in advance, the complex terrain three-dimensional model can be obtained by obtaining local images of the complex terrain areas marked accordingly in each aerial image and performing fine modeling of the complex terrain content according to the scope of the complex terrain area.

[0075] In one embodiment, reference Figure 3 As shown, Figure 3 This is an example flow chart for modeling a complex terrain 3D model. Step S40 of performing refined modeling of the complex terrain content of the complex terrain area to obtain a complex terrain 3D model may include:

[0076] S401 : capturing a local image from the aerial image according to the marked complex terrain area.

[0077] Specifically, for each aerial image, a local image is cut out of the complex terrain area according to the pre-marked area; Figure 4 As shown, Figure 4 This is an example of a complex terrain content annotation diagram. The dotted box in the figure is the complex terrain content range. This range is marked by a maximum circumscribed rectangular box. When performing subsequent 3D modeling, the complex terrain content within the rectangular box is rendered.

[0078] S402: Calculate position parameters of each of the local images in the aerial image.

[0079] Specifically, after each local image is captured, its position parameters in the aerial image are calculated so that it can be aligned with the corresponding position of the main three-dimensional model when the three-dimensional model is subsequently merged.

[0080] S403: Divide the local image into a plurality of local image sets corresponding to complex terrain areas according to the location parameters.

[0081] Specifically, the target area of ​​the UAV aerial photography mission may include multiple complex terrain areas. Here, an image set is established for each complex terrain area, so as to be used for three-dimensional reconstruction of the complex terrain area.

[0082] S404: Perform a three-dimensional model based on the local image set to obtain a three-dimensional model of the complex terrain corresponding to each complex terrain area.

[0083] For example, the Gaussian splattering technique may be used to reconstruct the three-dimensional model, thereby obtaining a three-dimensional model of the complex terrain corresponding to each complex terrain region.

[0084] As in the solution of the above-mentioned embodiment, a lightweight target detection algorithm model deployed on a drone is used to process the aerial images in advance when taking them, identify the target objects contained in the aerial images and mark them; during three-dimensional modeling, high-quality and refined three-dimensional modeling is performed on the complex terrain content within the marked range; further, Gaussian splashing technology is used to refine the rendering of the three-dimensional model. Gaussian splashing technology can achieve scene reconstruction through probability cloud and can achieve local precise rendering; since the accuracy of Gaussian splashing is related to the smoothness characteristics of the Gaussian kernel, and the higher the accuracy, the more computing resources are required, and the processing efficiency is lower, therefore, only the complex terrain area is processed, and the theme three-dimensional model is processed by conventional modeling, so that the surveying and mapping data of complex terrain can be processed at a lower cost.

[0085] S50: Generate an overall three-dimensional model map of the target area based on the main three-dimensional model and the complex terrain three-dimensional model.

[0086] Exemplarily, the main body three-dimensional model and the complex terrain three-dimensional model are synthesized using three-dimensional modeling software to generate a final overall three-dimensional model diagram.

[0087] In one embodiment, reference Figure 5 As shown, Figure 5 This is an example flow chart for building an overall three-dimensional model. Step S50 of generating an overall three-dimensional model of the target area based on the main three-dimensional model and the complex terrain three-dimensional model may include:

[0088] S501, obtaining a first extreme point on the edge line of the complex terrain three-dimensional model; wherein the first extreme point includes the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions.

[0089] S502, obtaining the second extreme points of the main body three-dimensional model along the edge line of the occluded area and the transparent area; wherein the second extreme points include the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions.

[0090] S503: Align and merge the main body three-dimensional model and the complex terrain three-dimensional model according to the first extreme point and the second extreme point to obtain an overall three-dimensional model map of the target area.

[0091] Specifically, when synthesizing two 3D models, the main 3D model and each complex terrain 3D model are first imported through 3D modeling software. Since the position of each complex terrain 3D model in the main 3D model is known, the first extreme point on the edge line of the complex terrain 3D model and the second extreme point on the edge line of the main 3D model along the occlusion area and the transparent area are obtained for alignment, so as to align the complex terrain 3D model to the main 3D model. Since the main 3D model is obtained by superimposing the basic 3D model and the occlusion layer, the occlusion layer and the complex terrain 3D model are superimposed on the basic 3D model, and the final overall 3D model image is obtained by using 3D model synthesis technology.

[0092] Furthermore, based on the overall three-dimensional model, the elevation data of each location can be further obtained, and then the elevation data can be used to generate a surveying map of the target area; using three-dimensional modeling software, full three-dimensional textures and realistic textures can be obtained to perfectly restore the actual scene. At the same time, high-resolution maps can be generated, and precise measurements of length, area, and volume can be performed on the three-dimensional model to generate elevation contour lines, etc.

[0093] As in the solution of the above embodiment, using a lower 3D modeling level to model the main body 3D model can greatly reduce the computing burden and resource consumption when modeling the 3D model, and improve the modeling efficiency. At the same time, using a higher 3D modeling level to model the complex terrain 3D model can enhance the visual focus and facilitate the refined modeling and presentation of complex terrain. Overall, it reduces the complexity and professional requirements of surveying and mapping involving complex terrain, so that resource consumption can be reduced during modeling and rendering, thereby adapting to low-performance equipment and promoting the promotion and use of drone surveying and mapping technology.

[0094] In order to make the technical solution of this application clearer, more embodiments are described below.

[0095] In view of the fact that the surveying and mapping technology based on UAV aerial photography has high requirements for the performance of UAV equipment and the professionalism of surveying and mapping personnel, a control scheme for UAV aerial photography surveying and mapping is proposed. It does not require professional surveying and mapping UAV equipment and can also realize UAV aerial photography surveying and mapping on low-performance UAV equipment, especially on civilian UAVs with low hardware performance, thereby reducing the professional requirements for surveying and mapping personnel.

[0096] Accordingly, the complex terrain surveying and mapping method of the present application may further include the following steps before controlling the UAV to perform the aerial photography task to obtain multiple aerial images in step S10:

[0097] (1) Use a drone to capture several real-life images of the target area at a preset location.

[0098] In this step, before surveying, a drone is first used to capture a number of real-scene images containing the target area. The drone can be controlled to fly to a preset height and capture real-scene images at a set angle.

[0099] In one embodiment, the above step (1) may specifically include:

[0100] A map interface is imported through the controller of the drone; several shooting point locations are selected on the map interface in response to the user's setting operation; the drone is controlled to fly to the shooting point locations, and the camera of the drone is controlled to capture several real-life images containing the target area at a preset angle.

[0101] Specifically, the map interface is imported into the map interface through the controller of the drone in combination with the positioning information. Since the map interface lacks terrain information, the shooting point locations for shooting real-life images can be designed through the map interface. Exemplarily, the controller can be a device with a touch screen, or it can be connected to a smartphone and used as a screen, or it can use a smartphone as a controller, etc. The user can perform setting operations on the map interface and select several shooting point locations as the location points for shooting real-life images. Several real-life images containing the target area. According to the set shooting point locations, the drone is controlled to fly quickly to the shooting point locations and shoot several real-life images at a preset angle. The real-life images include the range of the target area; the target area refers to the area that needs to be surveyed and mapped; usually, the drone can be controlled to shoot real-life images at a set altitude and perpendicular to the ground above the target area.

[0102] As in the above embodiment, by quickly setting a number of shooting point positions on the map interface and shooting real scene images containing the target area, an initial topographic map containing the target area can be generated.

[0103] (2) Inputting the real scene image into a pre-trained deep learning model to obtain an initial topographic map containing the target area.

[0104] In this step, the pre-trained deep learning model is used to input the captured real-life image into the deep learning model to generate an initial topographic map containing the target area.

[0105] In one embodiment, the above step (2) may specifically include:

[0106] (2.1) Extracting image feature information of the real scene image, and inputting the image features into a pre-trained deep learning model to perform depth prediction and edge detection to obtain terrain feature information.

[0107] As an example, a deep learning model can be trained by the following methods, specifically including the following:

[0108] (1) Establish a convolutional neural network model.

[0109] A Convolutional Neural Network (CNN) structure, such as a U-Net neural network, can be used. Alternatively, a Transformer-based model, such as a Vision Transformer model, or a GAN (Generative Adversarial Network) model can be used.

[0110] (2) Using the labeled aerial images and the corresponding topographic map dataset to train the convolutional neural network model.

[0111] By constructing a dataset of annotated aerial images and corresponding topographic maps, paired aerial images and digital elevation model (DEM) data can be collected, and real-time 3D tools can be used to generate synthetic virtual terrain and simulated aerial image datasets. Furthermore, the data can be enhanced, rotated, and brightness adjusted, and the generated dataset can be used to train convolutional neural network models.

[0112] Exemplarily, the training process mainly involves multi-scale feature fusion and geographic space embedding. By integrating shallow texture features and deep semantic features, longitude and latitude coordinate channels are added to the decoder, and training is performed under a set physical constraint loss function. At the same time, the difference between the generated terrain map and the real terrain is measured by evaluating the model and using the mean square error or structural similarity index.

[0113] (3) cyclically training the depth information prediction capability and edge detection capability of the convolutional neural network model, and optimizing the parameters of the convolutional neural network model.

[0114] By cyclically training the convolutional neural network model's depth information prediction capability, that is, the ability to predict and identify terrain height, the model can preliminarily identify the elevation parameters of the terrain; by cyclically training the convolutional neural network model's edge detection capability, the edge detection can realize the identification and segmentation of different terrain blocks, and realize the generation of a terrain map composed of different terrain blocks from the image.

[0115] For example, a progressive training approach using the PyTorch framework (an open-source deep learning framework for machine learning and deep learning) can be employed. During parameter tuning, the optimizer and learning rate strategy can be configured, and the dynamic range of the batch size (the number of samples processed by the model during a single training session) can be set. Alternatively, multimodal input fusion can be employed for training, such as integrating LiDAR point clouds to aid training.

[0116] As in the solution of the above embodiment, a deep learning model is trained through data collection and processing, model selection and training, post-processing conversion, evaluation and optimization, etc., which can improve the robustness and accuracy of the deep learning model.

[0117] (2.2) Acquire an initial topographic map including the target area based on the topographic feature information and the map interface.

[0118] As an example, the process of obtaining the initial topographic map in step (2.2) above may include:

[0119] First, image feature information of the real scene image is extracted, and the image features are input into a pre-trained deep learning model for depth prediction and edge detection to obtain terrain feature information.

[0120] For example, texture features, grayscale, color and other image parameters can be extracted from real-scene images; these image features are input into a pre-trained deep learning model to detect depth information and edge information to obtain corresponding terrain feature information.

[0121] Then, an initial topographic map including the target area is obtained according to the topographic feature information and a map interface.

[0122] Among them, the topographic map can show in detail the surface undulation, geographical location, shape, etc. In this embodiment, an initial topographic map is generated based on the terrain feature information combined with the map interface. The initial topographic map contains the terrain feature information of the target area; it can include elements such as scale, coordinate grid, contour lines, and isobaths. The terrain features include various land features and landform features, such as houses, roads, farmland, lakes, mountains, hills, plains, depressions, cliffs, gullies, etc.

[0123] In this step, the initial topographic map is not a high-precision topographic map, but a topographic map containing certain topographic information of the target area. The initial topographic map is used to configure the aerial photography parameters of the UAV.

[0124] As in the solution of the above embodiment, a pre-trained deep learning model is used to assist in generating an initial topographic map, so that the terrain feature information of the target area can be quickly obtained, providing fast and accurate data support for the subsequent configuration of the drone's aerial photography parameters.

[0125] Furthermore, as an embodiment, after the real-scene image is input into a pre-trained deep learning model to obtain an initial topographic map containing the target area, the target area can be determined on the initial topographic map and the topographic information of the initial topographic map can be edited in response to an editing operation performed by the user.

[0126] In this step, users can perform editing operations on the initial topographic map. Since the initial topographic map covers a large area, the target area for aerial survey can be selected on the initial topographic map. After selecting the target area, users can edit the terrain information on the initial topographic map. Because the terrain information on the initial topographic map is derived through a deep learning model and has limited accuracy and detail, users can use editing operations to correct the terrain information or customize it as needed.

[0127] In one embodiment, determining the target area on the initial topographic map and editing the topographic information of the initial topographic map may include:

[0128] (a) Selecting an aerial photography range of a target area on the initial topographic map according to a range selection operation by a user.

[0129] Specifically, the user performs a range selection operation on the initial topographic map to select the aerial photography range of the target area; in conventional technology, the target area is usually selected by the user on the imported map interface, and the map interface is only used as a reference.

[0130] (b) Dividing the initial topographic map within the aerial photography range into blocks and setting each block to an editable state.

[0131] The initial topographic map within the aerial photography range can be divided into blocks according to the features of the landforms and topography, and each block can be set to an editable state; for example, it can be divided into blocks according to houses and roads, farmlands and hills, rivers and lakes, etc.; at the same time, each block can be set to an editable state, and users can edit its landform type, elevation, area and other parameters.

[0132] In practical applications, such as Figure 6 As shown, Figure 6 This is an example of an initial topographic map diagram. The dotted line in the figure is a block diagram. Different terrain types can be divided into blocks. When dividing the blocks, the fineness and specific division method of the blocks can be set according to needs. For example, a large piece of farmland can be divided into one block, or it can be divided into multiple blocks according to the ridges. A large area of ​​houses can be divided into one block, or each house can be divided into a block.

[0133] (c) Editing the terrain information of the blocks according to the user's editing operation.

[0134] For example, users can edit the terrain of each block, such as adjusting landforms and topography information, or setting elevation parameters, etc. In the editing program, different editing options can be provided for users to choose from, and parameter ranges can be provided for users to adjust, thereby achieving rapid editing of terrain information.

[0135] As in the solution of the above embodiment, the user can select the target area on the initial terrain map and edit the terrain information, which can provide fast and accurate data support for the subsequent configuration of the drone's aerial photography parameters, and can customize the terrain information according to needs, thereby facilitating the adjustment of the configuration of the aerial photography parameters.

[0136] (3) Obtaining equipment parameters of the UAV, obtaining aerial photography parameters of the UAV based on the equipment parameters and the initial terrain map, and generating an aerial photography task based on the aerial photography parameters and configuring it in the UAV.

[0137] In this step, the drone's equipment parameters are obtained, and the drone's aerial photography parameters are obtained based on the equipment parameters and the initial terrain map. For example, the aerial photography parameters may include: flight path (route), altitude, overlap rate (such as heading overlap rate and lateral overlap rate), ground resolution, flight speed, obstacle avoidance mode, and camera parameters (shutter, ISO, and aperture).

[0138] Conventional drone surveying and mapping technology requires users to manually set parameters such as altitude, overlap rate, and flight path, requiring users to have professional surveying and mapping knowledge. If high-precision surveying and mapping is required, it is more dependent on professional drone equipment, which cannot be achieved with low-performance drones. In particular, for some users of low-level professional and low-performance civilian drones, it is difficult to carry out surveying and mapping tasks quickly and conveniently.

[0139] The solution of this embodiment utilizes the equipment parameters and initial terrain map of the UAV, combined with the correspondence between pre-set terrain information and aerial photography parameters, to automatically calculate the aerial photography parameters of the UAV; further, the user can also modify and adjust the calculated aerial photography parameters, and finally generate an aerial photography task based on the obtained aerial photography parameters and configure it into the UAV.

[0140] As in the solution of the above embodiment, the initial topographic map contains terrain feature information. When setting the aerial photography parameters of the drone, it avoids relying entirely on the user to configure based on professional knowledge. The parameters can be automatically calculated based on the equipment parameters and the terrain feature information of the corresponding area, reducing the requirements for the user's drone aerial photography and surveying-related professional knowledge in surveying and mapping, and improving the usability of drone surveying and mapping technology.

[0141] In some embodiments, considering that low-performance drones lack RTK (Real-time kinematic) equipment positioning, in order to achieve the positioning function requirements during aerial photography, the complex terrain surveying and mapping method of the present application may also include the following:

[0142] The drone obtains the satellite positioning data of the controller; the drone is positioned according to the satellite positioning data and the navigation positioning information calculated by the drone; wherein, when the satellite positioning signal of the drone is lost, the navigation positioning information is used as the positioning position of the drone.

[0143] For example, the satellite positioning data of the mobile phone connected to the controller, such as GPS (Global Positioning System) data, Beidou positioning data, etc., can be used, combined with the real-time positioning compensation algorithm on the drone. When the satellite positioning signal of the drone is lost, the navigation and positioning information of the drone can be calculated through the satellite positioning data of the mobile phone based on the visual-inertial odometry (VIO) algorithm and the fusion of camera and IMU (Inertial Measurement Unit) data.

[0144] As in the solution of the above embodiment, by constructing a positioning system that uses the satellite positioning data of a mobile phone, the navigation positioning information of the drone is calculated using a real-time positioning compensation algorithm in the absence of professional RTK equipment, ensuring that the drone can be accurately positioned even when the satellite positioning signal is lost.

[0145] An embodiment of a system for surveying and mapping complex terrain is described below.

[0146] This application is a complex terrain mapping system, reference Figure 7 As shown, Figure 7 This is a structural diagram of an example complex terrain surveying and mapping system, including: a drone and a terminal device; the drone is used to perform aerial photography tasks to obtain multiple aerial images, use a deployed pre-trained target detection algorithm model to identify complex terrain content from the aerial images, and mark the complex terrain area corresponding to the complex terrain content; the terminal device is used to import the aerial images, use three-dimensional modeling software to perform conventional modeling on the aerial images to obtain a main three-dimensional model; wherein, the main three-dimensional model includes the main part except the complex terrain area; the complex terrain area marked accordingly in each aerial image is obtained, and the complex terrain content of the complex terrain area is refinedly modeled to obtain a complex terrain three-dimensional model; and an overall three-dimensional model map of the target area is generated based on the main three-dimensional model and the complex terrain three-dimensional model.

[0147] For example, the drone may be a civilian drone, its controller may be a device with a touch screen or a smart phone, and the computer may be a common personal computer.

[0148] It should be noted that the complex terrain surveying and mapping system of this embodiment utilizes a complex terrain surveying and mapping method provided by the embodiment of the present application. For the complex terrain surveying and mapping process, please refer to the description of the corresponding complex terrain surveying and mapping method shown in the previous text, which will not be repeated here.

[0149] The technical solution of the above embodiment, when using drone aerial photography, first pre-marks complex terrain areas, and then combines the main 3D model with the refined modeling of the complex terrain areas during 3D modeling to obtain an overall 3D model of the target area. This reduces the amount of data processing and the complexity and specialization required for surveying and mapping involving complex terrain, thereby promoting the popularization and use of drone surveying and mapping technology. Furthermore, by using real-life images and deep learning models to generate an initial topographic map, users do not need to perform specialized drone route planning and other configuration operations. Instead, they edit terrain information on the initial topographic map to configure drone aerial photography parameters. This achieves responsiveness to surveying and mapping needs based on low-cost drone equipment, reduces the difficulty of parameter configuration for drone aerial photography and mapping technology, and improves the ease of use of drone surveying and mapping technology.

[0150] Embodiments of a computer device and a computer-readable storage medium are described below.

[0151] The present application provides a technical solution for a computer device to implement functions related to a method for surveying and mapping complex terrain. The computer device of this embodiment includes one or more processors, a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by one or more processors, and the one or more programs are configured to perform the steps of the method for surveying and mapping complex terrain in any embodiment.

[0152] like Figure 8 As shown, Figure 8 is a block diagram of an example computer device; the computer device 100 may include one or more of the following components: a processing component 102, a memory 104, a power component 106, a multimedia component 108, an audio component 109, an input / output (I / O) interface 112, a sensor component 114, and a communication component 116.

[0153] Processing component 102 generally controls the overall operation of computer device 100, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.

[0154] The memory 104 is configured to store various types of data to support operations in the computer device 100, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0155] The power supply assembly 106 provides power to the various components of the computer device 100 .

[0156] The multimedia component 109 includes a screen that provides an output interface between the computer device 100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In some embodiments, the multimedia component 108 includes a front camera and / or a rear camera.

[0157] The audio component 109 is configured to output and / or input audio signals.

[0158] I / O interface 112 provides an interface between processing component 102 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0159] The sensor assembly 114 includes one or more sensors for providing various aspects of status assessment for the computer device 100. The sensor assembly 114 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact.

[0160] The communication component 116 is configured to facilitate wired or wireless communication between the computer device 100 and other devices. The computer device 100 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof.

[0161] This application provides a technical solution for a computer-readable storage medium for implementing functions related to a method for surveying and mapping complex terrain. The computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded by a processor to execute the method for surveying and mapping complex terrain of any embodiment.

[0162] In an exemplary embodiment, the computer-readable storage medium may be a non-transitory computer-readable storage medium including instructions, such as a memory including instructions. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0163] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for surveying and mapping complex terrain, characterized in that: include: Control the UAV to perform aerial photography missions and obtain multiple aerial images; Using a pre-trained target detection algorithm model deployed on a drone to identify complex terrain content from aerial images, and marking the complex terrain areas corresponding to the complex terrain content; Acquire the aerial image and import it into a terminal device, and perform conventional modeling on the aerial image using 3D modeling software to obtain a main body 3D model; wherein the main body 3D model includes the main body except for the complex terrain area; Obtaining the complex terrain areas marked in the corresponding aerial images, and performing detailed modeling on the complex terrain content of the complex terrain areas to obtain a three-dimensional model of the complex terrain; An overall three-dimensional model map of the target area is generated based on the main three-dimensional model and the complex terrain three-dimensional model.

2. The method for surveying and mapping complex terrain according to claim 1, characterized in that: The aerial image is conventionally modeled using 3D modeling software to obtain a main 3D model; wherein the main 3D model includes the main part except the complex terrain area, including: Importing the aerial image into a 3D modeling software for conventional modeling to obtain a basic 3D model; Generate an occlusion layer according to the marked complex terrain area; wherein the complex terrain area is the occlusion area, and the remaining area is the transparent area; The occlusion layer is superimposed on the basic three-dimensional model for rendering to obtain a main three-dimensional model.

3. The method for surveying and mapping complex terrain according to claim 2, characterized in that: Refined modeling of the complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model includes: intercepting a local image from the aerial image according to the marked complex terrain area; Calculating position parameters of each of the local images in the aerial image; Dividing the local image into a plurality of local image sets corresponding to complex terrain areas according to the location parameters; A three-dimensional model is generated based on the local image set to obtain a three-dimensional model of the complex terrain corresponding to each complex terrain area.

4. The method for surveying and mapping complex terrain according to claim 3, characterized in that: Generating an overall three-dimensional model map of the target area according to the main three-dimensional model and the complex terrain three-dimensional model includes: Obtain a first extreme point on the edge line of the complex terrain three-dimensional model; wherein the first extreme point includes the uppermost point, the lowermost point, the leftmost point, and the rightmost point in four directions; Obtaining second extreme points of the main body three-dimensional model along the edge line of the occluded area and the transparent area; wherein the second extreme points include the uppermost point, the lowermost point, the leftmost point, and the rightmost point in four directions; The main body three-dimensional model and the complex terrain three-dimensional model are aligned and merged according to the first extreme point and the second extreme point to obtain an overall three-dimensional model map of the target area.

5. The method for surveying and mapping complex terrain according to any one of claims 1 to 4, characterized in that: Before controlling the drone to perform aerial photography missions and obtain multiple aerial images, the following steps are also required: Use a drone to capture several real-life images of the target area at a preset location; Inputting the real scene image into a pre-trained deep learning model to obtain an initial topographic map containing the target area; Obtain device parameters of the drone, obtain aerial photography parameters of the drone based on the device parameters and the initial terrain map, and generate an aerial photography task based on the aerial photography parameters and configure it in the drone.

6. The method for surveying and mapping complex terrain according to claim 5, characterized in that: After inputting the real scene image into a pre-trained deep learning model to obtain an initial topographic map containing the target area, the method further includes: In response to an editing operation performed by a user, the target area is determined on the initial topographic map and topographic information of the initial topographic map is edited.

7. The method for surveying and mapping complex terrain according to claim 6, characterized in that: Determining the target area on the initial topographic map and editing the topographic information of the initial topographic map include: Selecting an aerial photography range of a target area on the initial topographic map according to a range selection operation by a user; Dividing the initial topographic map within the aerial photography range into blocks and setting each block to an editable state; The terrain information of the blocks is edited according to the user's editing operation.

8. A complex terrain surveying and mapping system, characterized in that: include: drones and terminal devices; The drone is used to perform aerial photography tasks to obtain multiple aerial images, identify complex terrain content from the aerial images using a deployed pre-trained target detection algorithm model, and mark the complex terrain areas corresponding to the complex terrain content; The terminal device is used to import the aerial image, and use the three-dimensional modeling software to perform conventional modeling on the aerial image to obtain a main three-dimensional model; wherein, the main three-dimensional model includes the main part except the complex terrain area; obtain the complex terrain area correspondingly marked in each aerial image, and perform detailed modeling on the complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model; generate an overall three-dimensional model map of the target area based on the main three-dimensional model and the complex terrain three-dimensional model.

9. A computer device comprising: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the steps of the complex terrain surveying and mapping method described in any one of claims 1-7.

10. A computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the method for surveying and mapping complex terrain according to any one of claims 1 to 7.

Images