Method and system for mapping complex terrain, computer device and medium thereof

Abstract

The application relates to a complex terrain mapping method, a complex terrain mapping system, computer equipment and a computer readable storage medium. The method comprises the following steps: controlling a UAV to perform an aerial photography task to obtain multiple aerial photography images; using a pre-trained target detection algorithm model arranged on the UAV to identify complex terrain content from the aerial photography images, and marking the corresponding complex terrain area of the complex terrain content; obtaining the aerial photography images and importing them into a terminal device, using a three-dimensional modeling software to perform conventional modeling on the aerial photography images to obtain a main three-dimensional model; obtaining the complex terrain area marked in each aerial photography image, and performing fine modeling on the complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model; and generating an overall three-dimensional model diagram of a target area according to the main three-dimensional model and the complex terrain three-dimensional model. The technical scheme reduces the professional degree requirement of mapping complexity containing complex terrain, and promotes the popularization and use of UAV mapping technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicle surveying and mapping technology, and in particular to a complex terrain surveying and mapping method, a complex terrain surveying and mapping system, a computer device and a computer readable storage medium. BACKGROUND

[0002] With the development of unmanned aerial vehicle technology, the surveying and mapping technology based on unmanned aerial vehicle aerial photography has also developed rapidly. The digital aerial photography technology based on the unmanned aerial vehicle platform has shown its unique advantages and has broad prospects in basic surveying and mapping, land resource investigation and monitoring, land use dynamic monitoring, digital city construction and emergency disaster surveying and mapping data acquisition.

[0003] In unmanned aerial vehicle aerial surveying, local complex terrain is often encountered. The data processing amount of complex terrain is several times that of ordinary plains. In order to ensure that the data error is within a certain range, although powerful and advanced equipment is used, the surveying and mapping requirements of complex terrain in most scenes can be met. However, the data processing of aerial images in these technical solutions is complex, the data volume is large, the processing cost is high, and the degree of specialization is high, which affects the popularization and use of unmanned aerial vehicle surveying and mapping technology. SUMMARY

[0004] In order to solve one of the above defects, the present application provides a complex terrain surveying and mapping method, a complex terrain surveying and mapping system, a computer device and a computer readable storage medium, which realizes surveying and mapping of complex terrain at a lower cost.

[0005] A complex terrain surveying and mapping method, comprising:

[0006] controlling an unmanned aerial vehicle to perform an aerial photography task to obtain a plurality of aerial images;

[0007] using a pre-trained target detection algorithm model deployed on the unmanned aerial vehicle to identify complex terrain content from the aerial images and label a complex terrain region corresponding to the complex terrain content;

[0008] obtaining the aerial images labeled with the complex terrain region and importing them into a terminal device, and using a three-dimensional modeling software to perform regular modeling on the aerial images to obtain a main three-dimensional model; wherein the main three-dimensional model comprises a main part excluding the complex terrain region;

[0009] obtaining the complex terrain region labeled by each aerial image, and performing fine modeling on the complex terrain content of the complex terrain region to obtain a complex terrain three-dimensional model;

[0010] generating an overall three-dimensional model of a target region according to the main three-dimensional model and the complex terrain three-dimensional model.

[0011] In one embodiment, the aerial image is subjected to regular modeling by using three-dimensional modeling software to obtain a main three-dimensional model; wherein the main three-dimensional model comprises a main part excluding a complex terrain area, comprising:

[0012] The aerial image is imported into three-dimensional modeling software for regular modeling to obtain a basic three-dimensional model;

[0013] An occlusion layer is generated according to the labeled complex terrain area; wherein the complex terrain area is an occlusion area, and the remaining area is a transparent area;

[0014] The occlusion layer is superimposed on the basic three-dimensional model for rendering to obtain a main three-dimensional model.

[0015] In one embodiment, the complex terrain content of the complex terrain area is subjected to fine modeling to obtain a complex terrain three-dimensional model, comprising:

[0016] A local image is intercepted from the aerial image according to the labeled complex terrain area;

[0017] The position parameters of each local image in the aerial image are calculated;

[0018] The local images are divided into a plurality of local image sets corresponding to the complex terrain areas according to the position parameters;

[0019] The complex terrain three-dimensional model corresponding to each complex terrain area is obtained by three-dimensional modeling according to the local image set.

[0020] In one embodiment, the overall three-dimensional model of the target area is generated according to the main three-dimensional model and the complex terrain three-dimensional model, comprising:

[0021] A first extreme point on the edge line of the complex terrain three-dimensional model is obtained; wherein the first extreme point comprises the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions;

[0022] A second extreme point on the edge line of the main three-dimensional model along the occlusion area and the transparent area is obtained; wherein the second extreme point comprises the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions;

[0023] The main three-dimensional model and the complex terrain three-dimensional model are aligned and merged according to the first extreme point and the second extreme point to obtain the overall three-dimensional model of the target area.

[0024] In one embodiment, before the unmanned aerial vehicle is controlled to perform the aerial photography task to obtain a plurality of aerial images, further comprising:

[0025] A plurality of real scene images containing the target area are photographed by the unmanned aerial vehicle at a preset position;

[0026] inputting the live image into a pre-trained deep learning model to obtain an initial terrain map containing the target region;

[0027] obtaining device parameters of the UAV, obtaining aerial photography parameters of the UAV according to the device parameters and the initial terrain map, and generating an aerial photography task configuration into the UAV according to the aerial photography parameters.

[0028] In one embodiment, after inputting the live image into a pre-trained deep learning model to obtain an initial terrain map containing the target region, the method further comprises:

[0029] determining the target region on the initial terrain map and editing terrain information of the initial terrain map in response to an editing operation performed by a user.

[0030] In one embodiment, determining the target region on the initial terrain map and editing terrain information of the initial terrain map comprises:

[0031] selecting an aerial photography range of the target region on the initial terrain map according to a range selection operation of the user;

[0032] blocking the initial terrain map within the aerial photography range and setting each block to be editable;

[0033] editing terrain information of the blocks according to an editing operation of the user.

[0034] A surveying and mapping system for complex terrain, comprising a UAV and a terminal device.

[0035] The UAV is configured to perform an aerial photography task to obtain a plurality of aerial photography images, identify complex terrain content from the aerial photography images using a deployed pre-trained target detection algorithm model, and label a complex terrain region corresponding to the complex terrain content.

[0036] The terminal device is configured to import the aerial photography images labeled with the complex terrain region, perform regular modeling on the aerial photography images using a three-dimensional modeling software to obtain a main three-dimensional model, wherein the main three-dimensional model comprises a main part excluding the complex terrain region, obtain the complex terrain region labeled for each aerial photography image, perform fine modeling on the complex terrain content of the complex terrain region to obtain a complex terrain three-dimensional model, and generate an overall three-dimensional model of the target region according to the main three-dimensional model and the complex terrain three-dimensional model.

[0037] A computer device, comprising:

[0038] one or more processors;

[0039] a memory;

[0040] one or more application programs stored in the memory and configured for execution by the one or more processors, the one or more programs configured for performing the steps of the above-described method for surveying complex terrain.

[0041] A computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set loaded and executed by a processor to perform the steps of the above-described method for surveying complex terrain.

[0042] The technical solution of the above-described embodiment controls the UAV to perform a flight task to obtain multiple aerial images; a pre-trained target detection algorithm model deployed on the UAV is used to identify complex terrain content from the aerial images and label the complex terrain content corresponding to the complex terrain region; then the aerial images are obtained and imported into a terminal device, and a three-dimensional modeling software on the terminal device is used to perform three-dimensional modeling on the aerial images to obtain a main three-dimensional model; the complex terrain content of the complex terrain region is finely modeled according to the labeled complex terrain region to obtain a complex terrain three-dimensional model; finally, an overall three-dimensional model of the target region is generated according to the main three-dimensional model and the complex terrain three-dimensional model; this technical solution first labels the complex terrain region when the UAV is used for aerial photography, and then combines the main three-dimensional model and the fine modeling of the complex terrain region to obtain the overall three-dimensional model of the target region when three-dimensional modeling is performed, thereby reducing the complexity and professional degree of surveying of the complex terrain, reducing resource consumption when modeling and rendering are performed, and thus adapting to low-performance devices, thereby promoting the popularization and use of UAV surveying technology.

[0043] Further, an initial terrain map is generated using real scene images and a deep learning model, the user does not need to perform professional UAV flight path planning and other configuration operations, and instead performs terrain information editing operations on the initial terrain map to configure UAV aerial photography parameters, thereby realizing response to surveying requirements on the basis of low-cost UAV devices, reducing the parameter configuration difficulty of UAV aerial photography surveying technology, and improving the ease of use of UAV surveying technology.

[0044] Additional aspects and advantages of the application will be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above-described and / or additional aspects and advantages of the application will become apparent to those skilled in the art from the following description, which proceeds with reference to the drawings.

[0046] Figure 1 is a flowchart of a method for surveying complex terrain of an embodiment.

[0047] Figure 2 Fig. 1 is a modeling flow chart of a subject three-dimensional model of an example;

[0048] Figure 3 Fig. 2 is a modeling flow chart of a complex terrain three-dimensional model of an example;

[0049] Figure 4 Fig. 3 is a content labeling schematic diagram of a complex terrain of an example;

[0050] Figure 5 Fig. 4 is a modeling flow chart of an overall three-dimensional model diagram of an example;

[0051] Figure 6 Fig. 5 is an initial terrain map schematic diagram of an example;

[0052] Figure 7 Fig. 6 is a structural schematic diagram of a surveying system of a complex terrain of an example;

[0053] Figure 8 Fig. 7 is a block diagram of a computer device of an example. DETAILED DESCRIPTION

[0054] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals and characters in different figures denote the same components. The embodiments described below are examples for explaining the present application and are not intended to limit the present application.

[0055] It should be understood that, unless specifically stated otherwise, the singular forms "a," "an," and "the" include plural forms. It should be further understood that the terms "includes," "including," "comprises" and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0056] The technical scheme of the present application provides a complex terrain mapping scheme, which can realize mapping of complex terrain under the condition of using low-performance unmanned aerial vehicle equipment; an initial terrain map can be generated by using real scene images and a deep learning model, without the need for professional unmanned aerial vehicle route planning and other configuration operations by the user, and instead of terrain information editing operation on the initial terrain map, unmanned aerial vehicle aerial photography parameter configuration is realized, the response mapping demand is realized on the basis of low-cost unmanned aerial vehicle equipment, the parameter configuration difficulty of unmanned aerial vehicle aerial photography mapping technology is reduced, and the usability of unmanned aerial vehicle mapping technology is improved; when using unmanned aerial vehicle aerial photography, the complex terrain region is first labeled in advance, and the overall three-dimensional model graph of the target region is obtained by combining the main three-dimensional model and the fine modeling of the complex terrain region during three-dimensional modeling, thereby reducing the data processing amount, reducing the professional degree requirement of mapping complexity containing complex terrain, enabling resource consumption to be reduced during modeling and rendering, thereby adapting to low-performance equipment, promoting the popularization and use of unmanned aerial vehicle mapping technology, and thus being more conducive to the application of unmanned aerial vehicle mapping technology in various scenarios.

[0057] Reference Figure 1 As shown in the figure, Figure 1 is a complex terrain mapping method flowchart of an embodiment, mainly including the following steps:

[0058] S10, controlling the unmanned aerial vehicle to perform an aerial photography task to obtain multiple aerial photography images.

[0059] In this step, the unmanned aerial vehicle is controlled to perform an aerial photography task to obtain multiple aerial photography images. The unmanned aerial vehicle can perform an aerial photography task, fly along a flight path, and take aerial photography images according to the set camera parameters.

[0060] S20, using a pre-trained target detection algorithm model deployed on the unmanned aerial vehicle to identify complex terrain content from the aerial photography images and label the complex terrain region corresponding to the complex terrain content.

[0061] In this step, a pre-trained target detection algorithm model deployed on the unmanned aerial vehicle is used to identify complex terrain content from the aerial photography images and label the complex terrain region corresponding to the complex terrain content.

[0062] Illustratively, the target detection algorithm model can be a YOLO-Fastest model, a corresponding YOLO-Fastest model can be pre-trained according to the terrain type, the YOLO-Fastest model can be deployed into the unmanned aerial vehicle, real-time aerial photography images taken by the camera can be obtained, each aerial photography image can be detected by using the YOLO-Fastest model, the complex terrain content contained in the aerial photography image can be identified, and then local labeling can be performed, which can usually be labeled by using the minimum circumscribed rectangle.

[0063] In the above solution, by deploying a target detection algorithm model on the drone, complex terrain content can be directly identified and quickly labeled from aerial images. This ensures the accuracy of labeling based on the complex terrain conditions required for different purposes. It also facilitates the accurate identification of complex terrain areas during subsequent 3D modeling, thereby improving the accuracy of modeling.

[0064] S30, acquire the aerial image and import it into the terminal device, and use 3D modeling software to perform conventional modeling on the aerial image to obtain a main 3D model; wherein, the main 3D model includes the main body part excluding complex terrain areas.

[0065] In this step, all aerial images captured by the drone are extracted and then imported into a terminal device for processing. For example, the terminal device can be a computer or similar equipment; the 3D modeling software can be Pix4D, Blender, Maya, or 3ds Max.

[0066] Generally, 3D modeling levels are categorized into white model, target element outline modeling, conventional modeling, refined modeling, and high-refinement modeling. Each level differs significantly in detail, resource consumption, and application scenarios, leading to varying requirements for equipment cost, performance, and processing efficiency in surveying systems. Using aerial imagery and 3D modeling software for conventional 3D modeling yields a main 3D model of the target area, excluding areas with complex terrain. This main 3D model serves as the foundation for describing the target area. Processing it using conventional methods reduces data processing complexity, lowers processing costs, and simplifies the required level of specialization.

[0067] In one embodiment, reference Figure 2 As shown, Figure 2 This is a flowchart illustrating the modeling process for a main 3D model. Step S30, which involves using 3D modeling software to perform conventional modeling on the aerial image to obtain the main 3D model, may include:

[0068] S301, the aerial image is imported into 3D modeling software for conventional modeling to obtain a basic 3D model. Specifically, a basic 3D model can be obtained using 3D modeling, rendering, and production software for conventional modeling.

[0069] S302, Generate an occlusion layer based on the marked complex terrain area; wherein the complex terrain area is the occlusion area, and the remaining areas are transparent areas.

[0070] Specifically, a layer identical to the aerial image is generated in the marked complex terrain area as an occlusion layer, the complex terrain area is set as the occlusion area, and the remaining areas are set as transparent areas.

[0071] S303, the occlusion layer is superimposed on the base 3D model and rendered to obtain the main 3D model. Specifically, the main 3D model is obtained by superimposing and blending the occlusion layer with the base 3D model.

[0072] As in the above embodiment, some aerial images contain complex terrain content. When the main 3D model of these complex terrain contents is obtained through 3D modeling, it is not rendered in detail. Since the main 3D model is the main part other than the complex terrain area, this part usually does not involve complex terrain. By performing conventional modeling and rendering during 3D modeling, it is possible to quickly model with lower resource consumption on a lower performance device, thereby improving modeling efficiency.

[0073] S40: Obtain the complex terrain areas marked in each aerial image, and perform detailed modeling of the complex terrain content of the complex terrain areas to obtain a three-dimensional model of the complex terrain.

[0074] In this step, since the complex terrain areas on each aerial image containing complex terrain content have been marked in advance, the complex terrain 3D model can be obtained by acquiring local images of the corresponding marked complex terrain areas in each aerial image and refining the complex terrain content according to the range of the complex terrain area.

[0075] In one embodiment, reference Figure 3 As shown, Figure 3 This is an example flowchart of a complex terrain 3D modeling process. Step S40, which involves refining the complex terrain content of the complex terrain area to obtain a complex terrain 3D model, may include:

[0076] S401, extract a local image from the aerial image based on the marked complex terrain area.

[0077] Specifically, for each aerial image, a partial image is extracted from the complex terrain area based on pre-labeled details; for example... Figure 4 As shown, Figure 4 This is an example of a diagram illustrating the annotation of complex terrain content. The dashed box in the diagram represents the range of complex terrain content, which is marked by a maximum bounding rectangle. During subsequent 3D modeling, the complex terrain content within the bounding rectangle is rendered.

[0078] S402, calculate the position parameters of each of the local images in the aerial image.

[0079] Specifically, after each local image is cropped, its position parameters in the aerial image are calculated, so that it can be aligned with the corresponding position of the main 3D model during subsequent 3D model merging.

[0080] S403, divide the local image into a plurality of local image sets corresponding to complex terrain regions according to the position parameters.

[0081] Specifically, for the target region of the UAV aerial photography task, it can include a plurality of complex terrain regions, and in this regard, an image set is established for each complex terrain region, thereby being used for three-dimensional reconstruction of the complex terrain region.

[0082] S404, three-dimensional modeling is performed according to the local image set to obtain a complex terrain three-dimensional model corresponding to each complex terrain region.

[0083] Exemplarily, the three-dimensional model can be reconstructed by using the Gaussian splatting technology, so as to obtain the complex terrain three-dimensional model corresponding to each complex terrain region.

[0084] As the scheme of the above embodiment, a lightweight target detection algorithm model deployed on the UAV is used to, when the aerial photography image is photographed, pre-process the photographed aerial photography image, identify and label the target object contained in the aerial photography image; when three-dimensional modeling is performed, the complex terrain content in the labeling range is subjected to high-quality and fine three-dimensional modeling; further, the Gaussian splatting technology is used to fine render the three-dimensional model, the Gaussian splatting technology can realize scene reconstruction in the form of a probability cloud and can realize local accurate rendering; since the accuracy of the Gaussian splatting is related to the smoothing characteristic of the Gaussian kernel, and the higher the accuracy is, the more the required computing resources are and the lower the processing efficiency is, therefore, only the complex terrain region is processed, and the main three-dimensional model is processed by using the conventional modeling, so that the surveying and mapping data of the complex terrain can be processed at a lower cost.

[0085] S50, generate an overall three-dimensional model map of the target region according to the main three-dimensional model and the complex terrain three-dimensional model.

[0086] Exemplarily, the main three-dimensional model and the complex terrain three-dimensional model are synthesized by using a three-dimensional modeling software to generate the final overall three-dimensional model map.

[0087] In one embodiment, referring to Figure 5 , the Figure 5 is an example of an overall three-dimensional model modeling flowchart, and the step S50 of generating an overall three-dimensional model map of the target region according to the main three-dimensional model and the complex terrain three-dimensional model can include:

[0088] S501, obtain a first extreme point on an edge line of the complex terrain three-dimensional model; wherein the first extreme point includes the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions.

[0089] S502, acquiring a second extreme value point of the main three-dimensional model along the edge line of the occlusion area and the transparent area; wherein the second extreme value point includes the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions.

[0090] S503, aligning and merging the main three-dimensional model and the complex terrain three-dimensional model according to the first extreme value point and the second extreme value point to obtain the overall three-dimensional model graph of the target area.

[0091] Specifically, when synthesizing two three-dimensional models, first, the main three-dimensional model and each complex terrain three-dimensional model are imported through a three-dimensional modeling software. Since the positions of each complex terrain three-dimensional model in the main three-dimensional model are known, the complex terrain three-dimensional model is aligned to the main three-dimensional model by acquiring the first extreme value point of the complex terrain three-dimensional model along the edge line and the second extreme value point of the main three-dimensional model along the edge line of the occlusion area and the transparent area. Since the main three-dimensional model is obtained by stacking the base three-dimensional model and the occlusion layer, the occlusion layer and the complex terrain three-dimensional model are stacked on the base three-dimensional model, and the final overall three-dimensional model graph is obtained by using three-dimensional model synthesis technology.

[0092] Further, based on the overall three-dimensional model graph, the elevation data of each position can be further acquired, and then the elevation data is used to generate a surveying and mapping graph of the target area. The three-dimensional modeling software can acquire full three-dimensional texture and realistic texture, perfectly restore the scene, and also can generate a high-resolution map, accurately measure the length, area and volume on the three-dimensional model, and generate an elevation contour line.

[0093] As the scheme of the above embodiment, the main three-dimensional model is modeled by using a lower three-dimensional modeling level, which can greatly reduce the calculation burden and resource consumption in three-dimensional modeling, improve the modeling efficiency, and at the same time, the complex terrain three-dimensional model is modeled by using a higher three-dimensional modeling level, which can improve the visual focus, facilitate fine modeling and presentation of the complex terrain, and overall reduce the surveying and mapping complexity and professional degree requirement of the complex terrain, so that the resource consumption can be reduced during modeling and rendering, thereby adapting to low-performance devices and promoting the popularization and use of unmanned aerial vehicle surveying and mapping technology.

[0094] In order to make the technical scheme of the present application clearer, more embodiments are described below.

[0095] In view of the situation that the unmanned aerial vehicle surveying and mapping technology based on aerial photography of unmanned aerial vehicles has high requirements on the performance of unmanned aerial vehicle devices and the professionalism of surveying and mapping personnel, a control scheme for unmanned aerial vehicle aerial photography surveying and mapping is proposed, which can realize unmanned aerial vehicle aerial photography surveying and mapping on low-performance unmanned aerial vehicle devices without professional surveying and mapping unmanned aerial vehicle devices, especially on low-performance civil unmanned aerial vehicles, and reduce the professional requirement on surveying and mapping personnel.

[0096] Accordingly, the complex terrain mapping method of the present application, before the step S10 of controlling the unmanned aerial vehicle to perform the aerial photography task to obtain a plurality of aerial photographs, can further include the following steps:

[0097] (1) Using the unmanned aerial vehicle to take a plurality of real scene images containing the target area at a preset position.

[0098] In this step, before mapping, a plurality of real scene images containing the target area are first taken by the unmanned aerial vehicle. The unmanned aerial vehicle can be controlled to fly to a preset height position and take real scene images at a set angle.

[0099] In one embodiment, the above step (1) can specifically include:

[0100] Importing a map interface through the controller of the unmanned aerial vehicle; selecting a plurality of shooting point positions on the map interface in response to the user's setting operation; controlling the unmanned aerial vehicle to fly to the shooting point positions and controlling the camera of the unmanned aerial vehicle to take a plurality of real scene images containing the target area at a preset angle.

[0101] Specifically, the map interface is imported through the controller of the unmanned aerial vehicle combined with positioning information. Since the map interface lacks terrain information, the shooting point positions for taking real scene images can be designed through the map interface. The controller can be a device with a touch screen, or it can be a smart phone connected as a screen, or it can be a smart phone used as a controller, etc. The user can set the operation on the map interface to select a plurality of shooting point positions as the position points for taking real scene images. A plurality of real scene images containing the target area. According to the set shooting point positions, the unmanned aerial vehicle is controlled to quickly fly to the shooting point positions to take a plurality of real scene images at a preset angle, and the real scene images contain the range of the target area; the target area refers to the area that needs to be mapped; usually, the unmanned aerial vehicle can be controlled to take real scene images at a set height and perpendicular to the ground direction above the target area.

[0102] As in the above embodiment, by quickly setting a plurality of shooting point positions on the map interface and taking real scene images containing the target area, an initial terrain map containing the target area can be generated.

[0103] (2) Inputting the real scene images into a pre-trained deep learning model to obtain an initial terrain map containing the target area.

[0104] In this step, the pre-trained deep learning model is used to input the taken real scene images into the deep learning model to generate an initial terrain map containing the target area.

[0105] In one embodiment, the above step (2) can specifically include:

[0106] (2.1) Extract image feature information of the real scene image, and input the image feature into a pre-trained deep learning model for deep prediction and edge detection to obtain terrain feature information.

[0107] As an example, for the deep learning model, it can be trained by the following method, specifically including the following:

[0108] (I) Establish a convolutional neural network model.

[0109] The convolutional neural network (CNN) structure can be used, such as the U-Net neural network, in addition, the model based on Transformer can also be used, such as the Vision Transformer model, or combined with the GAN (generative adversarial network) model, etc.

[0110] (ii) Use the labeled aerial photo and corresponding terrain map dataset to train the convolutional neural network model.

[0111] By constructing the labeled aerial photo and corresponding terrain map dataset, paired aerial image and digital elevation model (DEM) data can be collected, and real-time 3D tools are used to generate synthetic virtual terrain and simulated aerial photo dataset; further, the data can be enhanced, rotated, brightness adjusted, etc.; thus, the generated dataset is used to train the convolutional neural network model.

[0112] Exemplarily, the training process mainly includes multi-scale feature fusion and geographical space embedding, by integrating shallow texture features and deep semantic features, adding latitude and longitude coordinate channels in the decoder, and training under the set physical constraint loss function; at the same time, the model is evaluated and the mean square error or structural similarity index is used to measure the difference between the generated terrain map and the real terrain.

[0113] (III) Cycle training of the depth information prediction ability and edge detection ability of the convolutional neural network model, and parameter optimization of the convolutional neural network model.

[0114] By cyclically training the depth information prediction ability of the convolutional neural network model, that is, the prediction and recognition ability of the terrain height, the model can preliminarily recognize the elevation parameters of the terrain; by cyclically training the edge detection ability of the convolutional neural network model, the recognition and segmentation of different terrain blocks can be realized through edge detection, and the image is generated into a terrain map composed of different terrain blocks.

[0115] Exemplarily, a progressive training scheme of the PyTorch framework (an open-source deep learning framework for machine learning and deep learning) can be adopted, and during the parameter tuning process, an optimizer and a learning rate strategy are configured, and a dynamic adjustment range of the Batch Size (the number of samples processed by the model in one training process) is set. In addition, a multi-modal input fusion method can also be used for training, such as combining integrated LiDAR point cloud for auxiliary training.

[0116] According to the scheme of the above embodiment, the deep learning model is trained through data collection and processing, model selection and training, post-processing conversion, evaluation and optimization, which can improve the robustness and accuracy of the deep learning model.

[0117] (2.2) Obtain an initial terrain map containing the target area according to the terrain feature information and the map interface.

[0118] As an embodiment, the process of obtaining the initial terrain map in step (2.2) can include:

[0119] First, the image feature information of the real scene image is extracted, and the image features are input into the pre-trained deep learning model for depth prediction and edge detection to obtain terrain feature information.

[0120] Exemplarily, texture features, gray scale, color and other image parameters can be extracted from the real scene image; these image features are input into the pre-trained deep learning model for detection of depth information and edge information to obtain corresponding terrain feature information.

[0121] Then, an initial terrain map containing the target area is obtained according to the terrain feature information and the map interface.

[0122] The terrain map can represent the relief form and geographical position, shape, etc. in detail. In this embodiment, the initial terrain map is generated according to the terrain feature information in combination with the map interface, and the initial terrain map contains the terrain feature information of the target area; it can include scale, coordinate grid, contour line, depth contour, etc. The terrain features include various ground features and landform features, such as houses, roads, farmland, lakes, mountains, hills, plains, depressions, cliffs, gullies, etc.

[0123] In this step, the initial terrain map is not a high-precision terrain map, but a terrain map containing certain terrain information of the target area. The initial terrain map is used to configure the aerial photography parameters of the unmanned aerial vehicle.

[0124] As the scheme of the above embodiment, the pre-trained deep learning model is used to assist in generating the initial terrain map, so that the terrain feature information of the target area can be quickly obtained, and fast and accurate data support is provided for subsequent configuration of the aerial photography parameters of the unmanned aerial vehicle.

[0125] Furthermore, as an embodiment, after inputting the real-scene image into a pre-trained deep learning model to obtain an initial topographic map containing the target area, the target area can be determined on the initial topographic map and the topographic information of the initial topographic map can be edited in response to an editing operation performed by the user.

[0126] In this step, users can perform editing operations on the initial topographic map. Since the initial topographic map includes a relatively large area, the target area to be surveyed by aerial measurement can be selected on it. After selecting the target area, users can edit the topographic information on the initial topographic map. Because the topographic information on the initial map is obtained through a deep learning model and its accuracy and detail are limited, the user's editing operations can correct the topographic information or customize it according to needs.

[0127] In one embodiment, the terrain information for determining the target area on the initial terrain map and editing the initial terrain map may include:

[0128] (a) Select the aerial photography range of the target area on the initial topographic map according to the user's range selection operation.

[0129] Specifically, users select the target area for aerial photography by making a range selection operation on the initial topographic map; while in conventional technology, the target area is usually selected by the user by drawing a box on the imported map interface, which is only used as a reference.

[0130] (b) Divide the initial topographic map within the aerial photography range into blocks and set each block to be editable.

[0131] The initial topographic map within the aerial photography area can be divided into blocks based on land features and landform characteristics, and each block can be set to an editable state. For example, blocks can be divided by houses and roads, by farmland and hills, by rivers and lakes, etc. At the same time, each block can be set to an editable state, allowing users to edit its land feature type, elevation, area and other parameters.

[0132] In practical applications, such as Figure 6 As shown, Figure 6 This is an example initial topographic map. The dotted lines in the map are block diagrams, which can divide different terrain types into blocks. When dividing blocks, the fineness and specific division method of the blocks can be set according to the needs. For example, a large area of ​​farmland can be divided into one block, or it can be divided into multiple blocks according to the field ridges. A large area of ​​houses can be divided into one block, or each house can be divided into a block.

[0133] (c) editing the terrain information of the sub-block according to the editing operation of the user.

[0134] For example, the user can edit the terrain of each sub-block, such as adjusting the ground object, topographic information, or setting the elevation parameter, etc. In the editing program, different editing options can be provided for the user to choose, and the parameter range can be provided for the user to adjust; so as to realize the rapid editing of the terrain information.

[0135] As the scheme of the above embodiment, the user can frame the target area on the initial topographic map and edit the terrain information, which can provide quick and accurate data support for subsequent configuration of the aerial photography parameters of the unmanned aerial vehicle, and can customize the terrain information according to the needs, thereby facilitating the adjustment of the configuration of the aerial photography parameters.

[0136] (3) obtaining the device parameters of the unmanned aerial vehicle, obtaining the aerial photography parameters of the unmanned aerial vehicle according to the device parameters and the initial topographic map, and generating an aerial photography task configuration according to the aerial photography parameters and configuring the unmanned aerial vehicle.

[0137] In this step, the device parameters of the unmanned aerial vehicle are obtained, and the aerial photography parameters of the unmanned aerial vehicle are obtained according to the device parameters and the initial topographic map; for example, the aerial photography parameters can include: flight path (route), flight height, overlap rate (such as forward overlap rate and lateral overlap rate), ground resolution, flight speed, obstacle avoidance mode, and camera parameters (shutter, ISO, and aperture), etc.

[0138] In the conventional unmanned aerial vehicle surveying and mapping technology, the user needs to manually set the flight height, overlap rate, flight path and other parameters, which requires the user to have professional surveying and mapping knowledge; if high-precision surveying and mapping is required, it is more dependent on professional unmanned aerial vehicle equipment, and low-performance unmanned aerial vehicles cannot achieve it, especially for users with low professional level and low-performance civilian unmanned aerial vehicles, it is difficult to quickly and conveniently carry out surveying and mapping tasks.

[0139] The scheme of the embodiment uses the device parameters of the unmanned aerial vehicle and the initial topographic map, and combines the corresponding relationship between the pre-set terrain information and the aerial photography parameters, to automatically calculate the aerial photography parameters of the unmanned aerial vehicle; further, the user can also modify and adjust the calculated aerial photography parameters, and finally generate an aerial photography task configuration according to the obtained aerial photography parameters and configure the unmanned aerial vehicle.

[0140] As the scheme of the above embodiment, the initial topographic map contains terrain feature information, and when setting the aerial photography parameters of the unmanned aerial vehicle, it avoids completely relying on the user to configure according to professional knowledge, and can be automatically calculated according to the device parameters and the terrain feature information of the corresponding area, which reduces the requirement for the user's professional knowledge related to the aerial photography and surveying of the unmanned aerial vehicle in surveying and mapping, and improves the ease of use of the unmanned aerial vehicle surveying and mapping technology.

[0141] In some embodiments, considering the lack of RTK (Real-time kinematic) positioning equipment on low-performance drones, the complex terrain mapping method of this application may further include the following to meet the positioning requirements during aerial photography:

[0142] The UAV acquires satellite positioning data from the controller; the UAV is positioned based on the satellite positioning data and navigation positioning information calculated by the UAV; wherein, when the UAV loses its satellite positioning signal, the navigation positioning information is used as the UAV's positioning location.

[0143] For example, satellite positioning data from the mobile phone connected to the controller, such as GPS (Global Positioning System) data, BeiDou positioning data, etc., can be used. Combined with the real-time positioning compensation algorithm on the drone, when the drone loses its satellite positioning signal, the navigation and positioning information of the drone can be calculated based on the visual-inertial odometry (VIO) algorithm and the fusion of camera and IMU (Inertial Measurement Unit) data, using the satellite positioning data from the mobile phone.

[0144] As described in the above embodiments, by constructing a positioning system that utilizes satellite positioning data from a mobile phone, the system can calculate the navigation and positioning information of the drone using a real-time positioning compensation algorithm in the absence of professional RTK equipment, thus ensuring that the drone can achieve accurate positioning even when satellite positioning signals are lost.

[0145] The following describes an example of a mapping system for complex terrain.

[0146] This application provides a mapping system for complex terrain, referencing... Figure 7 As shown, Figure 7 This is a schematic diagram of an example complex terrain mapping system, including: a drone and a terminal device; the drone is used to perform aerial photography tasks to acquire multiple aerial images, and uses a pre-trained target detection algorithm model to identify complex terrain content from the aerial images and label the complex terrain regions corresponding to the complex terrain content; the terminal device is used to import the aerial images, and use 3D modeling software to perform conventional modeling on the aerial images to obtain a main 3D model; wherein, the main 3D model includes the main body part excluding the complex terrain regions; the complex terrain regions corresponding to the labels in each aerial image are acquired, and the complex terrain content of the complex terrain regions is refined to obtain a complex terrain 3D model; based on the main 3D model and the complex terrain 3D model, an overall 3D model map of the target area is generated.

[0147] For example, the drone can be a civilian drone, its controller can be a device with a touch screen or a smartphone, and the computer can be a commonly used personal computer.

[0148] It should be noted that the complex terrain mapping system of this embodiment utilizes a complex terrain mapping method provided by the embodiment of this application. For the specific mapping process of complex terrain, please refer to the description of the corresponding complex terrain mapping method shown above, which will not be repeated here.

[0149] The technical solution described above, when using UAVs for aerial photography, first pre-marks complex terrain areas. During 3D modeling, it combines the main 3D model with refined modeling of the complex terrain areas to obtain a comprehensive 3D model of the target area. This reduces the amount of data processing and lowers the professional requirements for surveying complex terrain, promoting the widespread use of UAV surveying technology. Furthermore, it generates an initial topographic map using real-world images and deep learning models. Users no longer need to perform professional UAV flight path planning and other configuration operations; instead, they can edit terrain information on the initial topographic map to configure UAV aerial photography parameters. This allows for responding to surveying needs with low-cost UAV equipment, reducing the difficulty of parameter configuration for UAV aerial surveying technology and improving its usability.

[0150] The following describes embodiments of computer devices and computer-readable storage media.

[0151] This application provides a technical solution for a computer device to implement functions related to a surveying method for complex terrain. The computer device of this embodiment includes one or more processors, a memory, and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform steps for a surveying method for complex terrain in any embodiment.

[0152] like Figure 8 As shown, Figure 8 This is a block diagram of an example computer device; the computer device 100 may include one or more of the following components: a processing component 102, a memory 104, a power supply component 106, a multimedia component 108, an audio component 110, an input / output (I / O) interface 112, a sensor component 114, and a communication component 116.

[0153] Processing component 102 typically controls the overall operation of computer device 100, such as operations associated with display, telephone calls, data communication, camera operation, and recording operation.

[0154] The memory 104 is configured to store various types of data to support the operation of the computer device 100. The memory 104 can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0155] The power component 106 provides power for various components of the computer device 100.

[0156] The multimedia component 108 includes a screen providing an output interface between the computer device 100 and the user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). In some embodiments, the multimedia component 108 includes a front camera and / or a rear camera.

[0157] The audio component 110 is configured to output and / or input an audio signal.

[0158] The I / O interface 112 provides an interface between the processing component 102 and peripheral interface modules, which can be a keyboard, a click wheel, a button, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button and a lock button.

[0159] The sensor component 114 includes one or more sensors for providing status assessment of various aspects of the computer device 100. The sensor component 114 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact.

[0160] The communication component 116 is configured to facilitate wired or wireless communication between the computer device 100 and other devices. The computer device 100 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof.

[0161] The present application provides a computer readable storage medium to implement the functions of the complex terrain mapping method. The computer readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the complex terrain mapping method of any embodiment.

[0162] In an exemplary embodiment, the computer readable storage medium can be a non-transitory computer readable storage medium including instructions, such as a memory including instructions, for example. The non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk and an optical data storage device, etc.

[0163] The above merely provides part of the embodiments of the present application, and it should be pointed out that, for those skilled in the art, some improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method of mapping complex terrain, characterized by, The method comprises the following steps: controlling the UAV to perform a flight task to obtain a plurality of aerial images; using a pre-trained target detection algorithm model deployed on the UAV to identify complex terrain content from the aerial images and label the complex terrain region corresponding to the complex terrain content; obtaining the aerial images labeled with the complex terrain region and importing them into a terminal device, using a three-dimensional modeling software to perform regular modeling on the aerial images to obtain a main three-dimensional model, including: importing the aerial images labeled with the complex terrain region into the three-dimensional modeling software to perform regular modeling to obtain a basic three-dimensional model; generating an occlusion layer according to the labeled complex terrain region; wherein the complex terrain region is an occlusion region and the remaining region is a transparent region; superimposing the occlusion layer on the basic three-dimensional model to obtain a main three-dimensional model; wherein the main three-dimensional model includes the main part excluding the complex terrain region; obtaining the complex terrain region labeled in each aerial image, and performing fine modeling on the complex terrain content of the complex terrain region to obtain a complex terrain three-dimensional model, including: cutting out local images from the aerial images labeled with the complex terrain region according to the labeled complex terrain region; calculating the position parameters of each local image in the aerial image; dividing the local images into a plurality of local image sets corresponding to the complex terrain region according to the position parameters; performing three-dimensional modeling according to the local image set to obtain a complex terrain three-dimensional model corresponding to each complex terrain region; generating an overall three-dimensional model of the target region according to the main three-dimensional model and the complex terrain three-dimensional model.

2. The method of mapping complex terrain of claim 1, wherein, Generating an overall three-dimensional model of the target region according to the main three-dimensional model and the complex terrain three-dimensional model comprises: obtaining a first extreme point on the edge line of the complex terrain three-dimensional model; wherein the first extreme point includes the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions; obtaining a second extreme point on the edge line of the main three-dimensional model along the occlusion region and the transparent region; wherein the second extreme point includes the uppermost point, the lowermost point, the leftmost point and the rightmost point in four directions; aligning and merging the main three-dimensional model and the complex terrain three-dimensional model according to the first extreme point and the second extreme point to obtain an overall three-dimensional model of the target region.

3. The method of mapping complex terrain of claim 1 or 2, wherein, Before controlling the UAV to perform a flight task to obtain a plurality of aerial images, the method further comprises: using the UAV to take a plurality of real scene images containing the target region at a preset position; inputting the real scene images into a pre-trained deep learning model to obtain an initial terrain map containing the target region; obtaining device parameters of the UAV, obtaining flight parameters of the UAV according to the device parameters and the initial terrain map, and generating a flight task configuration into the UAV according to the flight parameters.

4. The method of mapping complex terrain of claim 3, wherein, After inputting the real scene images into a pre-trained deep learning model to obtain an initial terrain map containing the target region, the method further comprises: in response to an editing operation performed by a user, determining the target region on the initial terrain map and editing the terrain information of the initial terrain map.

5. The method of mapping complex terrain of claim 4, wherein, Determine the target area on the initial topographic map and edit the topographic information of the initial topographic map, comprising: Selecting a flight range of the target area on the initial topographic map according to a range selection operation of a user; Dividing the initial topographic map in the flight range into blocks and setting each block as editable; Editing the topographic information of the blocks according to an editing operation of the user.

6. A complex terrain mapping system characterized by, Comprise: A UAV and a terminal device; The UAV is used to perform a flight task to obtain a plurality of aerial images, use a deployed pre-trained target detection algorithm model to identify complex terrain content from the aerial images, and label a complex terrain area corresponding to the complex terrain content; The terminal device is used to import the aerial images labeled with the complex terrain area, use a three-dimensional modeling software to perform regular modeling on the aerial images to obtain a main three-dimensional model, comprising: importing the aerial images labeled with the complex terrain area into the three-dimensional modeling software to perform regular modeling to obtain a basic three-dimensional model; generating an occlusion layer according to the labeled complex terrain area; wherein the complex terrain area is an occlusion area, and the remaining area is a transparent area; superimposing the occlusion layer on the basic three-dimensional model to perform rendering to obtain a main three-dimensional model; wherein the main three-dimensional model comprises a main part excluding the complex terrain area; obtaining the complex terrain area labeled in each aerial image, and performing fine modeling on the complex terrain content of the complex terrain area to obtain a complex terrain three-dimensional model, comprising: cutting out local images from the aerial images labeled with the complex terrain area according to the labeled complex terrain area; calculating the position parameters of each local image in the aerial image; dividing the local images into a plurality of local image sets corresponding to the complex terrain area according to the position parameters; performing three-dimensional modeling according to the local image sets to obtain a complex terrain three-dimensional model corresponding to each complex terrain area; and generating an overall three-dimensional model of the target area according to the main three-dimensional model and the complex terrain three-dimensional model.

7. A computer device, comprising: One or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs are configured to perform the steps of the complex terrain mapping method of any one of claims 1-5.

8. A computer-readable storage medium, the storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to perform the steps of the complex terrain mapping method of any one of claims 1-5.

Images