A method and system for filling in missing modeling areas of a geological survey aerial image

Abstract

The application discloses a kind of geological survey aerial photograph image modeling area missing completion method and system, belong to geological survey three-dimensional modeling technical field;S1 selects missing area;S2 selects adjacent area;S3 calculates similarity;S4 generates missing area model;S5 similarity discrimination;S6 integrates mark.This geological survey aerial photograph image modeling area missing completion method is based on adjacent area reference completion, using the mode of imitating learning network generation, can better complete missing area, fully play the role of temporary completion to ensure that subsequent steps proceed smoothly, and there is determined mark, and subsequent re-modeling is carried out to improve three-dimensional model after convenient subsequent rephotographing aerial photograph.

Description

Technical Field

[0001] This invention belongs to the field of geological surveying and mapping 3D modeling technology, specifically relating to a method and system for filling in missing areas in geological surveying aerial image modeling. Background Technology

[0002] Existing technologies include solutions for 3D modeling based on UAV oblique photography, in which UAV flight path planning is a crucial step. A typical example is a UAV coverage path planning method based on a 3D model disclosed in Chinese Patent Application No. 202111121543.5, which can ensure high detection coverage, high efficiency, and low cost.

[0003] However, in practice, for field scenes, especially when using drone oblique photography for geological exploration, there are often situations where harsh environments make it difficult for drones to take aerial photos along the predetermined route. For example, a wind tunnel may form in a local area, causing abnormally strong airflow. In such cases, gaps may appear in the drone's aerial photography path, resulting in missing images during 3D modeling.

[0004] In general, missing data in 3D modeling can lead to a severe situation where the entire model is missing. However, this is rarely the case in geological mapping; most often, it involves missing data in a specific local area. However, in the processing of geological mapping systems, after 3D modeling, geological analysis and other operations are often performed directly based on the resulting 3D model. In this case, missing local areas of the 3D model often cause two problems: firstly, during model optimization, the missing areas are stitched together, resulting in a distorted model that deviates significantly from reality and becomes unusable; secondly, missing local areas can cause abnormal null values ​​during geological analysis, leading to errors and the system crashing. Therefore, temporarily filling in missing areas during 3D modeling is necessary.

[0005] Regarding this kind of completion operation, existing technologies include a method for automatic modeling of photovoltaic BIM roofs based on UAV oblique photogrammetry, disclosed in Chinese Patent Application No. CN202310075323.6. This method projects the segmented individual triangular faces onto a plane, determines the range of the grayscale image of the projected triangular face, and calculates the grayscale values ​​of the corresponding grids based on the elevations of the three endpoints to form a grayscale image. Noise reduction processing is then performed on the grayscale image through erosion and dilation calculations to complete the grayscale of the hollowed-out areas. This method is clearly unsuitable for temporarily completing missing parts in localized areas during 3D modeling. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method for completing missing areas in geological survey aerial image modeling. This method is based on neighboring area reference completion and uses a learning network generation approach to effectively complete missing areas, providing temporary completion to ensure the smooth progress of subsequent steps. Furthermore, it provides definite markers to facilitate remodeling after subsequent aerial image acquisition to improve the 3D model.

[0007] The present invention is achieved through the following technical solutions.

[0008] This invention provides a method for filling in missing areas in geological mapping aerial image modeling, comprising the following steps: S1. Select missing areas: Select the missing areas in the 3D model obtained by integrating and modeling aerial images, where modeling cannot be completed due to the lack of aerial images. S2. Selecting adjacent regions: Expand the missing region according to a preset distance, and then delete the missing region from the model region after expanding the missing region to obtain adjacent regions; S3. Calculate similarity: Traverse and calculate the regions in the 3D model whose similarity to the adjacent regions is greater than a preset threshold within a preset ratio range, and obtain a list of similar regions arranged from high to low similarity. Take the first item in the list of similar regions as the reference region. S4. Generate missing region model: Using the reference region as a reference, a three-dimensional model of the missing region is generated using an imitation learning network; S5. Similarity determination: Combine the 3D models of adjacent regions and missing regions into a generated region. Calculate the similarity between the generated region and the reference region. If the similarity is less than the preset threshold, delete the current item in the similar region list and return to step S3 to retrieve the reference region. If the weak similarity is greater than the preset threshold, proceed to the next step. S6. Integration Marker: Integrate the 3D model of the generated region into the 3D model obtained by integrating the aerial image, and mark the 3D model part of the generated region with a specified color.

[0009] The preset distance is 1 to 1.3 times the minimum diameter of the missing region.

[0010] The preset ratio range is (1-α) to (1+α), where α is a preset ratio parameter, and the preset ratio parameter takes values ​​from 0.2 to 0.4.

[0011] The preset threshold is 0.7.

[0012] The similarity was calculated using the Mahalanobis distance.

[0013] The specified color is yellow, and the specified color is used to identify the model as being colored and rendered.

[0014] Before step S1, the following steps are also included: S0.1. Acquire multi-source images: Obtain aerial images and positioning data from data transmitted back by multiple UAVs; S0.2. Integrated Modeling: Based on aerial images and positioning data, integrated modeling is performed to obtain a three-dimensional model.

[0015] This invention also provides a system for completing missing regions in geological survey aerial image modeling, including a data caching module, a three-dimensional analysis module, a missing region generation model, and a similarity calculation model; The data caching module is used to cache the three-dimensional model obtained by integrating and modeling aerial images in the geological mapping aerial image modeling region missing completion method as described above. The three-dimensional analysis module is used to combine the missing area generation model and the similarity calculation model to realize steps S1 to S6 in the geological mapping aerial image modeling region missing completion method as described above; The missing area generation model is used to implement step S4 in the geological mapping aerial image modeling missing area completion method described above; The similarity calculation model is used to perform similarity calculations in steps S3 and S5 of the geological mapping aerial image modeling region missing completion method described above.

[0016] It also includes a communication module and a 3D modeling module; The communication module is used to implement step S0.1 as described above; The 3D modeling module is used to implement step S0.2 as described above.

[0017] The beneficial effects of this invention are as follows: based on neighboring region reference completion and using the imitation learning network generation method, it can better complete the missing regions, fully play the role of temporary completion to ensure the smooth progress of subsequent steps, and has a definite identifier to facilitate subsequent aerial image re-modeling to improve the 3D model. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating at least one embodiment of the present invention; Figure 2 This is a schematic diagram of module connections according to another embodiment of the present invention. Detailed Implementation

[0019] Example 1: The first embodiment of the present invention relates to, as follows Figure 1 The method for completing missing regions in geological mapping aerial image modeling, as shown, includes the following steps: S1. Select missing areas: Select the missing areas in the 3D model obtained by integrating and modeling aerial images, where modeling cannot be completed due to the lack of aerial images. S2. Selecting adjacent regions: Expand the missing region according to a preset distance, and then delete the missing region from the model region after expanding the missing region to obtain adjacent regions; S3. Calculate similarity: Traverse and calculate the regions in the 3D model whose similarity to the adjacent regions is greater than a preset threshold within a preset ratio range, and obtain a list of similar regions arranged from high to low similarity. Take the first item in the list of similar regions as the reference region. S4. Generate missing region model: Using the reference region as a reference, a three-dimensional model of the missing region is generated using an imitation learning network; S5. Similarity determination: Combine the 3D models of adjacent regions and missing regions into a generated region. Calculate the similarity between the generated region and the reference region. If the similarity is less than the preset threshold, delete the current item in the similar region list and return to step S3 to retrieve the reference region. If the weak similarity is greater than the preset threshold, proceed to the next step. S6. Integration Marker: Integrate the 3D model of the generated region into the 3D model obtained by integrating the aerial image, and mark the 3D model part of the generated region with a specified color.

[0020] Existing imitation learning networks are mainly used to generate action sequences or execute strategies, and do not belong to image processing or 3D generation methods. A typical example is the deep reinforcement learning strategy protection and defense method and device based on imitation learning and attention mechanism disclosed in Chinese Patent Application No. CN202110592426.0. However, in this application, the generated model can be regarded as a single-step execution strategy, and the model data is regarded as strategy data. Therefore, the existing imitation learning network is directly adopted, and the number of input and output parameters is adjusted accordingly to adapt to the above steps. Preliminary tests have shown that the generation of 3D models can also be completed. Since the imitation learning network in this application is mainly an adaptive modification of the existing technical solution, further optimization and improvement of the network structure needs to be carried out in the subsequent research and development stage. Therefore, the imitation learning network will not be described in detail in this application.

[0021] This implementation method is essentially based on the fundamental idea that terrain has similarities. It finds similar complete regions around the missing area and then builds a model of the missing area using these complete regions. The main reason why the similar complete regions cannot be directly copied proportionally to the missing area is that while terrain has similarities, it does not have consistency. Therefore, it can be assumed that when the surrounding areas are similar, the core area must also be similar. However, terrain changes have their inherent laws. For areas that have not been fully surveyed, these inherent laws may be unknown or cannot be expressed in a simple form, and it is difficult to summarize a usable form in the short term. Therefore, by imitating the digital black-box representation of the learning network, better results can be achieved. However, the constraint on the generated results mainly lies in step S5. Given that step S3 calculates the similarity of the surrounding areas, step S5 essentially calculates the similarity of the generated area. However, calculating the similarity only for the generated area has not yielded ideal results in actual testing, and ensuring data consistency is also quite cumbersome. Therefore, combining the similarity calculation of the surrounding areas and the generated area as a whole can ensure data consistency in a simple step, with excellent results.

[0022] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0023] The second embodiment of the present invention is largely the same as the first embodiment, mainly in the further optimization of implementation details, wherein the preset distance is 1 to 1.3 times the minimum diameter of the missing region.

[0024] Furthermore, the preset ratio range is (1-α) to (1+α), where α is a preset ratio parameter, and the preset ratio parameter takes values ​​from 0.2 to 0.4.

[0025] Furthermore, the preset threshold is 0.7.

[0026] Furthermore, similarity is calculated using Mahalanobis distance.

[0027] Furthermore, specify the color as yellow, and use the specified color identifier to color the rendering model.

[0028] Generally, after marking the missing areas with a specified color, if you need to update the 3D model for the missing areas, you can directly use the coloring data to accurately find the missing areas, delete the missing areas, and then overwrite them with the updated 3D model. The operation is extremely convenient (only one line of script is needed).

[0029] The third embodiment of the present invention is largely the same as the first embodiment, mainly in the form of further preferred improvements. Before step S1, the following steps are also included: S0.1. Acquire multi-source images: Obtain aerial images and positioning data from data transmitted back by multiple UAVs; S0.2. Integrated Modeling: Based on aerial images and positioning data, integrated modeling is performed to obtain a three-dimensional model.

[0030] It is readily apparent that this embodiment essentially further defines the 3D model obtained by integrating and modeling aerial images. Generally, preliminary geological mapping modeling is currently conducted using drone aerial photography. In practice, to improve efficiency, multiple drones are used in parallel from multiple directions, enabling the acquisition of images of most areas within a single flight time. Therefore, the multi-drone aerial photography method is particularly suitable for geological mapping, which involves acquiring aerial images from the data transmitted back by multiple drones. The integrated modeling can utilize publicly available technical solutions, such as the engineering geological monitoring system and multi-phase aerial mapping 3D geological modeling method disclosed in Chinese Patent Application No. 202210527039.3.

[0031] The fourth embodiment of the present invention relates to, for example Figure 2 The system shown is a geological mapping aerial image modeling region missing completion system, including a data caching module, a 3D analysis module, a missing region generation model, and a similarity calculation model; The data caching module is used to cache the three-dimensional model obtained by integrating and modeling aerial images as described in the first and second embodiments above. The 3D analysis module is used to combine the missing region generation model and the similarity calculation model to implement steps S1 to S6 in the first and second embodiments described above. The missing region generation model is used to implement step S4 as described in the first and second embodiments above. The similarity calculation model is used to implement the similarity calculation of steps S3 and S5 in the first and second embodiments described above.

[0032] All modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0033] The fifth embodiment of the present invention is largely the same as the fourth embodiment, mainly in the further optimization of the module settings, and also includes a communication module and a three-dimensional modeling module; The communication module is used to implement step S0.1 as described in the third embodiment above; The 3D modeling module is used to implement step S0.2 of the third embodiment described above.

[0034] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A method for filling in missing areas in geological survey aerial image modeling, characterized in that, Includes the following steps: S1. Select missing areas: Select the missing areas in the 3D model obtained by integrating and modeling aerial images, where modeling cannot be completed due to the lack of aerial images. S2. Selecting adjacent regions: Expand the missing region according to a preset distance, and then delete the missing region from the model region after expanding the missing region to obtain adjacent regions; S3. Calculate similarity: Traverse and calculate the regions in the 3D model whose similarity to the adjacent regions is greater than a preset threshold within a preset ratio range, and obtain a list of similar regions arranged from high to low similarity. Take the first item in the list of similar regions as the reference region. S4. Generate missing region model: Using the reference region as a reference, a three-dimensional model of the missing region is generated using an imitation learning network; S5. Similarity determination: Combine the 3D models of adjacent regions and missing regions into a generated region, calculate the similarity between the generated region and the reference region. If the similarity is less than the preset threshold, delete the current item in the similar region list and return to step S3 to retrieve the reference region. If the similarity is greater than the preset threshold, proceed to the next step. S6. Integration Marker: Integrate the 3D model of the generated region into the 3D model obtained by integrating the aerial image, and mark the 3D model part of the generated region with a specified color.

2. The method for completing missing areas in geological mapping aerial image modeling according to claim 1, characterized in that, The preset distance is 1 to 1.3 times the minimum diameter of the missing region.

3. The method for completing missing areas in geological mapping aerial image modeling according to claim 1, characterized in that, The preset ratio range is (1-α) to (1+α), where α is a preset ratio parameter, and the preset ratio parameter takes values ​​from 0.2 to 0.

4.

4. The method for completing missing areas in geological mapping aerial image modeling according to claim 1, characterized in that, The preset threshold is 0.

7.

5. The method for completing missing areas in geological mapping aerial image modeling according to claim 1, characterized in that, The similarity was calculated using the Mahalanobis distance.

6. The method for completing missing areas in geological mapping aerial image modeling according to claim 1, characterized in that, The specified color is yellow, and the specified color is used to identify the model as being colored and rendered.

7. The method for completing missing areas in geological mapping aerial image modeling according to claim 1, characterized in that, Before step S1, the following steps are also included: S0.

1. Acquire multi-source images: Obtain aerial images and positioning data from data transmitted back by multiple UAVs; S0.

2. Integrated Modeling: Based on aerial images and positioning data, integrated modeling is performed to obtain a three-dimensional model.

8. A system for completing missing areas in geological survey aerial image modeling, applicable to the method described in any one of claims 1-7, characterized in that, It includes a data caching module, a 3D analysis module, a missing region generation model, and a similarity calculation model; The data caching module is used to cache the three-dimensional model obtained by integrating and modeling aerial images in the geological mapping aerial image modeling region missing completion method as described above. The three-dimensional analysis module is used to combine the missing area generation model and the similarity calculation model to realize steps S1 to S6 in the geological mapping aerial image modeling region missing completion method as described above; The missing area generation model is used to implement step S4 in the geological mapping aerial image modeling missing area completion method described above; The similarity calculation model is used to perform similarity calculations in steps S3 and S5 of the geological mapping aerial image modeling region missing completion method described above.

9. A geological mapping aerial image modeling region missing completion system according to claim 8, characterized in that, It also includes a communication module and a 3D modeling module; The communication module is used to implement step S0.1 as described above; the three-dimensional modeling module is used to implement step S0.2 as described above.

Images