Normalized digital surface model refinement method based on deep learning

Abstract

The embodiment of the invention discloses a normalized digital surface model refinement method based on deep learning, and relates to the technical field of photogrammetry, and the method comprises the steps: processing a top image of a pyramid image through a deep learning model, and obtaining a digital surface model estimation result; a next layer of image is selected for processing to obtain an initial estimation result, a smooth image, an edge image, a difference image and a directional diagram are generated, first refining, second refining, third refining and fourth refining are carried out, and refining results are obtained respectively; processing the fourth refining result and the initial estimation result based on a directional diagram to obtain a final refining result; and outputting a corresponding digital surface model estimation result under the condition that the resolution corresponding to the final refining result is equal to the initial remote sensing image. According to the method, the complex relation between pixels can be comprehensively captured, the sensitivity of the model to image details is improved, the precision of an estimation result can be gradually improved through multi-step refinement, and a ground object height distribution diagram with higher precision can be generated.

Description

Technical Field

[0001] This application relates to the field of photogrammetry technology, and particularly to a method for refining a normalized digital surface model based on deep learning. Background Art

[0002] With the rapid development of urbanization in China, the urban built environment is changing rapidly. While the social economy is continuously developing, it also brings new challenges to urban monitoring. In photogrammetry, by using the normalized digital surface model estimation method, the normalized digital surface model of the target area can be obtained, from which information such as the outline and height of buildings can be extracted, providing a large-scale technology for urban development monitoring. However, due to the characteristics of optical satellite images, such as large range, multiple levels, and complex semantics, traditional estimation methods are difficult to directly obtain the normalized digital surface model of the target area.

[0003] Currently, the estimation of the normalized digital surface model can be achieved by using deep learning algorithms. However, since remote sensing images often cover a large area and have a large resolution size, it is difficult to achieve refined estimation of the normalized digital surface model across the entire image range through a single match. Summary of the Invention

[0004] An embodiment of this application provides a method for refining a normalized digital surface model based on deep learning to solve the defects of the above-mentioned related technologies. The technical solution is as follows:

[0005] In a first aspect, an embodiment of this application provides a method for refining a normalized digital surface model based on deep learning, including:

[0006] Obtain the initial remote sensing image of the target area and process it to obtain a pyramid image;

[0007] Based on the top-layer image of the pyramid image, obtain the estimated result of the normalized digital surface model of the top layer through the trained deep learning model;

[0008] Select the next-layer image, obtain the initial estimated result of the normalized digital surface model of the corresponding layer through the deep learning model, and generate a smoothing map, an edge map, a difference map, and a direction map;

[0009] Based on the difference map and the direction map, perform the first refinement on the initial estimated result and output the first refinement result; perform the second refinement based on the direction map and output the second refinement result; perform the third refinement based on the edge map and output the third refinement result; perform the fourth refinement based on the smoothing map and output the fourth refinement result;

[0010] Based on the direction map, process the fourth refinement result and the initial estimated result to obtain the final refinement result;

[0011] If the resolution corresponding to the final refinement result is less than that of the initial remote sensing image, go to the step of selecting the next-layer image; otherwise, output the normalized digital surface model estimation result corresponding to the final refinement result.

[0012] In an alternative embodiment of the first aspect, the obtaining of the initial remote sensing image of the target area and processing to obtain the pyramid image includes:

[0013] Obtain the initial remote sensing image of the target area, perform downsampling layer by layer on the basis of the initial remote sensing image based on a preset resolution multiple and a preset number of layers to obtain images of the preset number of layers, and construct the pyramid image arranged from the bottom layer to the top layer in the order of the resolution of each layer of image from large to small;

[0014] Wherein, the resolution of each layer of image in the pyramid image is a preset multiple of the immediately adjacent upper layer of image, and the resolution of the bottom layer image is the same as that of the initial remote sensing image.

[0015] In an alternative embodiment of the first aspect, the generating of the smooth map, the edge map, the difference map and the direction map includes:

[0016] In the image corresponding to the initial estimation result, judge the height difference between the central pixel and other pixels within the preset pixel window, mark the central pixel without height difference as a smooth pixel, generate the smooth map according to the distribution of the smooth pixels, and generate the difference map based on the distribution of the height difference;

[0017] Based on the other pixel with the smallest height difference from the central pixel within the preset pixel window, mark the direction in which the central pixel points to the other pixel with the smallest height difference, and generate the direction map based on the distribution of the directions;

[0018] If the height difference between every two adjacent pixels is greater than the height difference threshold, mark the corresponding pixels as edge pixels and generate the edge map.

[0019] In an alternative embodiment of the first aspect, the first refinement of the initial estimation result based on the difference map and the direction map and outputting the first refinement result includes:

[0020] Perform weighted calculation on the initial estimation result by combining the difference map and the direction map, and apply the formula:

[0021]

[0022] Wherein, is the first refinement result, H map is the initial estimation result, Diff mapis the difference map, is the weight corresponding to each direction in the direction map, sum() represents summation, and Dir op is the direction operation parameter.

[0023] In an alternative scheme of the first aspect, the second refinement is performed based on the direction map, and the second refinement result is output, including:

[0024] Obtain the maximum direction weight in the direction map, perform weighted calculation on the first refinement result, and output the second refinement result, applying the formula:

[0025]

[0026] where, is the second refinement result, is the maximum direction weight.

[0027] In an alternative scheme of the first aspect, the third refinement is performed based on the edge map, and the third refinement result is output, including:

[0028] Determine the distribution of edge pixels according to the edge map;

[0029] If it is an edge pixel, perform weighted calculation in combination with the second refinement result corresponding to the edge pixel. If it is a pixel of other types, do not use the second refinement result corresponding to the pixel of other types, applying the formula:

[0030]

[0031] where, is the third refinement result, e map is the edge map.

[0032] In an alternative scheme of the first aspect, the fourth refinement is performed based on the smoothed map, and the fourth refinement result is output, including:

[0033] Determine the distribution of smoothed pixels according to the smoothed map;

[0034] If it is a smoothed pixel, perform an average convolution operation on the pixels around the smoothed pixel of the third refinement result, applying the formula:

[0035]

[0036] where, is the fourth refinement result, S ,ap is the smoothed map, and ave() is the average convolution.

[0037] In an alternative solution of the first aspect, processing the fourth refinement result and the initial estimation result based on the direction diagram to obtain a final refinement result includes:

[0038] Obtain the maximum direction weight in the direction diagram;

[0039] If the maximum direction weight is less than 0.5, output the initial estimation result as the final refinement result;

[0040] Otherwise, perform a weighted calculation on the fourth refinement result and the initial estimation result based on the maximum direction weight to obtain the final refinement result, and apply the formula:

[0041]

[0042] where is the final refinement result.

[0043] In a second aspect, an embodiment of the present application further provides a deep learning-based normalized digital surface model refinement device, including:

[0044] A pyramid image generation module for obtaining an initial remote sensing image of a target area and processing it to obtain a pyramid image;

[0045] A top-layer image processing module for processing the top-layer image of the pyramid image through a trained deep learning model to obtain an estimated result of the top-layer normalized digital surface model;

[0046] An image processing module for selecting the next-layer image, processing it through the deep learning model to obtain an initial estimation result of the normalized digital surface model of the corresponding layer, and generating a smoothing map, an edge map, a difference map, and a direction map;

[0047] A refinement processing module for performing a first refinement on the initial estimation result based on the difference map and the direction map and outputting a first refinement result; the refinement processing module is further configured to perform a second refinement based on the direction map and output a second refinement result; the refinement processing module is further configured to perform a third refinement based on the edge map and output a third refinement result; the refinement processing module is further configured to perform a fourth refinement based on the smoothing map and output a fourth refinement result;

[0048] The refinement processing module is further configured to process the fourth refinement result and the initial estimation result based on the direction map to obtain a final refinement result;

[0049] The result output module is used to go to the step of selecting the next-layer image of the image processing module when the resolution corresponding to the final refined result is less than that of the initial remote sensing image; otherwise, the result output module outputs the normalized digital surface model estimation result corresponding to the final refined result.

[0050] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the method provided in the first aspect or any implementation manner of the first aspect of the embodiments of the present application.

[0051] The beneficial effects brought by the technical solutions provided by some embodiments of the present application at least include:

[0052] A method for refining a normalized digital surface model based on deep learning provided by an embodiment of the present application can comprehensively capture the complex relationships between pixels, improve the sensitivity of the model to image details, and gradually improve the accuracy of the estimation result through multi-step refinement steps, thereby being able to generate a more accurate ground object height distribution map. In addition, by applying a deep learning model at multiple resolution levels, the adaptability of the model to different-scale features can be improved. Combining pyramid images and deep learning technologies can achieve efficient and accurate normalized digital surface model estimation and obtain a more accurate ground object height distribution map. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0054] Figure 1 is a flowchart of a method for refining a normalized digital surface model based on deep learning provided by an embodiment of the present application;

[0055] Figure 2 is a schematic diagram of pyramid image processing of a method for refining a normalized digital surface model based on deep learning provided by an embodiment of the present application;

[0056] Figure 3 is a schematic diagram of the structure of a device for refining a normalized digital surface model based on deep learning provided by an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions and advantages of this application clearer, the following will clearly and completely describe the technical solutions in this application in conjunction with the accompanying drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0059] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned accompanying drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other steps or modules inherent to these processes, methods, products or devices.

[0060] It should be noted that the terms "first / second" involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects. Understandably, "first / second" can be interchanged in a specific order or sequence when permitted. It should be understood that the objects distinguished by "first / second" can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those described or illustrated here.

[0061] It should be noted that Digital Surface Model Refinement refers to the process of further processing and optimizing the preliminary Digital Surface Model (DSM) obtained through various remote sensing technologies. The preliminary DSM usually contains the height information of the terrain surface and all objects on the ground surface, such as buildings, trees, etc., but may have problems such as insufficient accuracy, noise interference or data holes.

[0062] Currently, some related technologies can analyze remote sensing images through a deep learning network to more efficiently extract the height distribution of ground objects in the target area. However, since remote sensing images often cover a large geographical range and the image resolution is large, it is difficult to directly process them through a deep learning model, which has high requirements for the training, operation and efficiency of the model, and it is difficult to train a deep learning model with high precision and low cost to meet the requirements.

[0063] When this application innovatively applies the deep learning model to the entire satellite image, it also combines the pyramid strategy for multi-level matching and result refinement, enabling the deep learning model to start refining from the low-resolution image in the pyramid image, reducing the computational load of the model. Through multi-scale processing, the model can better capture image features at different scales, thereby improving the accuracy of the refined normalized digital surface model result.

[0064] The following will elaborate on this application in detail with specific embodiments.

[0065] Next, in combination with Figure 1 , this application will introduce a method for refining the normalized digital surface model based on deep learning provided by the embodiments of this application. For details, please refer to Figure 1 , Figure 1 FIG. Figure 1 shows a schematic flowchart of a method for refining the normalized digital surface model based on deep learning provided by the embodiments of this application. As

[0066] shown, this method includes the following steps:

[0067] S101, Obtain the initial remote sensing image of the target area and process it to obtain the pyramid image;

[0068] S102, Based on the top layer image of the pyramid image, process it through the trained deep learning model to obtain the estimated result of the normalized digital surface model of the top layer;

[0069] S103, Select the next layer of image, process it through the deep learning model to obtain the initial estimated result of the normalized digital surface model of the corresponding layer, and generate a smooth map, an edge map, a difference map, and a direction map;

[0070] S104, Based on the difference map and the direction map, perform the first refinement on the initial estimated result and output the first refinement result; perform the second refinement based on the direction map and output the second refinement result; perform the third refinement based on the edge map and output the third refinement result; perform the fourth refinement based on the smooth map and output the fourth refinement result;

[0071] S105, Based on the direction map, process the fourth refinement result and the initial estimated result to obtain the final refinement result;

[0072] In some embodiments, in S101, the obtaining the initial remote sensing image of the target area and processing it to obtain the pyramid image includes:

[0073] Obtain the initial remote sensing image of the target area, perform layer-by-layer downsampling on the basis of the initial remote sensing image based on a preset resolution multiple and a preset number of layers to obtain images of the preset number of layers, and construct the pyramid images arranged from the bottom layer to the top layer in the order of the resolution of each layer of images from large to small;

[0074] Among them, the resolution of each layer of images in the pyramid images is a preset multiple of the immediately adjacent upper layer of images, and the resolution of the bottom layer image is the same as that of the initial remote sensing image.

[0075] In some embodiments, in S103, the generating of the smooth map, the edge map, the difference map and the direction map includes:

[0076] In the image corresponding to the initial estimation result, judge the height difference between the central pixel and other pixels within the preset pixel window through the preset pixel window, mark the central pixel without height difference as a smooth pixel, generate the smooth map according to the distribution of the smooth pixels, and generate the difference map based on the distribution of the height difference;

[0077] Based on the other pixel with the smallest height difference between the central pixel and within the preset pixel window, mark the direction in which the central pixel points to the other pixel with the smallest height difference, and generate the direction map based on the distribution of the directions;

[0078] If the height difference between every two adjacent pixels is greater than the height difference threshold, mark the corresponding pixels as edge pixels and generate the edge map.

[0079] Specifically, the judgment of edge pixels can apply the formula:

[0080]

[0081] where i, j represent two adjacent pixels, h i represents the height value corresponding to pixel i, h j represents the height value corresponding to pixel j, and d is a preset parameter; [] represents the Iverson bracket, that is, the pixel value that satisfies the condition in the bracket is 1, and the pixel value that does not satisfy the condition is 0.

[0082] Exemplarily, the pyramid image of S101 is as shown Figure 2 on the left. Relationship mining and edge preservation can be performed according to the images of the pyramid images to obtain the normalized digital surface model estimation result, as well as the smooth map, the edge map, the difference map and the direction map respectively.

[0083] In some embodiments, the deep learning model can be pseudo-labeled based on the refined normalized digital surface model estimation result nDSM and the nDSM estimation result before refinement. Pseudo-Labeling is a semi-supervised learning method that can utilize unlabeled data to enhance the training effect of the model, can be used to alleviate the problem of scarce labeled data, and can fully utilize a large amount of unlabeled data.

[0084] In some embodiments, in S104, it specifically includes the following sub-steps:

[0085] S1041, first refining the initial estimation result based on the difference map and the direction map, and outputting a first refined result, including:

[0086] Performing weighted calculation on the initial estimation result by combining the difference map and the direction map, and applying the formula:

[0087]

[0088] where, is the first refined result, H map is the initial estimation result, Diff map is the difference map, is the weight corresponding to each direction in the direction map, sum() represents summation, Dir op is the direction operation parameter.

[0089] S1042, second refining based on the direction map and outputting a second refined result, including:

[0090] Obtaining the maximum direction weight in the direction map, performing weighted calculation on the first refined result, and outputting a second refined result, and applying the formula:

[0091]

[0092] where, is the second refined result, is the maximum direction weight.

[0093] S1043, third refining based on the edge map and outputting a third refined result, including:

[0094] Determining the distribution of edge pixels according to the edge map;

[0095] If it is an edge pixel, then perform weighted calculation by combining the second refined result corresponding to the edge pixel. If it is a pixel of other types, then do not use the second refined result corresponding to the pixel of other types, and apply the formula:

[0096]

[0097] Among them, is the third refinement result, E map is the edge map.

[0098] S1044, perform the fourth refinement based on the smoothed map and output the fourth refinement result, including:

[0099] Determine the distribution of smoothed pixels according to the smoothed map;

[0100] If it is a smoothed pixel, perform an average convolution operation on the pixels around the smoothed pixel of the third refinement result, and apply the formula:

[0101]

[0102] Among them, is the fourth refinement result, S map is the smoothed map, and ave() is the average convolution.

[0103] In some embodiments, in S105, the process of processing the fourth refinement result and the initial estimation result based on the direction map to obtain the final refinement result includes:

[0104] Obtain the maximum direction weight in the direction map;

[0105] If the maximum direction weight is less than 0.5, no confidence weighting is performed, and the initial estimation result is directly output as the final refinement result;

[0106] Otherwise, confidence weighting is performed. Specifically, based on the maximum direction weight, weighted calculation is performed on the fourth refinement result and the initial estimation result to obtain the final refinement result, and the formula is applied:

[0107]

[0108] Among them, is the final refinement result.

[0109] Furthermore, perform the steps of S106, and compare the resolution corresponding to the final refinement result and the resolution of the initial remote sensing image;

[0110] In the case where the resolution corresponding to the final refinement result is less than the initial remote sensing image, go to S103 and subsequent steps;

[0111] In the case where the resolution corresponding to the final refinement result is equal to the initial remote sensing image, the final refinement result can be output.

[0112] Furthermore, a ground object height distribution map of the corresponding target area can be generated in combination with the final refinement result.

[0113] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the method embodiment of the present application.

[0114] Next, please refer to Figure 3 , which is a schematic structural diagram of a deep learning-based normalized digital surface model refinement apparatus provided by an exemplary embodiment of the present application. The apparatus can be implemented as all or part of a terminal through software, hardware, or a combination of both, and can also be integrated as an independent module on a server. A deep learning-based normalized digital surface model refinement apparatus in an embodiment of the present application can be applied to a terminal or the cloud. The apparatus 30 includes a pyramid image generation module 301, a top-layer image processing module 302, an image processing module 303, a refinement processing module 304, and a result output module 305, where:

[0115] The pyramid image generation module 301 is configured to obtain an initial remote sensing image of a target area and process it to obtain a pyramid image;

[0116] The top-layer image processing module 302 is configured to process the top-layer image of the pyramid image through a trained deep learning model to obtain an estimated result of the normalized digital surface model at the top layer;

[0117] The image processing module 303 is configured to select the next-layer image, process it through the deep learning model to obtain an initial estimated result of the normalized digital surface model of the corresponding layer, and generate a smoothing map, an edge map, a difference map, and a direction map;

[0118] The refinement processing module 304 is configured to perform a first refinement on the initial estimated result based on the difference map and the direction map, and output a first refinement result; the refinement processing module 304 is further configured to perform a second refinement based on the direction map and output a second refinement result; the refinement processing module 304 is further configured to perform a third refinement based on the edge map and output a third refinement result; the refinement processing module 304 is further configured to perform a fourth refinement based on the smoothing map and output a fourth refinement result;

[0119] The refinement processing module 304 is further configured to process the fourth refinement result and the initial estimated result based on the direction map to obtain a final refinement result;

[0120] The result output module 305 is configured to, when the resolution corresponding to the final refined result is less than that of the initial remote sensing image, go to the step of selecting the next-layer image of the image processing module 303; otherwise, the result output module 305 outputs the normalized digital surface model estimation result corresponding to the final refined result.

[0121] It should be noted that when the device 30 provided in the above embodiment executes a method for refining a normalized digital surface model based on deep learning, only the above division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the embodiment of a method for refining a normalized digital surface model based on deep learning belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be elaborated here.

[0122] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method in any of the above embodiments are implemented.

[0123] Please refer to Figure 4 , which is a structural block diagram of an electronic device provided in an embodiment of the present application.

[0124] As Figure 4 shown, the electronic device 400 includes a processor 401 and a memory 402.

[0125] In an embodiment of the present application, the processor 401 is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine. The processor 401 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).

[0126] The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0127] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments of the present application, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 401 to implement the method in the embodiments of the present application.

[0128] In some embodiments, the electronic device 400 further includes: a peripheral device interface 403 and at least one peripheral device 404. The processor 401, the memory 402, and the peripheral device interface 403 may be connected through a bus or signal lines. Each peripheral device 404 may be connected to the peripheral device interface 403 through a bus, signal lines, or a circuit board. Specifically, the peripheral device 404 includes: a display screen, a camera, and an audio circuit. The peripheral device interface 403 may be used to connect at least one I / O (Input / Output) related peripheral device to the processor 401 and the memory 402.

[0129] In some embodiments of the present application, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 may be implemented on a separate chip or circuit board. The embodiments of the present application do not make specific limitations in this regard.

[0130] The block diagram of the electronic device structure shown in the embodiments of the present application does not constitute a limitation on the electronic device 400. The electronic device 400 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component layout.

[0131] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method in any of the foregoing embodiments are implemented. Among them, the computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the part that contributes to the related technologies can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A normalized digital surface model refinement method based on deep learning, characterized in that: include: Obtain the initial remote sensing image of the target area and process it to obtain a pyramid image; Processing the top image of the pyramid image using a trained deep learning model to obtain a normalized digital surface model estimation result of the top layer; Selecting the next layer of images, processing them with the deep learning model to obtain an initial estimation result of the normalized digital surface model of the corresponding layer, and generating a smoothing map, an edge map, a difference map, and a direction map; Performing a first refinement on the initial estimation result based on the difference map and the direction map, and outputting a first refinement result; Performing a second refinement based on the directional map, and outputting a second refinement result; Performing a third refinement based on the edge map, and outputting a third refinement result; Performing a fourth refinement based on the smoothing graph, and outputting a fourth refinement result; Processing the fourth refinement result and the initial estimation result based on the directional map to obtain a final refinement result; If the resolution corresponding to the final refinement result is smaller than that of the initial remote sensing image, proceed to the step of selecting the next layer of images; Otherwise, the normalized digital surface model estimation result corresponding to the final refinement result is output.

2. The method for refining a normalized digital surface model based on deep learning according to claim 1, characterized in that: The step of obtaining an initial remote sensing image of the target area and processing the image to obtain a pyramid image includes: Acquire an initial remote sensing image of the target area, perform layer-by-layer downsampling on the basis of the initial remote sensing image based on a preset resolution multiple and a preset number of layers to obtain an image of the preset number of layers, and construct the pyramid image arranged from the bottom layer to the top layer in descending order of the resolution of each layer of the image; The resolution of each layer of the pyramid image is a preset multiple of the image of the immediately previous layer, and the bottom layer image has the same resolution as the initial remote sensing image.

3. The method for refining a normalized digital surface model based on deep learning according to claim 1, characterized in that: The generating of the smoothing map, the edge map, the difference map and the direction map comprises: In the image corresponding to the initial estimation result, a height difference between a central pixel and other pixels in the preset pixel window is determined through a preset pixel window, a central pixel without a height difference is recorded as a smooth pixel, the smoothing map is generated according to the distribution of the smooth pixels, and the difference map is generated based on the distribution of the height difference; Mark the direction in which the central pixel points to the other pixels with the smallest height difference based on other pixels with the smallest height difference between the central pixel and the preset pixel window, and generate the direction map based on the distribution of directions; If the height difference between every two adjacent pixels is greater than a height difference threshold, the corresponding pixels are marked as edge pixels to generate the edge map.

4. The method for refining a normalized digital surface model based on deep learning according to claim 3, characterized in that: The first refining of the initial estimation result based on the difference map and the direction map, and outputting a first refinement result, comprises: The initial estimation result is weightedly calculated in combination with the difference map and the direction map, and the formula is applied: in, is the first refinement result, H map is the initial estimation result, Diff map is the difference map, is the weight corresponding to each direction in the directional map, sum() represents the sum, Dir op Direction operation parameter.

5. The method for refining a normalized digital surface model based on deep learning according to claim 4, characterized in that: The performing a second refinement based on the directional map and outputting a second refinement result includes: Obtain the maximum directional weight in the directional map, perform weighted calculation on the first refinement result, output the second refinement result, and apply the formula: in, is the second refinement result, is the maximum directional weight.

6. The method for refining a normalized digital surface model based on deep learning according to claim 5, characterized in that: The performing a third thinning based on the edge map and outputting a third thinning result includes: determining the distribution of edge pixels according to the edge map; If it is an edge pixel, the second refinement result corresponding to the edge pixel is combined for weighted calculation. If it is another type of pixel, the second refinement result corresponding to the other type of pixel is not used, and the formula is applied: in, is the third refinement result, E map is the edge map.

7. The method for refining a normalized digital surface model based on deep learning according to claim 6, characterized in that: The performing a fourth refinement based on the smoothing graph and outputting a fourth refinement result comprises: Determining the distribution of smoothed pixels according to the smoothed map; If it is a smooth pixel, an average convolution operation is performed on the pixels around the smooth pixel of the third refinement result, and the formula is applied: in, is the fourth refinement result, S map is the smoothed image, and ave() is the average convolution.

8. The method for refining a normalized digital surface model based on deep learning according to claim 7, characterized in that: The processing of the fourth refinement result and the initial estimation result based on the directional map to obtain a final refinement result includes: Obtaining the maximum directional weight in the directional pattern; If the maximum directional weight is less than 0.5, outputting the initial estimation result as the final refinement result; Otherwise, the fourth refinement result and the initial estimation result are weighted based on the maximum direction weight to obtain the final refinement result, applying the formula: in, is the final refinement result.

9. A normalized digital surface model refinement device based on deep learning, characterized in that: include: The pyramid image generation module is used to obtain the initial remote sensing image of the target area and process it to obtain the pyramid image; A top-level image processing module, used for obtaining a normalized digital surface model estimation result of the top level by processing the top-level image of the pyramid image through a trained deep learning model; An image processing module is used to select the next layer of images, obtain the initial estimation result of the normalized digital surface model of the corresponding layer through the deep learning model, and generate a smoothing map, an edge map, a difference map and a direction map; A refinement processing module, configured to perform a first refinement on the initial estimation result based on the difference map and the direction map, and output a first refinement result; the refinement processing module is further configured to perform a second refinement based on the direction map, and output a second refinement result; the refinement processing module is further configured to perform a third refinement based on the edge map, and output a third refinement result; the refinement processing module is further configured to perform a fourth refinement based on the smoothing map, and output a fourth refinement result; The refinement processing module is further used to process the fourth refinement result and the initial estimation result based on the directional map to obtain a final refinement result; A result output module, used for, when the resolution corresponding to the final refinement result is smaller than that of the initial remote sensing image, switching to the step of selecting the next layer of images in the image processing module; Otherwise, the result output module outputs the normalized digital surface model estimation result corresponding to the final refinement result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images