Satellite image digital surface model optimization method based on residual neural network

Abstract

The application discloses a satellite image digital surface model optimization method based on a residual neural network, belongs to the technical field of computer vision, photogrammetry and remote sensing, and converts a complex DSM reconstruction problem into an accurate correction problem of initial DSM errors by introducing a residual learning paradigm, so that the training difficulty and instability of a deep learning model are significantly reduced. A multi-scale residual network is constructed by fusing a hollow convolution, a channel attention mechanism and a double output head, and a composite target function is combined for training, so that high-fidelity recovery of a terrain macrostructure, micro details and sharp edges is realized. The confidence map output by the network itself is innovatively used to guide adaptive post-processing, so that the optimal trade-off between noise suppression and detail preservation in space is realized. Through an overlapping block and a cosine window fusion strategy, block effects during processing of a large range of images are effectively avoided, and product precision and visual quality are significantly improved, so that the method has a good engineering application prospect.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, photogrammetry and remote sensing, and particularly relates to a method for optimizing digital surface models of satellite images based on residual neural networks. Background Technology

[0002] Initial DSMs (Digital Surface Models) generated from satellite stereo image pairs using traditional dense matching (such as semi-global matching algorithms) generally suffer from significant noise, blurred building edges, distorted vegetation areas, and loss of small features, making them unsuitable for high-precision applications. While filtering-based post-processing methods can suppress noise, they often lead to edge smoothing and loss of detail. On the other hand, methods that rely entirely on deep learning to reconstruct the DSM end-to-end from the image ignore the valuable geometric priors contained in the initial DSM, resulting in high training difficulty and insufficient stability.

[0003] This invention proposes an optimization framework based on residual learning. Its core idea is to avoid directly learning the complex mapping from remote sensing imagery to a high-precision digital signal processing model (DSM). Instead, it utilizes a residual neural network to learn the residual relationship between "stereo imagery and initial DSM" and "high-precision ground truth DSM." This method transforms the optimization task into "precise correction" of the initial DSM, significantly reducing the model learning difficulty. It aims to achieve high-precision geometric correction, detail feature recovery, and adaptive noise suppression of the initial DSM, effectively improving the detail restoration capability and product usability of the DSM in complex scenes. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for optimizing digital surface models based on residual deep learning networks. This aims to solve the problems of noise, distortion and loss of detail in the initial DSM and achieve high-precision, high-fidelity adaptive optimization of DSM data.

[0005] In a first aspect, the present invention provides a method for optimizing digital surface models of satellite imagery based on residual neural networks, comprising the following steps:

[0006] Step S1: Acquire stereo image pair data from satellite images and perform initial dense matching to generate an initial digital surface model;

[0007] Step S2: Manually edit and optimize the initial digital surface model to obtain the true digital surface model;

[0008] Step S3: Using the stereo image pair data, the initial digital surface model, and the ground truth digital surface model as input, construct and train a multi-scale residual neural network, the network being configured to learn the residual between the initial digital surface model and the ground truth digital surface model;

[0009] Step S4: Divide the satellite image to be processed and the corresponding initial digital surface model into blocks to obtain several sub-image blocks;

[0010] Step S5: Input each sub-image block into the trained multi-scale residual neural network in sequence for processing to obtain the corresponding residual prediction sub-image;

[0011] Step S6: Stitch and merge all residual prediction subplots to obtain a complete residual prediction plot;

[0012] Step S7: Add the complete residual prediction map to the initial digital surface model to obtain a preliminary optimized digital surface model;

[0013] Step S8: Perform confidence-based adaptive post-processing on the initially optimized digital surface model to output the final optimized digital surface model.

[0014] Furthermore, in the satellite image digital surface model optimization method based on residual neural networks described in this invention, the process of constructing and training a multi-scale residual neural network in step S3 includes:

[0015] Step S31, Data Preprocessing and Alignment: Using the RPC parameters optimized by the regional network adjustment, the stereo image pairs are orthorectified to generate pixel-level aligned forward and backward orthorectified images.

[0016] Step S32: Construct a residual neural network with an encoder-decoder structure: The network includes an encoder, a decoder, and skip connections connecting the two; the encoder includes a multi-level downsampling module for extracting multi-scale features; the decoder includes a multi-level upsampling module for reconstructing spatial details; and a channel attention module is embedded in the skip connections to enhance the expression of key feature channels.

[0017] Step S33, Model Training and Optimization: The three-channel data consisting of the initial digital surface model, the forward orthophoto, and the rearward orthophoto are used as network inputs. The ground truth digital surface model is used as the supervision label, and the network is trained end-to-end with a composite loss function.

[0018] Furthermore, in the satellite image digital surface model optimization method based on residual neural network of the present invention, in step S32, the encoder includes at least five levels of downsampling modules, wherein at least two levels of downsampling modules employ dilated convolutional layers to expand the receptive field; and the network adopts a dual-output head design, including a main output head and an auxiliary output head, wherein the main output head is used to output the residual prediction map, and the auxiliary output head is used to output the confidence map representing the prediction determinism.

[0019] Furthermore, in the satellite image digital surface model optimization method based on residual neural network described in this invention, in step S33, the composite loss function... L total L1 norm loss L L1 Multi-scale structural similarity loss L MS-SSIM and gradient loss L Gradient The weighted summation is expressed as follows:

[0020] L total = λ 1· L L1 + λ 2· L MS-SSIM + λ 3· L Gradient

[0021] in, λ 1. λ 2. λ 3 represents a configurable weighting coefficient; L L1 Norm loss is responsible for constraining pixel-level absolute elevation errors, ensuring the basic accuracy of optimization. L MS-SSIM For multi-scale structural similarity loss, the similarity between the predicted results and the ground truth in terms of terrain structure, texture, etc. is measured by multiple scales to maintain the consistency of the macroscopic and microscopic structure of the terrain. L Gradient As a gradient loss, by calculating and minimizing the difference between the first-order gradient (or elevation change) between the predicted DSM and the ground DSM, the network is forced to learn and generate sharp elevation edges, thereby effectively suppressing the non-physical smoothing effect common in conventional methods at the edges of buildings and terrain.

[0022] Furthermore, in the satellite image digital surface model optimization method based on residual neural network of the present invention, step S33, training the network using a course learning strategy, specifically includes: in the first training stage, training the network using training sample blocks with a first size; and in the subsequent second training stage, training the network using training sample blocks with a second size, wherein the second size is smaller than the first size.

[0023] Furthermore, in the satellite image digital surface model optimization method based on residual neural network of the present invention, the confidence-based adaptive post-processing in step S8 includes: obtaining the confidence map output by the multi-scale residual neural network; dividing the processing area into high-confidence area and low-confidence area according to the confidence map; applying a higher filtering intensity to the low-confidence area than applying a higher filtering intensity to the high-confidence area; and for data missing areas, performing weighted interpolation on the optimized values ​​of surrounding pixels based on the confidence map to complete the filling.

[0024] Furthermore, in the satellite image digital surface model optimization method based on residual neural network described in this invention, the block processing in step S4 adopts a sliding window block strategy with overlapping regions; in the fusion processing in step S6, the overlapping predicted regions from different sub-image blocks are fused by weighted averaging based on cosine window function to eliminate inter-block stitching traces.

[0025] Secondly, the present invention provides a satellite image digital surface model optimization system based on residual neural network, including a digital surface model processing module, a residual neural network model construction module and an optimization output module;

[0026] The digital surface model processing module is used to acquire stereo image pair data of satellite images, perform initial dense matching, and generate an initial digital surface model.

[0027] The initial digital surface model is then manually edited and optimized to obtain a true digital surface model.

[0028] The satellite imagery to be processed and the corresponding initial digital surface model are divided into blocks to obtain several sub-image blocks;

[0029] The residual neural network model building module is used to construct and train a multi-scale residual neural network with the stereo image pair data, the initial digital surface model and the ground value digital surface model as inputs. The network is configured to learn the residual between the initial digital surface model and the ground value digital surface model.

[0030] Each sub-image block is sequentially input into the trained multi-scale residual neural network for processing to obtain the corresponding residual prediction sub-image; and all residual prediction sub-images are stitched together and fused to obtain the complete residual prediction image.

[0031] The complete residual prediction map is then added to the initial digital surface model to obtain a preliminary optimized digital surface model.

[0032] The optimization output module is used to perform confidence-based adaptive post-processing on the initially optimized digital surface model and output the final optimized digital surface model.

[0033] Thirdly, the present invention provides a satellite image digital surface model optimization device based on residual neural network, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to implement the satellite image digital surface model optimization method based on residual neural network as described in the first aspect when the computer program is executed.

[0034] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the satellite image digital surface model optimization method based on residual neural networks as described in the first aspect.

[0035] The satellite imagery digital surface model (DSM) optimization method based on residual neural networks described in this invention transforms the complex DSM reconstruction problem into a precise correction problem of the initial DSM error by introducing a residual learning paradigm, significantly reducing the training difficulty and instability of deep learning models. By constructing a multi-scale residual network integrating dilated convolution, channel attention mechanisms, and dual output heads, and training it with a composite objective function consisting of L1 loss, structural similarity loss, and gradient loss, high-fidelity restoration of macroscopic terrain structure, microscopic details, and sharp edges is achieved. The method innovatively utilizes the confidence map output by the network itself to guide adaptive post-processing, achieving an optimal spatial trade-off between noise suppression and detail preservation. Furthermore, the overlapping block segmentation and cosine window fusion strategy effectively avoids block artifacts when processing large-scale images. This method forms a complete, end-to-end high-precision DSM optimization solution, significantly improving product accuracy and visual quality, and possesses promising engineering application prospects. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the satellite image digital surface model optimization method based on residual neural network according to an embodiment of the present invention;

[0037] Figure 2 This is a comparison chart of the DSM effect before and after optimization in a typical urban scene, as described in an embodiment of the present invention.

[0038] Figure 3 This is a comparison chart of the DSM effect before and after optimization in a mountainous terrain scenario, as described in an embodiment of the present invention. Detailed Implementation

[0039] The satellite image digital surface model optimization method based on residual neural network described in this invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0040] This embodiment discloses a method for optimizing digital surface models of satellite imagery based on residual neural networks, including the following steps:

[0041] Step S1: Acquire stereo image pair data from satellite images and perform initial dense matching to generate an initial digital surface model;

[0042] Step S2: Manually edit and optimize the initial digital surface model to obtain the true digital surface model;

[0043] Step S3: Using the stereo image pair data, the initial digital surface model, and the ground truth digital surface model as input, construct and train a multi-scale residual neural network, the network being configured to learn the residual between the initial digital surface model and the ground truth digital surface model;

[0044] Step S4: Divide the satellite image to be processed and the corresponding initial digital surface model into blocks to obtain several sub-image blocks;

[0045] Step S5: Input each sub-image block into the trained multi-scale residual neural network in sequence for processing to obtain the corresponding residual prediction sub-image;

[0046] Step S6: Stitch and merge all residual prediction subplots to obtain a complete residual prediction plot;

[0047] Step S7: Add the complete residual prediction map to the initial digital surface model to obtain a preliminary optimized digital surface model;

[0048] Step S8: Perform confidence-based adaptive post-processing on the initially optimized digital surface model to output the final optimized digital surface model.

[0049] This embodiment uses stereo image pairs from the Gaofen-7 satellite as an example for illustration. Figure 1 As shown, the specific implementation process includes:

[0050] Acquire stereo image pairs (forward-looking and backward-looking panchromatic images) from the Gaofen-7 satellite and the initial DSM generated by a semi-global matching algorithm.

[0051] A regional network adjustment and orthorectification technique based on rational polynomial coefficient parameters is employed to eliminate the effects of terrain undulation and sensor geometric distortion, generating pixel-level precisely aligned forward-looking orthophotos, backward-looking orthophotos, and an initial DSM. After orthorectification, sensor geometric distortion and terrain displacement are eliminated, ensuring geometric consistency between the image and the DSM.

[0052] The orthophoto and the initial DSM are synchronously segmented into image block-DSM block pairs of the same size. The currently processed "forward image block + backward image block + initial DSM block" are used together as multimodal input.

[0053] The input data block is fed into a pre-trained multi-scale residual network for inference. This network learns detailed features through an encoder-decoder structure and outputs a residual map and a confidence map. The residual map represents the elevation difference (Δh) that needs to be corrected for the current DSM block; the confidence map represents the network's estimate of the reliability of the residual prediction results, with higher values ​​indicating more reliable predictions.

[0054] For low-confidence areas (such as dense vegetation and shadows), apply stronger filtering to suppress noise; for high-confidence areas (such as flat ground and regular buildings), maintain weak filtering to preserve sharp edges and details.

[0055] For areas with missing data, interpolation is performed to fill the gaps based on confidence weights. Once all data blocks have been processed, the data is then merged and output.

[0056] All processed DSM blocks are stitched together. For overlapping areas between blocks, a weighted fusion algorithm (such as a cosine window function) is used to smooth the transition, eliminating obvious seams or "block effects." The residual map Δh predicted by the network is added to the initial DSM, and after the above adaptive post-processing process, the final optimized high-precision DSM product is output.

[0057] In this embodiment, two sets of Gaofen-7 stereo image pairs were sampled for testing to verify the effectiveness of the current network. The tests were conducted in a typical urban scene (Example 1) and a mountainous terrain scene (Example 2). The processing results for the typical urban scene are as follows: Figure 2 As shown, the left image (a) is the test image of Sample 1, i.e., the initial DSM; the right image (b) is the predicted image of Sample 1, i.e., the optimized DSM. The mountainous terrain scene is as follows... Figure 3 As shown, the left image (a) is the test image of Sample 2, i.e. the initial DSM; the right image (b) is the predicted image of Sample 2, i.e. the optimized DSM.

[0058] Experimental results show that the method described in this embodiment can significantly improve the quality of DSM. In typical urban scenarios, this method achieves effective noise suppression through confidence-guided adaptive post-processing, outperforming traditional filtering methods and baseline deep learning models in terms of detail preservation and noise suppression. In mountainous terrain scenarios, the method described in this embodiment can significantly improve the ability to restore subtle terrain features such as steep slopes, ridgelines, and gullies, demonstrating its applicability for improving the details of DSM in mountainous areas.

[0059] The satellite image digital surface model optimization method based on residual neural network described in this invention not only improves the numerical accuracy of DSM, but also shows significant advantages in detail restoration, edge sharpening and noise suppression in complex scenes, providing an effective deep learning solution for the production of high-fidelity digital surface models.

[0060] This second embodiment discloses a satellite image digital surface model optimization system based on residual neural network, including a digital surface model processing module, a residual neural network model construction module, and an optimization output module;

[0061] The digital surface model processing module is used to acquire stereo image pair data of satellite images, perform initial dense matching, and generate an initial digital surface model.

[0062] The initial digital surface model is then manually edited and optimized to obtain a true digital surface model.

[0063] The satellite imagery to be processed and the corresponding initial digital surface model are divided into blocks to obtain several sub-image blocks;

[0064] The residual neural network model building module is used to construct and train a multi-scale residual neural network with the stereo image pair data, the initial digital surface model and the ground value digital surface model as inputs. The network is configured to learn the residual between the initial digital surface model and the ground value digital surface model.

[0065] Each sub-image block is sequentially input into the trained multi-scale residual neural network for processing to obtain the corresponding residual prediction sub-image; and all residual prediction sub-images are stitched together and fused to obtain the complete residual prediction image.

[0066] The complete residual prediction map is then added to the initial digital surface model to obtain a preliminary optimized digital surface model.

[0067] The optimization output module is used to perform confidence-based adaptive post-processing on the initially optimized digital surface model and output the final optimized digital surface model.

[0068] The implementation process of the satellite image digital surface model optimization system based on residual neural network described in this embodiment is the same as the satellite image digital surface model optimization method based on residual neural network described in Embodiment 1 above, and will not be repeated here.

[0069] This embodiment three discloses a satellite image digital surface model optimization device based on residual neural network, including a memory and a processor; the memory is used to store a computer program; the processor is used to implement the satellite image digital surface model optimization method based on residual neural network as described in embodiment one when the computer program is executed. The specific optimization method steps are the same as those in embodiment one, and will not be repeated here.

[0070] This embodiment four discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the satellite image digital surface model optimization method based on residual neural network as described in embodiment one. The specific optimization method steps are the same as those in embodiment one, and will not be repeated here.

[0071] The computer described in this application embodiment can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. The computer-readable storage medium can be any usable medium that a computer can read, or a data storage device such as a server or data center that integrates one or more usable media. The usable medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile optical disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)). The software formed by the computer's stored code can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media that are mature in the art.

[0072] In the various embodiments of this application, the functional modules can be integrated into one processing unit or module, or each module can exist physically separately, or two or more modules can be integrated into one unit or module. In the above embodiments, they can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated.

[0073] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for optimizing digital surface models of satellite imagery based on residual neural networks, characterized in that, Includes the following steps: Step S1: Acquire stereo image pair data from satellite images and perform initial dense matching to generate an initial digital surface model; Step S2: Manually edit and optimize the initial digital surface model to obtain the true digital surface model; Step S3: Using the stereo image pair data, the initial digital surface model, and the ground truth digital surface model as input, construct and train a multi-scale residual neural network, the network being configured to learn the residual between the initial digital surface model and the ground truth digital surface model; Step S4: Divide the satellite image to be processed and the corresponding initial digital surface model into blocks to obtain several sub-image blocks; Step S5: Input each sub-image block into the trained multi-scale residual neural network in sequence for processing to obtain the corresponding residual prediction sub-image; Step S6: Stitch and merge all residual prediction subplots to obtain a complete residual prediction plot; Step S7: Add the complete residual prediction map to the initial digital surface model to obtain a preliminary optimized digital surface model; Step S8: Perform confidence-based adaptive post-processing on the initially optimized digital surface model to output the final optimized digital surface model.

2. The satellite image digital surface model optimization method based on residual neural network according to claim 1, characterized in that, In step S3, the process of constructing and training a multi-scale residual neural network includes: Step S31, Data Preprocessing and Alignment: Using the RPC parameters optimized by the regional network adjustment, the stereo image pairs are orthorectified to generate pixel-level aligned forward and backward orthorectified images. Step S32: Construct a residual neural network with an encoder-decoder structure: The network includes an encoder, a decoder, and skip connections connecting the two; the encoder includes a multi-level downsampling module for extracting multi-scale features; the decoder includes a multi-level upsampling module for reconstructing spatial details; and a channel attention module is embedded in the skip connections to enhance the expression of key feature channels. Step S33, Model Training and Optimization: The three-channel data consisting of the initial digital surface model, the forward orthophoto, and the rearward orthophoto are used as network inputs. The ground truth digital surface model is used as the supervision label, and the network is trained end-to-end with a composite loss function.

3. The satellite image digital surface model optimization method based on residual neural network according to claim 2, characterized in that, In step S32, the encoder includes at least five downsampling modules, wherein at least two downsampling modules employ dilated convolutional layers to expand the receptive field; and the network adopts a dual-output head design, including a main output head and an auxiliary output head, wherein the main output head is used to output a residual prediction map, and the auxiliary output head is used to output a confidence map representing the prediction determinism.

4. The satellite image digital surface model optimization method based on residual neural network according to claim 2, characterized in that, In step S33, the composite loss function L total L1 norm loss L L1 Multi-scale structural similarity loss L MS-SSIM and gradient loss L Gradient The weighted summation is expressed as follows: L total = λ 1· L L1 + λ 2· L MS-SSIM + λ 3· L Gradient in λ 1. λ 2. λ 3 represents a configurable weighting coefficient.

5. The satellite image digital surface model optimization method based on residual neural network according to claim 2 or 3, characterized in that, In step S33, the network is trained using a course learning strategy, specifically including: in the first training stage, the network is trained using training sample blocks with a first size; in the subsequent second training stage, the network is trained using training sample blocks with a second size, where the second size is smaller than the first size.

6. The satellite image digital surface model optimization method based on residual neural network according to claim 1, characterized in that, The confidence-based adaptive post-processing described in step S8 includes: Obtain the confidence map of the output of the multi-scale residual neural network; Based on the confidence map, the processing area is divided into a high-confidence area and a low-confidence area; The filtering intensity applied to the low-confidence region is higher than the filtering intensity applied to the high-confidence region; For regions with missing data, weighted interpolation is performed on the optimized values ​​of surrounding pixels based on the confidence map to complete the filling.

7. The satellite image digital surface model optimization method based on residual neural network according to claim 1 or 6, characterized in that, The block segmentation process in step S4 adopts a sliding window block segmentation strategy with overlapping regions; the fusion process in step S6 uses a weighted average based on a cosine window function to fuse the overlapping predicted regions from different sub-image blocks in order to eliminate inter-block stitching traces.

8. A satellite imagery digital surface model optimization system based on residual neural networks, characterized in that: It includes a digital surface model processing module, a residual neural network model construction module, and an optimization output module; The digital surface model processing module is used to acquire stereo image pair data of satellite images, perform initial dense matching, and generate an initial digital surface model. The initial digital surface model is then manually edited and optimized to obtain a true digital surface model. The satellite imagery to be processed and the corresponding initial digital surface model are divided into blocks to obtain several sub-image blocks; The residual neural network model building module is used to construct and train a multi-scale residual neural network with the stereo image pair data, the initial digital surface model and the ground value digital surface model as inputs. The network is configured to learn the residual between the initial digital surface model and the ground value digital surface model. Each sub-image block is sequentially input into the trained multi-scale residual neural network for processing to obtain the corresponding residual prediction sub-image; and all residual prediction sub-images are stitched together and fused to obtain the complete residual prediction image. The complete residual prediction map is then added to the initial digital surface model to obtain a preliminary optimized digital surface model. The optimization output module is used to perform confidence-based adaptive post-processing on the initially optimized digital surface model and output the final optimized digital surface model.

9. A satellite image digital surface model optimization device based on residual neural network, characterized in that, It includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the satellite image digital surface model optimization method based on residual neural network according to any one of claims 1-7 when the computer program is executed.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the satellite image digital surface model optimization method based on residual neural networks as described in any one of claims 1-7.

Images