Remote sensing image fusion method of multi-scale attention deep convolutional network based on 3D convolution

Abstract

The invention discloses a remote sensing image fusion method of a multi-scale attention deep convolutional network based on 3D convolution, and the method comprises the steps of fusing the high spectral resolution of a multispectral image with the high spatial resolution of a panchromatic image, so as to obtain a multispectral image with the high spatial resolution and the high spectral resolution. A 3D multi-scale attention deep convolutional network model (MSAC-Net) is designed by using a U-Net network structure framework in deep learning. In order to retain the spectral resolution in multiple spectrums, the whole model uses 3D convolution, and feature extraction is carried out on information in the spectral dimension; in order to capture more spatial details, an attention mechanism is introduced at the jump connection of the model to learn regional details. In the decoding stage of the model, a plurality of reconstruction layers containing multi-scale space information are introduced to calculate a reconstruction result, the model is encouraged to learn multi-scale representation of different levels, and multi-level reference is provided for a final fusion result. And the fusion result of the final image is effectively improved.

Description

technical field [0001] The invention belongs to the field of information technology and relates to image processing technology, in particular to a remote sensing image fusion method based on a 3D convolution multi-scale attention depth convolution network. Background technique [0002] Remote sensing satellites can acquire panchromatic (PAN) images of the same scene while taking multispectral (MS) images. Among them, multispectral images are rich in spectral information, but have low spatial resolution and poor clarity, while panchromatic images has high spatial resolution but low spectral resolution; the spatial and spectral resolutions of the two contradict each other. Combining the advantages of the two to obtain a multispectral image with high spatial and spectral resolution is currently a great demand. [0003] At present, deep learning has been widely used in various research fields, providing a new solution to various fields. Among them, in the field of deep learnin...

AI Technical Summary

Problems solved by technology

[0006] In order to reduce the workload of image processing and improve the accuracy of image fusion by making full use of the correlation between each pixel and each band of the multispectral image and the high spatial resolution of the panchromatic image, the purpose of the present invention is to: Provides a remote sensing image fusion method based on deep learning 3D multi-scale attention deep convolutional network (MSAC-Net), adopts 3D convolution method, uses deep learning model while preserving the spectral details of multispectral images, using attention The force mechanism extracts spatial details from panchromatic images, and learns the final result with multiple intermediate-scale results to obtain the required high-resolution multispectral image, to solve the problem of incomplete fusion of remote sensing images and the quality of fusion in the existing technology. and the problem of poor fusion effect

Images