A Deep Learning and Model-Driven Multispectral and Hyperspectral Image Fusion Method

Abstract

The invention discloses a model-driven deep multi / hyperspectral image fusion method. The technical key of the present invention lies in the new model of network construction and network training (or testing). First, in the construction stage of the deep network, according to the generation mechanism of low-resolution images, a new multi- / hyperspectral image fusion model is established, an iterative solution algorithm for the model is designed, and then the iterative steps of the algorithm are correspondingly expanded into network modules , to establish a multi / hyperspectral image fusion network (MS / HS Fusion Net, MHF-net); in the training and testing phase of the network, the present invention provides a training mode in which response coefficients and low-resolution images are simultaneously input into the network, which is the first invention An effective method for deep multi / hyperspectral image fusion in scenarios where the response coefficients of training / test data are inconsistent. The deep multi / hyperspectral image fusion network of the present invention has obvious interpretability, generalization and strong practical application significance.

Description

technical field [0001] The invention belongs to the technical fields of image processing, remote sensing and deep learning, and particularly relates to a multispectral and hyperspectral image fusion method based on deep learning and model driving. Background technique [0002] Compared with RGB images, hyperspectral imaging technology can obtain scenes in continuous spectrum, which can better record the information of real scenes, and has important applications in remote sensing detection and other fields. However, in practical situations, due to equipment limitations, hyperspectral imaging systems can only provide image data with high spatial resolution but low spectral numbers or hyperspectral images with low spatial resolution. Therefore, the fusion of multispectral and hyperspectral images, that is, fusing the actually collected HrMS images and LrHS images to generate ideal HrHS images is a very practical problem, which has attracted extensive attention in the scientific...

AI Technical Summary

Problems solved by technology

However, most of the existing deep hyperspectral fusion methods simply borrow existing networks in the field of image processing, ignoring the domain knowledge of multispectral and hyperspectral image fusion problems.

In particular, the relevant deep learning method ignores the generative models of HrMS images and LrHS images, and does not consider the low rank of hyperspectral images in the spectral dimension, which is obviously not conducive to the improvement of fusion effect

Images