RGB image spectrum reconstruction method and system, storage medium and application

Abstract

The invention belongs to the technical field of hyperspectral image processing, and discloses an RGB image spectrum reconstruction method and system, a storage medium and application, and the method comprises the steps: constructing a backbone network of a hybrid 2D-3D deep residual attention network with structural tensor constraints; constructing a residual attention module, wherein the residualattention module comprises a plurality of 2-D residual attention modules and 3-D residual attention modules; respectively introducing a 2-D channel attention mechanism and a 3-D waveband attention mechanism into the 2-D deep residual attention network and the 3-D deep residual attention network; in combination with pixel values and structural differences of the hyperspectral image, adopting a mode of combining a structure tensor and MRAE as a loss function, and a finer constraint is formed. According to the method, end-to-end mapping from the RGB image to the hyperspectral image is realized,the characteristic response of the channel and the waveband dimension is self-adaptively recalibrated, the discriminant learning ability is enhanced, and the finer and more accurate hyperspectral image can be recovered in the training process.

Description

technical field [0001] The invention belongs to the technical field of hyperspectral image processing, and in particular relates to a RGB image spectral reconstruction method, system, storage medium and application. Background technique [0002] Currently, hyperspectral sensors record reflectance or transmittance values ​​in hundreds or thousands of bands from the infrared spectrum to the ultraviolet spectrum. Different from traditional RGB images, each pixel in hyperspectral images contains a continuous spectrum with rich spectral features. In fact, rich spectral features have been widely explored in various tasks, such as object tracking, image classification, scene segmentation, and hyperspectral band selection, etc. Nevertheless, most existing hyperspectral devices utilize 2-D sensors to capture 3-D data by scanning along spatial or spectral dimensions, which would require more exposure time, hindering their further application in dynamic scenes. Additionally, costly h...

AI Technical Summary

Problems solved by technology

Therefore, the relationship between the bands of the reconstructed hyperspectral image may not be exactly the same as the real one

On the other hand, most CNN-based spectral reconstruction models can build deeper networks only by stacking convolutional layers, and seldom exploit the interdependence between feature channels, thus limiting the representation ability of CNN and failing to extract higher-level contextual features of

[0003] Through the above analysis, the existing problems and defects of the existing technology are: most of the CNN-based spectral super-resolution algorithms are almost all focused on the spatial feature extraction based on the two-dimensional CNN, but do not simulate the correlation between the bands at the same time. The inherent interdependence between feature maps is also rarely exploited, limiting the representational power of CNNs and failing to extract higher-level contextual features

Images