A hyperspectral image compression and reconstruction method, device and equipment based on autoencoder

Abstract

The present invention discloses a hyperspectral image compression and reconstruction method, device and equipment based on an autoencoder, which comprises: obtaining a Houston image data set and performing preprocessing including random cropping to generate an image block of a preset size as an original hyperspectral image; inputting the original hyperspectral image into an autoencoder model including an encoder and a decoder and training based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; inputting a hyperspectral image to be compressed into the hyperspectral compression model, encoding the hyperspectral image to be compressed by the encoder to obtain compressed data, and decoding the compressed data by the decoder to obtain a reconstructed hyperspectral image.

Description

Technical Field

[0001] The present invention relates to the field of image compression technology, and in particular to a hyperspectral image compression and reconstruction method, device and equipment based on an autoencoder. Background Art

[0002] Hyperspectral remote sensing image compression can significantly reduce the amount of hyperspectral image data, reduce data transmission and storage costs, and make real-time processing of large-scale data sets possible. Through compression, researchers can also use limited computer resources more efficiently to analyze and process hyperspectral images, which is of great significance for promoting the application of hyperspectral images in agriculture, urban planning, environmental monitoring and other fields.

[0003] In recent years, hyperspectral image compression methods based on deep networks have emerged continuously. However, hyperspectral image (HSI) compression methods based on deep autoencoders, such as compression methods based on two-dimensional convolution autoencoders, ignore the feature extraction of the spectral dimension, resulting in low quality of decoded and reconstructed images. The three-dimensional convolution operation involves many parameters and high computational cost. In addition, due to the characteristics of the ground objects themselves, the limitation of spatial resolution, and the influence of atmospheric environment and other factors, the phenomenon of mixed pixels is common in hyperspectral images. However, conventional autoencoders usually ignore the problem of mixed pixels in hyperspectral images. Therefore, these methods cannot effectively protect the spectral information of mixed pixels during the reconstruction process, thereby affecting the reconstruction quality of the decompressed image and subsequent analysis applications. Summary of the invention

[0004] In view of this, the purpose of the present invention is to propose a hyperspectral image compression and reconstruction method, device and equipment based on autoencoder, aiming to solve how to reduce the amount of hyperspectral image data and the computational cost, effectively extract spectral dimensional features and protect the spectral information of mixed pixels, so as to improve the quality of reconstructed hyperspectral images and the reliability of their subsequent analysis applications.

[0005] To achieve the above object, the present invention provides a hyperspectral image compression and reconstruction method based on an autoencoder, the method comprising:

[0006] Obtain the Houston image dataset and perform preprocessing including random cropping to generate image blocks of preset size as the original hyperspectral image;

[0007] The original hyperspectral image is input into an autoencoder model including an encoder and a decoder and trained based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; further comprising:

[0008] Performing spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail;

[0009] Performing depth separation convolution and channel downsampling on the hyperspectral image through the latent coding branch to obtain a first latent coding;

[0010] The first thumbnail and the first latent code are combined and compressed by Huffman coding to obtain a compressed file;

[0011] Performing Huffman inverse coding on the compressed file to obtain a second thumbnail and a second latent code;

[0012] Perform channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map;

[0013] spatially upsampling the second latent code through the latent representation decoding branch, and unmixing and reconstructing the obtained decoded feature map and the apparent feature map to obtain a reconstructed result image;

[0014] The hyperspectral image to be compressed is input into the hyperspectral compression model, the hyperspectral image to be compressed is encoded by the encoder to obtain compressed data, and the compressed data is decoded by the decoder to obtain a reconstructed hyperspectral image.

[0015] Preferably, the thumbnail generation branch includes two spatial downsampling layers and one channel downsampling layer; the step of performing spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail includes:

[0016] Performing a convolution operation on the original hyperspectral image through a spatial downsampling layer to obtain a spatial feature map;

[0017] A convolution operation is performed on the spatial feature map through a channel downsampling layer to obtain the first thumbnail.

[0018] Preferably, the potential coding branch includes a channel downsampling layer, a plurality of DSC modules and a channel pooling layer; the hyperspectral image is subjected to deep separation convolution and channel downsampling by the potential coding branch to obtain a first potential coding, including:

[0019] Perform convolution and ReLU activation on the original hyperspectral image through a channel downsampling layer to obtain a channel feature map;

[0020] Performing dimension expansion on the channel feature map by the DSC module to obtain an expanded feature map;

[0021] The extended feature map is subjected to channel average pooling by the channel pooling layer to obtain the first latent code.

[0022] Preferably, the DSC module includes at least a first DSC module and a second DSC module; and the dimension expansion of the channel feature map by the DSC module to obtain the expanded feature map includes:

[0023] Performing dimension expansion on the channel feature map by the first DSC module to obtain a dimensional feature map; further comprising:

[0024] use , performing spatial convolution on the channel feature map using a kernel function with a spatial step size of 2 to obtain a first convolution feature map;

[0025] use Performing spectral convolution on the first convolution feature map using a kernel function to obtain a second convolution feature map;

[0026] use Performing point convolution and ReLU activation on the second convolution feature map using a kernel function to obtain the dimensional feature map;

[0027] The dimensional feature map is dimensionally expanded by the second DSC module to obtain the expanded feature map.

[0028] Preferably, the thumbnail decoding branch includes a channel upsampling layer and two spatial upsampling layers; the step of performing channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map includes:

[0029] Use multiple channels through the channel upsampling layer Convolve the second thumbnail with a convolution kernel to obtain an intermediate feature map;

[0030] The spatial upsampling layer uses a size of , and perform spatial upsampling on the intermediate feature map twice using a deconvolution with a step size of 2 to obtain the apparent feature map.

[0031] Preferably, the latent representation decoding branch includes a plurality of spatial upsampling layers and a demixing and reconstruction module, and the demixing and reconstruction module includes a first convolutional subnetwork and a second convolutional subnetwork; the spatial upsampling of the second latent code by the latent representation decoding branch, demixing and reconstructing the obtained decoded feature map and the apparent feature map to obtain a reconstructed result image includes:

[0032] Performing multiple spatial upsampling on the second latent code through multiple spatial upsampling layers of the latent representation decoding branch to obtain a decoded feature map;

[0033] Fusing the decoded feature map with the apparent feature map to obtain a fused feature map;

[0034] Inputting the fused feature map into the first convolutional subnetwork to obtain an abundance map;

[0035] The abundance map is input into the second convolutional subnetwork for image reconstruction to obtain the reconstructed result image.

[0036] Preferably, the preset loss function includes using a reconstruction error loss, wherein the reconstruction error loss is , where represents the total number of samples, represents the reconstructed image, Represents the original hyperspectral image.

[0037] To achieve the above object, the present invention also provides a hyperspectral image compression and reconstruction device based on an autoencoder, the device comprising:

[0038] A preprocessing unit, used for acquiring the Houston image dataset and performing preprocessing including random cropping to generate image blocks of a preset size as original hyperspectral images;

[0039] A model training unit is used to input the original hyperspectral image into an autoencoder model including an encoder and a decoder and perform training based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; the model training unit further includes:

[0040] A first encoding unit, configured to perform spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail;

[0041] A second encoding unit, configured to perform deep separation convolution and channel downsampling on the hyperspectral image through the potential encoding branch to obtain a first potential encoding;

[0042] An entropy coding unit, configured to combine the first thumbnail and the first latent code and compress them through Huffman coding to obtain a compressed file;

[0043] An inverse encoding unit, used for performing Huffman inverse encoding on the compressed file to obtain a second thumbnail and a second latent code;

[0044] A first decoding unit, configured to perform channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map;

[0045] A second decoding unit is used to spatially upsample the second latent code through the latent representation decoding branch, and unmix and reconstruct the obtained decoded feature map and the apparent feature map to obtain a reconstructed result image;

[0046] The compression and reconstruction unit is used to input the hyperspectral image to be compressed into the hyperspectral compression model, encode the hyperspectral image to be compressed by the encoder to obtain compressed data, and decode the compressed data by the decoder to obtain a reconstructed hyperspectral image.

[0047] In order to achieve the above objectives, the present invention also proposes a hyperspectral image compression and reconstruction device based on an autoencoder, comprising a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a hyperspectral image compression and reconstruction method based on an autoencoder as described in the above embodiment.

[0048] In order to achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of a hyperspectral image compression and reconstruction method based on an autoencoder as described in the above embodiment.

[0049] In order to achieve the above objectives, the present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a hyperspectral image compression and reconstruction method based on an autoencoder as described in the above embodiment.

[0050] Beneficial effects:

[0051] The above scheme achieves effective compression and high-quality reconstruction of hyperspectral images by designing a dual-branch encoder-decoder structure, that is, by constructing an encoder including a thumbnail generation branch and a latent coding branch, and a decoder including a thumbnail decoding branch and a latent representation decoding branch. The thumbnail generation branch is responsible for capturing the spatial and channel characteristics of the image, while the latent coding branch deeply explores the internal representation of the image. The combination of the two achieves comprehensive capture and efficient representation of hyperspectral image features. This design not only reduces the amount of data, but also ensures that the reconstructed image can retain the key information of the original image, providing reliable data support for the subsequent application of hyperspectral images.

[0052] The above scheme adopts a convolutional autoencoder, and reduces the dimension of the hyperspectral image by combining the convolution layer and the maximum pooling layer in the encoder, obtains its features and latent codes, and uses the features and latent codes as the compression results; in the decompression process, the latent codes are input into the decoder, and the hyperspectral image is reconstructed by the decoder. By introducing deep separable convolution to make full use of the spatial-spectral correlation of the hyperspectral image, the encoder can make good use of the spatial and spectral correlation of the hyperspectral image, reduce the computational cost, and effectively retain the key information of the hyperspectral image and remove redundant information. In view of the common phenomenon of mixed pixels in hyperspectral images, unmixing reconstruction is introduced in the decoder to retain the spectral mixing information of HSI, further improving the quality of the reconstructed image.

[0053] The above scheme, by constructing the specific structure of the thumbnail generation branch, including the spatial downsampling layer and the channel downsampling layer, can effectively capture the spatial and channel characteristics of the hyperspectral image and provide key information for subsequent compression and reconstruction tasks.

[0054] The above scheme, by constructing the structure of the latent coding branch, includes the use of deep separable convolution (DSC) blocks for feature extraction, which effectively reduces the number of model parameters and reduces the computational cost while maintaining a high feature extraction capability; while maintaining a high feature extraction capability, the model can handle larger data sets and improve the performance of compression and reconstruction tasks. Furthermore, the implementation process of the DSC module includes steps such as spatial convolution, spectral convolution, and point convolution, which can realize multi-dimensional feature extraction of hyperspectral images, enhance the feature representation capability of the model, and thus improve the quality of the reconstructed image.

[0055] The above scheme, by constructing the structure and implementation process of the thumbnail decoding branch, including the channel upsampling layer and the spatial upsampling layer, can restore the spatial and channel characteristics of the thumbnail, providing key information for the subsequent unmixing and reconstruction with the latent coding. At the same time, spatial upsampling is achieved through deconvolution operations, further improving the spatial resolution of the reconstructed image.

[0056] The above scheme constructs the structure and implementation process of the latent representation decoding branch. It introduces an unmixing and reconstruction module to retain the spectral mixing information of HSI, and adds unmixing constraints to the designed network model, so that the decoder can generate a hyperspectral image that conforms to the actual physical meaning when reconstructing the image. By fusing and reconstructing the decoded feature map and the apparent feature map, the problem of mixed pixels in the hyperspectral image is effectively solved. At the same time, the abundance map is obtained through the first convolutional subnetwork, which further enhances the spectral fidelity of the reconstructed image; the image is reconstructed through the second convolutional subnetwork to obtain a high-quality reconstructed result image. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A schematic flow chart of a hyperspectral image compression and reconstruction method based on an autoencoder provided in one embodiment of the present invention.

[0059] Figure 2 An overall schematic diagram of hyperspectral image compression and reconstruction based on an autoencoder provided by an embodiment of the present invention.

[0060] Figure 3 A schematic diagram of a network framework of an autoencoding model provided in one embodiment of the present invention.

[0061] Figure 4 A schematic diagram of the structure of an encoder module provided in one embodiment of the present invention.

[0062] Figure 5 A schematic diagram of the structure of a decoder module provided in one embodiment of the present invention.

[0063] Figure 6 A schematic diagram of the structure of a demixing and reconstruction module provided in one embodiment of the present invention.

[0064] Figure 7 A schematic diagram of the structure of a hyperspectral image compression and reconstruction device based on an autoencoder provided in one embodiment of the present invention.

[0065] The realization of the purpose, functional features and advantages of the invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0068] The content of the present invention is described in detail below with reference to embodiments.

[0069] Reference Figure 1 The figure is a flow chart of a hyperspectral image compression and reconstruction method based on an autoencoder provided in one embodiment of the present invention.

[0070] In this embodiment, the method includes:

[0071] S11, obtain the Houston image dataset and perform preprocessing including random cropping to generate image blocks of a preset size as the original hyperspectral image.

[0072] In this embodiment, the Houston image dataset is obtained. The spatial size of the DFC2018 Houston image dataset is 1202 4172, 48 effective bands. The Houston image dataset is randomly cropped and divided into 256 256 image blocks, and the physical storage size of each image block is about 6.25MB. In this embodiment, these image blocks are divided into training set, test set and validation set according to the ratio of 8:1:1. The training set images are used as the original hyperspectral images of the training samples. , assuming the original hyperspectral image ,in The number of channels is , The length is H and the width is W.

[0073] S12, inputting the original hyperspectral image into an autoencoder model including an encoder and a decoder and training based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; further comprising:

[0074] S12-1, performing spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail ;

[0075] S12-2, performing deep separation convolution and channel downsampling on the hyperspectral image through the potential coding branch to obtain a first potential coding ;

[0076] S12-3, combining the first thumbnail and the first latent code and compressing them through Huffman coding to obtain a compressed file ;

[0077] S12-4, performing Huffman inverse coding on the compressed file to obtain a second thumbnail and a second latent code;

[0078] S12-5, performing channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map ;

[0079] S12-6, spatially upsampling the second latent code through the latent representation decoding branch, and obtaining a decoded feature map The reconstructed image is obtained by unmixing and reconstructing the apparent feature map. .

[0080] Reference Figure 2 As shown in the figure, the compression process includes: neural network coding and entropy coding; neural network coding refers to the encoder converting Encoded as , entropy coding pair Perform lossless compression to output compressed files The decoding process includes: inverse encoding and neural network decoding and reconstruction; first, Restoring low-dimensional representation , and then through the decoder Reconstructed as , Represents the reconstructed hyperspectral image.

[0081] Reference Figure 3 As shown, the encoder includes: thumbnail generation branch and the latent coding branch .in, Perform spatial and channel downsampling on the original hyperspectral image to generate thumbnails ; Perform deep separation convolution and channel downsampling on hyperspectral images to generate low-dimensional latent codes ; and composition .

[0082] Further, the thumbnail generation branch includes two spatial downsampling layers and one channel downsampling layer; in step S12-1, the original hyperspectral image is spatially and channel-downsampled by the thumbnail generation branch to obtain a first thumbnail, including:

[0083] S12-1-1, performing a convolution operation on the original hyperspectral image through a spatial downsampling layer to obtain a spatial feature map ;

[0084] S12-1-2, performing a convolution operation on the spatial feature map through a channel downsampling layer to obtain the first thumbnail .

[0085] Reference Figure 4 As shown, in this embodiment, the thumbnail generation branch It contains 2 spatial downsampling layers and 1 channel downsampling layer; each spatial downsampling layer uses a spatial step size of 2. The kernel is convolved on the original hyperspectral image to obtain ; The channel downsampling layer uses 3 nuclear Perform convolution and transform the feature map Downsample to 3 channels to get thumbnails .

[0086] Further, the potential coding branch includes a channel downsampling layer, multiple DSC modules and a channel pooling layer; in step S12-2, the hyperspectral image is subjected to deep separation convolution and channel downsampling by the potential coding branch to obtain a first potential coding, including:

[0087] S12-2-1, convolve the original hyperspectral image through the channel downsampling layer and perform ReLU activation to obtain a channel feature map ;

[0088] S12-2-2, performing dimension expansion on the channel feature map through the DSC module to obtain an expanded feature map ;

[0089] S12-2-3, performing channel average pooling on the extended feature map through the channel pooling layer to obtain the first potential code .

[0090] Further, the DSC module includes at least a first DSC module and a second DSC module; in step S12-2-2, the dimension of the channel feature map is expanded by the DSC module to obtain an expanded feature map, including:

[0091] (1) Dimensionally expanding the channel feature map by the first DSC module to obtain a dimensional feature map; further comprising:

[0092] (1-1) Adoption , performing spatial convolution on the channel feature map using a kernel function with a spatial step size of 2 to obtain a first convolution feature map;

[0093] (1-2) Adoption Performing spectral convolution on the first convolution feature map using a kernel function to obtain a second convolution feature map;

[0094] (1-3) Adoption Performing point convolution and ReLU activation on the second convolution feature map using a kernel function to obtain the dimensional feature map;

[0095] (2) Dimensionally expand the dimensional feature map through the second DSC module to obtain the expanded feature map.

[0096] In this embodiment, the potential coding branch Contains 1 channel downsampling layer, DSC modules and 1 channel downsampling layer. In this embodiment Set to 5. The channel downsampling layer uses b layers of size Verification After convolution, ReLU activation is performed, and the output feature map is The DSC module is a lightweight depth-wise separable convolution, where The 3D convolution is decomposed into several steps, including spatial convolution, spectral convolution and point convolution. Specifically:

[0097] set up Indicates DSC module, its input ,in, , and is the height, width and number of channels of the feature map, Represents the depth of the feature map. For each internal processing of DSK, for any block in the K DSC modules, its input is , the output is The first DSC module has an input of , after expanding one dimension, we get ,Right now , .

[0098] In spatial convolution, the input feature map use , kernel function with spatial step size of 2 Convolution is performed, and the kernel function is , that is,

[0099] , get the feature map .

[0100] In spectral convolution, use The kernel function For feature maps Convolution is performed, and the kernel function is , that is,

[0101] , get the feature map .

[0102] In point convolution, the feature map Use The kernel function Perform convolution and ReLU activation, that is,

[0103] , For the The feature map output of each DSC module is , where k represents a value varying from 0-K.

[0104] In this embodiment, , It is set to 5, that is, there are 5 DSC modules in total, and the output feature map depth of each DSC module is 64.

[0105] In the channel pooling layer, the feature map Perform channel average pooling along the first dimension to obtain , where K represents the quantity, for example, K=5, then .

[0106] Then the encoder output and Merge into , losslessly compressed by Huffman coding (entropy coding) to obtain a compressed file .

[0107] Further, the thumbnail decoding branch includes a channel upsampling layer and two spatial upsampling layers; in step S12-5, the second thumbnail is subjected to channel and spatial upsampling by the thumbnail decoding branch to obtain an apparent feature map, including:

[0108] S12-5-1, using multiple channels through the channel upsampling layer The convolution kernel of is used to convolve the second thumbnail to obtain an intermediate feature map ;

[0109] S12-5-2, through the spatial upsampling layer using size , deconvolution with a step size of 2 performs two spatial upsampling on the intermediate feature map to obtain the apparent feature map .

[0110] Further, the potential representation decoding branch includes multiple spatial upsampling layers and a demixing and reconstruction module, and the demixing and reconstruction module includes a first convolutional subnetwork and a second convolutional subnetwork; in step S12-6, the second potential code is spatially upsampled by the potential representation decoding branch, and the obtained decoded feature map is demixed and reconstructed with the apparent feature map to obtain a reconstructed result image, including:

[0111] S12-6-1, performing multiple spatial upsampling on the second latent code through multiple spatial upsampling layers of the latent representation decoding branch to obtain a decoding feature map ;

[0112] S12-6-2, fusing the decoded feature map with the apparent feature map to obtain a fused feature map ;

[0113] S12-6-3, inputting the fused feature map into the first convolutional subnetwork to obtain an abundance map;

[0114] S12-6-4, inputting the abundance map into the second convolutional subnetwork for image reconstruction to obtain the reconstructed result image .

[0115] In this embodiment, the compressed file Obtained through Huffman inverse coding , further Input to the decoder for decoding and reconstruction, first Decompose into and . Reference Figure 5 As shown, the decoder includes: thumbnail decoding branch and the latent representation decoding branch . Thumbnail decoding branch Contains 1 channel upsampling layer and 2 spatial upsampling layers; latent representation decoding branch It consists of K spatial upsampling layers and an unmixing and reconstruction module.

[0116] Input thumbnail To the thumbnail decoding branch , in the channel upsampling layer, by using indivual The convolution kernel Perform convolution and Expanded from 3 channels to channels, that is, , get the feature map In the spatial upsampling layer, by using a size of , deconvolution with a step size of 2, for the feature map Perform two spatial upsampling to obtain the feature map .

[0117] In the latent representation decoding branch In the example, the spatial upsampling layer uses a size of , deconvolution with a step size of 2, for the feature map implement Sub-space upsampling to obtain feature maps , The latent coding branch of the encoder The number of downsampling in The same (since the output feature maps h and w during encoding are 1 / 2^K of the original H and W, they must be restored to the original H and W through upsampling during decoding). And decode branches by thumbnail What you get Fusion to obtain feature map , the feature map Enter the unmixing and reconstruction module. That is, .

[0118] Reference Figure 6 As shown, the unmixing and reconstruction module includes and . First go through Output abundance map , and then Input to In, by Reconstruct the reconstructed image .in, Includes a convolutional layer and Softmax; The convolutional layer in The convolution kernel The convolutional layer outputs a feature map ; Then use the Softmax function along the first dimension to ensure that the output is non-negative and the sum along the first dimension is 1, and get the dimension Abundance diagram of , corresponding to the four end members in That is, .

[0119] The convolutional layer is indivual The kernel function Composed of Reconstruct the reconstructed image .Right now, .

[0120] Extract the original hyperspectral image by using the VCA algorithm before training The spectral curves of the four end members of: trees, soil, water, and buildings; The weights of the convolutional layers in are initialized, and the weights are updated by back propagation of gradients during training.

[0121] The reconstruction error loss is used to train the autoencoder model, that is,

[0122] ;

[0123] in, is the total number of samples. During training, the ADAM optimizer is used to find the global minimum from the error generated by the loss function, and the learning rate is dynamically adjusted through StepLR.

[0124] S13, inputting the hyperspectral image to be compressed into the hyperspectral compression model, encoding the hyperspectral image to be compressed by the encoder to obtain compressed data, and decoding the compressed data by the decoder to obtain a reconstructed hyperspectral image.

[0125] The present invention proposes a dual-branch codec structure and introduces a deep separable convolution module to fully utilize the spatial-spectral correlation of hyperspectral images. In view of the common phenomenon of mixed pixels in hyperspectral images, this embodiment introduces a demixing and reconstruction module in the decoder to retain the spectral mixing information of HSI. The algorithm adopts a convolutional autoencoder to reduce the dimension of the hyperspectral image through a combination of the convolution layer and the maximum pooling layer in the encoder, obtain its features and apparent latent code, and use the latent code as the compression result. During the decompression process, the latent code is input into the decoder, and the hyperspectral image is reconstructed by the decoder, which further improves the quality of the reconstructed image and provides reliable data support for the subsequent application of hyperspectral images.

[0126] Reference Figure 7 FIG. 1 is a schematic diagram of the structure of a hyperspectral image compression and reconstruction device based on an autoencoder provided by an embodiment of the present invention.

[0127] In this embodiment, the device 20 includes:

[0128] A preprocessing unit 21 is used to obtain the Houston image data set and perform preprocessing including random cropping to generate image blocks of a preset size as original hyperspectral images;

[0129] The model training unit 22 is used to input the original hyperspectral image into an autoencoder model including an encoder and a decoder and perform training based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; the model training unit 22 further includes:

[0130] A first encoding unit, configured to perform spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail;

[0131] A second encoding unit, configured to perform deep separation convolution and channel downsampling on the hyperspectral image through the potential encoding branch to obtain a first potential encoding;

[0132] An entropy coding unit, configured to combine the first thumbnail and the first latent code and compress them through Huffman coding to obtain a compressed file;

[0133] An inverse encoding unit, used for performing Huffman inverse encoding on the compressed file to obtain a second thumbnail and a second latent code;

[0134] A first decoding unit, configured to perform channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map;

[0135] A second decoding unit is used to spatially upsample the second latent code through the latent representation decoding branch, and unmix and reconstruct the obtained decoded feature map and the apparent feature map to obtain a reconstructed result image;

[0136] The compression and reconstruction unit 23 is used to input the hyperspectral image to be compressed into the hyperspectral compression model, encode the hyperspectral image to be compressed by the encoder to obtain compressed data, and decode the compressed data by the decoder to obtain a reconstructed hyperspectral image.

[0137] Each unit module of the device 20 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the above corresponding steps for details.

[0138] The embodiment of the present invention further provides a hyperspectral image compression and reconstruction device based on an autoencoder, the device comprising the hyperspectral image compression and reconstruction device based on an autoencoder as described above, wherein the hyperspectral image compression and reconstruction device based on an autoencoder can be used Figure 7 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0139] The device includes: a mobile phone, a digital camera, a tablet computer, or other devices with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.

[0140] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.

[0141] The input unit can be used to receive input digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.

[0142] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel, and optionally, the display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface may cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event, and then the processor provides a corresponding visual output on the display panel according to the type of touch event.

[0143] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not installed in a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The hyperspectral image compression and reconstruction method based on the autoencoder is shown. The computer readable storage medium can be a read-only memory, a disk or an optical disk, etc.

[0144] The embodiment of the present invention also provides a computer program product, including a computer program / instruction, which is loaded and executed by a processor to implement Figure 1 A hyperspectral image compression and reconstruction method based on autoencoder is shown.

[0145] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, equipment embodiment and storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0146] Furthermore, in this document, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0147] The above description shows and describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention, through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A hyperspectral image compression and reconstruction method based on autoencoder, characterized in that: The method comprises: Obtain the Houston image dataset and perform preprocessing including random cropping to generate image blocks of preset size as the original hyperspectral image; The original hyperspectral image is input into an autoencoder model including an encoder and a decoder and trained based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; further comprising: Performing spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail; Performing depth separation convolution and channel downsampling on the hyperspectral image through the latent coding branch to obtain a first latent coding; Furthermore, the potential coding branch includes a channel downsampling layer, multiple DSC modules and a channel pooling layer; the hyperspectral image is subjected to deep separation convolution and channel downsampling by the potential coding branch to obtain a first potential coding, including: Perform convolution and ReLU activation on the original hyperspectral image through a channel downsampling layer to obtain a channel feature map; Performing dimension expansion on the channel feature map by the DSC module to obtain an expanded feature map; Performing channel average pooling on the extended feature map through the channel pooling layer to obtain the first latent code; Further, the DSC module includes at least a first DSC module and a second DSC module; the dimension expansion of the channel feature map by the DSC module to obtain the extended feature map includes: Performing dimension expansion on the channel feature map by the first DSC module to obtain a dimensional feature map; further comprising: use , performing spatial convolution on the channel feature map using a kernel function with a spatial step size of 2 to obtain a first convolution feature map; use Performing spectral convolution on the first convolution feature map using a kernel function to obtain a second convolution feature map; use Performing point convolution and ReLU activation on the second convolution feature map using a kernel function to obtain the dimensional feature map; Performing dimension expansion on the dimensional feature map by the second DSC module to obtain the expanded feature map; The first thumbnail and the first latent code are combined and compressed by Huffman coding to obtain a compressed file; Performing Huffman inverse coding on the compressed file to obtain a second thumbnail and a second latent code; Perform channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map; spatially upsampling the second latent code through the latent representation decoding branch, and unmixing and reconstructing the obtained decoded feature map and the apparent feature map to obtain a reconstructed result image; Furthermore, the potential representation decoding branch includes multiple spatial upsampling layers and a demixing and reconstruction module, and the demixing and reconstruction module includes a first convolutional subnetwork and a second convolutional subnetwork; the second potential code is spatially upsampled by the potential representation decoding branch, and the obtained decoded feature map is demixed and reconstructed with the apparent feature map to obtain a reconstructed result image, including: Performing multiple spatial upsampling on the second latent code through multiple spatial upsampling layers of the latent representation decoding branch to obtain a decoded feature map; Fusing the decoded feature map with the apparent feature map to obtain a fused feature map; Inputting the fused feature map into the first convolutional subnetwork to obtain an abundance map; Inputting the abundance map into the second convolutional subnetwork to perform image reconstruction to obtain the reconstructed result image; The hyperspectral image to be compressed is input into the hyperspectral compression model, the hyperspectral image to be compressed is encoded by the encoder to obtain compressed data, and the compressed data is decoded by the decoder to obtain a reconstructed hyperspectral image.

2. The hyperspectral image compression and reconstruction method based on autoencoder according to claim 1 is characterized in that: The thumbnail generation branch includes 2 spatial downsampling layers and 1 channel downsampling layer; The step of performing spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail includes: Performing a convolution operation on the original hyperspectral image through a spatial downsampling layer to obtain a spatial feature map; A convolution operation is performed on the spatial feature map through a channel downsampling layer to obtain the first thumbnail.

3. The hyperspectral image compression and reconstruction method based on autoencoder according to claim 1 is characterized in that: The thumbnail decoding branch includes a channel upsampling layer and two spatial upsampling layers; The step of performing channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map includes: Use multiple channels through the channel upsampling layer Convolve the second thumbnail with a convolution kernel to obtain an intermediate feature map; The spatial upsampling layer uses a size of , and perform spatial upsampling on the intermediate feature map twice using a deconvolution with a step size of 2 to obtain the apparent feature map.

4. The hyperspectral image compression and reconstruction method based on autoencoder according to claim 1 is characterized in that: The preset loss function includes using a reconstruction error loss, wherein the reconstruction error loss is , where represents the total number of samples, represents the reconstructed image, Represents the original hyperspectral image.

5. A hyperspectral image compression and reconstruction device based on an autoencoder, characterized in that: The device comprises: A preprocessing unit, used for acquiring the Houston image dataset and performing preprocessing including random cropping to generate image blocks of a preset size as original hyperspectral images; A model training unit is used to input the original hyperspectral image into an autoencoder model including an encoder and a decoder and perform training based on a preset loss function to obtain a trained hyperspectral compression model, wherein the network structure of the encoder includes a thumbnail generation branch and a potential encoding branch, and the network structure of the decoder includes a thumbnail decoding branch and a potential representation decoding branch; the model training unit further includes: A first encoding unit, configured to perform spatial and channel downsampling on the original hyperspectral image through the thumbnail generation branch to obtain a first thumbnail; A second encoding unit, configured to perform deep separation convolution and channel downsampling on the hyperspectral image through the potential encoding branch to obtain a first potential encoding; Furthermore, the potential coding branch includes a channel downsampling layer, a plurality of DSC modules and a channel pooling layer; the second coding unit is further used for: Perform convolution and ReLU activation on the original hyperspectral image through a channel downsampling layer to obtain a channel feature map; Performing dimension expansion on the channel feature map by the DSC module to obtain an expanded feature map; Performing channel average pooling on the extended feature map through the channel pooling layer to obtain the first latent code; Further, the DSC module includes at least a first DSC module and a second DSC module; the dimension expansion of the channel feature map by the DSC module to obtain the extended feature map includes: Performing dimension expansion on the channel feature map by the first DSC module to obtain a dimensional feature map; further comprising: use , performing spatial convolution on the channel feature map using a kernel function with a spatial step size of 2 to obtain a first convolution feature map; use Performing spectral convolution on the first convolution feature map using a kernel function to obtain a second convolution feature map; use Performing point convolution and ReLU activation on the second convolution feature map using a kernel function to obtain the dimensional feature map; Performing dimension expansion on the dimensional feature map by the second DSC module to obtain the expanded feature map; An entropy coding unit, configured to combine the first thumbnail and the first latent code and compress them through Huffman coding to obtain a compressed file; An inverse encoding unit, used for performing Huffman inverse encoding on the compressed file to obtain a second thumbnail and a second latent code; A first decoding unit, configured to perform channel and spatial upsampling on the second thumbnail through the thumbnail decoding branch to obtain an apparent feature map; A second decoding unit is used to spatially upsample the second latent code through the latent representation decoding branch, and unmix and reconstruct the obtained decoded feature map and the apparent feature map to obtain a reconstructed result image; Furthermore, the potential representation decoding branch includes a plurality of spatial upsampling layers and a demixing and reconstruction module, the demixing and reconstruction module includes a first convolutional subnetwork and a second convolutional subnetwork; the second decoding unit is further used for: Performing multiple spatial upsampling on the second latent code through multiple spatial upsampling layers of the latent representation decoding branch to obtain a decoded feature map; Fusing the decoded feature map with the apparent feature map to obtain a fused feature map; Inputting the fused feature map into the first convolutional subnetwork to obtain an abundance map; Inputting the abundance map into the second convolutional subnetwork to perform image reconstruction to obtain the reconstructed result image; The compression and reconstruction unit is used to input the hyperspectral image to be compressed into the hyperspectral compression model, encode the hyperspectral image to be compressed by the encoder to obtain compressed data, and decode the compressed data by the decoder to obtain a reconstructed hyperspectral image.

6. A hyperspectral image compression and reconstruction device based on an autoencoder, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a hyperspectral image compression and reconstruction method based on an autoencoder as claimed in any one of claims 1 to 4.

7. A computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps of a hyperspectral image compression and reconstruction method based on an autoencoder as claimed in any one of claims 1 to 4.

Images