A Hyperspectral Image Super-Resolution Method and System Based on Latent Space Diffusion Model

Abstract

This invention discloses a hyperspectral image super-resolution method and system based on a latent space diffusion model. The method involves upsampling and grouping a low-resolution hyperspectral image, using an encoder network to reduce the dimensionality and obtain low-dimensional prior latent space information, then inputting this information into a multi-scale conditional encoder to obtain conditional information. The low-dimensional prior latent space information is concatenated with Gaussian noise channels, and the diffusion model, guided by the conditional information, transforms the Gaussian noise into high-frequency residual latent space information. This residual information is then added to the low-dimensional prior latent space information to obtain low-dimensional latent space information with high-frequency details. Finally, a pre-trained decoder network is used for decoding and reconstruction to obtain the final high-resolution hyperspectral super-resolution image. This invention aims to address the problems of excessive smoothness and lack of texture detail in traditional hyperspectral image super-resolution methods based on convolutional neural networks, as well as the challenges faced when applying diffusion models to hyperspectral image super-resolution.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a hyperspectral image super-resolution method and system based on a latent space diffusion model. Background Technology

[0002] In recent years, with the rapid development of remote sensing technology and sensors, acquiring remote sensing image data has become increasingly easier. Hyperspectral images, in particular, have shown great application potential in fields such as land cover classification, environmental monitoring, mineral identification, and agricultural analysis due to their rich spectral information. However, hyperspectral images often suffer from insufficient spatial resolution, which limits their application in areas such as land cover identification.

[0003] To overcome this shortcoming, much research has focused on hyperspectral image super-resolution techniques. Current hyperspectral image super-resolution methods can be broadly categorized into two types: image fusion-based hyperspectral image super-resolution and single-image-based hyperspectral image super-resolution. Image fusion-based methods typically yield high-quality, high-resolution hyperspectral images, but these methods rely on high spatial resolution auxiliary images, significantly increasing the complexity of the imaging system and severely limiting their practical applications. Single-image super-resolution methods include interpolation methods, sparse representation-based methods, and deep learning-based methods. Although interpolation methods are simple and fast, they easily lead to hyperspectral image blurring, affecting image quality. Sparse representation-based methods are computationally complex and have poor robustness.

[0004] Deep learning methods are currently the mainstream hyperspectral image super-resolution methods, such as VDSR, 3DFCNN, EDSR, MCNet, and EUNet.Among them, the VDSR method references Kim J, Lee JK, Lee K M. Accurate imagesuper-resolution using very deep convolutional networks[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 1646-1654; the 3DFCNN method references Mei S, Yuan X, Ji J, et al. Hyperspectral imagespatial super-resolution via 3D full convolutional neural network[J]. RemoteSensing, 2017, 9(11): 1139, EDSR method reference Lim B, Son S, Kim H, et al.Enhanced deep residual networks for single image super-resolution[C] / / Proceedings of the IEEE conference on computer vision and pattern recognitionworkshops. 2017: 136-144; MCNet method reference Li Q, Wang Q, Li X. Mixed 2D / 3Dconvolutional network for hyperspectral image super-resolution[J]. Remotesensing, 2020, 12(10): 1660; EUNet method reference Liu D, Li J, Yuan Q, et al. An efficient unfolding network with disentangled spatial-spectral representation for hyperspectral image super-resolution[J]. Information Fusion, 2023, 94:92-111. However, although these methods have achieved relatively good super-resolution results, the images reconstructed by existing methods are often relatively smooth and lack texture details.

[0005] Diffusion models, as an emerging generative modeling approach, have attracted considerable attention due to their significant performance in image generation. These models generate high-quality images by progressively removing noise and possess the ability to handle high-dimensional data and complex distributions, making them well-suited for hyperspectral image super-resolution tasks. However, applying diffusion models to hyperspectral image super-resolution faces several challenges. The high dimensionality and scarcity of hyperspectral data often lead to difficulties in model training and convergence. Furthermore, diffusion models are computationally expensive, and their iterative inference process results in long generation times. These factors severely hinder the application of diffusion models in hyperspectral image super-resolution. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a hyperspectral image super-resolution method and system based on a latent space diffusion model, which addresses the above-mentioned problems in the existing technology. This invention aims to solve the problems of excessive smoothness and lack of texture detail information in traditional hyperspectral image super-resolution methods based on convolutional neural networks, as well as the challenges faced when applying diffusion models to hyperspectral image super-resolution.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A hyperspectral image super-resolution method based on a latent space diffusion model includes obtaining a final high-resolution hyperspectral super-resolution image from a low-resolution hyperspectral image using a hyperspectral image super-resolution network composed of an encoder network, a multi-scale conditional encoder, a diffusion model, and a decoder network: S1, the low-resolution hyperspectral image is upsampled and grouped according to the number of bands; S2, each group of images is sequentially passed through a pre-trained encoder network to reduce the spectral dimension to obtain low-dimensional prior latent space information, and then encoded using a multi-scale conditional encoder to obtain conditional information; S3, the low-dimensional prior latent space information is concatenated with Gaussian noise channels, and then the Gaussian noise is converted into high-frequency residual latent space information under the guidance of the conditional information using a diffusion model; S4, the high-frequency residual latent space information is added to the low-dimensional prior latent space information to obtain low-dimensional latent space information with high-frequency details; S5, each group of low-dimensional latent space information is decoded and restored through a pre-trained decoder network to obtain the final high-resolution hyperspectral super-resolution image.

[0009] Optionally, the encoder network in step S2 includes a 5×5 convolutional layer, multiple residual modules, and a 5×5 convolutional layer connected in sequence. The data input to the encoder network first passes through a 5×5 convolutional layer to obtain preliminary features. The preliminary features then pass through multiple residual modules and are added together. Finally, the data passes through a 5×5 convolutional layer to obtain the final output of the encoder network. The decoder network includes a 5×5 convolutional layer, a first group of multiple residual modules, a 5×5 convolutional layer, a 5×5 convolutional layer, a second group of multiple residual modules, and a 5×5 convolutional layer connected in sequence. The data input to the decoder network first passes through a 5×5 convolutional layer to obtain preliminary features. The preliminary features then pass through the first group of multiple residual modules and are added together. Finally, the data passes through a 5×5 convolutional layer to obtain the final output of the encoder network. A 5×5 convolutional layer is used to obtain deep feature information. Multiple deep feature information is concatenated to obtain deep image features. The deep image features are first passed through a 5×5 convolutional layer to obtain reconstructed image features. The reconstructed image features are then passed through a second set of multiple residual modules and added to the reconstructed image features. Finally, a 5×5 convolutional layer is passed to obtain reconstructed image information. The reconstructed image information is then added to the deep image features to obtain the output of the decoder network. The residual module includes a 5×5 convolutional layer, a LeakyReLU activation function, and a 5×5 convolutional layer. The data input to the residual module is processed by a 5×5 convolutional layer, a LeakyReLU activation function, and a 5×5 convolutional layer, and then added to the data input to the residual module to obtain the output data of the residual module.

[0010] Optionally, the multi-scale conditional encoder in step S2 includes a 3×3 convolutional layer and multiple spatial spectral prior modules connected in sequence. The data input to the multi-scale conditional encoder first passes through the 3×3 convolutional layer to extract preliminary features, and then the preliminary features pass through multiple spatial spectral prior modules to obtain conditional information at multiple different scales. The spatial spectral prior module includes a 3×3 convolutional layer, multiple spatial modules, multiple spatial spectral modules, and a 3×3 downsampling convolutional layer connected in sequence. The data input to the spatial spectral prior module first passes through the 3×3 convolutional layer to extract initial depth features. The initial depth features are sequentially processed through three spatial modules and added to generate intermediate depth features. These intermediate depth features are then processed through multiple spatial spectral modules and added to both the intermediate and initial depth features to obtain enhanced depth features. Finally, the enhanced depth features are passed through a 3×3 downsampling convolutional layer to obtain the output conditional information. Each spatial module consists of a 3×3 convolutional layer, a LeakyReLU activation function, another 3×3 convolutional layer, yet another 3×3 convolutional layer, a LeakyReLU activation function, and a final 3×3 convolutional layer connected in sequence. The data entering the spatial module first passes through a first 3×3 convolutional layer and a LeakyReLU activation function, then through a second 3×3 convolutional layer, and is added to the initial input data to obtain preliminary processed features. Subsequently, these preliminary processed features pass through a third 3×3 convolutional layer, a LeakyReLU activation function, and a final 3×3 convolutional layer, and are then added back to the preliminary processed features to form the final output features of the spatial module. The spatial spectral module includes one 3×3 convolutional layer, a LeakyReLU activation function, and one 3×3 convolutional layer connected in sequence. The spatial spectral module consists of a 1×1 convolutional layer, a LeakyReLU activation function, a 1×1 convolutional layer, and a channel attention module. The data input to the spatial spectral module first passes through the first 3×3 convolutional layer and the LeakyReLU activation function, then through the second 3×3 convolutional layer, and is added to the initial input data to generate preliminary features. These preliminary features then pass through the first 1×1 convolutional layer, the LeakyReLU activation function, the second 1×1 convolutional layer, and the channel attention module, and are added to the preliminary features to form the final output features of the spatial spectral module.

[0011] Optionally, the diffusion model in step S3 includes a first 3×3 convolutional layer, a U-shaped network, a second 3×3 convolutional layer, and a temporal encoder. The first 3×3 convolutional layer, the U-shaped network, and the second 3×3 convolutional layer are connected sequentially, and the temporal encoder is connected in parallel with the U-shaped network. The network structure of the U-shaped network includes three parts: an encoder, a bottom bottleneck layer, and a decoder. The encoder part consists of four consecutive conditional denoising modules and a downsampling layer, used to extract features layer by layer and gradually reduce the spatial dimension of the feature map to complete feature compression. The bottom bottleneck layer consists of one conditional denoising module, whose function is to connect the encoder and the decoder. The encoder performs deep processing on the compressed features. The decoder consists of four consecutive conditional denoising modules and an upsampling layer, used to recover the spatial dimension of the feature map layer by layer. The decoder is directly connected to the corresponding encoder layer through skip connections. The temporal encoder includes a sinusoidal encoding operation, a multilayer perceptron (MLP) consisting of two linear layers and a SiLU activation function, and a reshaping operation. The time step t input to the temporal encoder is first sinusoidally encoded, and then the encoded result is input into the MLP for processing. Finally, the reshaping operation adjusts the shape of the temporal embedding to achieve effective fusion of temporal information and image features.

[0012] Optionally, the conditional denoising module processes the input features as follows: the features input to the conditional denoising module are sequentially processed through layer normalization, a 1×1 convolutional layer, a 3×3 depthwise convolutional layer, and a simple gate unit to generate first-level features; the first-level features are processed through a lightweight channel attention module and then through a 1×1 convolutional layer to obtain second-level features; the second-level features are added to the features input to the conditional denoising module to obtain intermediate features; the intermediate features are further processed through layer normalization, a 1×1 convolutional layer, a simple gate unit, and a 1×1 convolutional layer, and then added to the intermediate features to generate the output features of the conditional denoising module; The simple gating unit processes the input features by dividing the input data into two parts along the channel direction and performing element-wise multiplication on the two sub-data. The lightweight channel attention module processes the input features by applying global average pooling to the input features of dimension H×W×C and then transforming them into 1×C. Then, a one-dimensional convolutional layer of length 3 is used to perform convolution in dimension C. The weights obtained after convolution are transformed in dimension and then multiplied element-wise with the input features of the lightweight channel attention module to obtain the output features of the lightweight channel attention module.

[0013] Optionally, training a hyperspectral image super-resolution network is included before step S1:

[0014] S101, pre-trained autoencoder network consisting of both an encoder network and a decoder network;

[0015] S102 combines the encoder and decoder networks in the pre-trained autoencoder network, uses a multi-scale conditional encoder to obtain conditional information, and uses the conditional information to guide the training of the diffusion model in the low-dimensional residual latent space, finally completing the training of the entire hyperspectral image super-resolution network.

[0016] Optionally, in step S101, when pre-training the autoencoder network consisting of both the encoder network and the decoder network, the process includes training the autoencoder network using the training set, and determining the optimal network parameters of the autoencoder network by optimizing a preset first loss function to minimize the loss function. The functional expression of the first loss function is as follows:

[0017] ,

[0018] ,

[0019] ,

[0020] ,

[0021] in, The first loss function is preset. For absolute value error loss, For spectral angle loss, For spectral gradient loss, and For weight constants, These represent the total height, total width, and total number of bands of a high-resolution hyperspectral image. This represents a high-resolution hyperspectral reference image in the training set. Indicates will The high-resolution hyperspectral reconstructed image obtained from the input autoencoder network, In high-resolution hyperspectral reconstructed images Pixel value at that location, For high-resolution hyperspectral reference images Pixel value at that location, For the height coordinates, width coordinates, and band coordinates of a high-resolution hyperspectral image, In high-resolution hyperspectral reconstructed images Spectral gradient at that location, For high-resolution hyperspectral reference images The spectral gradient at the location; in step S102, when using the multi-scale conditional encoder to obtain conditional information and guide the training of the diffusion model in the low-dimensional residual latent space using conditional information, the expression of the second loss function is:

[0022] ,

[0023] In the above formula, For the second loss function, In the high-resolution hyperspectral reference image predicted by the diffusion model at the i-th time step Noise at that location In the high-resolution hyperspectral reference image added for the i-th time step The noise at the location is such that the time step i ranges from 1 to 2000.

[0024] Furthermore, the present invention also provides a hyperspectral image super-resolution system based on a latent space diffusion model, comprising an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model.

[0025] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model via a processor.

[0026] Furthermore, the present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model via a processor.

[0027] Compared with existing technologies, this invention has the following main advantages: 1. Addressing the problems of high computational resource consumption and slow inference speed in current diffusion models during training and inference, this invention designs a simple and efficient diffusion model that significantly reduces resource requirements. While maintaining high generation quality, it significantly improves inference speed, breaking through the limitations of traditional models in resource utilization and inference efficiency, and improving the practicality of the system. 2. This invention proposes a multi-scale conditional encoder, which can fully utilize prior knowledge and enhance the accuracy of the generation process. By integrating prior knowledge across multiple scales, the multi-scale conditional encoder more accurately guides the diffusion process, significantly improving the realism and consistency of the generated images, while effectively avoiding common model defects such as over-smoothing. 3. Considering the potential loss of detail during latent space transformation, this invention designs a simple and efficient grouped autoencoder. This grouped autoencoder effectively minimizes the potential loss during latent space transformation, ensuring that the texture and edge details of the generated images are rich and natural. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the network structure of the hyperspectral image super-resolution network in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the encoder network structure of the autoencoder in an embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of the decoder network structure of the autoencoder in an embodiment of the present invention.

[0032] Figure 5 This is a schematic diagram of the network structure of the spatial spectral prior module in an embodiment of the present invention. The multi-scale conditional encoder is composed of four spatial spectral prior modules connected in series.

[0033] Figure 6 This is a schematic diagram of the network structure of the diffusion model in an embodiment of the present invention.

[0034] Figure 7 This is a schematic diagram of the network structure of the conditional denoising module in an embodiment of the present invention.

[0035] Figure 8 This is a schematic diagram of the super-resolution results of different super-resolution methods on the Chikusei dataset images in the experiment of this embodiment of the invention. Among them, (a) is a low-resolution four-fold bitriplexed hyperspectral image, (b) is a four-fold super-resolution image using the 3DFCNN method, (c) is a four-fold super-resolution image using the VDSR method, (d) is a four-fold super-resolution image using the EDSR method, (e) is a four-fold super-resolution image using the MCNet method, (f) is a four-fold super-resolution image using the EUNet method, (g) is a four-fold super-resolution image using the method of this embodiment, and (h) is a high-resolution hyperspectral reference image.

[0036] Figure 9 This is a schematic diagram of the super-resolution results of different super-resolution methods on Houton dataset images in the experiments of this embodiment of the invention. Among them, (a) is a low-resolution four-fold bitriplexed hyperspectral image, (b) is a four-fold super-resolution image using the 3DFCNN method, (c) is a four-fold super-resolution image using the VDSR method, (d) is a four-fold super-resolution image using the EDSR method, (e) is a four-fold super-resolution image using the MCNet method, (f) is a four-fold super-resolution image using the EUNet method, (g) is a four-fold super-resolution image using the method of this embodiment, and (h) is a high-resolution hyperspectral reference image.

[0037] Figure 10This is a schematic diagram of the super-resolution results of different super-resolution methods on the Pavia Center dataset images in the experiment of this embodiment of the invention. Among them, (a) is a low-resolution four-fold bitriplexed hyperspectral image, (b) is a four-fold super-resolution image using the 3DFCNN method, (c) is a four-fold super-resolution image using the VDSR method, (d) is a four-fold super-resolution image using the EDSR method, (e) is a four-fold super-resolution image using the MCNet method, (f) is a four-fold super-resolution image using the EUNet method, (g) is a four-fold super-resolution image using the method of this embodiment, and (h) is a high-resolution hyperspectral reference image. Detailed Implementation

[0038] This invention aims to address the problems of excessive smoothness and lack of texture detail in traditional hyperspectral image super-resolution methods based on convolutional neural networks, as well as the challenges faced when applying diffusion models to hyperspectral image super-resolution. It utilizes a lightweight diffusion model to efficiently generate high-frequency residual detail information, accelerates image generation, reduces computational resource requirements, and improves the practicality of the diffusion model, thereby achieving excellent super-resolution results.

[0039] like Figure 1 and Figure 2 As shown, the hyperspectral image super-resolution method based on the latent space diffusion model in this embodiment includes obtaining a final high-resolution hyperspectral super-resolution image from a low-resolution hyperspectral image using a hyperspectral image super-resolution network composed of an encoder network, a multi-scale conditional encoder, a diffusion model, and a decoder network: S1, the low-resolution hyperspectral image is upsampled and then grouped according to the number of bands; S2, each group of images is sequentially passed through a pre-trained encoder network to reduce the spectral dimension to obtain low-dimensional prior latent space information, and then encoded using a multi-scale conditional encoder to obtain conditional information; S3, the low-dimensional prior latent space information is concatenated with the Gaussian noise channel, and then the Gaussian noise is converted into high-frequency residual latent space information under the guidance of the conditional information using the diffusion model; S4, the high-frequency residual latent space information is added to the low-dimensional prior latent space information to obtain low-dimensional latent space information with high-frequency details; S5, each group of low-dimensional latent space information is decoded and restored through a pre-trained decoder network to obtain the final high-resolution hyperspectral super-resolution image.

[0040] In step S1, when upsampling the low-resolution hyperspectral image, the desired upsampling method can be selected as needed. For example, as an optional implementation, this embodiment uses a bitriangular interpolation algorithm for interpolation upsampling. After upsampling the low-resolution hyperspectral image in step S1, when grouping it according to the number of bands, the grouping strategy divides the hyperspectral image into several groups based on the number of bands, with some band overlap between groups; for example... Figure 2As shown, in this embodiment, the number of groups is N, with one hyperspectral image yielding groups 1 to N. After processing both the high-resolution hyperspectral reference image and the bitriarch interpolated hyperspectral image using an encoder, the two are subtracted to obtain the low-dimensional residual latent space information. Since the noise addition and denoising processes of the diffusion model rely on noise of the same dimension as the original data, training directly on the original data is inefficient. Therefore, this embodiment uses an autoencoder to map the pixel space of the hyperspectral image to the low-dimensional latent space, effectively reducing computational resource requirements. Complex data distributions increase the training difficulty of the diffusion model; the residual latent space, which better conforms to a Gaussian distribution, helps reduce the training difficulty of the diffusion model, allowing it to focus more on learning spatial spectral details. Training and inference within this low-dimensional residual latent space significantly reduces learning and computational complexity.

[0041] like Figure 3 As shown, the encoder network in step S2 includes a 5×5 convolutional layer, multiple residual modules (which can be selected according to actual needs; in this embodiment, there are 3) connected in sequence, and a 5×5 convolutional layer. The data input to the encoder network first passes through a 5×5 convolutional layer to obtain preliminary features. The preliminary features then pass through multiple residual modules in sequence and are added to the preliminary features. Finally, the data passes through a 5×5 convolutional layer to obtain the final output of the encoder network.

[0042] like Figure 4 As shown, the decoder network includes a 5×5 convolutional layer, a first group of multiple (selectable according to actual needs, specifically 3 in this embodiment) residual modules, a 5×5 convolutional layer, a second group of multiple (selectable according to actual needs, specifically 3 in this embodiment) residual modules, and a 5×5 convolutional layer connected in sequence. The data input to the decoder network first passes through a 5×5 convolutional layer to obtain preliminary features. The preliminary features then pass through the first group of multiple residual modules and are added to the preliminary features. Finally, they pass through a 5×5 convolutional layer to obtain deep feature information. The multiple deep feature information is concatenated to obtain deep image features. The deep image features then pass through a 5×5 convolutional layer to obtain reconstructed image features. The reconstructed image features then pass through the second group of multiple residual modules and are added to the reconstructed image features. Finally, they pass through a 5×5 convolutional layer to obtain reconstructed image information. The reconstructed image information is then added to the deep image features to obtain the output of the decoder network.

[0043] In this embodiment, the residual module includes a 5×5 convolutional layer, a LeakyReLU activation function, and a 5×5 convolutional layer. The data input to the residual module is processed sequentially through a 5×5 convolutional layer, a LeakyReLU activation function, and a 5×5 convolutional layer, and then added to the data input to the residual module to obtain the output data of the residual module.

[0044] like Figure 5 As shown, the multi-scale conditional encoder in step S2 includes a 3×3 convolutional layer and multiple spatial spectral prior modules connected in sequence (which can be selected according to actual needs; in this embodiment, there are 4 modules). The data input to the multi-scale conditional encoder first extracts preliminary features through the 3×3 convolutional layer, and then the preliminary features are sequentially passed through multiple spatial spectral prior modules to obtain conditional information at multiple different scales.

[0045] like Figure 5 As shown, the spatial spectrum prior module includes a 3×3 convolutional layer, multiple spatial modules (selectable according to actual needs, specifically 3 in this embodiment), multiple spatial spectrum modules (selectable according to actual needs, specifically 3 in this embodiment), and a 3×3 downsampling convolutional layer connected in sequence. The data input to the spatial spectrum prior module first passes through the 3×3 convolutional layer to extract the initial depth features; the initial depth features pass through the 3 spatial modules in sequence and are added to the initial depth features to generate intermediate depth features; the intermediate depth features are then processed by multiple spatial spectrum modules and added to the intermediate depth features and the initial depth features to obtain enhanced depth features; finally, the enhanced depth features pass through the 3×3 downsampling convolutional layer to obtain the output conditional information.

[0046] like Figure 5 As shown, the spatial module consists of a 3×3 convolutional layer, a LeakyReLU activation function, another 3×3 convolutional layer, another 3×3 convolutional layer, a LeakyReLU activation function, and a final 3×3 convolutional layer connected in sequence. The data input to the spatial module first passes through the first 3×3 convolutional layer and the LeakyReLU activation function in sequence, then through the second 3×3 convolutional layer, and is added to the initial input data to obtain preliminary processed features. Subsequently, these preliminary processed features pass through the third 3×3 convolutional layer, the LeakyReLU activation function, and the final 3×3 convolutional layer in sequence, and are then added to the preliminary processed features to form the final output features of the spatial module.

[0047] like Figure 5As shown, the spatial spectral module consists of a 3×3 convolutional layer, a LeakyReLU activation function, a 3×3 convolutional layer, a 1×1 convolutional layer, a LeakyReLU activation function, a 1×1 convolutional layer, and a channel attention module connected in sequence. The data input to the spatial spectral module first passes through the first 3×3 convolutional layer and the LeakyReLU activation function, then through the second 3×3 convolutional layer, and is added to the initial input data to generate preliminary features. This preliminary feature then passes through the first 1×1 convolutional layer, the LeakyReLU activation function, the second 1×1 convolutional layer, and the channel attention module, and is added to the preliminary feature to form the final output feature of the spatial spectral module.

[0048] like Figure 6 As shown, the diffusion model in step S3 includes a first 3×3 convolutional layer, a U-shaped network, a second 3×3 convolutional layer, and a temporal encoder. The first 3×3 convolutional layer, the U-shaped network, and the second 3×3 convolutional layer are connected in sequence, and the temporal encoder is connected in parallel with the U-shaped network. The U-shaped network consists of 9 conditional denoising blocks. Due to the use of conditional denoising blocks, the model has fewer parameters and faster training and inference speeds. In this embodiment, the U-shaped network structure includes three parts: an encoder, a bottom bottleneck layer, and a decoder. The encoder consists of four consecutive conditional denoising modules and a downsampling layer, used to extract features layer by layer and gradually reduce the spatial dimension of the feature map to achieve feature compression. The bottom bottleneck layer consists of one conditional denoising module, which connects the encoder and decoder to perform deep processing on the features compressed by the encoder. The decoder consists of four consecutive conditional denoising modules and an upsampling layer, used to restore the spatial dimension of the feature map layer by layer. The decoder is directly connected to the corresponding encoder layer through skip connections (used to achieve feature fusion and effective utilization of multi-scale information, thereby significantly improving the accuracy and robustness of the model). See [link to documentation]. Figure 6 It can be seen that the output terminals of the 1st to 4th conditional denoising modules are all connected to a downsampling module, and the output terminals of the 5th to 8th conditional denoising modules are all connected to an upsampling module. The output of the 5th conditional denoising module is added to the output element of the 4th conditional denoising module and used as the input of the 6th conditional denoising module. The output of the 6th conditional denoising module is added to the output element of the 3rd conditional denoising module and used as the input of the 7th conditional denoising module. The output of the 7th conditional denoising module is added to the output element of the 2nd conditional denoising module and used as the input of the 8th conditional denoising module. The output of the 8th conditional denoising module is added to the output element of the 1st conditional denoising module and used as the input of the 9th conditional denoising module.

[0049] like Figure 6As shown, the time encoder sequentially includes a sinusoidal encoding operation, a multilayer perceptron (MLP) consisting of two linear layers and a SiLU activation function, and a reshape operation. The time step t input to the time encoder is first sinusoidally encoded, and then the encoded result is input into the MLP for processing. Finally, the shape of the time embedding is adjusted through the reshape operation to enable the effective fusion of time information and image features in the subsequent process.

[0050] like Figure 7 As shown, the conditional denoising module processes the input features as follows: the features input to the conditional denoising module are sequentially processed through layer normalization, a 1×1 convolutional layer, a 3×3 depthwise convolutional layer, and a simple gating unit. The first-level features are generated by the gate; the first-level features are passed through the lightweight channel attention module and then through a 1×1 convolutional layer to obtain the second-level features; the second-level features are added to the features of the input conditional denoising module to obtain the intermediate features; the intermediate features are further processed by layer normalization, 1×1 convolutional layer, simple gate unit and 1×1 convolutional layer and then added to the intermediate features to generate the output features of the conditional denoising module; the processing of the input features by the simple gate unit includes: dividing the input data into two parts along the channel direction and performing element-wise multiplication on the two sub-data; the processing of the input features by the lightweight channel attention module includes: applying global average pooling to the input features with dimensions H×W×C and then performing dimension transformation to transform them into 1×C, then using a 1-dimensional convolutional layer of length 3 to perform convolution in the dimension of C, and then performing dimension transformation on the weights obtained after convolution and multiplying them element-wise with the input features of the lightweight channel attention module to obtain the output features of the lightweight channel attention module.

[0051] In this embodiment, training a hyperspectral image super-resolution network is included before step S1:

[0052] S101, pre-trained autoencoder network consisting of both an encoder network and a decoder network;

[0053] S102 combines the encoder and decoder networks in the pre-trained autoencoder network, uses a multi-scale conditional encoder to obtain conditional information, and uses the conditional information to guide the training of the diffusion model in the low-dimensional residual latent space, finally completing the training of the entire hyperspectral image super-resolution network.

[0054] In step S101 of this embodiment, when pre-training the autoencoder network composed of both an encoder network and a decoder network, the autoencoder network is determined by optimizing the loss function to minimize it during training using the training set. The training set consists of high-resolution hyperspectral reference images. The loss function refers to the error between the high-resolution hyperspectral reconstructed image obtained by inputting the high-resolution hyperspectral reference image into the autoencoder network and the corresponding high-resolution hyperspectral reference image. Specifically, in step S101 of this embodiment, when pre-training the autoencoder network composed of both an encoder network and a decoder network, the process includes training the autoencoder network using the training set, and determining the optimal network parameters of the autoencoder network by optimizing a preset first loss function to minimize it. The loss function uses L1 error, spectral angle loss, and spectral gradient loss as weighted sums. Specifically, the function expression of the first loss function is:

[0055] ,

[0056] ,

[0057] ,

[0058] ,

[0059] in, The first loss function is preset. For absolute value error loss, For spectral angle loss, For spectral gradient loss, and These are weight constants (in this embodiment, the values ​​are 0.5 and 0.3 respectively). These represent the total height, total width, and total number of bands of a high-resolution hyperspectral image. This represents a high-resolution hyperspectral reference image in the training set. Indicates will The high-resolution hyperspectral reconstructed image obtained from the input autoencoder network, In high-resolution hyperspectral reconstructed images Pixel value at that location, For high-resolution hyperspectral reference images Pixel value at that location, For the height, width, and channel coordinates of a high-resolution hyperspectral image, In high-resolution hyperspectral reconstructed images Spectral gradient at that location, For high-resolution hyperspectral reference images The spectral gradient at the location; the training set consists of low-resolution images and corresponding high-resolution hyperspectral reference image pairs. The first stage of autoencoder training only uses the high-resolution hyperspectral reference image (ground-truth). The process is "high-resolution hyperspectral reference image → autoencoder → high-resolution hyperspectral reconstructed image". Then, the high-resolution hyperspectral reference image and the high-resolution hyperspectral reconstructed image are used for loss. Because this process is a process from the same resolution to the same resolution, I call the result image "high-resolution hyperspectral reconstructed image" rather than "high-resolution hyperspectral super-resolution image".

[0060] The diffusion model is determined by optimizing the loss function to minimize it during training using a training set. The training set includes low-resolution hyperspectral images and corresponding high-resolution hyperspectral reference images. The loss function refers to the error between the noise predicted by the diffusion model and the corresponding added noise after feeding the high-resolution hyperspectral reference image and the low-resolution hyperspectral image into the entire model, using conditional information obtained through a multi-scale conditional encoder, and using this conditional information to guide the diffusion model training in the low-dimensional residual latent space. In step S102 of this embodiment, when using conditional information obtained through a multi-scale conditional encoder to guide the diffusion model training in the low-dimensional residual latent space, the expression for the second loss function is:

[0061] ,

[0062] In the above formula, This is the second loss function (L1 error). In the high-resolution hyperspectral reference image predicted by the diffusion model at the i-th time step Noise at that location In the high-resolution hyperspectral reference image added for the i-th time step The noise at the time step i ranges from 1 to 2000. During the training of the diffusion model in the second stage, noise is added to the reference image. The network predicts the noise added to the high-resolution hyperspectral reference image, while the low-resolution hyperspectral image serves only as conditional information to guide the model in denoising. After the model is trained, the inference process only needs to use the low-resolution hyperspectral image to guide the diffusion model in converting the Gaussian noise image into a high-resolution hyperspectral super-resolution image.

[0063] This embodiment uses the Chikusei, Houston, and Pavia Center datasets to verify the practicality and effectiveness of the hyperspectral image super-resolution method based on the latent space diffusion model. The experimental procedure is described below. The low-resolution hyperspectral images in the experiment are obtained by bitriangular interpolation downsampling of actual hyperspectral images, with the original actual hyperspectral image serving as the high-resolution hyperspectral reference image. The high-resolution hyperspectral reference image and the corresponding low-resolution hyperspectral image are used to form the training and test sets. The method in this embodiment is mainly compared with one traditional super-resolution method (bitriangular interpolation) and five deep learning methods: 3DFCNN, VDSR, EDSR, MCNet, and EUNet. The results are as follows: Figure 8 , Figure 9 and Figure 10 As shown. Figure 8 In the image, (a) is a low-resolution 4x super-resolution image using bitriplex interpolation, (b) is a 4x super-resolution image using the 3DFCNN method, (c) is a 4x super-resolution image using the VDSR method, (d) is a 4x super-resolution image using the EDSR method, (e) is a 4x super-resolution image using the MCNet method, (f) is a 4x super-resolution image using the EUNet method, (g) is a 4x super-resolution image using the method described in this embodiment, and (h) is a high-resolution hyperspectral reference image. Figure 9 In the image, (a) is a low-resolution 4x super-resolution image using bitriplex interpolation, (b) is a 4x super-resolution image using the 3DFCNN method, (c) is a 4x super-resolution image using the VDSR method, (d) is a 4x super-resolution image using the EDSR method, (e) is a 4x super-resolution image using the MCNet method, (f) is a 4x super-resolution image using the EUNet method, (g) is a 4x super-resolution image using the method described in this embodiment, and (h) is a high-resolution hyperspectral reference image. Figure 10 In the image, (a) is a low-resolution 4x super-resolution image using bitriplex interpolation, (b) is a 4x super-resolution image using the 3DFCNN method, (c) is a 4x super-resolution image using the VDSR method, (d) is a 4x super-resolution image using the EDSR method, (e) is a 4x super-resolution image using the MCNet method, (f) is a 4x super-resolution image using the EUNet method, (g) is a 4x super-resolution image using the method described in this embodiment, and (h) is a high-resolution hyperspectral reference image. See also... Figure 8 , Figure 9 and Figure 10 It can be seen that, compared with the high-resolution hyperspectral reference image, Figure 8 Some details in (a) to (f) are lost, due to Figure 8 As can be seen from (g) in the embodiments of the present invention, the spectrum of the method is well preserved. Figure 9 (a)~(f) and Figure 10 The super-resolution images in (a) to (f) have blurred edges and significant loss of detail. Figure 9 (g) and Figure 10 As can be seen from (g) in this embodiment, the method described in this embodiment preserves details well. The super-resolution image clearly shows that the difference between the super-resolution image and the high-resolution hyperspectral reference image is small, indicating good spectral preservation and detail injection.

[0064] By analyzing the super-resolution image, an intuitive understanding of the super-resolution results can be obtained. This embodiment further evaluates the image super-resolution results using objective evaluation metrics. The method described in this embodiment evaluates the super-resolution results using six metrics: MPSNR (Mean Peak Signal-to-Noise Ratio), MSSIM (Mean Structural Similarity), CC (Correlation Coefficient), RMSE (Root Mean Square Error), SAM (Spectral Angle Mapping), and ERGAS (Relative Global Dimensionless Error). The optimal values ​​for RMSE, SAM, and ERGAS are 0; the optimal values ​​for MSSIM and CC are 1; and the optimal value for MPSNR is positive infinity. The objective evaluation metrics for super-resolution results of different methods on different datasets are shown in Tables 1, 2, and 3.

[0065] Table 1: Objective Evaluation Metrics of Super-Resolution Results for Test Images in the Chikusei Dataset

[0066]

[0067] Table 2: Objective Evaluation Metrics for Super-Resolution Results of Test Images in the Houston Dataset

[0068]

[0069] Table 3: Objective Evaluation Metrics of Super-Resolution Results for Test Images on the Pavia Center Dataset

[0070]

[0071] As can be seen from Tables 1 to 3, the method described in this embodiment outperforms other methods in all metrics on the Chikusei, Houston, and Chikusei datasets. Table 4 shows the average time required for the method to infer a 128×128 resolution image when performing a 4x super-resolution experiment on the Houston dataset.

[0072] Table 4: Analysis of Super-Resolution Efficiency Indicators for Test Images on the Houston Dataset

[0073]

[0074] Figure 4 The efficiency of this embodiment is demonstrated by its inference time.

[0075] The ablation experiment in this embodiment was a 4x super-resolution experiment conducted on the Houston dataset. The experimental results are shown in Tables 5, 6 and 7.

[0076] Table 5: Validity of Low-Dimensional Residual Latent Space

[0077]

[0078] As can be seen from Table 5, the method in this embodiment achieved the best results, where "w / o Auto-encoder" means removing the autoencoder and "w / o residual space" means removing the residual space.

[0079] Table 6: Effectiveness of Multi-Scale Conditional Encoders and Efficient Diffusion Models

[0080]

[0081] Table 6 shows that the multi-scale encoder combined with the efficient diffusion model in this embodiment not only achieves the best super-resolution performance but also significantly reduces the number of model parameters. Among them, " f Encoder "" indicates a multi-scale conditional encoder, "SEDNet" indicates an efficient diffusion model network, and "UNet" indicates a network structure commonly used in diffusion models.

[0082] Table 7. Effectiveness of the Conditional Denoising Module

[0083]

[0084] In Table 7, "NAFNet" serves as the baseline for module design. "Simple CA→ECA" indicates replacing the Simple Channel Attention module (Simple CA) in NAFNet with the Efficient Channel Attention module (ECA), and "ECA→SECA" indicates replacing the Efficient Channel Attention module with a Lightweight Channel Attention module, which is the improvement in this embodiment. The baseline and Model-1 are two comparison models in this embodiment, respectively. Table 7 further verifies the effectiveness of the conditional denoising module in this embodiment. The diffusion model constructed by the conditional denoising module not only has fewer parameters but also achieves excellent super-resolution performance.

[0085] In summary, this embodiment involves upsampling and grouping low-resolution hyperspectral images. Each group is then sequentially encoded using an encoder network to obtain low-dimensional prior latent space information. This low-dimensional prior latent space information is input into a multi-scale conditional encoder to obtain conditional information. The low-dimensional prior latent space information is concatenated with Gaussian noise channels, and a diffusion model, guided by the conditional information, transforms the Gaussian noise into high-frequency residual latent space information. This residual latent space information is then added to the low-dimensional prior latent space information to obtain low-dimensional latent space information with high-frequency details. Finally, a pre-trained decoder network decodes and restores the image to obtain the final high-resolution hyperspectral super-resolution image. This embodiment employs a grouping strategy, combining an autoencoder to transform the hyperspectral pixel space into a low-dimensional residual latent space, and utilizes a lightweight diffusion model to efficiently generate high-frequency residual detail information. This method not only accelerates image generation but also significantly reduces computational resource requirements, greatly improving the practicality of the diffusion model. This method achieves excellent super-resolution results and can be widely applied in remote sensing image processing, target detection, and target recognition.

[0086] Furthermore, this embodiment also provides a hyperspectral image super-resolution system based on a latent space diffusion model, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model.

[0087] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model via a processor.

[0088] Furthermore, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model via a processor.

[0089] Those skilled in the art will understand that the technical solutions provided by the embodiments of this application may be in the form of a method, system, or computer program product. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create an implementation for the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A hyperspectral image super-resolution method based on a latent space diffusion model, characterized in that, This includes using a hyperspectral image super-resolution network, consisting of an encoder network, a multi-scale conditional encoder, a diffusion model, and a decoder network, to obtain a final high-resolution hyperspectral super-resolution image from a low-resolution hyperspectral image: S1, after upsampling the low-resolution hyperspectral image, group it according to the number of bands; S2, each group of images is sequentially passed through a pre-trained encoder network to reduce the dimensionality in the spectral dimension to obtain low-dimensional prior latent space information, and then the conditional information is obtained by encoding with a multi-scale conditional encoder. S3, after splicing the low-dimensional prior latent space information with the Gaussian noise channel, uses a diffusion model to transform the Gaussian noise into high-frequency residual latent space information under the guidance of conditional information; S4. Add the high-frequency residual latent space information to the low-dimensional prior latent space information to obtain low-dimensional latent space information with high-frequency details. S5. The low-dimensional latent space information of each group is decoded and restored through a pre-trained decoder network to obtain the final high-resolution hyperspectral super-resolution image. The diffusion model in step S3 includes a first 3×3 convolutional layer, a U-shaped network, a second 3×3 convolutional layer, and a temporal encoder. The first 3×3 convolutional layer, the U-shaped network, and the second 3×3 convolutional layer are connected sequentially, and the temporal encoder is connected in parallel with the U-shaped network. The U-shaped network structure includes an encoder, a bottom bottleneck layer, and a decoder. The encoder consists of four consecutive conditional denoising modules and a downsampling layer, used to extract features layer by layer and gradually reduce the spatial dimension of the feature map to achieve feature compression. The bottom bottleneck layer consists of one conditional denoising module, whose function is to connect the encoder and decoder. The encoder performs deep processing on the compressed features. The decoder consists of four consecutive conditional denoising modules and an upsampling layer, used to recover the spatial dimension of the feature map layer by layer. The decoder is directly connected to the corresponding encoder layer through skip connections. The temporal encoder includes a sinusoidal encoding operation, a multilayer perceptron (MLP) consisting of two linear layers and a SiLU activation function, and a reshaping operation. The time step t input to the temporal encoder is first sinusoidally encoded, and then the encoded result is input into the MLP for processing. Finally, the reshaping operation adjusts the shape of the temporal embedding to achieve effective fusion of temporal information and image features in the subsequent process.

2. The hyperspectral image super-resolution method based on the latent space diffusion model according to claim 1, characterized in that, The encoder network in step S2 comprises a 5×5 convolutional layer, multiple residual modules, and a 5×5 convolutional layer connected in sequence. The data input to the encoder network first passes through the 5×5 convolutional layer to obtain preliminary features. These preliminary features then pass through multiple residual modules and are added together. Finally, the data passes through a 5×5 convolutional layer to obtain the final output of the encoder network. The decoder network comprises a 5×5 convolutional layer, a first group of multiple residual modules, a 5×5 convolutional layer, a second group of multiple residual modules, and a 5×5 convolutional layer connected in sequence. The data input to the decoder network first passes through the 5×5 convolutional layer to obtain preliminary features. These preliminary features then pass through the first group of multiple residual modules and are added together. Finally, the data passes through a 5×5 convolutional layer. The convolutional layer obtains deep feature information. Multiple deep feature information is concatenated to obtain deep image features. The deep image features are first passed through a 5×5 convolutional layer to obtain reconstructed image features. The reconstructed image features are then passed through a second set of multiple residual modules and added to the reconstructed image features. Finally, they are passed through a 5×5 convolutional layer to obtain reconstructed image information. The reconstructed image information is then added to the deep image features to obtain the output of the decoder network. The residual module includes a 5×5 convolutional layer, a LeakyReLU activation function, and a 5×5 convolutional layer. The data input to the residual module is processed by a 5×5 convolutional layer, a LeakyReLU activation function, and a 5×5 convolutional layer, and then added to the data input to the residual module to obtain the output data of the residual module.

3. The hyperspectral image super-resolution method based on the latent space diffusion model according to claim 1, characterized in that, The multi-scale conditional encoder in step S2 includes a 3×3 convolutional layer and multiple spatial spectral prior modules connected in sequence. The data input to the multi-scale conditional encoder first passes through the 3×3 convolutional layer to extract preliminary features. Then, the preliminary features pass through multiple spatial spectral prior modules to obtain conditional information at multiple different scales. The spatial spectral prior module includes a 3×3 convolutional layer, multiple spatial modules, multiple spatial spectral modules, and a 3×3 downsampling convolutional layer connected in sequence. The data input to the spatial spectral prior module first passes through the 3×3 convolutional layer to extract initial depth features. The initial depth features pass through three spatial modules in sequence and are added to the initial depth features to generate intermediate depth features. The intermediate depth features are then processed by multiple spatial spectral modules and added to the intermediate depth features and the initial depth features to obtain enhanced depth features. Finally, the enhanced depth features are passed through a 3×3 downsampling convolutional layer to obtain the output conditional information. The spatial module includes a 3×3 convolutional layer, a LeakyReLU activation function, a 3×3 convolutional layer, another 3×3 convolutional layer, a LeakyReLU activation function, and a final 3×3 convolutional layer connected in sequence. The data input to the spatial module first passes through the first 3×3 convolutional layer and the LeakyReLU activation function in sequence, then through the second 3×3 convolutional layer, and is added to the initial input data to obtain preliminary processed features. Subsequently, these preliminary processed features pass through the third 3×3 convolutional layer, the LeakyReLU activation function, and the final 3×3 convolutional layer in sequence, and are then added to the preliminary processed features to form the final output features of the spatial module. The spatial spectral module comprises a 3×3 convolutional layer, a LeakyReLU activation function, a 3×3 convolutional layer, a 1×1 convolutional layer, a LeakyReLU activation function, a 1×1 convolutional layer, and a channel attention module connected in sequence. The data input to the spatial spectral module first passes through the first 3×3 convolutional layer and the LeakyReLU activation function, then through the second 3×3 convolutional layer, and is added to the initial input data to generate preliminary features. This preliminary feature then passes through the first 1×1 convolutional layer, the LeakyReLU activation function, the second 1×1 convolutional layer, and the channel attention module, and is added to the preliminary feature to form the final output feature of the spatial spectral module.

4. The hyperspectral image super-resolution method based on the latent space diffusion model according to claim 1, characterized in that, The conditional denoising module processes the input features as follows: the features input to the conditional denoising module are sequentially processed through layer normalization, a 1×1 convolutional layer, a 3×3 depthwise convolutional layer, and a simple gate unit to generate first-level features; the first-level features are processed through a lightweight channel attention module and then through a 1×1 convolutional layer to obtain second-level features; the second-level features are added to the features input to the conditional denoising module to obtain intermediate features; the intermediate features are further processed through layer normalization, a 1×1 convolutional layer, a simple gate unit, and another 1×1 convolutional layer, and then added back to the intermediate features to generate the output features of the conditional denoising module; The simple gating unit processes the input features by dividing the input data into two parts along the channel direction and performing element-wise multiplication on the two sub-data. The lightweight channel attention module processes the input features by applying global average pooling to the input features of dimension H×W×C and then transforming them into 1×C. Then, a one-dimensional convolutional layer of length 3 is used to perform convolution in dimension C. The weights obtained after convolution are transformed in dimension and then multiplied element-wise with the input features of the lightweight channel attention module to obtain the output features of the lightweight channel attention module.

5. The hyperspectral image super-resolution method based on the latent space diffusion model according to claim 1, characterized in that, Step S1 is preceded by training a hyperspectral image super-resolution network: S101, a pre-trained autoencoder network consisting of both an encoder network and a decoder network; S102 combines the encoder and decoder networks in the pre-trained autoencoder network, uses a multi-scale conditional encoder to obtain conditional information, and uses the conditional information to guide the training of the diffusion model in the low-dimensional residual latent space, finally completing the training of the entire hyperspectral image super-resolution network.

6. The hyperspectral image super-resolution method based on the latent space diffusion model according to claim 5, characterized in that, In step S101, pre-training the autoencoder network, which consists of both an encoder network and a decoder network, includes training the autoencoder network using a training set, and determining the optimal network parameters of the autoencoder network by optimizing a preset first loss function to minimize the loss function. The functional expression of the first loss function is as follows: ， ， ， ， in, The first loss function is preset. For absolute value error loss, For spectral angle loss, For spectral gradient loss, and For weight constants, These represent the total height, total width, and total number of bands of a high-resolution hyperspectral image. This represents a high-resolution hyperspectral reference image in the training set. Indicates will The high-resolution hyperspectral reconstructed image obtained from the input autoencoder network, For high-resolution hyperspectral reconstructed images Pixel value at that location, For high-resolution hyperspectral reference images Pixel value at that location, For the height coordinates, width coordinates, and band coordinates of a high-resolution hyperspectral image, In high-resolution hyperspectral reconstructed images Spectral gradient at that location, For high-resolution hyperspectral reference images The spectral gradient at the location; in step S102, when using the multi-scale conditional encoder to obtain conditional information and guide the training of the diffusion model in the low-dimensional residual latent space using conditional information, the expression of the second loss function is: ， In the above formula, For the second loss function, In the high-resolution hyperspectral reference image predicted by the diffusion model at the i-th time step Noise at that location In the high-resolution hyperspectral reference image added for the i-th time step The noise at the location is such that the time step i ranges from 1 to 2000.

7. A hyperspectral image super-resolution system based on a latent space diffusion model, comprising an interconnected microprocessor and a memory, characterized in that, The microprocessor is programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model as described in any one of claims 1 to 6 via a processor.

9. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the hyperspectral image super-resolution method based on the latent space diffusion model as described in any one of claims 1 to 6 via a processor.

Images