Hyperspectral image super-resolution method, device and system and storage medium

Abstract

The invention provides a hyperspectral image super-resolution method, device and system and a storage medium, and belongs to the technical field of image reconstruction, and the method comprises the steps: importing an original high-resolution hyperspectral image and an original low-resolution hyperspectral image; a high-resolution model and a low-resolution model are constructed, and high-resolution feature analysis is performed on an original high-resolution hyperspectral image through the high-resolution model to obtain high-resolution dictionary features and target high-resolution coefficient features. The technical problem that the quality of the generated result is poor in the process of generating the high-resolution hyperspectral image from the low-resolution hyperspectral image at present is solved, end-to-end mapping between the low-resolution hyperspectral image and the high-resolution hyperspectral image is achieved, and therefore the high-spatial-resolution hyperspectral image with higher quality is generated, and the image quality is improved. And the calculation cost and the calculation time are reduced.

Description

Technical Field

[0001] The present invention mainly relates to the field of image reconstruction technology, and in particular to a hyperspectral image super-resolution method, device, system and storage medium. Background Art

[0002] Hyperspectral images possess rich reflectance information and a continuous, wide spectral range. Thanks to this rich spectral information, hyperspectral images are widely used in various fields, including target detection, object classification, and remote sensing change detection. However, the acquisition of hyperspectral images is always subject to a trade-off between spatial fineness and spectral coverage. Due to the physical limitations of imaging sensors, it is impossible to obtain high-resolution hyperspectral images that simultaneously capture both spatial and spectral domains. Therefore, as a classic computer vision task, hyperspectral image super-resolution is a powerful alternative to directly acquiring high-resolution hyperspectral images in terms of cost-effectiveness, practicality, and sustainability.

[0003] In recent years, researchers have proposed a large number of single hyperspectral image super-resolution methods, including sparse representation, total variation, and low-rank priors. However, these methods typically model the super-resolution process as a complex optimization problem. While highly interpretable, they require a large number of iterative computations, resulting in high computational costs and a time-consuming process. Deep learning methods have garnered widespread attention in various fields of computer vision due to their powerful feature extraction and representation capabilities. Some scholars have utilized deep learning-based hyperspectral image super-resolution techniques to improve the subjective and objective quality of reconstructed images. Although deep learning-based methods offer a highly efficient optimization process, their inherent "black box" nature makes their decision-making process difficult to understand and verify, preventing their generalization and application in highly interpretable fields such as government and public policymaking. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and provide a hyperspectral image super-resolution method, device, system and storage medium.

[0005] The present invention solves the above-mentioned technical problem with the following technical solution: A hyperspectral image super-resolution method comprises the following steps:

[0006] Importing a plurality of original high-resolution hyperspectral images and an original low-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images;

[0007] Constructing a high-resolution model and a low-resolution model, and performing high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain a high-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images;

[0008] Performing low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images;

[0009] The low-resolution model is trained by using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model;

[0010] Importing a low-resolution image to be processed, reconstructing the low-resolution image to be processed using the super-resolution model to obtain a hyperspectral image super-resolution result.

[0011] Another technical solution of the present invention to solve the above technical problem is as follows: a hyperspectral image super-resolution device, comprising:

[0012] An import module, configured to import a plurality of original high-resolution hyperspectral images and an original low-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images;

[0013] a high-resolution feature analysis module, configured to construct a high-resolution model and a low-resolution model, and perform high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain a high-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images;

[0014] a low-resolution feature analysis module, configured to perform low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain a target low-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target low-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images;

[0015] a model training module, configured to train the low-resolution model using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model;

[0016] The import module is further used to import the low-resolution image to be processed;

[0017] The super-resolution result obtaining module is used to reconstruct the low-resolution image to be processed by using the super-resolution model to obtain a hyperspectral image super-resolution result.

[0018] Based on the above-mentioned hyperspectral image super-resolution method, the present invention also provides a hyperspectral image super-resolution system.

[0019] Another technical solution of the present invention to solve the above technical problem is as follows: A hyperspectral image super-resolution system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the hyperspectral image super-resolution method described above is implemented.

[0020] Based on the above-mentioned hyperspectral image super-resolution method, the present invention also provides a computer-readable storage medium.

[0021] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the hyperspectral image super-resolution method as described above is implemented.

[0022] The beneficial effects of the present invention are as follows: high-resolution dictionary features and target high-resolution coefficient features are obtained by analyzing the high-resolution features of the original high-resolution hyperspectral image through a high-resolution model, target low-resolution dictionary features and target low-resolution coefficient features are obtained by analyzing the low-resolution features of the original low-resolution hyperspectral image through a low-resolution model, a super-resolution model is obtained by training the low-resolution model through the high-resolution dictionary features, target high-resolution coefficient features, target low-resolution dictionary features and target low-resolution coefficient features, and a hyperspectral image super-resolution result is obtained by reconstructing the image of the low-resolution image to be processed through the super-resolution model, which solves the technical problem of poor quality of the generated results in the process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images, thereby generating higher quality high spatial resolution hyperspectral images, and reducing computing cost and computing time. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of a process for a hyperspectral image super-resolution method according to an embodiment of the present invention;

[0024] Figure 2One of the module schematic diagrams of the training model of the hyperspectral image super-resolution method provided by an embodiment of the present invention;

[0025] Figure 3 A second schematic diagram of a module of a training model for a hyperspectral image super-resolution method according to an embodiment of the present invention;

[0026] Figure 4 This is a module block diagram of the hyperspectral image super-resolution device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0028] Figure 1 A schematic diagram of a process for super-resolution of hyperspectral images provided by an embodiment of the present invention.

[0029] like Figure 1 As shown, a hyperspectral image super-resolution method includes the following steps:

[0030] Importing a plurality of original high-resolution hyperspectral images and an original low-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images;

[0031] Constructing a high-resolution model and a low-resolution model, and performing high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain a high-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images;

[0032] Performing low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images;

[0033] The low-resolution model is trained by using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model;

[0034] Importing a low-resolution image to be processed, reconstructing the low-resolution image to be processed using the super-resolution model to obtain a hyperspectral image super-resolution result.

[0035] It should be understood that a high-resolution hyperspectral image (ie, an original high-resolution hyperspectral image) is input to train the teacher network (ie, a high-resolution model) in a self-supervised learning manner.

[0036] It should be understood that the teacher network (i.e., the high-resolution model) is trained to obtain a high-resolution dictionary (i.e., high-resolution dictionary features) and sparse representation coefficients (i.e. target high resolution coefficient features).

[0037] Specifically, freeze the teacher network (i.e., high-resolution model) and train the student network (i.e., low-resolution model) to obtain a low-resolution high-dimensional dictionary (i.e., target low-resolution dictionary features) and sparse representation coefficients (i.e., target low-resolution coefficient features).

[0038] In the above embodiment, the high-resolution features of the original high-resolution hyperspectral image are analyzed by the high-resolution model to obtain high-resolution dictionary features and target high-resolution coefficient features, the low-resolution features of the original low-resolution hyperspectral image are analyzed by the low-resolution model to obtain target low-resolution dictionary features and target low-resolution coefficient features, the low-resolution model is trained by the high-resolution dictionary features, the target high-resolution coefficient features, the target low-resolution dictionary features and the target low-resolution coefficient features to obtain a super-resolution model, and the image of the low-resolution image to be processed is reconstructed by the super-resolution model to obtain a hyperspectral image super-resolution result, which solves the technical problem of poor quality of the generated results in the process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images, thereby generating higher quality high spatial resolution hyperspectral images, and reducing computational cost and computational time.

[0039] Optionally, as an embodiment of the present invention, the high-resolution model includes a plurality of first ESSA Block networks and a high-resolution coefficient generation network;

[0040] The process of performing high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain high-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes:

[0041] Performing feature extraction of high-resolution dictionary features on each of the original high-resolution hyperspectral images through the plurality of first ESSA Block networks to obtain high-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images;

[0042] The high-resolution coefficient generation network is used to extract the high-resolution coefficient features of each of the original high-resolution hyperspectral images, thereby obtaining target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images.

[0043] Preferably, the number of the first ESSA Block networks may be 5.

[0044] Specifically, the first ESSA Block network belongs to the prior art and adopts the ESSA Block network in the document: Zhang M, Zhang C, Zhang Q, et al. Essaformer: Efficient transformer for hyperspectralimage super-resolution[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision.2023:23073-23084.

[0045] It should be understood that it includes two parts: a dictionary generation network and a coefficient generation network. The dictionary generation network consists of 5 ESSA Blocks (i.e., the first ESSA Block network), which learns the corresponding high-resolution dictionary according to the input high-resolution hyperspectral image (i.e., the original high-resolution hyperspectral image). (i.e., high-resolution dictionary features).

[0046] In the above embodiment, high-resolution feature analysis is performed on the original high-resolution hyperspectral image through a high-resolution model to obtain high-resolution dictionary features and target high-resolution coefficient features, which solves the technical problem of poor quality of generated results in the current process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, and realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images.

[0047] Optionally, as an embodiment of the present invention, the high-resolution coefficient generation network includes a first fully connected layer, a plurality of TInv Block layers, a first 1D attention layer, and a second fully connected layer;

[0048] The process of extracting high-resolution coefficient features from each of the original high-resolution hyperspectral images through the high-resolution coefficient generation network to obtain target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes:

[0049] Performing feature extraction processing on each of the original high-resolution hyperspectral images through the first fully connected layer, obtaining original high-resolution hyperspectral features corresponding to each of the original high-resolution hyperspectral images;

[0050] Performing feature extraction on each of the original high-resolution hyperspectral features through the plurality of TInv Block layers to obtain feature-extracted high-resolution hyperspectral features corresponding to each of the original high-resolution hyperspectral images;

[0051] updating the weights of the high-resolution hyperspectral features after feature extraction by the first 1D attention layer to obtain updated high-resolution hyperspectral features corresponding to the original high-resolution hyperspectral images;

[0052] Feature extraction is performed on each of the updated high-resolution hyperspectral features through the second fully connected layer to obtain a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images.

[0053] It should be understood that the first and second fully connected layers, also known as Linear layers, are also called fully connected layers or densely connected layers in the field of deep learning. Linear layers are a fundamental component of neural networks, mapping input features to a new feature space, which helps the model learn complex relationships between input and output. A multilayer perceptron (MLP) can be constructed by stacking multiple Linear layers and combining them with nonlinear activation functions to enhance the model's expressive power.

[0054] Specifically, the TInv Block layer belongs to the prior art and adopts the TInv Block layer in the document: Wang J, Lu T, Huang X, et al. Pan-sharpening via conditional invertible neural network[J]. Information Fusion, 2024, 101: 101980.

[0055] It should be understood that the first 1D attention layer is a key component for processing sequence data (such as time series, audio, text), which allows the model to focus on important parts of the sequence.

[0056] Specifically, the coefficient network (i.e., the high-resolution coefficient generation network) consists of a Linear layer (i.e., the first fully connected layer and the second fully connected layer), a TInv Block (i.e., the TInv Block layer) and a 1D attention layer (i.e., the first 1D attention layer), and learns the corresponding sparse representation coefficients according to the input high-resolution hyperspectral image (i.e., the original high-resolution hyperspectral image). (i.e. target high resolution coefficient features).

[0057] In the above embodiment, the high-resolution coefficient generation network is used to extract the high-resolution coefficient features of the original high-resolution hyperspectral image to obtain the target high-resolution coefficient features, which solves the technical problem of poor quality of the generated results in the current process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images, thereby generating higher-quality high-spatial resolution hyperspectral images and reducing computational cost and computational time.

[0058] Optionally, as an embodiment of the present invention, the low-resolution model includes a low-resolution dictionary generation network and a low-resolution coefficient generation network;

[0059] The process of performing low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes:

[0060] Performing feature extraction of low-resolution dictionary features on each of the original low-resolution hyperspectral images through the low-resolution dictionary generation network to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images;

[0061] The low-resolution coefficient generation network is used to extract the low-resolution coefficient features of each of the original low-resolution hyperspectral images, thereby obtaining target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images.

[0062] In the above embodiment, a low-resolution feature analysis is performed on the original low-resolution hyperspectral image through a low-resolution model to obtain a target low-resolution dictionary feature and a target low-resolution coefficient feature, thereby realizing an end-to-end mapping between the low-resolution hyperspectral image and the high-resolution hyperspectral image, thereby generating a higher-quality high-spatial-resolution hyperspectral image and reducing the computational cost and computational time.

[0063] Optionally, as an embodiment of the present invention, the low-resolution dictionary generation network includes multiple second ESSA Block networks and upsampling layers;

[0064] The process of extracting low-resolution dictionary features from each of the original low-resolution hyperspectral images using the low-resolution dictionary generation network to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images includes:

[0065] Performing feature extraction on each of the original low-resolution hyperspectral images through a plurality of the second ESSA Block networks to obtain original low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images;

[0066] The upsampling layer performs upsampling processing on each of the original low-resolution dictionary features to obtain a target low-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images.

[0067] Preferably, the number of the plurality of second ESSA Block networks may be 3.

[0068] It should be understood that the second ESSA Block network belongs to the prior art and adopts the ESSA Block network in the document: Zhang M, Zhang C, Zhang Q, et al. Essaformer: Efficient transformer for hyperspectralimage super-resolution[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision.2023:23073-23084.

[0069] Specifically, a student network (i.e., low-resolution model) is constructed that is similar to the teacher network (i.e., high-resolution model) but has a lighter dictionary and coefficient network. During the training of the student network (i.e., low-resolution model), the teacher network (i.e., high-resolution model) is frozen, and a low-resolution hyperspectral image (i.e., the original low-resolution hyperspectral image) is input to generate a dictionary (i.e., target low-resolution dictionary features) and coefficients (i.e., target low-resolution coefficient features) of the low-resolution image. Since low-resolution and high-resolution images differ in spatial scale, the dictionary of the low-resolution image needs to be upsampled to align it with the dictionary of the high-resolution image to generate a high-dimensional low-resolution dictionary.

[0070] In the above embodiment, the low-resolution dictionary features are extracted from the original low-resolution hyperspectral images through the low-resolution dictionary generation network to obtain the target low-resolution dictionary features, thereby realizing end-to-end mapping between the low-resolution hyperspectral images and the high-resolution hyperspectral images, thereby generating higher-quality high-spatial resolution hyperspectral images and reducing computational cost and computational time.

[0071] Optionally, as an embodiment of the present invention, the low-resolution coefficient generation network includes a third fully connected layer, a second 1D attention layer, and a fourth fully connected layer;

[0072] The process of extracting low-resolution coefficient features from each of the original low-resolution hyperspectral images through the low-resolution coefficient generation network to obtain target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes:

[0073] Performing feature extraction on each of the original low-resolution hyperspectral images through the third fully connected layer to obtain original low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images;

[0074] updating the weights of the original low-resolution coefficient features respectively through the second 1D attention layer, obtaining updated low-resolution coefficient features corresponding to the original high-resolution hyperspectral images;

[0075] Feature extraction is performed on each of the updated low-resolution coefficient features through the fourth fully connected layer to obtain target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images.

[0076] It should be understood that the third and fourth fully connected layers, also known as Linear layers, are also called fully connected layers or densely connected layers in the field of deep learning. Linear layers are a fundamental component of neural networks, mapping input features to a new feature space, which helps the model learn complex relationships between input and output. Multilayer perceptrons (MLPs) can be constructed by stacking multiple Linear layers and combining them with nonlinear activation functions to enhance the model's expressive power.

[0077] It should be understood that the second 1D attention layer is a key component for processing sequence data (such as time series, audio, text), which allows the model to focus on important parts of the sequence.

[0078] In the above embodiment, the low-resolution coefficient generation network is used to extract the low-resolution coefficient features of the original low-resolution hyperspectral image to obtain the target low-resolution coefficient features, thereby realizing end-to-end mapping between the low-resolution hyperspectral image and the high-resolution hyperspectral image, thereby generating a higher-quality high-spatial-resolution hyperspectral image and reducing the computational cost and computational time.

[0079] Optionally, as an embodiment of the present invention, the process of training the low-resolution model using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model includes:

[0080] Performing feature multiplication on each of the target low-resolution dictionary features and the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images to obtain a first reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images;

[0081] Performing feature multiplication on each of the high-resolution dictionary features and the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images to obtain a second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images;

[0082] Extracting spectral features of each of the first reconstructed high-resolution hyperspectral images through a 3×3 convolutional layer to obtain spectral features corresponding to each of the original high-resolution hyperspectral images and spectral channel dimensions corresponding to each of the original high-resolution hyperspectral images;

[0083] Performing feature extraction of spatial features on each of the first reconstructed high-resolution hyperspectral images through a 1×1 convolutional layer to obtain spatial features corresponding to each of the original high-resolution hyperspectral images and spatial channel dimensions corresponding to each of the original high-resolution hyperspectral images;

[0084] Normalizing each of the spectral features through a layer normalization layer to obtain a spectral query vector corresponding to each of the original high-resolution hyperspectral images;

[0085] Normalizing each of the spatial features through the layer normalization layer to obtain a spatial query vector corresponding to each of the original high-resolution hyperspectral images;

[0086] Performing feature extraction on each of the spectral query vectors through a convolutional layer to obtain a spectral key vector corresponding to each of the original high-resolution hyperspectral images;

[0087] Performing feature extraction on each of the spatial query vectors through the convolutional layer to obtain a spatial key vector corresponding to each of the original high-resolution hyperspectral images;

[0088] Performing feature extraction on each of the spectral key vectors through a depthwise separable convolutional layer to obtain a spectral numerical vector corresponding to each of the original high-resolution hyperspectral images;

[0089] Performing feature extraction on each of the spatial key vectors through the depthwise separable convolutional layer to obtain a spatial numerical vector corresponding to each of the original high-resolution hyperspectral images;

[0090] The spectral attention corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the spatial query vectors, the spectral channel dimension corresponding to each of the original high-resolution hyperspectral images, and the spectral key vector corresponding to each of the original high-resolution hyperspectral images through the first formula. The first formula is:

[0091]

[0092] Among them, CA j is the spectral attention corresponding to the j-th original high-resolution hyperspectral image, SoftMax() is the SoftMax function, is the spatial query vector corresponding to the j-th original high-resolution hyperspectral image, is the spectral key vector corresponding to the j-th original high-resolution hyperspectral image, is the spectral channel dimension corresponding to the j-th original high-resolution hyperspectral image;

[0093] The spatial attention corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the spectral query vectors, the spatial channel dimension corresponding to each of the original high-resolution hyperspectral images, and the spatial key vector corresponding to each of the original high-resolution hyperspectral images through the second formula. The second formula is:

[0094]

[0095] Among them, SA j is the spatial attention corresponding to the j-th original high-resolution hyperspectral image, SoftMax() is the SoftMax function, is the spectral query vector corresponding to the j-th original high-resolution hyperspectral image, is the spatial key vector corresponding to the j-th original high-resolution hyperspectral image, is the spatial channel dimension corresponding to the j-th original high-resolution hyperspectral image;

[0096] The target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the spectral features, the spectral attention corresponding to each of the original high-resolution hyperspectral images, the spectral numerical vector corresponding to each of the original high-resolution hyperspectral images, the spatial features corresponding to each of the original high-resolution hyperspectral images, the spatial attention corresponding to each of the original high-resolution hyperspectral images, and the spatial numerical vector corresponding to each of the original high-resolution hyperspectral images through the third formula. The third formula is:

[0097]

[0098] in, is the target high-resolution hyperspectral image corresponding to the jth original high-resolution hyperspectral image, Conv() is the convolution process, f FFN () is the feedforward neural network function, γ and β are both learnable parameters, CA j is the spectral attention corresponding to the j-th original high-resolution hyperspectral image, is the spatial numerical vector corresponding to the j-th original high-resolution hyperspectral image, is the spatial feature corresponding to the jth original high-resolution hyperspectral image, SA j is the spatial attention corresponding to the j-th original high-resolution hyperspectral image, is the spectral numerical vector corresponding to the jth original high-resolution hyperspectral image, is the spectral feature corresponding to the jth original high-resolution hyperspectral image;

[0099] Calculating a loss function for each of the high-resolution dictionary features, the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, and the target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, respectively, to obtain a target distillation loss function corresponding to each of the original high-resolution hyperspectral images;

[0100] The parameters of the low-resolution model are updated according to all the target distillation loss functions to obtain a super-resolution model.

[0101] It should be understood that the high-dimensional low-resolution dictionary of the student network (i.e., target low-resolution dictionary features) and sparse representation coefficients (i.e., the target low-resolution coefficient feature) is multiplied to obtain the initial reconstructed hyperspectral image (i.e., high-resolution hyperspectral image after the first reconstruction).

[0102] Specifically, the dictionary learned by the student network (i.e., the target low-resolution dictionary features) and the coefficients (i.e., the target low-resolution coefficient features) are multiplied and restored to the image space to obtain the initial reconstructed high-resolution hyperspectral image. (i.e., high-resolution hyperspectral image after the first reconstruction).

[0103] It should be understood that the high-resolution dictionary (i.e., high-resolution dictionary features) and sparse representation coefficients (i.e. target high resolution coefficient feature) multiplied to obtain a high resolution image (i.e., high-resolution hyperspectral image after the second reconstruction).

[0104] Specifically, the dictionary (i.e., high-resolution dictionary features) and coefficients (i.e., target high-resolution coefficient features) learned by the teacher network are restored to the image space through the inverse singular value decomposition process to obtain the reconstructed high-resolution hyperspectral image. (i.e., high-resolution hyperspectral image after the second reconstruction).

[0105] It should be understood that optimizing the initial reconstructed hyperspectral image (i.e., the spatial features) and spectral features of the high-resolution hyperspectral image after the first reconstruction to obtain the final hyperspectral image (i.e. target high-resolution hyperspectral image).

[0106] Specifically, the initial reconstruction of the student network (i.e., the high-resolution hyperspectral image after the first reconstruction) is refined for spatial and spectral features. First, 3×3 (i.e., 3×3 convolutional layer) generates spectral features, and 1×1 convolution (i.e., 1×1 convolutional layer) generates spatial features. Then, spatial tokens Q are constructed from the spatial and spectral features respectively. spa (i.e. spectral query vector), K spa (i.e. spectral key vector), V spa (i.e., spectral numerical vector), and spectral tokens Q spe (i.e. spatial query vector), K spe (i.e., spatial key vector), V spe (i.e., spatial numerical vectors). The spatial and spectral features of the image are restored through the cross-attention mechanism.

[0107] Specifically, the dictionary branch of the student network needs to upsample the low-resolution dictionary to restore it to a scale aligned with the high-resolution dictionary. Therefore, the initial reconstructed image of the student network (i.e., the high-resolution hyperspectral image after the first reconstruction) is:

[0108]

[0109] It should be understood that the dictionary of the teacher network (i.e., high-resolution dictionary features) and coefficients (i.e., target high-resolution coefficient features) supervise and distill the high-dimensional low-resolution dictionary of the student network (i.e., target low-resolution dictionary features) and sparse representation coefficients (i.e. target low-resolution coefficient features) learning.

[0110] Specifically, considering the reconstruction accuracy of the student network and the unique spectral and spatial characteristics of the image, it is necessary to reconstruct the image of the student network. Fine-tuning and optimization are performed to restore more ideal spatial and spectral fidelity. Therefore, a spatial-spectral cross attention module is proposed, which consists of spatial and spectral branches. Specifically, 3×3 (i.e., 3×3 convolutional layer) and 1×1 convolution (i.e., 1×1 convolutional layer) are used to extract the spectral and spatial features of the image, respectively. spe (i.e. spectral characteristics) and X spa (i.e. spatial features), input into the spatial-spectral cross attention module, after layer normalization (Layer Normalization) (i.e. layer normalization layer), convolution (Conv) (i.e. convolution layer) and depth-wise separable convolution (i.e. depth-wise separable convolution layer), obtain the respective query vectors Q (i.e. spatial query vector and spectral query vector), key vectors K (i.e. spatial key vector and spectral key vector) and value vectors V (i.e. spatial value vector and spectral value vector), which are respectively:

[0111] Q spa ,K spa =W QK X spa ,V spa =W V X spa ,

[0112] Q spe ,K spe =X QK X spe ,V spe =W V X spe ,

[0113] Among them, W QK , W V is a learnable linear transformation from X spa Get Q spa ,K spa , V spa , from X spe Get Q spe ,K spe , V spe The calculation formulas for spatial attention SA (Spatial Atention) and spectral attention CA (Channel Attention) are as follows:

[0114]

[0115] Where d1 and d2 represent X spa and X speThe channel dimension of , prevents the gradient from disappearing. Since K and Q come from two different inputs, the two attentions fuse information of other dimensions. The final output is as follows:

[0116] Y spa =f FFN (γ(CA×V spa )+X spa ),

[0117] Y spe =f FFN (β(SA×V spe )+X spe ),

[0118] Among them, β and γ are learnable parameters, FFN represents feedforward neural network, and the final reconstructed image of the student network is (i.e. the target high-resolution hyperspectral image) is:

[0119]

[0120] In the above embodiment, a super-resolution model is obtained by training a low-resolution model through all high-resolution dictionary features, all target high-resolution coefficient features, all target low-resolution dictionary features, and all target low-resolution coefficient features, thereby restoring the spectral and texture details of the hyperspectral image. This solves the technical problem of poor quality of the generated results in the current process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images, thereby generating higher-quality high-spatial resolution hyperspectral images and reducing computational cost and computational time.

[0121] Optionally, as an embodiment of the present invention, the process of performing loss function calculation on each of the high-resolution dictionary features, the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, and the target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images to obtain the target distillation loss function corresponding to each of the original high-resolution hyperspectral images includes:

[0122] The target distillation loss function corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the high-resolution dictionary features, the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, and the target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images through the fourth formula. The fourth formula is:

[0123]

[0124] in,

[0125] in, is the target distillation loss function corresponding to the jth original high-resolution hyperspectral image, α, δ, and ε are all balance coefficients, is the target high-resolution hyperspectral image corresponding to the j-th original high-resolution hyperspectral image, is the second reconstructed high-resolution hyperspectral image corresponding to the j-th original high-resolution hyperspectral image, is the dictionary distillation loss function corresponding to the j-th original high-resolution hyperspectral image, is the coefficient distillation loss function corresponding to the j-th original high-resolution hyperspectral image, is the high-resolution dictionary feature corresponding to the j-th original high-resolution hyperspectral image, is the target low-resolution dictionary feature corresponding to the j-th original high-resolution hyperspectral image, is the target high-resolution coefficient feature corresponding to the j-th original high-resolution hyperspectral image, is the target low-resolution coefficient feature corresponding to the j-th original high-resolution hyperspectral image, and ‖‖1 is the L1 norm.

[0126] It should be understood that according to the high-resolution dictionary obtained through the teacher network (i.e., high-resolution dictionary features) and coefficients (i.e., target high-resolution coefficient features), using relational knowledge distillation to supervise and guide student network dictionaries (i.e., target low-resolution dictionary features) and coefficients The learning and updating of the target low-resolution coefficient feature is as follows:

[0127]

[0128] Among them, L dicrepresents the loss of dictionary distillation (i.e., dictionary distillation loss function), L coe Represents the loss of coefficient distillation (i.e., coefficient distillation loss function).

[0129] Specifically, the distillation loss function of the student network (i.e., the target distillation loss function) is:

[0130]

[0131] Where α, δ and ε are balance coefficients.

[0132] In the above embodiment, the loss function is calculated for each high-resolution dictionary feature, target high-resolution coefficient feature, target low-resolution dictionary feature, target low-resolution coefficient feature, second reconstructed high-resolution hyperspectral image, and target high-resolution hyperspectral image to obtain a target distillation loss function, which solves the current technical problem of poor quality of generated results in the process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images, thereby generating higher quality high spatial resolution hyperspectral images and reducing computational cost and computational time.

[0133] Optionally, as another embodiment of the present invention, the present invention includes: according to the dictionary learning theory, a high-dimensional tensor can be decomposed into the product of sparse coefficients and low-rank subspaces, that is, the product of sparse representation coefficients and low-rank dictionaries. In order to achieve effective supervision of dictionary and coefficient learning, this method adopts the idea of ​​knowledge distillation and is divided into two stages: teacher network and student network. The dictionary and coefficients of high-resolution images learned by the teacher network are used to supervise the learning of dictionary and coefficients in the process of low-resolution image reconstruction. Specifically, the teacher network is trained to obtain the dictionary of high-resolution images. and sparse representation coefficients Multiply the high-resolution dictionary and the sparse representation coefficient to obtain a high-resolution image Freeze the teacher network and train the student network to obtain low-resolution dictionary and sparse representation coefficients Upsample the dictionary to obtain a high-dimensional low-resolution dictionary Teacher Network Dictionary and coefficients Dictionary for Supervising Student Networks and coefficients Learning and updating; dictionary of student network and coefficients Multiply to obtain the initial reconstructed high-resolution hyperspectral image Spatial-spectral cross-attention module refinement The spatial and spectral characteristics of the super-resolution reconstruction are used to obtain the high spatial resolution hyperspectral image. The present invention solves the technical problem of poor quality of the generated results in the current process of generating high-resolution hyperspectral images from low-resolution hyperspectral images, and realizes end-to-end mapping between low-resolution hyperspectral images and high-resolution hyperspectral images, thereby generating higher-quality high-spatial resolution hyperspectral images.

[0134] Optionally, as another embodiment of the present invention, the present invention includes the following steps:

[0135] Step S1: Train the teacher network to obtain a high-resolution dictionary and sparse representation coefficients

[0136] Step S2: High-resolution dictionary and sparse representation coefficients Multiply to get a high-resolution image

[0137] Step S3: Freeze the teacher network and train the student network to obtain a low-resolution high-dimensional dictionary and sparse representation coefficients

[0138] Step S4: Dictionary of the teacher network and coefficients High-dimensional low-resolution dictionary for supervising student networks and sparse representation coefficients study;

[0139] Step S5: High-dimensional low-resolution dictionary of the student network and sparse representation coefficients Multiply to get the initial reconstructed hyperspectral image

[0140] Step S6: Spatial-spectral cross attention module optimizes the initial reconstructed hyperspectral image The spatial and spectral characteristics of the final hyperspectral image are obtained

[0141] Optionally, as another embodiment of the present invention, the present invention addresses the problems of complex optimization process and poor interpretability in existing deep learning-based hyperspectral image super-resolution technology, and proposes a new framework by using sparse representation modeling theory, i.e., assuming that three-dimensional hyperspectral images can be decomposed into the product of a low-rank dictionary and sparse coefficients.

[0142] Optionally, as another embodiment of the present invention, compared with the existing methods, the present invention has advantages and positive effects: the present invention is composed of a teacher network, a student network and spatial-spectral cross attention. The teacher network is trained to obtain a high-resolution dictionary and sparse representation coefficients. The teacher network is frozen, and the student network is trained to obtain a low-resolution high-dimensional dictionary and sparse representation coefficients. The dictionary and coefficients of the teacher network supervise and distill the dictionary and sparse representation coefficient learning of the student network. The spatial-spectral cross attention module optimizes the spatial and spectral features of the student network to reconstruct the hyperspectral image, and obtains the final hyperspectral image. The present invention obtains the true value of the dictionary and coefficients from the self-supervised learning of the teacher stage, guides the lightweight student network to learn the dictionary and coefficients, and the spatial-spectral cross attention module ensures the fidelity of the spectral and spatial information of the reconstruction results of the student network.

[0143] Optionally, as another embodiment of the present invention, the environment used in the example of the present invention is: the CPU of the server is Xeon(R)Silver 4316CPU, NVIDIA GeForce RTX 4090D GPU, Ubuntu 20.04.5 operating system, Pytorch 2.5.1+cu118 compilation environment, Python 3.11.

[0144] Alternatively, as another embodiment of the present invention, according to the dictionary learning theory, a high-dimensional tensor can be decomposed into the product of sparse coefficients and low-rank subspaces, that is, the product of sparse representation coefficients and low-rank dictionaries. Combining dictionary learning and spectral low-rank properties, the hyperspectral image I can be decomposed into

[0145] I=D ic C oe ,

[0146] D ic represents the low-rank dictionary, C oe Represents the coefficient.

[0147] The high-resolution multispectral image is represented as I hr ∈R C×H×W , the corresponding low-resolution hyperspectral image I lr ∈R C×h×w Where H, h and W, w represent height and width, and C represents the number of spectral bands of the hyperspectral image. hr and I lr With the same spectral resolution and different spatial resolution. Considering that the true value of the dictionary and coefficient of the hyperspectral image is difficult to obtain, the high-resolution hyperspectral image I is used. hr The teacher network is trained in a self-supervised manner. The teacher network mainly consists of a coefficient learning branch and a dictionary learning branch, which are used to obtain the dictionary of high-resolution images. and coefficients And restore to high-resolution hyperspectral image The loss function L of the teacher network t as follows:

[0148]

[0149] in,

[0150] Alternatively, as another embodiment of the present invention, Figure 2 As shown in the figure, the advantage of the knowledge distillation of the present invention is that it can transfer knowledge from a complex and large teacher network to a smaller and simpler student network. Therefore, although the student network is mainly composed of a dictionary learning branch and a coefficient learning branch, it is more lightweight. It should be noted that the dictionary branch of the student network needs to upsample the low-resolution dictionary to restore it to a scale aligned with the high-resolution dictionary. Therefore, the initial reconstructed image of the student network is:

[0151]

[0152] Alternatively, as another embodiment of the present invention, Figure 3 As shown, the present invention is based on a high-resolution dictionary obtained through a teacher network and coefficients Supervise and guide student web dictionaries using relational knowledge diction and coefficients The specific formula for learning and updating is as follows:

[0153]

[0154] Among them, L dic represents the loss of dictionary distillation, L coe represents the loss of coefficient distillation.

[0155] Alternatively, as another embodiment of the present invention, compared to existing methods, the advantages and positive effects of the present invention are as follows: To alleviate the uninterpretability of hyperspectral image super-resolution based on deep learning, the present invention combines the advantages of dictionary learning and deep learning to decompose the hyperspectral image reconstruction task into the task of learning sparse representation coefficients and a low-rank dictionary. To generate supervision for the dictionary and coefficients, a self-supervised teacher network is constructed. The real dictionary and coefficients are learned from high-resolution images to guide the training of a lightweight student network, thereby restoring the spectral and texture details of the hyperspectral image.

[0156] Figure 4 This is a module block diagram of a hyperspectral image super-resolution device provided by an embodiment of the present invention.

[0157] Alternatively, as another embodiment of the present invention, Figure 4 As shown, a hyperspectral image super-resolution device includes:

[0158] An import module, configured to import a plurality of original high-resolution hyperspectral images and an original low-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images;

[0159] a high-resolution feature analysis module, configured to construct a high-resolution model and a low-resolution model, and perform high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain a high-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images;

[0160] a low-resolution feature analysis module, configured to perform low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain a target low-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target low-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images;

[0161] a model training module, configured to train the low-resolution model using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model;

[0162] The import module is further used to import the low-resolution image to be processed;

[0163] The super-resolution result obtaining module is used to reconstruct the low-resolution image to be processed by using the super-resolution model to obtain a hyperspectral image super-resolution result.

[0164] Alternatively, another embodiment of the present invention provides a hyperspectral image super-resolution system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the hyperspectral image super-resolution method described above is implemented. The system may be a computer or other system.

[0165] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the hyperspectral image super-resolution method as described above is implemented.

[0166] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0167] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0169] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of the embodiments of the present invention.

[0170] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hyperspectral image super-resolution method, characterized in that: The steps include: Importing a plurality of original high-resolution hyperspectral images and an original low-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images; Constructing a high-resolution model and a low-resolution model, and performing high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain a high-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images; Performing low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images; The low-resolution model is trained by using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model; Importing a low-resolution image to be processed, reconstructing the low-resolution image to be processed using the super-resolution model to obtain a hyperspectral image super-resolution result.

2. The hyperspectral image super-resolution method according to claim 1, characterized in that: The high-resolution model includes a plurality of first ESSA Block networks and a high-resolution coefficient generation network; The process of performing high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain high-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes: Performing feature extraction of high-resolution dictionary features on each of the original high-resolution hyperspectral images through the plurality of first ESSA Block networks to obtain high-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images; The high-resolution coefficient generation network is used to extract the high-resolution coefficient features of each of the original high-resolution hyperspectral images, thereby obtaining target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images.

3. The hyperspectral image super-resolution method according to claim 2, characterized in that: The high-resolution coefficient generation network includes a first fully connected layer, multiple TInv Block layers, a first 1D attention layer, and a second fully connected layer; The process of extracting high-resolution coefficient features from each of the original high-resolution hyperspectral images through the high-resolution coefficient generation network to obtain target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes: Performing feature extraction processing on each of the original high-resolution hyperspectral images through the first fully connected layer, obtaining original high-resolution hyperspectral features corresponding to each of the original high-resolution hyperspectral images; Performing feature extraction on each of the original high-resolution hyperspectral features through the plurality of TInv Block layers to obtain feature-extracted high-resolution hyperspectral features corresponding to each of the original high-resolution hyperspectral images; updating the weights of the high-resolution hyperspectral features after feature extraction by the first 1D attention layer to obtain updated high-resolution hyperspectral features corresponding to the original high-resolution hyperspectral images; Feature extraction is performed on each of the updated high-resolution hyperspectral features through the second fully connected layer to obtain a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images.

4. The hyperspectral image super-resolution method according to claim 1, characterized in that: The low-resolution model includes a low-resolution dictionary generation network and a low-resolution coefficient generation network; The process of performing low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images and target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes: Performing feature extraction of low-resolution dictionary features on each of the original low-resolution hyperspectral images through the low-resolution dictionary generation network to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images; The low-resolution coefficient generation network is used to extract the low-resolution coefficient features of each of the original low-resolution hyperspectral images, thereby obtaining target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images.

5. The hyperspectral image super-resolution method according to claim 4, characterized in that: The low-resolution dictionary generation network includes multiple second ESSA Block networks and upsampling layers; The process of extracting low-resolution dictionary features from each of the original low-resolution hyperspectral images using the low-resolution dictionary generation network to obtain target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images includes: Performing feature extraction on each of the original low-resolution hyperspectral images through a plurality of the second ESSA Block networks to obtain original low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images; The upsampling layer performs upsampling processing on each of the original low-resolution dictionary features to obtain a target low-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images.

6. The hyperspectral image super-resolution method according to claim 4, characterized in that: The low-resolution coefficient generation network includes a third fully connected layer, a second 1D attention layer, and a fourth fully connected layer; The process of extracting low-resolution coefficient features from each of the original low-resolution hyperspectral images through the low-resolution coefficient generation network to obtain target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images includes: Performing feature extraction on each of the original low-resolution hyperspectral images through the third fully connected layer to obtain original low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images; updating the weights of the original low-resolution coefficient features respectively through the second 1D attention layer, obtaining updated low-resolution coefficient features corresponding to the original high-resolution hyperspectral images; Feature extraction is performed on each of the updated low-resolution coefficient features through the fourth fully connected layer to obtain target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images.

7. The hyperspectral image super-resolution method according to claim 1, characterized in that: The process of training the low-resolution model by using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model includes: Performing feature multiplication on each of the target low-resolution dictionary features and the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images to obtain a first reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images; Performing feature multiplication on each of the high-resolution dictionary features and the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images to obtain a second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images; Extracting spectral features of each of the first reconstructed high-resolution hyperspectral images through a 3×3 convolutional layer to obtain spectral features corresponding to each of the original high-resolution hyperspectral images and spectral channel dimensions corresponding to each of the original high-resolution hyperspectral images; Performing feature extraction of spatial features on each of the first reconstructed high-resolution hyperspectral images through a 1×1 convolutional layer to obtain spatial features corresponding to each of the original high-resolution hyperspectral images and spatial channel dimensions corresponding to each of the original high-resolution hyperspectral images; Normalizing each of the spectral features through a layer normalization layer to obtain a spectral query vector corresponding to each of the original high-resolution hyperspectral images; Normalizing each of the spatial features through the layer normalization layer to obtain a spatial query vector corresponding to each of the original high-resolution hyperspectral images; Performing feature extraction on each of the spectral query vectors through a convolutional layer to obtain a spectral key vector corresponding to each of the original high-resolution hyperspectral images; Performing feature extraction on each of the spatial query vectors through the convolutional layer to obtain a spatial key vector corresponding to each of the original high-resolution hyperspectral images; Performing feature extraction on each of the spectral key vectors through a depthwise separable convolutional layer to obtain a spectral numerical vector corresponding to each of the original high-resolution hyperspectral images; Performing feature extraction on each of the spatial key vectors through the depthwise separable convolutional layer to obtain a spatial numerical vector corresponding to each of the original high-resolution hyperspectral images; The spectral attention corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the spatial query vectors, the spectral channel dimension corresponding to each of the original high-resolution hyperspectral images, and the spectral key vector corresponding to each of the original high-resolution hyperspectral images through the first formula. The first formula is: Among them, CA j is the spectral attention corresponding to the j-th original high-resolution hyperspectral image, SoftMax() is the SoftMax function, is the spatial query vector corresponding to the j-th original high-resolution hyperspectral image, is the spectral key vector corresponding to the j-th original high-resolution hyperspectral image, is the spectral channel dimension corresponding to the j-th original high-resolution hyperspectral image; The spatial attention corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the spectral query vectors, the spatial channel dimension corresponding to each of the original high-resolution hyperspectral images, and the spatial key vector corresponding to each of the original high-resolution hyperspectral images through the second formula. The second formula is: Among them, SA j is the spatial attention corresponding to the j-th original high-resolution hyperspectral image, SoftMax() is the SoftMax function, is the spectral query vector corresponding to the j-th original high-resolution hyperspectral image, is the spatial key vector corresponding to the j-th original high-resolution hyperspectral image, is the spatial channel dimension corresponding to the j-th original high-resolution hyperspectral image; The target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the spectral features, the spectral attention corresponding to each of the original high-resolution hyperspectral images, the spectral numerical vector corresponding to each of the original high-resolution hyperspectral images, the spatial features corresponding to each of the original high-resolution hyperspectral images, the spatial attention corresponding to each of the original high-resolution hyperspectral images, and the spatial numerical vector corresponding to each of the original high-resolution hyperspectral images through the third formula. The third formula is: in, is the target high-resolution hyperspectral image corresponding to the j-th original high-resolution hyperspectral image, Conv() is the convolution process, f FFN () is the feedforward neural network function, γ and β are both learnable parameters, CA j is the spectral attention corresponding to the j-th original high-resolution hyperspectral image, is the spatial numerical vector corresponding to the j-th original high-resolution hyperspectral image, is the spatial feature corresponding to the jth original high-resolution hyperspectral image, SA j is the spatial attention corresponding to the j-th original high-resolution hyperspectral image, is the spectral numerical vector corresponding to the j-th original high-resolution hyperspectral image, is the spectral feature corresponding to the jth original high-resolution hyperspectral image; Calculating a loss function for each of the high-resolution dictionary features, the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, and the target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, respectively, to obtain a target distillation loss function corresponding to each of the original high-resolution hyperspectral images; The parameters of the low-resolution model are updated according to all the target distillation loss functions to obtain a super-resolution model.

8. The hyperspectral image super-resolution method according to claim 7, characterized in that: The process of calculating the loss function for each of the high-resolution dictionary features, the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, and the target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images to obtain the target distillation loss function corresponding to each of the original high-resolution hyperspectral images includes: The target distillation loss function corresponding to each of the original high-resolution hyperspectral images is obtained by respectively calculating each of the high-resolution dictionary features, the target high-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution dictionary features corresponding to each of the original high-resolution hyperspectral images, the target low-resolution coefficient features corresponding to each of the original high-resolution hyperspectral images, the second reconstructed high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images, and the target high-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images through the fourth formula. The fourth formula is: in, in, is the target distillation loss function corresponding to the jth original high-resolution hyperspectral image, α, δ, and ε are all balance coefficients, is the target high-resolution hyperspectral image corresponding to the j-th original high-resolution hyperspectral image, is the second reconstructed high-resolution hyperspectral image corresponding to the j-th original high-resolution hyperspectral image, is the dictionary distillation loss function corresponding to the j-th original high-resolution hyperspectral image, is the coefficient distillation loss function corresponding to the j-th original high-resolution hyperspectral image, is the high-resolution dictionary feature corresponding to the j-th original high-resolution hyperspectral image, is the target low-resolution dictionary feature corresponding to the j-th original high-resolution hyperspectral image, is the target high-resolution coefficient feature corresponding to the j-th original high-resolution hyperspectral image, is the target low-resolution coefficient feature corresponding to the j-th original high-resolution hyperspectral image, and ‖‖1 is the L1 norm.

9. A hyperspectral image super-resolution device, characterized in that: include: An import module, configured to import a plurality of original high-resolution hyperspectral images and an original low-resolution hyperspectral image corresponding to each of the original high-resolution hyperspectral images; a high-resolution feature analysis module, configured to construct a high-resolution model and a low-resolution model, and perform high-resolution feature analysis on each of the original high-resolution hyperspectral images using the high-resolution model to obtain a high-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target high-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images; a low-resolution feature analysis module, configured to perform low-resolution feature analysis on each of the original low-resolution hyperspectral images using the low-resolution model to obtain a target low-resolution dictionary feature corresponding to each of the original high-resolution hyperspectral images and a target low-resolution coefficient feature corresponding to each of the original high-resolution hyperspectral images; a model training module, configured to train the low-resolution model using all the high-resolution dictionary features, all the target high-resolution coefficient features, all the target low-resolution dictionary features, and all the target low-resolution coefficient features to obtain a super-resolution model; The import module is further used to import the low-resolution image to be processed; The super-resolution result obtaining module is used to reconstruct the low-resolution image to be processed by using the super-resolution model to obtain a hyperspectral image super-resolution result.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the hyperspectral image super-resolution method according to any one of claims 1 to 8 is implemented.

Images