Spectral super-resolution reconstruction method based on dynamic dictionary learning

Abstract

The invention relates to the technical field of image reconstruction, in particular to a spectral super-resolution reconstruction method based on dynamic dictionary learning. Comprising the following steps: constructing a DLTN network architecture which comprises a feature coding unit, a space down-sampling unit, a multi-stage dictionary learning and sparse coding unit, a feature decoding unit and a global feature fusion mechanism; obtaining a hyperspectral training data set, preprocessing an input low-resolution RGB image, and inputting the image into a network for training; and inputting a to-be-processed low-resolution RGB image into the trained network, sequentially carrying out feature coding, space down-sampling, multi-level sparse coding and feature enhancement, multi-depth feature global fusion and feature decoding processing, and outputting a reconstructed hyperspectral image. The method has the advantages that dynamic dictionary learning and a lightweight Transform architecture are fused, feature sparse representation is realized through a DLSC module, and the calculation cost is reduced; while the reconstruction precision is ensured, the number of model parameters and the number of floating point operations are significantly reduced.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, and in particular to a spectral super-resolution reconstruction method based on dynamic dictionary learning. Background Technology

[0002] Traditional hyperspectral image acquisition often faces challenges such as high hardware costs, large storage and processing volumes, and limited spatial resolution, severely hindering the efficient acquisition and practical application of hyperspectral images. Spectral super-resolution, as a computational solution, achieves hyperspectral image reconstruction through algorithm optimization, demonstrating significant advantages in the efficient acquisition of hyperspectral images. Among these, the technique of reconstructing hyperspectral images based on RGB images has been widely used due to the ease of obtaining input data.

[0003] Currently, deep learning-based spectral super-resolution algorithms have become the mainstream technology direction, and various spectral super-resolution network models have emerged in related research to realize the reconstruction from RGB images to hyperspectral images. However, existing spectral super-resolution network models still have obvious technical defects in practical applications, including the following aspects: (1) Some models are difficult to balance the benchmark performance and computational efficiency of hyperspectral image reconstruction. While ensuring that the reconstruction effect is close to the mainstream level, the number of floating-point operations (FLOPs) and the number of parameters of the model remain high, resulting in high computational cost and low running efficiency, which cannot meet the needs of lightweight deployment; (2) In order to pursue lightweight and simplified network structure, some models are prone to problems such as loss of image detail features and gradient vanishing in deep networks, which are insufficient in expressing the spectral-spatial details of hyperspectral images. At the same time, the robustness of the model is poor, making it difficult to achieve efficient extraction and accurate representation of hyperspectral features.

[0004] Furthermore, existing technologies lack sufficient feature mining for the spectral similarity and spatial sparsity of hyperspectral images, and lack effective means to perform low-dimensional and sparse representation of hyperspectral features. This makes it impossible to retain key spectral-spatial details while filtering redundant spectral information, resulting in difficulty in simultaneously improving the accuracy and efficiency of hyperspectral image reconstruction. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a spectral super-resolution reconstruction method based on dynamic dictionary learning.

[0006] The purpose of this invention is to provide a spectral super-resolution reconstruction method based on dynamic dictionary learning, comprising the following steps: S1. Construct the DLTN network architecture; the DLTN network includes feature encoding units, spatial downsampling units, multi-level dictionary learning and sparse encoding units, feature decoding units, and a global feature fusion mechanism; S2. Obtain the hyperspectral training dataset, preprocess the input low-resolution RGB image and the corresponding high-resolution hyperspectral image, input them into the DLTN network for training and performance verification, and optimize the model parameters; S3. Input the low-resolution RGB image to be processed into the trained DLTN network, and process it sequentially through feature encoding, spatial downsampling, multi-level sparse coding and feature enhancement, multi-depth feature global fusion and feature decoding, and finally output a reconstructed hyperspectral image with high spatial resolution and high spectral fidelity.

[0007] Preferably, in step S1, the multi-level dictionary learning and sparse coding unit is composed of at least two cascaded dictionary learning and sparse coding modules. Except for the last dictionary learning and sparse coding module, each dictionary learning and sparse coding module is connected to a feature extension module.

[0008] Preferably, the feature expansion module achieves spectral spatial upsampling through multi-scale convolution, activation functions, and PixelShuffle layers.

[0009] Preferably, each dictionary learning and sparse coding module includes, in sequence, a multi-scale feature extraction submodule, a multi-head self-attention submodule, a dictionary reconstruction submodule, a channel attention submodule, a spectral attention submodule, and an enhanced residual submodule; The multi-scale feature extraction submodule uses multi-scale convolutional kernels to extract features in parallel and fuses feature information from different receptive fields. The multi-head self-attention submodule captures long-range dependencies between different spectral channels and spatial locations, achieving global contextual information fusion and supplementing global correlation information of spectral features; each head in the multi-head self-attention submodule independently performs self-attention calculations. The dictionary reconstruction submodule serves as the core execution unit, performing sparse encoding and reconstruction on the input features to uncover and enhance potential spectral details while preserving the spatial structure. The dictionary reconstruction submodule includes a coefficient prediction module and a sparse encoding module, and incorporates a dynamic spectral dictionary and a sparse encoding mechanism.

[0010] Preferably, the multi-scale feature extraction submodule uses convolutional kernels of 1×1, 3×3, and 5×5 sizes, with the number of convolutional kernels at each scale set to 32. In the feature fusion process, after concatenating the channel dimensions, 1×1 convolution is used to integrate the dimensions, ensuring that the output feature map size is consistent with the input.

[0011] Preferably, the working process of the multi-head self-attention submodule includes: inputting the feature map Fin∈R B×C×H×W Flattened into a sequence F∈R B×C×H×WVectors Q, K, and V are generated through linear transformation; Q, K, and V are divided into 4 heads and self-attention calculations are performed in parallel, with each head having a dimension of 8; self-attention scores are calculated and converted into attention weights through the Softmax function; the attention weights are weighted and summed with V, and then reshaped back to the original feature map shape (B, C, H, W) through linear transformation.

[0012] Preferably, the feature encoding unit is composed of a 3×3 convolutional layer, a normalization layer and a LeakyReLU activation function connected in sequence, which is used to map the basic visual features of the low-resolution RGB image into high-dimensional preliminary spectral features; The spatial downsampling unit is connected to the feature encoding unit. It downsamples the encoded spectral feature image through a 4×4 convolutional layer with a stride of 2 and outputs the downsampled feature image, which retains key feature information while compressing the spatial dimension.

[0013] Preferably, the training process in step S2 employs an Adam optimizer combined with a cosine annealing learning rate scheduling strategy, with the L1 loss function as the optimization objective; the parameters of the Adam optimizer are set to β1=0.9, β2=0.99, =10 -8 The initial learning rate is 0.0005.

[0014] Compared with the prior art, the present invention can achieve the following beneficial effects: This paper proposes a novel spectral super-resolution network (Dictionary Learning & Transformer Network, DLTN) for RGB image hyperspectral reconstruction, integrating dynamic dictionary learning and a lightweight Transformer module. Its core lies in combining the advantages of deep learning and dictionary learning, creatively designing a dictionary learning and sparse coding module. DLTN mainly consists of multiple cascaded DLSC modules and a feedforward network (FFN). The overall network adopts a multi-stage hierarchical encoder-decoder architecture, where the decoder can directly acquire feature information from different depths (including the encoder and each layer of the DLSC module), effectively avoiding the loss of image detail features and gradient vanishing problems in deep networks. Experimental results demonstrate that DLTN's benchmark performance is comparable to most current mainstream super-resolution algorithms, and while significantly reducing the number of floating-point operations (FLOPs) and model parameters, it achieves computational efficiency far exceeding similar algorithms, fully proving that DLTN is a highly promising lightweight spectral super-resolution network. Attached Figure Description

[0015] Figure 1This is a DLTN network architecture diagram provided according to an embodiment of the present invention; in the figure, (a) is the backbone network; (b) is the feature extension module; and (c) is the feedforward network (FFN); where SD represents spatial downsampling and SU represents spatial upsampling.

[0016] Figure 2 This is a structural diagram of the DLSC module provided according to an embodiment of the present invention.

[0017] Figure 3 This is a structural diagram of the multi-scale feature extraction (MF) submodule provided according to an embodiment of the present invention.

[0018] Figure 4 This is a structural diagram of the multi-head self-attention (MSA) submodule provided according to an embodiment of the present invention.

[0019] Figure 5 This is a structural diagram of the dictionary reconstruction (DR) submodule provided according to an embodiment of the present invention.

[0020] Figure 6 This is a structural diagram of the spectral attention (SA) submodule provided according to an embodiment of the present invention.

[0021] Figure 7 The images are the reconstructed images of the 30th band and the root mean square error image of two test images in the CAVE dataset provided according to embodiments of the present invention; (a) HSCNN; (b) HSRnet; (c) AWAN; (d) MST++; (e) SSRnet; (f) LTRN; (g) DLTN of the present invention; (h) reference image.

[0022] Figure 8 The images are reconstructed images of the 30th band and root mean square error images of two test images from the Harvard dataset provided in the embodiments of the present invention; (a) HSCNN; (b) HSRnet; (c) AWAN; (d) MST++; (e) SSRnet; (f) LTRN; (g) DLTN of the present invention; (h) reference image. Detailed Implementation

[0023] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0025] This invention provides a spectral super-resolution reconstruction method based on dynamic dictionary learning, aiming to effectively filter unnecessary and redundant spectral information in the reconstructed hyperspectral image while highlighting and preserving important spectral details and texture information. Drawing on the sparse representation concept in traditional dictionary learning, a dynamic spectral dictionary and sparse coding mechanism are introduced into DLTN. The designed dynamic dictionary is a learnable dictionary that can be adjusted in real time according to the spectral features of the input hyperspectral data. The basis vectors (i.e., "atoms") in the dictionary can adaptively update as the input spectral features change. Sparse coding further sparsifies the dynamic spectral dictionary, achieving compression and dimensionality reduction of the input features. Specifically, it includes the following steps: S1. Establishing the DLTN network architecture and configuring modules: See the architecture diagram. Figure 1 The DLTN (Dictionary Learning & Transformer Network) adopts a multi-stage hierarchical encoder-decoder architecture, including feature encoding units, spatial downsampling units, multi-level dictionary learning and sparse coding (MDLSC) units, feature decoding units, and a global feature fusion mechanism. The Dictionary Learning and Sparse Coding (DLSC) module is the core execution unit, and multiple DLSC modules are cascaded with a feedforward network (FFN) to form the core link. The specific module configuration is as follows: The feature encoding unit, as the front-end processing unit of the network, consists of a 3×3 convolutional layer, a normalization layer, and a Leaky ReLU activation function connected in sequence. It is used to extract initial features from the input low-resolution RGB image, mapping the basic visual features of the RGB image to preliminary features in a high-dimensional spectral feature space (encoding the spectral feature image X). EN This provides foundational data support for subsequent sparse coding and spectral reconstruction; the feature coding expression is as follows: ; In the formula, To encode spectral feature images, Representative feature encoding network, W and H These are the width and height of the space, respectively. Output the number of channels during the network process; The number of bands must be greater than that of the hyperspectral image to ensure that the network learns the correlation between spectral channels better.

[0026] The spatial downsampling unit is connected to the feature encoding unit. It downsamples the encoded spectral feature image through a 4×4 convolutional layer with a stride of 2, and outputs the downsampled feature image X. SD This reduces the computational complexity of subsequent modules while compressing the spatial dimension and retaining key feature information. If X is directly EN Inputting into the DLSC module may increase computational costs, which contradicts the original design intent of DLTN. Therefore, to reduce computational complexity, a 4x4 convolutional layer with a stride of 2 is added after the feature encoding module to achieve spatial downsampling. ; In the formula, X SD This is a feature image after spatial downsampling; Represents a spatial downsampling network; X EN Output tensors for the module.

[0027] The Multi-Level Dictionary Learning and Sparse Coding (MDLSC) unit connects to the spatial downsampling unit and consists of at least two cascaded DLSC modules. Except for the last DLSC module, each DLSC module is followed by a feature expansion module to perform feature expansion on the downsampled feature image X. SD The DLSC module employs multi-stage sparse coding processing to output multi-stage encoded features. Each DLSC module sequentially includes a multi-scale feature extraction (MF) submodule, a multi-head self-attention (MSA) submodule, a dictionary reconstruction (DR) submodule, a channel attention submodule, a spectral attention (SA) submodule, and an enhancement residual submodule. The structure of the DLSC module is as follows: Figure 2 As shown in the example. In a specific embodiment, in the feature expansion module, the number of output channels for the multi-scale convolution is set to 64, and the number of input channels for the PixelShuffle layer is set to 256, ensuring that the number of channels after upsampling matches the number of preceding features.

[0028] The multi-scale feature extraction (MF) submodule employs multi-scale convolutional kernels to extract features in parallel, fusing feature information from different receptive fields to enrich the representational power of input features and improve the accuracy of capturing spatial details in hyperspectral images; the module structure is as follows: Figure 3 As shown. In a specific embodiment, the number of convolutional kernels at each scale is set to 32, and the padding is set to 0, 1, and 2 respectively to ensure that the output feature map size is consistent with the input; the fusion process uses channel dimension concatenation followed by 1×1 convolution (output channel number 64) for dimension integration.

[0029] The multi-head self-attention (MSA) submodule captures long-range dependencies between different spectral channels and spatial locations, breaking through the limitations of the local receptive field in traditional convolution, achieving global contextual information fusion, and supplementing the global correlation information of spectral features; the module structure is as follows. Figure 4 As shown; Specifically, the MSA module first processes the input feature map Fin∈R B×C×H×W Flattened into a sequence F∈R B×C×H×W Then, through linear transformation, three vectors, Query, Key, and Value, are generated and defined as Q, K, and V, respectively, as shown in the following formula; the Q, K, and V vectors are further divided into multiple "heads", and each head independently performs self-attention calculation; ; ; ; ; in, w It is a weight matrix; h The number of bullish positions is 4; d It is the dimension of a single head, which is 8. For each head, the correlation between any two elements in the input sequence, i.e., the self-attention score, is calculated using its independent Q and K, as shown in the following formula: ; Among them, China It is the self-attention score; Indicates the transpose operation; This is a scaling factor, designed to avoid excessively large gradients; The self-attention score A is converted into attention weights using the Softmax function. This indicates the degree of attention the current query pays to other keys: ; The calculated attention weights With the corresponding header By performing weighted summation, the output matrix of the self-attention layer, which incorporates global contextual information, is finally obtained. See the following formula: ; in, Represents a linear layer; Finally, output features The features are reshaped back to their original shape (B, C, H, W) for integration with subsequent convolutional layers or other modules. It's important to emphasize that the MSA module incorporates numerous reshape operations to ensure dimensionality matching between the process feature data. Through a multi-head mechanism, the attention of different heads can learn different patterns of information association, such as capturing spatial dependencies at different scales or interactions between different spectral features. Finally, the outputs of multiple heads are concatenated and linearly transformed, reshaping the features back to their original spatial dimensions to output a spectral feature representation that enhances global perception and feature interrelationships, providing richer and more favorable input for subsequent spectral reconstruction tasks.

[0030] The Dictionary Reconstruction (DR) submodule, as the core execution unit of the DLSC module, aims to sparsely encode and reconstruct input features using a learnable spectral dictionary, thereby preserving spatial structure while uncovering and enhancing potential spectral details. It includes a Coefficient Prediction Module and a Sparse Encoding Module, with a built-in dynamic spectral dictionary and sparse encoding mechanism. The dynamic spectral dictionary uses orthogonal initialization and can adaptively update based on the spectral features of the input hyperspectral data, achieving adaptive feature extraction. The sparse encoding unit is configured with a Top-K sparsification strategy and an L1 norm normalization rule, selectively activating atoms in the dictionary to complete feature sparsification and output the sparse representation of the hyperspectral features. The module structure is as follows: Figure 5 As shown; The coefficient prediction module maps input features to a dictionary coefficient space, acting as an encoder "from features to dictionary coordinates." The first 1×1 convolutional layer performs initial channel mixing and feature extraction on the input features, followed by BN and LeakyReLU layers to enhance non-linear expressive power. The key role of MSA is to break the limitations of local receptive fields, traversing different spatial locations and feature channels to capture long-range dependencies and global contextual information in the input signal, enabling the model to understand more complex patterns. The final 1×1 convolutional layer maps features to a dimension consistent with the dictionary size, providing a foundation for subsequent sparse encoding and reconstruction. Thus, for each spatial location in the input signal, the network outputs a vector containing N values, representing the activation strength or weight of the signal at that location on each dictionary atom.

[0031] Sparse coding module: Learnable spectral dictionary It contains N fixed-dimensional atoms. The dictionary uses orthogonal initialization to enhance the independence between atoms in the spectral dictionary, ensuring gradient stability in the early stages of training and preventing gradient explosion or vanishing. Hyperspectral images typically exhibit high correlation and sparsity in the spectral dimension, meaning they can be represented by a linear combination of a small number of "atoms" in the spectral dictionary. The core idea of ​​sparse coding is feature extraction and refinement. This module achieves "reconstructing effective information with a small number of dictionary atoms," which is key to DLSC sparse representation and spectral detail generation. The sparse coding module employs a Top-K sparsity strategy: it finds the K largest coefficients out of N coefficients at each spatial location and forces the remaining coefficients to zero. Then, it performs L1 norm normalization on the remaining non-zero coefficients to maintain the relative proportion of activation values.

[0032] Next, the sparse coding coefficients are subjected to an outer product operation with the normalized dictionary atoms to remap the signal from the sparse representation space back to the original feature space. Finally, the dictionary reconstruction result is fused with the input features in a residual manner to output a reconstructed feature that combines the original representation with the "dictionary enhancement" representation.

[0033] The channel attention submodule captures the dependencies between channels, refines channel features, reduces redundant channel information, and strengthens the feature representation of key channels. The residual submodule is enhanced to alleviate the gradient vanishing problem in deep network training through residual connections, preserve shallow feature information, and improve the integrity of feature propagation. The Spectral Attention (SA) submodule assigns attention weights to the input spectral features, enhancing key spectral-spatial detail features in hyperspectral images, reducing redundant information, and improving the model's ability to discriminate effective spectral features. The module structure is as follows: Figure 6 As shown.

[0034] Brief Principle Description: Typically, increasing the number of network layers leads to the loss of image detail features and the vanishing gradient problem. To address this, DLSC introduces global residual connections to avoid these issues. The decoder can directly acquire feature information from different depths (including layers of the encoder and DLSC modules). This multi-level feature fusion helps recover detail features that may be lost during encoding, spectral dictionary learning, and sparse coding. Simultaneously, these direct skip paths provide a "shortcut" for backpropagating gradients, effectively mitigating the problem of gradient decay layer by layer in deep networks, ensuring training stability and convergence. Except for the last module, each DLSC module is followed by a feature expansion module, such as... Figure 1 As shown in Figure b, this module gradually expands and enhances the spatial resolution and information content of features through multi-scale convolution, activation functions, and PixelShuffle layers, thereby achieving spectral spatial upsampling.

[0035] S2. Obtain the hyperspectral training dataset. After preprocessing the input low-resolution RGB images and their corresponding high-resolution hyperspectral images, input them into the DLTN network for training and performance verification. Optimize the model parameters until the model converges, ensuring that reconstruction accuracy and lightweight characteristics meet the requirements. Input the data into the DLTN network for training, using the Adam optimizer combined with a cosine annealing learning rate scheduling strategy, and optimizing the model parameters with the L1 loss function as the primary loss. Specifically, this includes the following sub-steps: S21. Acquisition and Preprocessing of Hyperspectral Training Dataset: Dataset selection: Authoritative publicly available datasets in the field of spectral imaging (CAVE dataset or Harvard dataset) are selected. The datasets must contain pairs of low-resolution RGB images and corresponding high-resolution hyperspectral images. The CAVE dataset contains hyperspectral images of 32 scenes (31 bands, wavelength 400-700nm, spectral resolution 10nm), and the Harvard dataset contains hyperspectral images of 50 scenes (31 bands, wavelength 420-720nm, spectral resolution 10nm). Ensure that the data pairs cover the same area and are captured under the same conditions.

[0036] In some embodiments, the dataset can be divided into training set: validation set: test set = 3:1:1. For example, the CAVE dataset selects 20 images as the training set, 6 images as the validation set, and 6 images as the test set; the Harvard dataset is divided in the corresponding ratio to avoid data leakage and overfitting.

[0037] Data standardization: Pixel values ​​in low-resolution RGB images are normalized to the [0,1] interval; reflectance values ​​in high-resolution hyperspectral images are normalized using Z-Score (i.e., norm(x) = (x...)). μ) / σ, where μ is the band mean and σ is the band standard deviation), eliminates the dimensional differences between different bands and different data, and ensures training stability.

[0038] Image segmentation and data loading: The normalized image is cropped into 64×64 pixel image patches with a step size of 32 pixels. ReflectPadding is used to fill the edge areas to ensure uniform size. The generated image patches are loaded into the PyTorch DataLoader, the batch size is set to 32, and the shuffle mechanism is enabled to shuffle the data order to improve the model's generalization ability.

[0039] S22. Training parameter configuration: Core parameter settings: The number of DLSC modules is set to 2, the number of atoms N in the dynamic spectral dictionary is set to 32, and the K value in the Top-K sparsity strategy of the sparse coding module is set to 5; the number of output channels C of the feature coding unit is set to 64 (greater than 31 bands in the hyperspectral image to ensure spectral correlation learning).

[0040] Optimizer and learning rate configuration: The Adam optimizer is used, with parameters set to β1=0.9, β2=0.99, =10 -8 The initial learning rate is 0.0005; a cosine annealing learning rate scheduling strategy is adopted, and the learning rate is adaptively adjusted according to the cosine function after each training round to avoid the model getting trapped in local optima.

[0041] The design employs the L1 loss function (to reduce pixel-level errors between the reconstructed image and the real hyperspectral image); the training process is built based on the PyTorch deep learning framework; mixed precision training (AMP) is enabled to accelerate the training process; and the gradient clipping threshold is set to 1.0 to prevent gradient explosion.

[0042] S23. Network training process execution: Set differentiated training rounds for different datasets—1000 rounds for the CAVE dataset and 200 rounds for the Harvard dataset to avoid overfitting or underfitting.

[0043] In some embodiments, after every 50 training rounds, the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and spectral angle mapping (SAM) of the reconstructed image are calculated on the validation set, and the trends of training loss and validation loss are recorded. If the validation loss does not decrease for 50 consecutive rounds, the learning rate is automatically halved, and the optimal model parameters from the first 10 rounds are loaded to continue training. The model is considered to have converged and the number of training rounds has reached a preset value when any of the following conditions are met: the PSNR fluctuation on the validation set is less than 0.05 dB for 30 consecutive rounds, and the SAM fluctuation is less than 0.1 for 30 consecutive rounds; the training loss value is below 0.001 and tends to stabilize.

[0044] During training, the model parameters that achieve the best performance on the validation set (highest PSNR, lowest SAM) are saved in real time, including network weights, dynamic spectral dictionary parameters, optimizer states, etc., for subsequent reconstruction of the test image.

[0045] S24. Performance Validation and Parameter Tuning: Evaluate the performance of the optimal model on the test set. Key quantitative metrics include: Reconstruction accuracy metrics include peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and spectral angle mapping (SAM), ensuring that DLTN's reconstruction accuracy is superior to mainstream comparison algorithms (such as MST++ and LTRN). Lightweight performance metrics include the number of model parameters and the number of floating-point operations (FLOPs), ensuring that DLTN's parameter count (≤0.21M) and FLOPs (≤0.63G) are superior to mainstream comparison algorithms; In some embodiments, the hardware environment is equipped with an Intel i9-13900 CPU and an NVIDIA RTX 3080Ti GPU, providing powerful computational support for model training.

[0046] S3. Super-resolution reconstruction of the image to be tested: The low-resolution RGB image to be processed is input into the trained DLTN network, and sequentially processed through feature encoding, spatial downsampling, multi-level sparse coding and feature enhancement, multi-depth feature global fusion and feature decoding. End-to-end hyperspectral reconstruction inference is performed, and finally, a reconstructed hyperspectral image with high spatial resolution and high spectral fidelity is output, realizing end-to-end super-resolution reconstruction from a low-dimensional RGB image to a high-dimensional hyperspectral image. The specific steps are as follows: S31. Input Preprocessing and Feature Encoding: The low-resolution RGB image to be reconstructed is input into the DLTN network. The feature encoding unit performs initial feature extraction on the RGB image, completing the mapping from RGB visual features to preliminary high-dimensional spectral features. The encoded spectral feature map is output as the input data for the spatial downsampling unit.

[0047] S32. Spatial Downsampling: The spatial downsampling unit performs downsampling processing on the encoded spectral feature map through a 4×4 convolutional layer with a stride of 2, and outputs the downsampled feature image X. SD This provides dimension-adaptive feature inputs for subsequent multi-level sparse coding.

[0048] S33. Multi-level sparse coding and feature enhancement will downsample the feature image X. SD The input is a multi-level DLSC and FFN cascaded link, and feature processing is performed sequentially through each DLSC module and the feedforward network (FFN). Each level of processing performs the following sub-steps: S331. Multi-scale feature extraction: The MF submodule performs multi-scale convolution extraction and fusion on the input features to enrich the spatial details and spectral representation of the features; S332. Spectral Attention Weighting: The SA submodule calculates and assigns attention weights to the multi-scale fused features, enhancing key spectral-spatial detail features and filtering redundant information. S333. Global Context Fusion: The MSA submodule performs global self-attention calculation on the attention-weighted features to capture the long-distance dependence of spectral channels on spatial location and fuse global context information; S334. Channel Attention Enhancement: The channel attention submodule refines channel features, reduces channel redundancy, and improves the feature representation of effective channels; S335. Adaptive Dictionary Reconstruction and Sparse Coding: The dynamic spectral dictionary in the DR submodule adjusts the dictionary atom parameters in real time according to the distribution and attributes of the input spectral features. Then, the features extracted from the dictionary are sparsified through a sparse coding mechanism, retaining only the activation information of key features, to achieve a low-dimensional and sparse representation of hyperspectral features. S336. Feature Expansion and Transition Enhancement: The sparse features output by non-last-stage DLSC modules are supplemented with dimensions by the feature expansion module, and then input into the feedforward network (FFN) for nonlinear transformation and feature enhancement. The output is used as the input to the next-stage DLSC module.

[0049] S34. Global Fusion of Multi-Depth Features: The feature decoding unit receives the final sparse features output by the last-level DLSC module, and retrieves intermediate feature information from different depth layers of the feature encoding unit and each DLSC module through the global feature fusion mechanism. The features of multiple stages and multiple depths are fused to obtain a global fused feature that includes basic features, intermediate enhanced features and final sparse features, thus making up for the loss of features in multi-level processing.

[0050] S35. Hyperspectral Image Reconstruction Output: The feature decoding unit sequentially performs operations such as spectral channel mapping, spatial upsampling, and feature refinement reconstruction on the global fusion features, restoring the low-dimensional sparse hyperspectral features to high-resolution spectral features that match the number of target bands, and finally outputs the reconstructed high-resolution hyperspectral image, completing the entire reconstruction process from RGB image to hyperspectral image.

[0051] Throughout the inference process, each module adopts a lightweight design concept: dynamic dictionary learning avoids redundant parameter storage through adaptive updates, sparse coding mechanism reduces invalid computation, and multi-level module cascading balances feature extraction accuracy and computational cost, significantly reducing the number of floating-point operations (FLOPs) and parameters of the model, thus achieving lightweight end-to-end hyperspectral reconstruction.

[0052] To fully verify the performance advantages of the DLTN network of this invention in the spectral super-resolution reconstruction task, mainstream spectral super-resolution algorithms (AWAN, MST++, SSRnet, HSCNN, HSRnet, LTRN) were selected as comparison objects. Comparative experiments were carried out on public datasets, and quantitative evaluations were conducted from the aspects of reconstruction accuracy and lightweight characteristics. Combined with qualitative effect verification, the effectiveness of this invention was fully demonstrated.

[0053] Using the CAVE and Harvard datasets, four core quantization metrics were selected to compare reconstruction accuracy and lightweight characteristics. To ensure fairness and reproducibility, all methods were trained and evaluated on the same datasets, with consistent training, validation, and test sets. During model training, the optimal optimizer and learning rate configurations provided by the original authors of each model were strictly followed. The model complexity of different methods is shown in Table 1 below. Table 1. Model complexity of different methods

[0054] To visually demonstrate the performance of different algorithms on the hyperspectral image reconstruction task, we selected two representative test images from two datasets. Figure 7 , Figure 8 The reconstruction results of the 30th band of these images and the root mean square error (RMSE) plots are clearly displayed. To further reveal the algorithm's ability to handle details, a magnified view of a local region is provided. Through comparative observation, MST++, LTRN, and our DLTN method can all effectively recover image details, and the RMSE images of the three methods show almost no significant prominent error regions. This intuitively proves that DLTN's performance on SSR problems is comparable to mainstream algorithms. However, DLTN exhibits a significant advantage in model efficiency: DLTN's model parameters are reduced by 82% compared to MST++ and by 57.3% compared to LTRN. DLTN's model FLOPS are reduced by 51.7% compared to MST++ and by 17.1% compared to LTRN. This fully demonstrates DLTN's high efficiency and lightweight nature, significantly reducing computational costs while maintaining the quality of hyperspectral image reconstruction.

[0055] The key technical points and advantages of this invention are as follows: (1) It innovatively integrates dynamic spectral dictionary learning with the Transformer architecture to fully utilize the sparse and low-rank nature of hyperspectral images. Through efficient collaboration among multiple important modules, DLTN achieves high-quality reconstruction from RGB to hyperspectral images.

[0056] (2) A novel DLSC module is proposed, consisting of an MF module, an SA module, and an MSA module, which are used to sparsely represent spectral features to reduce computational cost. SA can improve the network's ability to discriminate spatial-spectral features of hyperspectral images. MSA and MF can effectively capture long-range spectral-spatial dependencies.

[0057] (3) Experiments on two public datasets show that, compared with the existing mainstream SSR algorithms, the DLTN network and reconstruction method of this invention can ensure the quality of spectral super-resolution while having lower computational cost, thus ensuring the computational efficiency and timeliness of the algorithm.

[0058] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0059] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A spectral super-resolution reconstruction method based on dynamic dictionary learning, characterized in that: Includes the following steps: S1. Construct the DLTN network architecture; the DLTN network includes feature encoding units, spatial downsampling units, multi-level dictionary learning and sparse encoding units, feature decoding units, and a global feature fusion mechanism; S2. Obtain the hyperspectral training dataset, preprocess the input low-resolution RGB image and the corresponding high-resolution hyperspectral image, input them into the DLTN network for training and performance verification, and optimize the model parameters; S3. Input the low-resolution RGB image to be processed into the trained DLTN network, and process it sequentially through feature encoding, spatial downsampling, multi-level sparse coding and feature enhancement, multi-depth feature global fusion and feature decoding, and finally output a reconstructed hyperspectral image with high spatial resolution and high spectral fidelity.

2. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 1, characterized in that: In step S1, the multi-level dictionary learning and sparse coding unit is composed of at least two cascaded dictionary learning and sparse coding modules. Except for the last dictionary learning and sparse coding module, each dictionary learning and sparse coding module is connected to a feature extension module.

3. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 2, characterized in that: The feature expansion module achieves spectral spatial upsampling through multi-scale convolution, activation functions, and a PixelShuffle layer.

4. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 2, characterized in that: Each dictionary learning and sparse coding module includes, in sequence, a multi-scale feature extraction submodule, a multi-head self-attention submodule, a dictionary reconstruction submodule, a channel attention submodule, a spectral attention submodule, and an enhanced residual submodule; The multi-scale feature extraction submodule uses multi-scale convolutional kernels to extract features in parallel and fuses feature information from different receptive fields. The multi-head self-attention submodule captures long-range dependencies between different spectral channels and spatial locations, achieving global contextual information fusion and supplementing global correlation information of spectral features; each head in the multi-head self-attention submodule independently performs self-attention calculations. The dictionary reconstruction submodule serves as the core execution unit, performing sparse encoding and reconstruction on the input features to uncover and enhance potential spectral details while preserving the spatial structure. The dictionary reconstruction submodule includes a coefficient prediction module and a sparse encoding module, and incorporates a dynamic spectral dictionary and a sparse encoding mechanism.

5. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 4, characterized in that: The multi-scale feature extraction submodule uses convolutional kernels of 1×1, 3×3, and 5×5 sizes, with the number of kernels at each scale set to 32. The feature fusion process involves concatenating the channel dimensions and then integrating them using 1×1 convolutions to ensure that the output feature map size is consistent with the input.

6. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 4, characterized in that: The working process of the multi-head self-attention submodule includes: inputting the feature map Fin∈R B×C×H×W Flattened into a sequence F∈R B ×C×H×W Vectors Q, K, and V are generated through linear transformation; Q, K, and V are divided into 4 heads and self-attention calculations are performed in parallel, with each head having a dimension of 8; self-attention scores are calculated and converted into attention weights through the Softmax function; the attention weights are weighted and summed with V, and then reshaped back to the original feature map shape (B, C, H, W) through linear transformation.

7. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 1, characterized in that: The feature encoding unit consists of a 3×3 convolutional layer, a normalization layer, and a LeakyReLU activation function connected in sequence, used to map the basic visual features of a low-resolution RGB image into preliminary high-dimensional spectral features; The spatial downsampling unit is connected to the feature encoding unit. It downsamples the encoded spectral feature image through a 4×4 convolutional layer with a stride of 2 and outputs the downsampled feature image, which retains key feature information while compressing the spatial dimension.

8. The spectral super-resolution reconstruction method based on dynamic dictionary learning according to claim 1, characterized in that: The training process in step S2 employs the Adam optimizer combined with a cosine annealing learning rate scheduling strategy, with the L1 loss function as the optimization objective; the parameters of the Adam optimizer are set to β1=0.9, β2=0.99, =10 -8 The initial learning rate is 0.0005.

Images