Hyperspectral image super-resolution method and device based on iterative reconstruction and spectral refining

Abstract

The invention provides a hyperspectral image super-resolution method and device based on iterative reconstruction and spectral refinement, and relates to the technical field of hyperspectral imagines.The method comprises the steps that firstly, a three-adjacent-window cube is intercepted in a sliding mode along the spectral dimension, and the three-adjacent-window cube is mapped into pseudo RGB three-channel features through a lightweight head adapter; then domain adaptation is carried out on the frozen pre-training network through LoRA low-rank decomposition, and corresponding three-band high-resolution prediction is output; for multiple predictions generated by different central windows in the same target wave band, obtaining a coarse super-resolution result by normalized fusion of'spectral position window weight * confidence coefficient '; and finally, the spectrum reconstruction decoder performs end-to-end refinement under the joint SAM spectrum angle constraint to obtain a final high-resolution hyperspectral image. The method aims to transfer a natural image pre-training model to a hundred-wave-band hyperspectral scene in a lossless mode with extremely low parameter increment, and synchronous jumping of space details and spectral fidelity is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral image, and particularly relates to a hyperspectral image super-resolution method and device based on iterative reconstruction and spectral refinement. BACKGROUND

[0002] In the fields of remote sensing, precision agriculture, environmental monitoring and geological exploration, hyperspectral image provides rich information of "graph-spectrum integration", but its spatial resolution is long-term limited by sensor technology, signal-to-noise ratio and satellite platform load capacity. In order to obtain higher spatial details on orbit, the academic and industrial circles generally use "super-resolution" method to reconstruct high-resolution hyperspectral data from single or multiple low-resolution hyperspectral cubes. However, the number of bands of hyperspectral image is usually more than one hundred, adjacent bands are highly correlated but have slight differences, and when the traditional natural image super-resolution technology is directly migrated to this field, the dilemma of coexistence of "spectral distortion" and "spatial artifact" is easily caused.

[0003] Early methods are represented by cube interpolation, sparse dictionary learning or low-rank tensor decomposition, only numerical fitting is done in the image domain, lacking of constraints on imaging physics and spectral continuity, the enlarged details are fuzzy, the edge sawtooth is obvious, and the spectral curve shape is systematically distorted, which is difficult to meet the stringent requirements of quantitative remote sensing inversion on radiation accuracy. With the rise of deep learning, end-to-end convolution or three-dimensional convolution network is used to map low-resolution cubes to target resolution at one time, although the visual sharpness is improved, but the model parameter quantity and the calculation quantity increase linearly or even quadratically with the bands, the demand for pairs of high-low resolution samples in the training stage is exponentially magnified; when the sensor platform is replaced, the spatial resolution or the spectral response function is slightly different, the network needs to be retrained from scratch, and the hardware cost and time cost make users difficult to bear.

[0004] To alleviate the problems of data and computing power, some researchers try to introduce pre-trained weights on natural image big data to give hyperspectral networks stronger spatial prior in the form of transfer learning. Common practices include: 1. Fine-tuning the entire pre-trained model, which can theoretically maximize the preservation of priori. However, hyperspectral samples are scarce, and it is easy to overfit. The full backward propagation of hundreds of billions of parameters is almost impossible to deploy in engineering; 2. Freeze the pre-trained backbone and only add a lightweight decoder or post-processing module to the periphery. Although the training overhead is greatly reduced, the backbone parameters are "locked", and any adaptive adjustment to the hyperspectral statistical characteristics is impossible. Although the spatial details are improved, there is a lack of explicit feedback channel for spectral consistency, resulting in systematic amplification of the spectral angle distance of different ground objects, and subsequent classification and target detection error chain amplification; 3. A more compromised "staged" strategy divides the process into "rough super-resolution-spectral correction" or "spatial-spectral decoupling three stages". The frozen network is used to generate a high-resolution intermediate image, and then an independent network is used to perform spectral shaping in the second stage. However, the gradient flow is truncated between stages, and the pre-trained network cannot perceive downstream spectral errors, resulting in a suboptimal balance of each processing; At the same time, the rich information at the feature level is repeatedly compressed in the round-trip quantization, introducing irreversible loss; Inference requires loading more than two sets of models in series, significantly increasing memory usage and engineering deployment complexity.

[0005] On the other hand, RGB-guided hyperspectral super-resolution has recently become a hot topic, that is, using high-resolution RGB or MSI images of the same scene as spatial prior, to improve texture details through feature transfer, matrix decomposition or joint learning. However, the RGB band coverage is only about 3 wide channels in the visible light range, which has a huge gap with the hundreds of narrow channels in the spectral dimension. If there is a lack of domain alignment and spectral fidelity constraints, it is easy to introduce false details in hyperspectral sensitive areas such as vegetation and water, even change the spectral shape, causing the paradox of seemingly clearer but actually more distorted.

[0006] In summary, existing technologies are trapped in a spatial-spectral dilemma: either sacrifice spectral fidelity for spatial sharpness, or maintain spectral consistency while struggling to break spatial limits; At the same time, the powerful spatial prior brought by pre-trained models has not been efficiently and low-riskly injected into the hyperspectral field, and the parameter size and training cost have become a huge threshold between industry, academia and research. Therefore, how to achieve adaptive fine-tuning in the hyperspectral domain without touching the original pre-trained weights; How to optimize spatial details and spectral consistency simultaneously within an end-to-end framework; How to complete the robust super-resolution of a hundred-band hyperspectral cube with the smallest parameter increment, the smallest training data, and the shortest inference delay, is still a core problem that needs to be broken through in the field.

[0007] Therefore, the present application is proposed. SUMMARY

[0008] The application provides a hyperspectral image super-resolution method and device based on iterative reconstruction and spectral refinement, which can at least partially improve the above problems.

[0009] To achieve the above object, the application adopts the following technical solutions: A hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement comprises the following steps: Obtaining a low-resolution hyperspectral image Extracting adjacent three-band sliding windows according to a preset spectral dimension, and inputting the adjacent three-band sliding windows into a head adapter to obtain pseudo RGB three-channel features; Freezing parameters of a pre-trained model, adapting the pre-trained model by using LoRA (Low-Rank Adaptation), inputting the pseudo RGB three-channel features into the adapted pre-trained model to obtain corresponding three bands, and inputting the three bands into a tail adapter to obtain predicted values corresponding to the three bands respectively, wherein the three bands are three different bands; Processing three predicted values of the same target band generated by different center windows by using a spectral window weighted fusion module to obtain a coarse HR result, and inputting the coarse HR result into a spectral reconstruction decoder to refine the coarse HR result and obtain a final high-resolution hyperspectral image.

[0010] The application further provides a hyperspectral image super-resolution device based on iterative reconstruction and spectral refinement, which comprises: An extraction unit configured to obtain a low-resolution hyperspectral image Extracting adjacent three-band sliding windows according to a preset spectral dimension, and inputting the adjacent three-band sliding windows into a head adapter to obtain pseudo RGB three-channel features; An adaptation unit configured to freeze parameters of a pre-trained model, adapt the pre-trained model by using LoRA, input pseudo RGB three-channel features into the adapted pre-trained model to obtain corresponding three bands, and input the three bands into a tail adapter to obtain predicted values corresponding to the three bands respectively, wherein the three bands are three different bands; An image generation unit configured to process three predicted values of the same target band generated by different center windows by using a spectral window weighted fusion module to obtain a coarse HR result, and input the coarse HR result into a spectral reconstruction decoder to refine the coarse HR result and obtain a final high-resolution hyperspectral image.

[0011] In summary, the present application aims at the long-standing "spatial-spectral dilemma" in the field of hyperspectral image super-resolution, as well as the high cost of pre-training large models, the gradient fracture of the stage process, and the lack of explicit constraints on spectral consistency. A complete technical solution of "sliding three-neighbor window + LoRA low-rank adaptation + spectral window weighted fusion + end-to-end spectral reconstruction" is proposed. By sliding and intercepting a three-band cube along the spectral dimension with a step size of 1, a small amount of LoRA increment is injected into the Q / K / V, attention output and MLP linear layer of the frozen IPT (Image Processing Transformer) / SwinIR (Swin Transformer for Image Restoration) natural image pre-training backbone, which can realize adaptive fine-tuning in the hyperspectral domain, avoid overfitting and algorithm disaster caused by full fine-tuning, and realize the adaptive fine-tuning of the same target band by three center windows. The prediction values are normalized and fused by "spectral position window weight x neighbor window consistency confidence" to obtain the coarse super-resolution result, and then the spectral-spatial joint refinement is performed under the joint supervision of the SAM (Spectral Angle Mapper) spectral angle loss of the spectral reconstruction decoder of the cascaded Spectral-Spatial Block, and finally the high-resolution hyperspectral cube maintaining the original radiometric relationship is output.

[0012] The whole process is end-to-end derivable, and only one forward inference is required, without external RGB assistance, which significantly reduces the model size, training time and memory occupation. The spatial details and spectral fidelity indicators on multiple public data sets are better than those of the existing full fine-tuning and multi-stage schemes, and the method can be directly deployed on satellites, airborne or edge terminals, providing a hyperspectral super-resolution solution that considers efficiency, accuracy and feasibility for quantitative remote sensing, precision agriculture, environmental monitoring, mineral mapping and other applications. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a flowchart of the hyperspectral image super-resolution method based on iterative reconstruction and spectral refining provided by the first embodiment of the present application.

[0014] Figure 2 is a whole framework diagram of the hyperspectral image super-resolution method based on iterative reconstruction and spectral refining provided by the embodiment of the present application.

[0015] Figure 3 is a rough super-resolution diagram of the spectral dimension sliding window provided by the embodiment of the present application.

[0016] Figure 4 is a spectral reconstruction decoder diagram provided by the embodiment of the present application.

[0017] Figure 5 This is a schematic diagram of a module of a hyperspectral image super-resolution device based on iterative reconstruction and spectral refinement provided in the second embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement, which can be executed by a hyperspectral image super-resolution device based on iterative reconstruction and spectral refinement (hereinafter referred to as the super-resolution device), specifically, by one or more processors within the super-resolution device, to implement the following method: S1, Acquire low-resolution hyperspectral image The adjacent three-band sliding windows are extracted according to the preset spectral dimensions, and the adjacent three-band sliding windows are input into the head adapter to obtain pseudo-RGB three-channel features. Specifically, step S1 further includes: acquiring low-resolution hyperspectral images. And according to the preset spectral dimensions, with a step size of 1, extract the sliding window of three adjacent bands. , among which, when At that time, the sliding window for three adjacent bands is ,when At that time, the sliding window for three adjacent bands is , This represents a specific band index in a hyperspectral image. For real numbers, For the length of low-resolution hyperspectral images, For the width of low-resolution hyperspectral images, This refers to the number of bands in a low-resolution hyperspectral image. Band Index The corresponding low-resolution hyperspectral image, Band Index The corresponding low-resolution hyperspectral image, Band Index The corresponding low-resolution hyperspectral image; The adjacent three-band sliding window is input into the head adapter to map pseudo-RGB three-channel features. The head adapter consists of a Conv block and a GELU activation function.

[0020] In this embodiment, low-resolution hyperspectral image X is input, and first, adjacent three-band sliding windows are extracted in a preset spectral dimension with a step of 1. The sliding mode ensures that the head and tail bands are also wrapped by context information, avoiding spectral jumps at the boundary. The adjacent three-band sliding window is sent to the same head adapter. The adapter is composed of a Conv block and a GELU activation function, and the weight is initialized with zero mean small perturbation; the GELU uses the standard form, smooths the negative value, and considers the nonlinearity and gradient liquidity. After the mapping, each set of three-band data is converted into pseudo-RGB three-channel features, the spatial resolution is unchanged, and the spectral dimension is compressed by 3, providing an input distribution consistent with natural images for the subsequent frozen IPT / SwinIR backbone, so that the spatial prior of the pre-trained weight can be directly reused without modification.

[0021] Among them, the adapter is only a simple adjustment of the data to adapt to the input to the ipt, and the preliminary reconstruction is realized by the pre-trained (IPT) model.

[0022] Please refer to Figure 3 , S2, the parameters of the pre-trained model are frozen, the pre-trained model is adapted by LoRA, the pseudo-RGB three-channel features are input into the adapted pre-trained model, the corresponding three bands are obtained, and the three bands are input into the tail adapter to obtain the predicted values corresponding to the three bands, wherein the three bands are three different bands. Specifically, step S2 further comprises: applying LoRA adaptation on the pre-training linear layer , and the formula is , R , L , wherein the original parameter is frozen, and the parameter , can be trained, is the original weight matrix of the pre-training, is the right matrix of the low-rank matrix, is the left matrix of the low-rank matrix, is the output vector of the linear layer, is the input vector of the linear layer, is the dimension of the output, is the dimension of the input, is the scaling, is the rank, and the classical value is 4-16, and zero perturbation initialization makes the initial ; Low-rank incremental adaptation is performed on the Q matrix or the K matrix or the V matrix of the pre-trained model, the attention output and the MLP linear layer. The parameter quantity after low-rank decomposition is , which is much lower than the original parameter quantity .

[0023] The pseudo- RGB three-channel feature is input into the pre-trained model after adaptation to obtain corresponding three bands; The three bands are input into the tail adapter to obtain high-resolution HSI output , wherein, represents the predicted value of the band , and the target band is predicted. , , When input respectively with as the center, three predicted values of the target band are obtained. , , wherein, is the predicted value of the target band by the spectral sliding window with the band as the center, is the predicted value of the target band by the spectral sliding window with the band as the center, is the predicted value of the target band by the spectral sliding window with the band as the center, and the tail adapter is composed of a Conv block and a GELU activation function.

[0024] In this embodiment, PEFT (Parameter-Efficient Fine-Tuning) performs low-rank incremental adaptation on the Q / K / V, attention output and MLP linear layer of IPT / SwinIR through LoRA, freezes the original weights, and only trains the increments.

[0025] Specifically, the pre-trained model selects IPT (Image Processing Transformer) or SwinIR that has been pre-trained in the natural image super-resolution task, and all original parameters of the pre-trained model are frozen and do not participate in gradient update. LoRA low-rank adaptation is applied on the Q matrix, K matrix, V matrix, attention output projection and MLP linear layer of the pre-trained model. Specifically, for any pre-trained linear layer, under the premise of keeping the original unchanged, new trainable parameters and are added, and the network output is consistent with the pre-training state. This bypass is applied to the Q, K, V projection matrix, attention output projection and two MLP dimension-up / dimension-down linear layers of IPT, forming a “layer-by-layer low-rank increment” structure. The parameter quantity is reduced from the original to .

[0026] The pseudo- RGB three-channel features are stacked along the batch dimension and fed into the adapted IPT. Since the initial state of weights is consistent with pre-training, the first forward result has rich natural image texture priori; the IPT output is still three channels, the spatial size is expanded, and the channel order is one-to-one corresponding to the input pseudo- RGB. The three-channel result is immediately fed into the tail adapter: symmetric to the head, the tail adapter is also composed of Conv + GELU, and the weight initialization strategy is consistent with the head. Its function is to map the "pseudo- RGB" high-resolution features output by the IPT back to the hyperspectral three-waveband, and obtain the high-resolution cubic fragments corresponding to the center wavelength of the input sliding window. Since the sliding window exists overlap in the spectral dimension, the same target waveband will receive three independent predictions. The three predictions are independent of each other, but share the same set of IPT and tail adapter weights, which not only avoids introducing additional parameter quantity, but also provides complementary information for subsequent fusion through different spectral context.

[0027] Please refer to Figure 4 , S3, the spectral window weighted fusion module is used to process the three prediction values to obtain a coarse HR result, and the coarse HR result is input into the spectral reconstruction decoder for refinement to obtain a final high-resolution hyperspectral image.

[0028] Specifically, step S3 further includes: using a spectral window weighted fusion module to fuse and process the three prediction values to obtain a coarse HR result of the entire image . Wherein, the coarse HR result of the target waveband at the coordinates is , , , , is the spectral sliding window , , , is a very small amount, is the standard deviation parameter of the Gaussian function, is the pixel coordinate, is the spectral sliding window centered on the th waveband, the predicted pixel value of the target waveband at the coordinates is , , . It should be noted that M k (t,i,j) in represents the pixel coordinates, which can be represented as Mk (t).

[0029] The coarse HR result is input into a spectral reconstruction decoder for refinement to obtain a final high-resolution hyperspectral image, specifically, the coarse HR result is input into a spectral reconstruction decoder for spatial-spectral joint refinement processing to obtain a final high-resolution hyperspectral image , the SRD is a spectral reconstruction decoder composed of a group of cascaded SSBs, specifically including 1. spectral convolution: kernel expanding the spectral receptive field; 2. spatial convolution: kernel depth separable spatial convolution; 3. point-by-point mixing: point convolution fusion channel.

[0030] In this embodiment, after completing three independent predictions, the “spectral window weighted fusion-spectral reconstruction” two-stage refinement process is entered, which aims to smooth the slight spectral jitter caused by different window centers, and further amplify spatial details and maintain radiation consistency. Then, the fusion output is performed according to the formula to obtain the coarse HR result. Among them, the spectral window weighted fusion, the confidence weight is obtained from one or more of the window consistency, and is normalized together with the spectral position window weight.

[0031] Subsequently, the coarse HR result is input into the spectral reconstruction decoder SRD for spatial-spectral joint refinement to obtain a final high-resolution hyperspectral image. Through the cascaded processing of SSB, the coarse HR result is further improved in spatial details and spectral continuity, and the output is the final high-resolution hyperspectral image, completing the super-resolution reconstruction from coarse to fine.

[0032] Among them, the present application adopts spectral sliding window input to perform multiple reconstructions on the same target waveband, and performs normalized fusion with “spectral position window weight x confidence”, and then obtains the final HR through spectral reconstruction and constraint.

[0033] Preferably, the pre-training model and the spectral reconstruction decoder are constrained by spectral consistency during training and fine-tuning, and at least one of the SAM constraint or the spectral smoothing regularity is included; and an end-to-end joint optimization is adopted, and the corresponding loss function formula is , , is a loss weight, and is 0.01, is a real hyperspectral image, is an L1 norm, is HxW, is the length of the image, is the width of the image, is the predicted spectral vector at the position ​​​For the real spectrum vector at the position is the L2 norm. is the L2 norm.

[0034] Specifically, in the embodiment, under the premise that the overall network is trainable, the pre-trained model and the spectral reconstruction decoder are fine-tuned synchronously instead of being trained in stages. The fine-tuning stage introduces at least a spectral consistency constraint to suppress the common shape drift between bands of hyperspectral images. In specific implementation, in addition to the pixel-level L1 loss, a SAM (Spectral Angle Mapper) constraint is additionally added; spectral smoothing regularization can also be superimposed according to data characteristics, but at least the SAM item is guaranteed to exist, thereby forming an end-to-end joint optimization target.

[0035] To sum up, the application converts the hundred-band hyperspectral super-resolution problem into an end-to-end pipeline of three-band sliding-low-rank increment-spectral weighted fusion-joint consistency constraint, seamlessly injects natural image prior into the hyperspectral domain through LoRA bypass without touching the original pre-trained weight, and realizes domain adaptation by compressing the parameter quantity to a very low proportion of the backbone network; the three-neighbor-window sliding mechanism obtains three complementary predictions for the same target band, and then normalizes and fuses them by using the spectral position weight x neighbor window consistency weight, thereby suppressing outliers and preserving edge sharpness, and the coarse result already has a smooth spectral curve; the subsequent SSB decoder further refines in spectral-spatial alternating convolution, is supervised by the SAM angle loss throughout the whole process, and ensures that the radiation shape is consistent with the true value. The entire scheme completes inference once forward, does not require external RGB assistance, greatly reduces the training and deployment cost, and can simultaneously outperform traditional full-volume fine-tuning and multi-stage methods in terms of spatial details and spectral fidelity, thereby providing a feasible, controllable and scalable technical path for real-time acquisition of high-quality hyperspectral images on board, airborne or edge.

[0036] Referring to Figure 5 The second embodiment of the application provides a hyperspectral image super-resolution device based on iterative reconstruction and spectral refinement, which comprises: The extraction unit 101 is configured to obtain a low-resolution hyperspectral image and extract adjacent three-band sliding windows in a preset spectral dimension, and input the adjacent three-band sliding windows into a head adapter to obtain pseudo-RGB three-channel features. The adaptation unit 102 is configured to freeze the parameters of a pre-trained model, adapt the pre-trained model by using LoRA, input the pseudo-RGB three-channel features into the adapted pre-trained model to obtain corresponding three bands, and input the three bands into a tail adapter to obtain prediction values corresponding to the three bands, respectively, wherein the three bands are three different bands. The image generation unit 103 is configured to process the three prediction values generated by different center windows for the same target waveband by using a spectral window weighting fusion module to obtain a coarse HR result, and input the coarse HR result into a spectral reconstruction decoder to refine the coarse HR result to obtain a final high-resolution hyperspectral image.

[0037] The above describes the preferred embodiments of the present application. It should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements are also considered to be within the protection scope of the present application.

Claims

1. A hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement, characterized in that, The method comprises the following steps: Acquiring low-resolution hyperspectral images and a preset spectral dimension is extracted, and the adjacent three-band sliding window is input into a head adapter to obtain a pseudo-RGB three-channel feature Freezing the parameters of the pre-trained model, adapting the pre-trained model by LoRA, inputting the pseudo RGB three-channel features into the adapted pre-trained model to obtain corresponding three bands, and inputting the three bands into the tail adapter to obtain the prediction values corresponding to the three bands respectively, wherein the three bands are three different bands; The three prediction values generated by different center windows for the same target band are processed by the spectral window weighted fusion module to obtain a coarse HR result, and the coarse HR result is input into the spectral reconstruction decoder for refinement to obtain a final high-resolution hyperspectral image.

2. The hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement according to claim 1, characterized in that, Acquiring low-resolution hyperspectral images The adjacent three-band sliding window is input into a head adapter to obtain a pseudo-RGB three-channel feature, specifically: Acquiring low-resolution hyperspectral image and extracting a three-adjacent-band sliding window with a step of 1 in the preset spectral dimension wherein when the three-adjacent-band sliding window is when the three-adjacent-band sliding window is , represents a specific band index of the hyperspectral image, is a real number, is a length of the low-resolution hyperspectral image, is a width of the low-resolution hyperspectral image, is a number of bands of the low-resolution hyperspectral image, is a low-resolution hyperspectral image corresponding to the band index , is a low-resolution hyperspectral image corresponding to the band index , is a low-resolution hyperspectral image corresponding to the band index . The adjacent three-band sliding window is input into the head adapter to obtain the pseudo RGB three-channel features, wherein the head adapter is composed of a Conv block and a GELU activation function.

3. The hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement according to claim 2, characterized in that, The parameters of the pre-trained model are frozen, and the pre-trained model is adapted by LoRA, specifically as follows: In pre-training linear layer The above applies LoRA adaptation, whose formula is , R , L Where the original parameters are frozen, and the parameters , can be trained, is the pre-trained original weight matrix, is the right matrix of the low-rank matrix, is the left matrix of the low-rank matrix, is the output vector of the linear layer, is the input vector of the linear layer, is the dimension of the output, is the dimension of the input, is the scaling, is the rank; Low-rank incremental adaptation is performed on the Q matrix or K matrix or V matrix of the pre-trained model, the attention output and the MLP linear layer, and the parameter quantity after low-rank decomposition is only , which is much lower than the original parameter quantity .

4. The hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement of claim 1, wherein, The pseudo RGB three-channel features are input into the adapted pre-trained model to obtain corresponding three bands, and the three bands are input into the tail adapter to obtain prediction values corresponding to the three bands respectively, wherein the three bands are three different bands, specifically as follows: The pseudo RGB three-channel features are input into the adapted pre-trained model to obtain corresponding three bands. The three bands are input into the tail adapter, and a high-resolution HSI output is obtained by mapping , , , , , , , , , , , , , , , , , , , , The tail adapter is composed of a Conv block and a GELU activation function.

5. The hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement of claim 1, wherein, The three prediction values generated by different center windows for the same target band are processed by the spectral window weighted fusion module to obtain a coarse HR result, specifically as follows: The three predicted values are fused by a spectral window weighting fusion module to obtain a coarse HR result of the whole image ; wherein the target waveband The coarse HR result at coordinates The coarse HR result at coordinates The formula is , , , , is a spectral sliding window , , , is a minimum amount, is a standard deviation parameter of a Gaussian function, is a pixel coordinate, is a spectral sliding window centered on the waveband, and The predicted pixel value at coordinates is , , , .

6. The hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement of claim 1, wherein, The coarse HR result is input into a spectral reconstruction decoder for refinement to obtain a final high-resolution hyperspectral image. Specifically, the coarse HR result is input into a spectral reconstruction decoder for spatial-spectral joint refinement processing to obtain a final high-resolution hyperspectral image , , The SRD is a spectral reconstruction decoder composed of a series of SSBs.

7. The hyperspectral image super-resolution method based on iterative reconstruction and spectral refinement according to claim 6, characterized in that, The method further comprises the following steps: The pre-trained model and the spectral reconstruction decoder are constrained by spectral consistency during training and fine-tuning, and at least one of the SAM constraint or the spectral smoothing regularity is included; And end-to-end joint optimization is adopted, and the corresponding loss function formula is , , is the loss weight, is the real hyperspectral image, is the L1 norm, is HxW, is the length of the image, is the width of the image, is the predicted spectral vector at position , is the real spectral vector at position , is the L2 norm.

8. A hyperspectral image super-resolution device based on iterative reconstruction and spectral refinement, characterized in that, The method comprises the following steps: An extraction unit is configured to acquire a low-resolution hyperspectral image and a preset spectrum dimension is extracted to obtain a pseudo RGB three-channel feature by inputting the adjacent three-band sliding window into a header adapter. The adaptation unit is configured to freeze the parameters of the pre-trained model, adapt the pre-trained model by LoRA, input the pseudo RGB three-channel features into the adapted pre-trained model to obtain corresponding three bands, and input the three bands into the tail adapter to obtain prediction values corresponding to the three bands respectively, wherein the three bands are three different bands. The image generation unit is configured to process the three prediction values generated by different center windows for the same target band by the spectral window weighted fusion module to obtain a coarse HR result, and input the coarse HR result into the spectral reconstruction decoder for refinement to obtain a final high-resolution hyperspectral image.

Images