High-resolution reconstruction method and device for hyperspectral image, equipment and medium

Abstract

The invention discloses a high-resolution reconstruction method and device for a hyperspectral image, equipment and a medium, and belongs to the technical field of image reconstruction. The image reconstruction network model comprises a local-based multi-scale spatial feature extraction module, a global-based self-attention mechanism spatial feature extraction module and a spatial spectral feature extraction module; obtaining a high-resolution hyperspectral image, and performing degradation processing to generate a low-resolution hyperspectral image; according to the similarity of adjacent spectrums, carrying out band-by-band grouping operation on the low-resolution hyperspectral image; and training the constructed image reconstruction network model according to the grouping result as training data, and performing low-resolution hyperspectral image reconstruction through the trained image reconstruction network model. According to the method, the complete features of the image can be better extracted, and real and effective super-resolution reconstruction of the hyperspectral image is completed.

Description

Technical Field

[0001] The present invention relates to the field of image reconstruction technology, and in particular to a method, device, equipment and medium for reconstructing a high-resolution hyperspectral image. Background Art

[0002] Super-resolution reconstruction of hyperspectral images is an important research direction in the fields of remote sensing and computer vision. Its core goal is to improve the spatial resolution of hyperspectral images through algorithms while maintaining the integrity of their rich spectral information. Hyperspectral imaging captures the fine spectral characteristics of a scene through hundreds of continuous and dense spectral bands, giving it unique advantages in fields such as material identification, environmental monitoring, and precision agriculture. However, due to the physical constraints of imaging equipment (such as sensor sensitivity, optical diffraction limit, and signal-to-noise ratio trade-offs), hyperspectral images often face an inherent contradiction between spatial resolution and spectral resolution: improving spectral resolution often requires increasing the spectral sampling interval of the sensor, resulting in a significant reduction in the spatial details of single-band images. This low spatial resolution seriously restricts the application value of hyperspectral data in tasks such as fine classification and small target detection.

[0003] Traditional hyperspectral image super-resolution methods rely primarily on multi-frame image fusion or optimization models based on prior knowledge. For example, these methods reconstruct a high-resolution image by registering multiple low-resolution images of the same scene, or perform joint spatial-spectral reconstruction using high-resolution panchromatic or multispectral images. However, these methods often have a high dependence on the alignment accuracy, noise level, and prior assumptions of the input data, and struggle to model complex, high-dimensional, spatial-spectral nonlinear relationships. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device, equipment and medium for reconstructing high-resolution hyperspectral images, which can better extract the complete features of the image and complete the real and effective super-resolution reconstruction of the hyperspectral image.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] The present invention provides a method for reconstructing a hyperspectral image with high resolution, comprising:

[0007] Constructing an image reconstruction network model, wherein the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module;

[0008] Acquire high-resolution hyperspectral images and perform degradation processing to generate low-resolution hyperspectral images;

[0009] According to the similarity of adjacent spectra, the low-resolution hyperspectral image is grouped band by band;

[0010] The constructed image reconstruction network model is trained using the grouping results as training data, and low-resolution hyperspectral image reconstruction is performed using the trained image reconstruction network model.

[0011] Optionally, the band-by-band grouping operation on the low-resolution hyperspectral image includes:

[0012] The current band and the four bands with the highest similarity to the current band are selected within the preset spectral range to obtain a five-band group:

[0013]

[0014] Where, Respectively Band images, Respectively Band images, Respectively Band images, For the The five-band group corresponding to each band, L is the total number of bands;

[0015] Each of the five-band groups is divided into three groups:

[0016] like ,but:

[0017] like ,but:

[0018] like ,but:

[0019] Where, Five-band group The grouping results.

[0020] Optionally, the multi-scale spatial feature extraction module includes three branches, each of which includes a first convolutional layer, an encoder group, and a U-Net network;

[0021] Each branch corresponds to a grouping result, and the first convolution layers of the three branches extract features from the grouping results to obtain feature images. ;

[0022] The encoder groups of the three branches respectively process the feature image Perform three downsampling operations to obtain three feature images of different sizes ;

[0023] The feature image The same size is added to obtain the feature image :

[0024]

[0025]

[0026]

[0027] Among them, the feature image The size of the halved;

[0028] The feature image The U-Net network of the third branch performs feature extraction and upsampling operations to obtain a feature image , the feature image After two more upsampling operations, the feature image Add together to get the output of the third branch;

[0029] The feature image With the feature image The result of the splicing operation is extracted and up-sampled by the U-Net network of the second branch to obtain the feature image , the feature image After another upsampling operation, the feature image Add together to get the output of the second branch;

[0030] The feature image With the feature image The result of the splicing operation is extracted and up-sampled by the U-Net network of the first branch and then combined with the feature image Add up to get the output of the first branch;

[0031] After splicing the outputs of the three branches, a multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

[0032] Optionally, the encoder group includes three serially connected encoders, each of which includes a second convolutional layer, a depth-separable convolutional layer, and a maximum pooling layer.

[0033] Optionally, the U-Net network includes a third convolutional layer, a residual module and an upsampling module, and the residual module includes two convolutional layers and a ReLU activation function layer.

[0034] Optionally, the self-attention mechanism spatial feature extraction module includes a Transformer layer, and the Transformer layer includes multiple encoder layers, and the encoder layer is used to:

[0035] The multi-scale feature image X output by the multi-scale spatial feature extraction module is subjected to a Reshape operation and a Permute dimension conversion operation:

[0036]

[0037]

[0038] In the formula, Reshape is the reshape operation, Permute is the Permute dimension conversion operation, are the feature images obtained by the Reshape operation and the Permute dimension conversion operation respectively; H, W, C are the height, width, and number of channels of the multi-scale feature image X, and B is the batch size;

[0039] Based on the multi-head self-attention mechanism, the linear projection of the feature image Z generates the Q value, K value and V value:

[0040]

[0041] Where, are the weight matrices corresponding to Q value, K value and V value respectively;

[0042] Split operations on Q value, K value and V value:

[0043]

[0044]

[0045]

[0046] Where, , h is the number of attention heads, are the Q value, K value, and V value corresponding to the i-th attention head, is the dimension of each attention head, ;

[0047] Calculate the attention value of each attention head:

[0048]

[0049] Where, is the attention value of the i-th attention head, and Softmax is the Softmax activation function;

[0050] Merge attention values ​​to get multi-head attention values :

[0051]

[0052] Where, is the attention value of the h-th attention head, is the projection matrix, Concat is the concatenation operation;

[0053] Multi-head attention value Features are obtained through residual connection and layer normalization :

[0054]

[0055] In the formula, Dropout is the regularization process, and LayerNorm is the layer normalization process;

[0056] feature The features are obtained by feedforward neural network :

[0057]

[0058] Where ReLU is the ReLU activation function, is the weight matrix, is the bias term;

[0059] feature Features are obtained through residual connection and layer normalization :

[0060]

[0061] Compute the output of the encoder layer:

[0062]

[0063] Where, is the feature image corresponding to the previous encoder layer ;

[0064] The outputs of all the encoder layers are summed to obtain the feature :

[0065]

[0066] Where, For the The output of the encoder layer, is the number of encoder layers;

[0067] Pair Features Perform the Reshape operation to obtain feature Y:

[0068] .

[0069] Optionally, the spatial spectrum feature extraction module includes a spatial spectrum fusion module, a fourth convolutional layer, a fifth convolutional layer, a context fusion module and a sixth convolutional layer; after the outputs of the self-attention mechanism spatial feature extraction module and the multi-scale spatial feature extraction module are added, the spatial spectrum fusion module performs spatial and spectral feature fusion to obtain a feature image The characteristic image The fourth convolution layer and the fifth convolution layer are used to extract features and obtain a feature image. The characteristic image The context fusion module and the sixth convolutional layer perform feature fusion and feature extraction to obtain a feature image ; Perform bicubic interpolation on any grouping result and then compare it with the feature image Add to get the reconstructed image.

[0070] In a second aspect, the present invention provides a device for reconstructing a hyperspectral image with high resolution, comprising:

[0071] A model construction module is configured to construct an image reconstruction network model, wherein the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module;

[0072] The sample processing module is configured to acquire a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; based on the similarity of adjacent spectra, the low-resolution hyperspectral image is grouped band by band;

[0073] A model training module is configured to train the constructed image reconstruction network model based on the grouping results as training data;

[0074] The image reconstruction module is configured to perform low-resolution hyperspectral image reconstruction through the trained image reconstruction network model.

[0075] In a third aspect, the present invention provides an electronic device, including a processor and a storage medium;

[0076] The storage medium is used to store instructions;

[0077] The processor is configured to operate according to the instructions to execute the steps of the above method.

[0078] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] The present invention provides a method, apparatus, device, and medium for high-resolution reconstruction of hyperspectral images. Multi-scale feature extraction is performed on hyperspectral image data to extract shallow features containing rich local information. These shallow features are further merged and extracted. Deep features containing global information are extracted through the Transformer layer. To maintain the fidelity of the hyperspectral image and subsequent spatial feature extraction, the obtained features are input into a spatial-spectral fusion module, where the generated spatial features are further fused with the spectral features. These operations extensively and fully extract hyperspectral image features from four aspects: global, local, spatial, and spectral. This captures and reconstructs image information to the greatest extent possible, restoring and reshaping the original image as much as possible, and achieving high fidelity and realism. Results on real hyperspectral datasets demonstrate that the super-resolution reconstruction achieved by the present invention is effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 This is a flow chart of a method for reconstructing a high-resolution hyperspectral image provided by an embodiment of the present invention;

[0082] Figure 2 It is a structural diagram of the image reconstruction network model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0083] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0084] Example 1:

[0085] like Figure 1 As shown, an embodiment of the present invention provides a method for reconstructing a hyperspectral image with high resolution, comprising the following steps:

[0086] Step S1, constructing an image reconstruction network model, the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module and a spatial spectral feature extraction module;

[0087] Step S2: obtaining a high-resolution hyperspectral image and performing degradation processing to generate a low-resolution hyperspectral image;

[0088] Step S3: performing a band-by-band grouping operation on the low-resolution hyperspectral image based on the similarity of adjacent spectra;

[0089] Step S4: training the constructed image reconstruction network model based on the grouping results as training data;

[0090] Step S5: reconstructing the low-resolution hyperspectral image using the trained image reconstruction network model.

[0091] Specifically in this embodiment, performing a band-by-band grouping operation on a low-resolution hyperspectral image includes:

[0092] Select the current band and the four bands with the highest similarity to the current band within the preset spectral range to obtain a five-band group:

[0093]

[0094] Where, Respectively Band images, Respectively Band images, Respectively Band images, For the The five-band group corresponding to each band, L is the total number of bands;

[0095] Divide each five-band group into three groups:

[0096] like ,but:

[0097] like ,but:

[0098] like ,but:

[0099] Where, Five-band group The grouping results.

[0100] like Figure 2 As shown in the figure, the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module.

[0101] (1) The multi-scale spatial feature extraction module consists of three branches, each of which includes the first convolutional layer, the encoder group and the U-Net network.

[0102] Each branch corresponds to a grouping result. The first convolutional layer of the three branches extracts features from the grouping results to obtain feature images. .

[0103] The encoder groups of the three branches respectively process the feature image Perform three downsampling operations to obtain three feature images of different sizes .

[0104] The feature image The same size is added to obtain the feature image :

[0105]

[0106]

[0107]

[0108] Among them, the feature image The size is halved in sequence.

[0109] Feature Image The feature image is obtained by performing feature extraction and upsampling operations on the U-Net network of the third branch. , feature image After two more upsampling operations and the feature image Add them together to get the output of the third branch.

[0110] Feature Image With feature image The result of the splicing operation is extracted and up-sampled by the U-Net network of the second branch to obtain the feature image , feature image After another upsampling operation and the feature image Add them together to get the output of the second branch.

[0111] Feature Image With the feature image The result of the splicing operation is extracted and up-sampled by the U-Net network of the first branch and then combined with the feature image Add them together to get the output of the first branch.

[0112] After splicing the outputs of the three branches, the multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

[0113] Specifically, the encoder group consists of three serially connected encoders, each of which includes a second convolutional layer, a depthwise separable convolutional layer, and a max pooling layer. The depthwise separable convolutional layer can better adapt to diverse input data and significantly reduce computational effort. The output feature map of the depthwise separable convolutional layer is input to a Reinforced Luminance (ReLU) activation function and finally passes through a max pooling layer with a 2×2 kernel size to extract salient image features from the feature map. The depthwise separable convolutional layer extracts features and changes the number of channels in the input feature map. The Reinforced Luminance (RELU) activation function adds nonlinearity, enabling the network to learn more complex features. The max pooling layer downsamples the feature map, reducing its spatial size (width and height) while retaining key features within the local region. Because the pooling operation downsamples the feature map, the size of the feature map is halved with each encoder layer, while other features remain unchanged. Each encoder layer outputs the current result, ultimately resulting in three feature outputs with gradually halved size.

[0114] Specifically, the U-Net network includes a third convolutional layer, a residual module, and an upsampling module. The residual module includes two convolutional layers and a ReLU activation function layer. The U-Net network implements multi-scale feature extraction.

[0115] (2) The self-attention mechanism spatial feature extraction module includes a Transformer layer, which includes multiple encoder layers. The encoder layer is used to:

[0116] Perform Reshape and Permute operations on the multi-scale feature image X output by the multi-scale spatial feature extraction module:

[0117]

[0118]

[0119] In the formula, Reshape is the reshape operation, Permute is the Permute dimension conversion operation, are the feature images obtained by the Reshape operation and the Permute dimension conversion operation respectively; H, W, C are the height, width, and number of channels of the multi-scale feature image X, and B is the batch size.

[0120] Based on the multi-head self-attention mechanism, the linear projection of the feature image Z generates the Q value, K value and V value:

[0121]

[0122] Where, are the weight matrices corresponding to the Q value, K value, and V value respectively.

[0123] Split operations on Q value, K value and V value:

[0124]

[0125]

[0126]

[0127] Where, , h is the number of attention heads, are the Q value, K value, and V value corresponding to the i-th attention head, is the dimension of each attention head, .

[0128] Calculate the attention value of each attention head:

[0129]

[0130] Where, is the attention value of the i-th attention head, and Softmax is the Softmax activation function.

[0131] Merge attention values ​​to get multi-head attention values :

[0132]

[0133] Where, is the attention value of the h-th attention head, is the projection matrix, and Concat is the concatenation operation.

[0134] Multi-head attention value Features are obtained through residual connection and layer normalization :

[0135]

[0136] In the formula, Dropout is the regularization process, and LayerNorm is the layer normalization process.

[0137] feature The features are obtained by feedforward neural network :

[0138]

[0139] Where ReLU is the ReLU activation function, is the weight matrix, is the bias term.

[0140] feature Features are obtained through residual connection and layer normalization :

[0141]

[0142] Compute the output of the encoder layer:

[0143]

[0144] Where, is the feature image corresponding to the previous encoder layer .

[0145] The output of all encoder layers is summed to obtain the feature :

[0146]

[0147] Where, For the The output of the encoder layer, is the number of encoder layers.

[0148] Pair Features Perform the Reshape operation to obtain feature Y:

[0149] .

[0150] (3) The spatial-spectral feature extraction module includes the spatial-spectral fusion module, the fourth convolutional layer, the fifth convolutional layer, the context fusion module, and the sixth convolutional layer; after the outputs of the self-attention mechanism spatial feature extraction module and the multi-scale spatial feature extraction module are added, the spatial-spectral fusion module performs spatial and spectral feature fusion to obtain the feature image. ; Feature image After the fourth convolution layer and the fifth convolution layer, feature extraction is performed to obtain the feature image ; Feature image The feature image is obtained by performing feature fusion and feature extraction through the context fusion module and the sixth convolutional layer. ; Perform bicubic interpolation on any grouping result and then compare it with the feature image Add to get the reconstructed image.

[0151] Specifically in this embodiment, training the constructed image reconstruction network model using the grouping results as training data includes:

[0152] The network parameters are iteratively updated through backpropagation and gradient descent until the loss function converges.

[0153] A composite loss function is used during the training process to evaluate the reconstruction quality: in addition to the basic pixel-level L1 loss (MAE), the spectral angle mapping loss (SAM Loss) ensures spectral fidelity. The gradient of the loss function with respect to the network parameters is calculated through automatic differentiation technology, and a gradient clipping strategy is used to prevent gradient explosion.

[0154] The training process utilizes parameter optimization and learning rate scheduling: The AdamW optimizer is used for parameter updates, and its momentum mechanism accelerates convergence. A cosine annealing learning rate strategy is also employed, with an initial learning rate of 3e-4 and periodic adjustments every 20 epochs. This ensures rapid initial convergence while enabling fine-tuning later in the training process.

[0155] The training process uses a validation monitoring and early stopping mechanism: after each training cycle, indicators such as PSNR, SSIM, and SAM are evaluated on an independent validation set. When the validation loss does not decrease for 10 consecutive epochs, early stopping is triggered and the model is rolled back to the optimal parameter snapshot.

[0156] The present invention was validated using balloon hyperspectral images from the CAVE dataset, a classic spectral image dataset. The balloon hyperspectral images have a spatial resolution of 512×512 pixels and contain 31 spectral bands with wavelengths ranging from 400 nm to 700 nm, with intervals of approximately 10 nm. Super-resolution reconstruction of the balloon hyperspectral images was performed using Bicubic, EDSR, SSPSR, and the present invention, respectively. The reconstruction results and performance indicators are shown in Table 1.

[0157] Table 1 Comparison of indicators of the method of the present invention and other methods

[0158]

[0159] As can be seen from Table 1, the hyperspectral image super-resolution reconstruction index of the present invention is significantly better than that of several other methods.

[0160] The above demonstrates that the present invention, through multi-scale feature extraction and fusion supplemented by the Transformer architecture, as well as the addition of a spatial-spectral fusion mechanism, can fully enable the super-resolution reconstruction network to learn the salient features of hyperspectral images, including texture and detail information, while maintaining the integrity of the spectral information, thereby effectively improving the effect of super-resolution reconstruction of hyperspectral images. This has proven the feasibility of the present invention in super-resolution reconstruction of hyperspectral images.

[0161] Example 2:

[0162] Based on the reconstruction method provided in Example 1, this embodiment of the present invention provides a high-resolution hyperspectral image reconstruction device, including:

[0163] A model construction module is configured to construct an image reconstruction network model, wherein the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module;

[0164] The sample processing module is configured to acquire a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; based on the similarity of adjacent spectra, the low-resolution hyperspectral image is grouped band by band;

[0165] A model training module is configured to train the constructed image reconstruction network model based on the grouping results as training data;

[0166] The image reconstruction module is configured to reconstruct low-resolution hyperspectral images through a trained image reconstruction network model.

[0167] Example 3:

[0168] Based on the reconstruction method provided in the first embodiment, an embodiment of the present invention provides an electronic device, including a processor and a storage medium;

[0169] The storage medium is used to store instructions;

[0170] The processor is configured to operate according to the instructions to execute the steps of the above method.

[0171] Example 4:

[0172] Based on the reconstruction method provided in the first embodiment, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0173] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0177] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for high-resolution reconstruction of a hyperspectral image, characterized in that: include: Constructing an image reconstruction network model, wherein the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module; Acquire high-resolution hyperspectral images and perform degradation processing to generate low-resolution hyperspectral images; According to the similarity of adjacent spectra, the low-resolution hyperspectral image is grouped band by band; The constructed image reconstruction network model is trained according to the grouping results as training data, and low-resolution hyperspectral image reconstruction is performed through the trained image reconstruction network model.

2. The method for high-resolution reconstruction of hyperspectral images according to claim 1, characterized in that: The band-by-band grouping operation of the low-resolution hyperspectral image comprises: The current band and four bands with the highest similarity to the current band are selected within the preset spectral range to obtain a five-band group: In the formula, Respectively Band images, Respectively Band images, Respectively Band images, For the The five-band group corresponding to the bands, L is the total number of bands; Each of the five-band groups is divided into three groups: like ,but: like ,but: like ,but: In the formula, Five-band group The grouping results.

3. The method for high-resolution reconstruction of hyperspectral images according to claim 1, characterized in that: The multi-scale spatial feature extraction module includes three branches, each of which includes a first convolutional layer, an encoder group and a U-Net network; Each branch corresponds to a grouping result, and the first convolution layers of the three branches respectively extract features from the grouping results to obtain feature images. ; The encoder groups of the three branches respectively process the feature image Perform three downsampling operations to obtain three feature images of different sizes ; The feature image The same size is added to obtain the feature image : Among them, the feature image The size of each is halved successively; The feature image The U-Net network of the third branch performs feature extraction and upsampling operations to obtain a feature image , the feature image After two more upsampling operations, the feature image Add together to get the output of the third branch; The feature image With the characteristic image The result of the splicing operation is extracted and upsampled by the U-Net network of the second branch to obtain the feature image , the feature image After another upsampling operation, the feature image Add together to get the output of the second branch; The feature image With the characteristic image The result of the splicing operation is extracted and up-sampled by the U-Net network of the first branch and then combined with the feature image Add together to get the output of the first branch; After splicing the outputs of the three branches, a multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

4. The method for high-resolution reconstruction of hyperspectral images according to claim 3, characterized in that: The encoder group includes three serially connected encoders, each of which includes a second convolution layer, a depth-separable convolution layer, and a maximum pooling layer.

5. The method for high-resolution reconstruction of hyperspectral images according to claim 3, characterized in that: The U-Net network includes a third convolutional layer, a residual module and an upsampling module, and the residual module includes two convolutional layers and a ReLU activation function layer.

6. The method for high-resolution reconstruction of hyperspectral images according to claim 1, characterized in that: The self-attention mechanism spatial feature extraction module includes a Transformer layer, and the Transformer layer includes multiple encoder layers, and the encoder layer is used to: The multi-scale feature image X output by the multi-scale spatial feature extraction module is subjected to a Reshape operation and a Permute dimension conversion operation: In the formula, Reshape is the reshape operation, Permute is the Permute dimension conversion operation, are the feature images obtained by the Reshape operation and the Permute dimension conversion operation respectively; H, W, C are the graphic height, graphic width and number of graphic channels of the multi-scale feature image X, and B is the batch size; Based on the multi-head self-attention mechanism, the feature image Z is linearly projected to generate Q value, K value and V value: In the formula, They are the weight matrices corresponding to the Q value, K value, and V value respectively; Split operations on Q value, K value and V value: In the formula, , h is the number of attention heads, are the Q value, K value, and V value corresponding to the i-th attention head, For the dimension of each attention head, ; Calculate the attention value of each attention head: In the formula, is the attention value of the i-th attention head, Softmax is the Softmax activation function; Merge attention values ​​to get multi-head attention values : In the formula, is the attention value of the h-th attention head, is the projection matrix, Concat is the concatenation operation; Multi-head attention value Features are obtained through residual connection and layer normalization : In the formula, Dropout is the regularization process, and LayerNorm is the layer normalization process; feature The features are obtained by feedforward neural network : Where ReLU is the ReLU activation function, is the weight matrix, is the bias term; feature Features are obtained through residual connection and layer normalization : Compute the output of the encoder layer: In the formula, is the feature image corresponding to the previous encoder layer ; The outputs of all the encoder layers are summed to obtain the feature : In the formula, For the The output of the encoder layer, is the number of encoder layers; Features Perform the Reshape operation to obtain feature Y: 。 7. The method for high-resolution reconstruction of hyperspectral images according to claim 1, characterized in that: The spatial-spectral feature extraction module includes a spatial-spectral fusion module, a fourth convolutional layer, a fifth convolutional layer, a context fusion module and a sixth convolutional layer; after the outputs of the self-attention mechanism spatial feature extraction module and the multi-scale spatial feature extraction module are added, the spatial-spectral fusion module performs spatial and spectral feature fusion to obtain a feature image. The characteristic image The fourth convolution layer and the fifth convolution layer are used to extract features and obtain a feature image. The characteristic image The context fusion module and the sixth convolutional layer perform feature fusion and feature extraction to obtain a feature image. ; Perform bicubic interpolation on any grouping result and then compare it with the feature image Add together to get the reconstructed image.

8. A high-resolution reconstruction device for hyperspectral images, characterized in that: include: A model building module is configured to build an image reconstruction network model, wherein the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module; A sample processing module is configured to acquire a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; According to the similarity of adjacent spectra, the low-resolution hyperspectral image is grouped band by band; A model training module is configured to train the constructed image reconstruction network model according to the grouping results as training data; The image reconstruction module is configured to perform low-resolution hyperspectral image reconstruction through the trained image reconstruction network model.

9. An electronic device, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images