A method, device, equipment and medium for high-resolution reconstruction of hyperspectral images

Abstract

The present invention discloses a method, device, equipment and medium for high-resolution reconstruction of hyperspectral images, belonging to the technical field of image reconstruction. The reconstruction method includes constructing an image reconstruction network model, which includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module; obtaining a high-resolution hyperspectral image and performing degradation processing to generate a low-resolution hyperspectral image; performing per-band grouping operations on the low-resolution hyperspectral image according to the similarity of adjacent spectra; training the constructed image reconstruction network model using the grouping results as training data, and reconstructing the low-resolution hyperspectral image through the trained image reconstruction network model. The present invention can better extract the complete features of the image and complete the super-resolution and effective reconstruction of the hyperspectral image.

Description

Technical Field

[0001] The present invention relates to the technical field of image reconstruction, and particularly relates to a method, device, equipment and medium for high-resolution reconstruction of hyperspectral images. Background Art

[0002] Super-resolution reconstruction of hyperspectral images is an important research direction in the fields of remote sensing and computer vision. Its core goal is to improve the spatial resolution of hyperspectral images through algorithms while maintaining the integrity of their rich spectral information. Hyperspectral imaging captures the fine spectral features of a scene through hundreds of continuous and dense spectral bands, giving it unique advantages in fields such as material identification, environmental monitoring, and precision agriculture. However, limited by the physical constraints of imaging devices (such as sensor sensitivity, optical diffraction limit, and signal-to-noise ratio trade-off), hyperspectral images usually face an inherent contradiction between spatial resolution and spectral resolution: improving spectral resolution often requires increasing the spectral sampling interval of the sensor, resulting in a significant reduction in the spatial details of single-band images. This low spatial resolution severely restricts the application value of hyperspectral data in tasks such as fine classification and small target detection.

[0003] Traditional hyperspectral image super-resolution methods mainly rely on multi-frame image fusion or optimization models based on prior knowledge. For example, high-resolution images are reconstructed by registering multiple low-resolution images of the same scene, or joint spatial-spectral reconstruction is performed using high-resolution panchromatic / multispectral images. However, such methods usually have high dependencies on the alignment accuracy, noise level, and prior assumptions of the input data, and it is difficult to model complex high-dimensional spatial-spectral non-linear relationships. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device, equipment and medium for high-resolution reconstruction of hyperspectral images, which can better extract the complete features of the images and complete the real and effective reconstruction of hyperspectral image super-resolution.

[0005] To achieve the above purpose, the present invention is implemented by the following technical solutions:

[0006] The present invention provides a method for high-resolution reconstruction of hyperspectral images, including:

[0007] Construct an image reconstruction network model, where the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module;

[0008] Obtain a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image;

[0009] Perform a band-by-band grouping operation on the low-resolution hyperspectral image according to the similarity of adjacent spectra;

[0010] Use the grouping result as training data to train the constructed image reconstruction network model, and perform low-resolution hyperspectral image reconstruction through the trained image reconstruction network model.

[0011] Optionally, the band-by-band grouping operation on the low-resolution hyperspectral image includes:

[0012] Select the current band and the four bands with the highest similarity to the current band within a preset spectral range to obtain a five-band group:

[0013]

[0014] wherein, are the images of the th band respectively, are the images of the th band respectively, are the images of the th band respectively, is the five-band group corresponding to the th band, and L is the total number of bands;

[0015] Divide each of the five-band groups into three groups:

[0016] If , then:

[0017]

[0018] If , then:

[0019]

[0020] If , then:

[0021]

[0022] wherein, is the grouping result of the five-band group .

[0023] Optionally, the multi-scale spatial feature extraction module includes three branches, and each branch includes a first convolutional layer, an encoder group, and a U-Net network;

[0024] Each branch corresponds to a grouping result, and the first convolutional layers of the three branches respectively extract features from the grouping results to obtain feature images ;

[0025] The encoder groups of the three branches respectively perform three downsampling operations on the feature image to obtain three feature images with different sizes ;

[0026] Add the feature images with the same size in the feature image to obtain a feature image :

[0027]

[0028]

[0029]

[0030] Among them, the sizes of the feature images are halved in sequence;

[0031] The feature image is subjected to feature extraction and upsampling operations by the U-Net network of the third branch to obtain a feature image , and the feature image is upsampled twice and then added to the feature image to obtain the output of the third branch;

[0032] The feature image is concatenated with the feature image , and the result is subjected to feature extraction and upsampling operations by the U-Net network of the second branch to obtain a feature image , and the feature image is upsampled once and then added to the feature image to obtain the output of the second branch;

[0033] The feature image is concatenated with the feature image , and the result is subjected to feature extraction and upsampling operations by the U-Net network of the first branch and then added to the feature image to obtain the output of the first branch;

[0034] After concatenating the outputs of the three branches, the multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

[0035] Optionally, the encoder group includes three encoders connected in series, and each encoder includes a second convolutional layer, a depthwise separable convolutional layer, and a max pooling layer.

[0036] Optionally, the U-Net network includes a third convolutional layer, a residual module, and an upsampling module, and the residual module includes two convolutional layers and a ReLU activation function layer.

[0037] Optionally, the self-attention mechanism spatial feature extraction module includes a Transformer layer, and the Transformer layer includes a plurality of encoder layers, and the encoder layers are used for:

[0038] Performing a Reshape reshaping operation and a Permute dimension conversion operation on the multi-scale feature image X output by the multi-scale spatial feature extraction module:

[0039]

[0040]

[0041] where Reshape is the Reshape reshaping operation, Permute is the Permute dimension conversion operation, are the feature images obtained by the Reshape reshaping operation and the Permute dimension conversion operation respectively; H, W, and C are the graphic height, graphic width, and graphic channel number of the multi-scale feature image X, and B is the batch size;

[0042] Based on the multi-head self-attention mechanism, linearly projecting the feature image Z to generate Q values, K values, and V values:

[0043]

[0044] where are the weight matrices corresponding to the Q value, K value, and V value respectively;

[0045] Performing a splitting operation on the Q value, K value, and V value:

[0046]

[0047]

[0048]

[0049] where , h is the number of attention heads, are the Q value, K value, and V value corresponding to the i-th attention head, is the dimension of each attention head, ;

[0050] Calculating the attention value of each attention head:

[0051]

[0052] In the formula, is the attention value of the i-th attention head, and Softmax is the Softmax activation function;

[0053] The attention values are combined to obtain the multi-head attention value :

[0054]

[0055] In the formula, is the attention value of the h-th attention head, is the projection matrix, and Concat is the concatenation operation;

[0056] The multi-head attention value is passed through residual connection and layer normalization to obtain the feature :

[0057]

[0058] In the formula, Dropout is the regularization process, and LayerNorm is the layer normalization process;

[0059] The feature is passed through a feed-forward neural network to obtain the feature :

[0060]

[0061] In the formula, ReLU is the ReLU activation function, is the weight matrix, is the bias term;

[0062] The feature is passed through residual connection and layer normalization to obtain the feature :

[0063]

[0064] Calculate the output of the encoder layer:

[0065]

[0066] In the formula, is the feature map corresponding to the previous encoder layer ;

[0067] The outputs of all the encoder layers are summed to obtain the feature :

[0068]

[0069] In the formula, is the The output of the encoder layer, where is the number of encoder layers;

[0070] For the feature perform a Reshape reshaping operation to obtain the feature Y:

[0071] .

[0072] Optionally, the spatial-spectral feature extraction module includes a spatial-spectral fusion module, a fourth convolutional layer, a fifth convolutional layer, a context fusion module, and a sixth convolutional layer; after adding the outputs of the self-attention mechanism spatial feature extraction module and the multi-scale spatial feature extraction module, perform spatial and spectral feature fusion through the spatial-spectral fusion module to obtain a feature image ; the feature image perform feature extraction through the fourth convolutional layer and the fifth convolutional layer to obtain a feature image ; the feature image perform feature fusion and feature extraction through the context fusion module and the sixth convolutional layer to obtain a feature image ; perform bicubic interpolation processing on any grouping result, and then add it to the feature image to obtain the reconstructed image.

[0073] In a second aspect, the present invention provides a high-resolution reconstruction device for hyperspectral images, including:

[0074] A model construction module, configured to construct an image reconstruction network model, where the image reconstruction network model includes a local-based multi-scale spatial feature extraction module, a global-based self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module;

[0075] A sample processing module, configured to obtain a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; perform a per-band grouping operation on the low-resolution hyperspectral image according to the similarity of adjacent spectra;

[0076] A model training module, configured to train the constructed image reconstruction network model according to the grouping result as training data;

[0077] An image reconstruction module, configured to perform low-resolution hyperspectral image reconstruction through the trained image reconstruction network model.

[0078] In a third aspect, the present invention provides an electronic device, including a processor and a storage medium;

[0079] The storage medium is used to store instructions;

[0080] The processor is used to operate according to the instructions to execute the steps of the above method.

[0081] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0082] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0083] The present invention provides a method, device, equipment and medium for high-resolution reconstruction of hyperspectral images. Through hyperspectral image data, multi-scale feature extraction is carried out to extract shallow features containing rich local information, and the shallow features are further merged and extracted. Deep features containing global information are extracted through the Transformer layer. Then, in order to maintain the fidelity of the hyperspectral image and subsequent spatial feature extraction, the obtained features are input into the spatial-spectral fusion module, and further fusion is carried out using the generated spatial features and spectral features; these operations widely and fully extract hyperspectral image features from four aspects: global, local, spatial and spectral, capture and reconstruct image information to the greatest extent, and restore and shape the original image as much as possible, achieving the effects of high vividness and high fidelity of the image. The implementation results on real hyperspectral data sets show that the super-resolution reconstruction effect obtained by the present invention is better. Description of the Drawings

[0084] Figure 1 is a schematic flowchart of a method for high-resolution reconstruction of a hyperspectral image provided by an embodiment of the present invention;

[0085] Figure 2 is a schematic structural diagram of an image reconstruction network model provided by an embodiment of the present invention. Detailed Embodiments

[0086] The present invention will be further described below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0087] Embodiment 1:

[0088] As Figure 1 shown, an embodiment of the present invention provides a method for high-resolution reconstruction of a hyperspectral image, including the following steps:

[0089] Step S1, construct an image reconstruction network model, the image reconstruction network model includes a local-based multi-scale spatial feature extraction module, a global-based self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module;

[0090] Step S2: Obtain a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image;

[0091] Step S3: According to the similarity of adjacent spectra, perform a band-by-band grouping operation on the low-resolution hyperspectral image;

[0092] Step S4: Use the grouping result as training data to train the constructed image reconstruction network model;

[0093] Step S5: Reconstruct the low-resolution hyperspectral image through the trained image reconstruction network model.

[0094] Specifically, in this embodiment, the band-by-band grouping operation on the low-resolution hyperspectral image includes:

[0095] Select the current band and the four bands with the highest similarity to the current band within a preset spectral range to obtain a five-band group:

[0096]

[0097] where are the images of the th band respectively, are the images of the th band respectively, are the images of the th band respectively, is the five-band group corresponding to the th band, and L is the total number of bands;

[0098] Divide each five-band group into three groups:

[0099] If , then:

[0100]

[0101] If , then:

[0102]

[0103] If , then:

[0104]

[0105] where is the grouping result of the five-band group .

[0106] For example Figure 2As shown in the figure, the image reconstruction network model includes a local-based multi-scale spatial feature extraction module, a global-based self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module.

[0107] (1) The multi-scale spatial feature extraction module includes three branches, and each branch includes a first convolutional layer, an encoder group, and a U-Net network.

[0108] Each branch corresponds to a grouping result, and the first convolutional layers of the three branches respectively extract features from the grouping results to obtain feature images .

[0109] The encoder groups of the three branches respectively perform three downsampling operations on the feature images to obtain three feature images of different sizes .

[0110] Add the feature images with the same size to obtain a feature image :

[0111]

[0112]

[0113]

[0114] Among them, the sizes of the feature images are halved in turn.

[0115] The feature image is subjected to feature extraction and upsampling operations by the U-Net network of the third branch to obtain a feature image , and the feature image is upsampled twice and then added to the feature image to obtain the output of the third branch.

[0116] The feature image is concatenated with the feature image , and the result is subjected to feature extraction and upsampling operations by the U-Net network of the second branch to obtain a feature image , and the feature image is upsampled once and then added to the feature image to obtain the output of the second branch.

[0117] The feature image is concatenated with the feature image , and the result is subjected to feature extraction and upsampling operations by the U-Net network of the first branch and then added to the feature image The outputs are added to obtain the output of the first branch.

[0118] After concatenating the outputs of the three branches, the multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

[0119] Specifically, the encoder group includes three encoders connected in series. Each encoder includes a second convolutional layer, a depthwise separable convolutional layer, and a max pooling layer. The depthwise separable convolutional layer can better adapt to different input data and significantly reduce the computational amount. The output feature map of the depthwise separable convolutional layer is input into the ReLu activation function, and finally passed through a max pooling layer with a pooling kernel size of 2×2 to extract the significant image features of the feature map. The role of the depthwise separable convolutional layer is to extract features and change the number of channels of its input feature map; the role of the activation function RELU is to increase non-linearity so that the network can learn more complex features; the max pooling layer mainly downsamples the feature map, reducing the spatial dimensions (width and height) of the feature map while retaining the main features within the local area. Since the pooling operation downsamples the feature map, that is, every time an encoder layer is passed, that is, every time a pooling operation is passed, the size of the feature map is halved and the others remain unchanged. The current result is output every time an encoder layer is passed, and finally three feature output results with gradually halved sizes are obtained.

[0120] Specifically, the U-Net network includes a third convolutional layer, a residual module, and an upsampling module. The residual module includes two convolutional layers and a ReLU activation function layer. The U-Net network realizes multi-scale feature extraction.

[0121] (2) The self-attention mechanism spatial feature extraction module includes a Transformer layer. The Transformer layer includes multiple encoder layers, and the encoder layers are used for:

[0122] Performing a Reshape reshaping operation and a Permute dimension conversion operation on the multi-scale feature image X output by the multi-scale spatial feature extraction module:

[0123]

[0124]

[0125] In the formula, Reshape is the Reshape reshaping operation, Permute is the Permute dimension conversion operation, are the feature images obtained by the Reshape reshaping operation and the Permute dimension conversion operation respectively; H, W, and C are the graphic height, graphic width, and graphic channel number of the multi-scale feature image X, and B is the batch size.

[0126] Based on the multi-head self-attention mechanism, linearly project the feature image Z to generate Q values, K values, and V values:

[0127]

[0128] In the formula, are the weight matrices corresponding to the Q value, K value, and V value respectively.

[0129] Perform segmentation operations on the Q value, K value, and V value:

[0130]

[0131]

[0132]

[0133] In the formula, , h is the number of attention heads, are the Q value, K value, and V value corresponding to the i-th attention head, is the dimension of each attention head, .

[0134] Calculate the attention value of each attention head:

[0135]

[0136] In the formula, is the attention value of the i-th attention head, and Softmax is the Softmax activation function.

[0137] Merge the attention values to obtain the multi-head attention value :

[0138]

[0139] In the formula, is the attention value of the h-th attention head, is the projection matrix, and Concat is the concatenation operation.

[0140] The multi-head attention value Obtain the feature through residual connection and layer normalization:

[0141]

[0142] In the formula, Dropout is the regularization process, and LayerNorm is the layer normalization process.

[0143] The feature Obtain the feature through the feed-forward neural network:

[0144]

[0145] Wherein, ReLU is the ReLU activation function, is the weight matrix, is the bias term.

[0146] Feature is obtained by residual connection and layer normalization to get feature :

[0147]

[0148] Calculate the output of the encoder layer:

[0149]

[0150] Wherein, is the feature image corresponding to the previous encoder layer .

[0151] Sum the outputs of all encoder layers to get feature :

[0152]

[0153] Wherein, is the th output of the encoder layer, is the number of encoder layers.

[0154] Perform a Reshape reshaping operation on feature to get feature Y:

[0155] .

[0156] (3) The spatial spectral feature extraction module includes a spatial spectral fusion module, a fourth convolutional layer, a fifth convolutional layer, a context fusion module, and a sixth convolutional layer; after adding the outputs of the self-attention mechanism spatial feature extraction module and the multi-scale spatial feature extraction module, spatial and spectral feature fusion is performed through the spatial spectral fusion module to obtain the feature image ; the feature image undergoes feature extraction through the fourth convolutional layer and the fifth convolutional layer to obtain the feature image ; the feature image undergoes feature fusion and feature extraction through the context fusion module and the sixth convolutional layer to obtain the feature image ; perform bicubic interpolation processing on any grouping result, and then add it to the feature image to obtain the reconstructed image.

[0157] Specifically, in this embodiment, training the constructed image reconstruction network model with the grouping result as training data includes:

[0158] Iteratively update the network parameters through backpropagation and gradient descent until the loss function converges.

[0159] The training process uses a composite loss function to evaluate the reconstruction quality: in addition to the basic pixel-level L1 loss (MAE), the Spectral Angle Mapping loss (SAM Loss) ensures spectral fidelity. Calculate the gradient of the loss function with respect to the network parameters through automatic differentiation technology, and adopt a gradient clipping strategy to prevent gradient explosion.

[0160] The training process adopts parameter optimization and learning rate scheduling: Use the AdamW optimizer to perform parameter updates, and its momentum mechanism can accelerate the convergence process. At the same time, adopt the cosine annealing learning rate strategy, set the initial learning rate to 3e-4, and perform periodic adjustment every 20 epochs, which can not only ensure fast convergence in the initial stage, but also fine-tune in the later stage.

[0161] The training process adopts validation monitoring and early stopping mechanism: After each training cycle, evaluate indicators such as PSNR, SSIM, and SAM on an independent validation set. When the validation loss does not decrease for 10 consecutive epochs, trigger early stopping and roll back to the best parameter snapshot.

[0162] The embodiment of the present invention uses the balloon hyperspectral image in the classic hyperspectral image dataset CAVE dataset for verification. The spatial resolution of the balloon hyperspectral image is 512×512 pixels, including 31 spectral bands, and the wavelength range covers 400nm to 700nm, with an interval of about 10nm. Bicubic, EDSR, SSPSR and the present invention are respectively used to perform super-resolution reconstruction on the balloon hyperspectral image, and the reconstruction effect and indicators are shown in Table 1.

[0163] Table 1 Comparison chart of indicators of the method of the present invention and other methods

[0164]

[0165] As can be seen from Table 1, the super-resolution reconstruction indicators of the hyperspectral images of the present invention are significantly better than several other methods.

[0166] The above confirms that the present invention can fully enable the super-resolution reconstruction network to learn the significant features of hyperspectral images, including texture and detail information, etc., while maintaining the integrity of spectral information, effectively improving the effect of hyperspectral image super-resolution reconstruction, and proving the feasibility of the present invention in hyperspectral image super-resolution reconstruction by means of multi-scale feature extraction and fusion supplemented by the Transformer architecture and adding a spatio-spectral fusion mechanism.

[0167] Example Two:

[0168] Based on the reconstruction method provided in Example One, an embodiment of the present invention provides a high-resolution reconstruction device for hyperspectral images, including:

[0169] A model construction module, configured to construct an image reconstruction network model, where the image reconstruction network model includes a local multi-scale spatial feature extraction module, a global self-attention mechanism spatial feature extraction module, and a spatial spectral feature extraction module;

[0170] A sample processing module, configured to obtain a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; according to the similarity of adjacent spectra, perform a per-band grouping operation on the low-resolution hyperspectral image;

[0171] A model training module, configured to train the constructed image reconstruction network model using the grouping result as training data;

[0172] An image reconstruction module, configured to perform reconstruction of the low-resolution hyperspectral image through the trained image reconstruction network model.

[0173] Example Three:

[0174] Based on the reconstruction method provided in Example One, an embodiment of the present invention provides an electronic device, including a processor and a storage medium;

[0175] The storage medium is used to store instructions;

[0176] The processor is used to operate according to the instructions to execute the steps of the above method.

[0177] Example Four:

[0178] Based on the reconstruction method provided in Example One, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0179] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0180] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0181] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0182] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0183] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principles of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A high-resolution reconstruction method for hyperspectral images, characterized in that, Including: Construct an image reconstruction network model, which includes a local-based multi-scale spatial feature extraction module, a global-based self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module; Obtain a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; According to the similarity of adjacent spectra, perform a band-by-band grouping operation on the low-resolution hyperspectral image; Use the grouping result as training data to train the constructed image reconstruction network model, and perform low-resolution hyperspectral image reconstruction through the trained image reconstruction network model; Among them, the multi-scale spatial feature extraction module includes three branches, and each branch includes a first convolutional layer, an encoder group, and a U-Net network; Each of the branches corresponds to a grouping result, and the first convolutional layers of the three branches respectively extract features from the grouping results to obtain feature images ; The encoder groups of the three branches respectively perform three downsampling operations on the feature image to obtain three feature images with different sizes ; Add the feature images with the same size in to obtain the feature image : ； ； ； Among them, the characteristic images are successively halved in size; The feature image is subjected to feature extraction and upsampling operations by the U-Net network of the third branch to obtain a feature image , the feature image is subjected to two more upsampling operations and then added to the feature image to obtain the output of the third branch; The feature image and the feature image are subjected to a splicing operation, and the result is subjected to feature extraction and upsampling operations by the U-Net network of the second branch to obtain a feature image , the feature image is subjected to another upsampling operation and then added to the feature image to obtain the output of the second branch; The feature image and the feature image are subjected to a splicing operation. After the result is subjected to feature extraction and upsampling operations by the U-Net network of the first branch, it is added to the feature image to obtain the output of the first branch; After splicing the outputs of the three branches, a multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

2. The high-resolution reconstruction method of hyperspectral images according to claim 1, wherein The band-by-band grouping operation on the low-resolution hyperspectral image includes: Select the current band and the four bands with the highest similarity to the current band within a preset spectral range to obtain a five-band group: ； wherein, are respectively the images of the th band, are respectively the images of the th band, are respectively the images of the th band, is the five - band group corresponding to the th band, and L is the total number of bands; Divide each of the five-band groups into three groups: If , then: ； If , then: ； If , then: ； In the formula, is the grouping result of the five-band group .

3. The high-resolution reconstruction method of hyperspectral images according to claim 1, characterized in that, The encoder group includes three serially connected encoders, and each encoder includes a second convolutional layer, a depthwise separable convolutional layer, and a max pooling layer.

4. The high-resolution reconstruction method for hyperspectral images according to claim 1, wherein The U-Net network includes a third convolutional layer, a residual module, and an upsampling module, and the residual module includes two convolutional layers and a ReLU activation function layer.

5. The high-resolution reconstruction method of hyperspectral images according to claim 1, characterized in that The self-attention mechanism spatial feature extraction module includes a Transformer layer, and the Transformer layer includes multiple encoder layers, and the encoder layers are used for: Perform a Reshape reshaping operation and a Permute dimension conversion operation on the multi-scale feature image X output by the multi-scale spatial feature extraction module: ； ； In the formula, Reshape is the Reshape reshaping operation, and Permute is the Permute dimension conversion operation. are the feature images obtained by the Reshape reshaping operation and the Permute dimension conversion operation respectively; H, W, and C are the graph height, graph width, and graph channel number of the multi-scale feature image X, and B is the batch size. Based on the multi-head self-attention mechanism, linearly project the feature image Z to generate Q values, K values, and V values: ； In the formula, are the weight matrices corresponding to the Q value, the K value, and the V value, respectively; Perform a splitting operation on the Q values, K values, and V values: ； ； ； In the formula, , where h is the number of attention heads, are the Q value, K value, and V value corresponding to the i-th attention head, is the dimension of each attention head, ; Calculate the attention values of each attention head: ； wherein, is the attention value of the i-th attention head, and Softmax is the Softmax activation function; Combine the attention values to obtain the multi-head attention value : ； wherein, is the attention value of the h-th attention head, is the projection matrix, and Concat is the concatenation operation; Multi-head attention value Obtain features through residual connection and layer normalization : ； In the formula, Dropout is regularization processing, and LayerNorm is layer normalization processing; Feature The feature is obtained by a feedforward neural network : ； where ReLU is the ReLU activation function, is the weight matrix, is the bias term; Feature The feature is obtained through residual connection and layer normalization : ； Calculate the output of the encoder layer: ； In the formula, is the feature image corresponding to the previous encoder layer ; The outputs of all the encoder layers are summed to obtain the features : ； In the formula, is the output of the -th encoder layer, is the number of encoder layers; For the feature perform a Reshape operation to obtain feature Y: 。 6. The high - resolution reconstruction method of hyperspectral images according to claim 1, characterized in that, The spatial spectral feature extraction module includes a spatial spectral fusion module, a fourth convolutional layer, a fifth convolutional layer, a context fusion module, and a sixth convolutional layer; after the outputs of the self-attention mechanism spatial feature extraction module and the multi-scale spatial feature extraction module are added together, spatial and spectral feature fusion is performed by the spatial spectral fusion module to obtain a feature image ; the feature image is subjected to feature extraction through the fourth convolutional layer and the fifth convolutional layer to obtain a feature image ; the feature image is subjected to feature fusion and feature extraction through the context fusion module and the sixth convolutional layer to obtain a feature image ; any grouping result is subjected to bicubic interpolation processing, and then added to the feature image to obtain a reconstructed image.

7. A high-resolution reconstruction device for hyperspectral images, characterized in that, Including: A model construction module configured to construct an image reconstruction network model, which includes a local-based multi-scale spatial feature extraction module, a global-based self-attention mechanism spatial feature extraction module, and a spatial-spectral feature extraction module; A sample processing module configured to obtain a high-resolution hyperspectral image and perform degradation processing to generate a low-resolution hyperspectral image; According to the similarity of adjacent spectra, perform a band-by-band grouping operation on the low-resolution hyperspectral image; A model training module configured to train the constructed image reconstruction network model using the grouping result as training data; An image reconstruction module configured to perform low-resolution hyperspectral image reconstruction through the trained image reconstruction network model; Among them, the multi-scale spatial feature extraction module includes three branches, and each branch includes a first convolutional layer, an encoder group, and a U-Net network; Each of the branches corresponds to a grouping result, and the first convolutional layers of the three branches respectively perform feature extraction on the grouping results to obtain feature images ; The encoder groups of the three branches respectively perform three downsampling operations on the feature image to obtain three feature images with different sizes ; Add the feature images with the same size in, and obtain the feature image : ； ； ； Among them, the feature images are successively halved in size; The feature image undergoes feature extraction and upsampling operations by the U-Net network of the third branch to obtain a feature image , the feature image after undergoing two more upsampling operations, is added to the feature image to obtain the output of the third branch; The feature image and the feature image are subject to a splicing operation, and the result is subjected to feature extraction and upsampling operations by the U-Net network of the second branch to obtain a feature image , and the feature image is subjected to another upsampling operation and then added to the feature image to obtain the output of the second branch; The feature image and the feature image are subjected to a splicing operation, and the result is subjected to feature extraction and upsampling operations by the U-Net network of the first branch and then added to the feature image to obtain the output of the first branch; After splicing the outputs of the three branches, a multi-scale feature image output by the multi-scale spatial feature extraction module is obtained.

8. An electronic device, characterized in that, It includes a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, wherein, When the program is executed by the processor, the steps of the method according to any one of claims 1-6 are implemented.

Images