Hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features

Abstract

The invention discloses a hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features, and mainly solves the problems that complex features in a spectral domain and a spatial domain cannot be fully extracted and the noise suppression effect is poor in the prior art. The scheme comprises the following steps: 1) sampling and normalizing an original image to generate a training data set; 2) respectively constructing a cross-domain feature extraction layer, a spectral differential feature extraction layer and a spatial differential feature extraction layer, and constructing a feature extraction sub-network based on differential features and cross-domain features by using the three feature extraction layers; 3) constructing a reconstruction network by using the shallow feature extraction layer and the plurality of feature extraction sub-networks, and training the reconstruction network by using the training data set; and 4) using the trained network to realize super-resolution reconstruction of a real hyperspectral image. According to the method, the cross-domain features of the spectral domain and the spatial domain of the hyperspectral image can be fully extracted and utilized, and the problem of detail distortion of the reconstructed image is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and further relates to a hyperspectral image super-resolution reconstruction technology, specifically a hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features, which can be used to supplement image detail information while amplifying a target image to achieve super-resolution reconstruction of a hyperspectral image. Background Art

[0002] With the rapid development of my country's aerospace industry and remote sensing detection technology, the demand for hyperspectral detection technology has gradually increased, and has developed into a cutting-edge technology with important scientific research value and application prospects. Hyperspectral images can obtain richer spectral information, that is, they have higher spectral resolution, which can better reflect the material, composition and structure of objects. However, the low spatial resolution of hyperspectral images limits the subsequent higher-level task processing effect. Therefore, the super-resolution reconstruction technology for hyperspectral images has important scientific research significance and application value. In the reconstruction technology, problems such as loss of reconstructed image details are often encountered. How to fully extract and utilize the spatial information in the image and the correlation between spectral bands has become an aspect that needs to be considered in the reconstruction process.

[0003] In their paper “Hyperspectral Image Super-Resolution Based on Feature Diversity Extraction”, Z. Jing et al. proposed a method for super-resolution reconstruction of hyperspectral images based on feature diversity extraction. First, the input image is passed through a convolution layer with a convolution kernel size of 3×3×3 to extract shallow features, and then the shallow features are passed through five feature diversity extraction modules in sequence. Each module includes four spatial-spectral feature extraction modules based on channel attention. Finally, the output features of the feature diversity extraction module are passed through an image reconstruction module based on improved 3D sub-pixel convolution to obtain the final result image. Although this method improves the reconstruction effect to a certain extent, there is still a problem that the convolutional neural network does not fully learn the image detail information, resulting in distortion of the reconstructed image details. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features to solve the problem that complex features in the spectral domain and spatial domain cannot be fully extracted and noise cannot be suppressed in hyperspectral image super-resolution reconstruction.

[0005] The technical idea to achieve the purpose of the present invention is: using the cross-domain connection between the spectral domain and spatial domain features of the hyperspectral image, the hyperspectral features are extracted and emphasized, and at the same time, the differential feature distribution of the spectral domain is used to reconstruct the spectral domain using the first-order differential features; for the feature information of the spatial domain, the Laplace operator is used to extract the spatial domain features, and the second-order differential features are used to suppress the noise of the spatial domain features. The present invention can fully extract and utilize the two cross-domain features of the spectral domain and the spatial domain of the hyperspectral image, thereby effectively improving the problem of detail distortion in the reconstructed image.

[0006] The present invention achieves the above-mentioned purpose by the following specific steps:

[0007] (1) Sampling and normalizing the images in the original data set to obtain low-resolution image blocks, and using the low-resolution image blocks to generate a training data set;

[0008] (2) Build a feature extraction sub-network based on differential features and cross-domain features:

[0009] (2.1) Construct a cross-domain feature extraction layer, whose structure is: the first convolution layer, the first sampling layer, the second convolution layer, the third convolution layer, the second sampling layer, the first mathematical operation layer, the fourth convolution layer, the second mathematical operation layer, the fifth convolution layer, the third sampling layer, the sixth convolution layer, the seventh convolution layer, the fourth sampling layer, the third mathematical operation layer, the eighth convolution layer, and the ninth convolution layer; wherein the first and second sampling layers are the spectral domain downsampling layer and the upsampling layer, respectively, the third and fourth sampling layers are the spatial domain downsampling layer and the upsampling layer, respectively, and all sampling layers use trilinear interpolation functions as sampling functions; the first to third mathematical operation layers are all point-by-point addition operations;

[0010] (2.2) constructing a spectral differential feature extraction layer, whose structure is: differential layer, first convolution layer, second convolution layer, first mathematical operation layer, second mathematical operation layer, first integration layer, third convolution layer, fourth convolution layer, first activation function layer, second activation function layer; wherein, the first mathematical operation layer is a point-by-point multiplication operation, the second mathematical operation layer is a point-by-point addition operation, the first activation function layer is implemented by using the exp(-|x|) exponential function; the second activation function layer is implemented by using the softmax function;

[0011] (2.3) Construct a spatial differential feature extraction layer, whose structure is: the first convolution layer, the second convolution layer, the second-order differential operator layer, the first mathematical operation layer, the first differential layer, the second differential layer, the second mathematical operation layer, the first activation function layer, the third mathematical operation layer, the fourth mathematical operation layer, the fifth mathematical operation layer, the sixth mathematical operation layer, the combination layer, the second activation function layer, and the third activation function layer. Among them, the second-order differential operator layer uses the Laplace operator as the second-order differential feature extraction operator; the first mathematical operation layer is a point-by-point subtraction operation, the second mathematical operation layer is a point-by-point Hessian matrix determinant operation, and the third to sixth mathematical operation layers use point-by-point multiplication operations; the first differential layer is applied in the spatial domain and uses a first-order difference operation; the second differential layer is applied in the spatial domain and uses a second-order difference operation; the combination layer uses a parallel method to combine features; the first activation function layer is implemented using the relu function, and the second and third activation function layers are implemented using the exp(-|x|) exponential function;

[0012] (2.4) Using the feature extraction layer constructed in steps (2.1)-(2.3), a feature extraction subnetwork based on differential features and cross-domain features is constructed, and its structure is as follows: cross-domain feature extraction layer, spectral differential feature extraction layer, spatial differential feature extraction layer, first combination layer, second combination layer, third combination layer, and mathematical operation layer;

[0013] (3) Using the shallow feature extraction layer and five feature extraction sub-networks based on differential features and cross-domain features, a hyperspectral image super-resolution reconstruction network based on differential features and cross-domain features is constructed;

[0014] (4) inputting the training data set into the hyperspectral image super-resolution reconstruction network built in step (3), iteratively updating the network parameters through back propagation and gradient descent until the loss function converges, and obtaining a trained super-resolution network based on differential features and cross-domain features;

[0015] (5) Use the trained super-resolution network based on differential features and cross-domain features to perform super-resolution reconstruction of real hyperspectral images.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] First, since the present invention constructs a hyperspectral image super-resolution reconstruction network based on differential features and cross-domain features, and utilizes the cross-domain connection between the spectral domain and spatial domain features of the hyperspectral image, the two cross-domain features are extracted and utilized, so that the network can fully learn the image detail information, alleviating the problem that the prior art cannot fully utilize the joint features of the spectral domain and spatial domain in the hyperspectral image; by effectively utilizing the cross-domain features of the spectral domain and spatial domain to perform super-resolution reconstruction of the hyperspectral image, the problem of detail distortion in the reconstructed image is overcome.

[0018] Second, the present invention constructs a spectral differential feature extraction layer, which can effectively improve the network's learning ability of the spectral domain feature distribution, thereby making full use of the differential characteristics of the spectral domain of the hyperspectral image, and using the first-order differential features to reconstruct the spectral domain, so that the spectral domain feature distribution curves of the reconstructed image and the reference image are closer, that is, the difference between the feature distribution of the hyperspectral image in the spectral domain after super-resolution reconstruction and the reference image is reduced.

[0019] Third, the present invention constructs a spatial differential feature extraction layer for the spatial domain, uses the Laplace operator to perform edge enhancement, and uses the second-order differential operator to suppress noise in the spatial domain features. By making full use of the differential characteristics of the hyperspectral image in the spatial domain, the noise phenomenon generated in the feature map after the differential operator performs feature extraction is eliminated, thereby greatly improving the detail information of the reconstructed image and achieving a better reconstruction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart for realizing the present invention;

[0021] Figure 2 Schematic diagram of the structure of the cross-domain feature extraction layer in the present invention;

[0022] Figure 3 Schematic diagram of the structure of the spectrum differential feature extraction layer in the present invention;

[0023] Figure 4 Schematic diagram of the structure of the spatial differential feature extraction layer in the present invention;

[0024] Figure 5 A schematic diagram of the Pavia data set used in an embodiment of the present invention;

[0025] Figure 6 Result diagrams of image reconstruction using the existing method and the method of the present invention in an embodiment of the present invention; wherein (a)-(d) are respectively a reconstructed image obtained by using the traditional bicubic interpolation method, a reconstructed image obtained by using an excellent deep learning method, a reconstructed image after being magnified 2 times using the method of the present invention, and a high-resolution reference image. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings.

[0027] Embodiment 1: Refer to the attached Figure 1-4 The present invention proposes a hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features, which specifically includes the following steps:

[0028] Step 1: Sample and normalize the images in the original data set to obtain low-resolution image blocks, and use the low-resolution image blocks to generate a training data set; the implementation steps are as follows:

[0029] (1.1) Downsampling the original data set by 1, 3 / 4, and 1 / 2, respectively, and using the downsampled images as the initial high-resolution images at 2, 4, and 8 magnification factors, respectively;

[0030] (1.2) The initial high-resolution images at magnification factors of 2, 4, and 8 are normalized respectively using the maximum normalization method to obtain high-resolution reference images at different magnification factors. The normalization process described in this embodiment is to limit the pixel values ​​of the initial high-resolution image to the interval [0, 1] to complete the unified normalization of the image.

[0031] (1.3) The high-resolution reference images with different magnifications are randomly cropped with overlaps. In this embodiment, the high-resolution reference images with magnification factors of 2, 4, and 8 are cropped into image blocks of sizes 64*64, 128*128, and 256*256, respectively; and the cropped image blocks are randomly rotated by angles of 90°, 180°, or 270°. The rotated image blocks are used as true reference labels, and then corresponding downsampling operations of 2, 4, and 8 times are performed on them to obtain low-resolution image blocks of size 32*32. All low-resolution image blocks and their corresponding true reference labels constitute a training data set.

[0032] Step 2. Build a feature extraction subnetwork based on differential features and cross-domain features:

[0033] (2.1)Reference Figure 2 , construct a cross-domain feature extraction layer, whose structure is: a first convolution layer, a first sampling layer, a second convolution layer, a third convolution layer, a second sampling layer, a first mathematical operation layer, a fourth convolution layer, a second mathematical operation layer, a fifth convolution layer, a third sampling layer, a sixth convolution layer, a seventh convolution layer, a fourth sampling layer, a third mathematical operation layer, an eighth convolution layer, and a ninth convolution layer; wherein the first and second sampling layers are spectral domain downsampling layers and upsampling layers, respectively, the third and fourth sampling layers are spatial domain downsampling layers and upsampling layers, respectively, and all sampling layers use trilinear interpolation functions as sampling functions; the first to third mathematical operation layers are all point-by-point addition operations; in this embodiment, the cross-domain feature extraction layer is set to have a size of 3×1×1 and a step size of 1 for the first to fourth convolution layers, a size of 1×3×3 and a step size of 1 for the fifth to eighth convolution layers, and a size of 1×1×1 and a step size of 1 for the ninth convolution layer.

[0034] (2.2)Reference Figure 3, construct a spectral differential feature extraction layer, whose structure is: differential layer, first convolution layer, second convolution layer, first mathematical operation layer, second mathematical operation layer, first integration layer, third convolution layer, fourth convolution layer, first activation function layer, second activation function layer; wherein, the first mathematical operation layer is a point-by-point multiplication operation, the second mathematical operation layer is a point-by-point addition operation, the first activation function layer is implemented by the exp(-|x|) exponential function; the second activation function layer is implemented by the softmax function; in this embodiment, the differential layer is a differential operation along the spectral dimension, the integration layer is an accumulation operation along the spectral dimension, the sizes of the first and third convolution layers are both 3×1×1, and the step size is 1; the sizes of the second and fourth convolution layers are both 1×3×3, and the step size is 1.

[0035] (2.3)Reference Figure 4 , construct a spatial differential feature extraction layer, whose structure is: the first convolution layer, the second convolution layer, the second-order differential operator layer, the first mathematical operation layer, the first differential layer, the second differential layer, the second mathematical operation layer, the first activation function layer, the third mathematical operation layer, the fourth mathematical operation layer, the fifth mathematical operation layer, the sixth mathematical operation layer, the combination layer, the second activation function layer, and the third activation function layer. Among them, the second-order differential operator layer uses the Laplace operator as the second-order differential feature extraction operator; the first mathematical operation layer is a point-by-point subtraction operation, the second mathematical operation layer is a point-by-point Hessian matrix determinant operation, and the third to sixth mathematical operation layers use point-by-point multiplication operations; the first differential layer is applied in the spatial domain and uses a first-order difference operation; the second differential layer is applied in the spatial domain and uses a second-order difference operation; the combination layer uses a parallel method to combine features; the first activation function layer is implemented using the relu function, and the second and third activation function layers are implemented using the exp(-|x|) exponential function; in this embodiment, the size of the first convolution layer is 3×1×1 and the step size is 1; the size of the second convolution layer is 1×3×1 and the step size is 1.

[0036] (2.4) Using the feature extraction layer constructed in steps (2.1)-(2.3), a feature extraction subnetwork based on differential features and cross-domain features is built, and its structure is: cross-domain feature extraction layer, spectral differential feature extraction layer, spatial differential feature extraction layer, first combination layer, second combination layer, third combination layer, mathematical operation layer; in this embodiment, the first to third combination layers are connected in parallel to perform feature combination, and the mathematical operation layer is a point-by-point addition operation.

[0037] Step 3. Use the shallow feature extraction layer and 5 sub-networks based on differential features and cross-domain features to build a hyperspectral image super-resolution reconstruction network based on differential features and cross-domain features; the shallow feature extraction layer consists of a convolutional layer and an activation layer, where the size of the convolutional layer is 3×3×3 and the step size is 1.

[0038] Step 4. Input the training data set into the hyperspectral image super-resolution reconstruction network built in step 3, and iteratively update the network parameters through back propagation and gradient descent until the loss function converges, thereby obtaining a trained super-resolution network based on differential features and cross-domain features;

[0039] Step 5. Use the trained super-resolution network based on differential features and cross-domain features to perform super-resolution reconstruction of the real hyperspectral image.

[0040] The present invention designs a hyperspectral image super-resolution reconstruction network based on differential features and cross-domain features, and designs a sub-network based on differential features and cross-domain feature extraction. By using the cross-domain features of the spectral domain and the spatial domain, the two features are organically combined to improve the network's learning ability for the spectral domain and spatial domain features. By using the spectral domain differential features, the network's learning of the distribution of spectral domain features is strengthened. By using the Laplace operator and the spatial domain differential features, the generation of noise is suppressed while enhancing the network's learning of texture detail features.

[0041] The effect of the present invention is further described below in conjunction with simulation experiments.

[0042] 1. Simulation conditions:

[0043] The simulation experiment of the present invention is carried out on a multi-card server. The server system is Ubuntu 22.04.4. The specific environment is configured with Python 3.8 and PyTorch 1.10. The software used is MATLAB and PyCharm. The hardware equipment includes four NVIDIA RTX 4090 graphics cards, the graphics card driver version is 535.171.04, and the unified computing device architecture (CUDA) version is 12.2.

[0044] 2. Simulation content:

[0045] Including: 1) the traditional bicubic interpolation method; 2) the existing deep learning hyperspectral image super-resolution method with excellent performance; 3) the method described in the present invention, that is, the hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features. Among them, the traditional bicubic interpolation method and the existing deep learning hyperspectral image super-resolution method with excellent performance are both classic methods that are widely cited in hyperspectral super-resolution reconstruction.

[0046] Reference Figure 5, is the Pavia dataset used in this experiment, which contains 102 spectral bands, and uses the ROSIS-03 sensor to obtain the hyperspectral remote sensing image of Pavia, Italy, with a spectral range of 0.43-0.86 microns and a spatial resolution of 1.3 meters. The spatial size of the Pavia dataset is 1096×1096 pixels, but only 1096×715 valid pixels are selected. In order to train and test more efficiently, we randomly selected a 144×144 pixel area from the Pavia dataset as a test image, and used the remaining area as a training image, as shown in the box in the figure. The results of the simulation experiment are shown in Table 1.

[0047] Table 1 Comparison of the method proposed in this invention with other algorithms

[0048]

[0049] Evaluation indicators: 1) Peak Signal-to-Noise Ratio (PSNR) is a commonly used indicator to measure image quality, especially in the field of image compression and image restoration. PSNR is calculated based on MSE (Mean Squared Error) and is used to compare the similarity between the original image and the distorted image. The higher the PSNR value, the smaller the image distortion and the better the image quality. The lower the PSNR value, the greater the image distortion and the worse the image quality; 2) MPSNR (Mean Peak Signal-to-Noise Ratio), that is, the average peak signal-to-noise ratio, is an indicator used to evaluate the quality of images or videos, especially in hyperspectral image processing. Since hyperspectral images contain multiple bands, MPSNR calculates PSNR for each band separately and then takes the average of these PSNR values ​​to evaluate the quality of the entire hyperspectral image; 3) Structural Similarity Index (SSIM) is an indicator used to measure the similarity between two images, which takes into account the brightness, contrast and structural information of the image. The value range of SSIM is between 0 and 1, and the larger the value, the better the image quality. When two images are exactly the same, the SSIM value is 1, indicating that the image quality is very good without any distortion; 4) Spectral Angle Mapper SAM (Spectral Angle Mapper) is an evaluation indicator for evaluating the similarity of spectral images. It works by calculating the spectral angle of each pixel. The spectral angle is the angle between two spectral vectors, which is usually used to represent the similarity between them. In SAM, the angle between the spectral vector of each pixel and the reference spectral vector (which can be the spectrum of a specific feature or a user-defined spectrum) is calculated. The smaller the angle, the more similar the spectrum of the pixel is to the reference spectrum, which means that the synthesized image is closer to the reference image.

[0050] Simulation 1: Use the traditional bicubic interpolation super-resolution method to reconstruct the test image at 2, 4, and 8 times magnification, and show the 2x magnification result. Figure 6 (a).

[0051] Simulation 2: Use deep learning hyperspectral image super-resolution method to reconstruct the test image at 2, 4 and 8 times magnification, and show the 2 times magnification result. Figure 6 (b).

[0052] Simulation 3: Using the improved spectral image super-resolution method of the present invention to reconstruct the test image at 2, 4 and 8 times magnification, the 2 times magnification result is shown as follows: Figure 6 (c).

[0053] 3. Result analysis:

[0054] a) Index analysis: From the data in Table 1, it can be seen that the peak signal-to-noise ratio (PSNR) and average peak signal-to-noise ratio (MPSNR) of the proposed method are much better than the traditional bicubic interpolation method and the existing excellent deep learning methods under the three method multiples. The structural similarity index (SSIM) and spectral angle mapping (SAM) indicators are also better than the existing excellent deep learning super-resolution reconstruction methods under the three method multiples.

[0055] Under a magnification of 2 times, the PSNR of the method proposed in the present invention is 36.188, which is 0.457 higher than the deep learning method and 3.782 higher than the bicubic interpolation method. The SSIM index is 0.9573, which is better than 0.9036 of the bicubic interpolation method and 0.9519 of the excellent deep learning method. Finally, the SAM index is 3.484, which is 0.111 and 0.886 lower than the deep learning method and the bicubic interpolation method, respectively, indicating that the reconstructed image of the method proposed in the present invention has better quality and less distortion.

[0056] b) Subjective graph analysis: Figure 6 (a), (b), (c), and (d) are the reconstructed images after being magnified 2 times by the traditional bicubic interpolation method, the excellent deep learning method, and the proposed method, as well as the high-resolution reference image. It can be seen that the image reconstructed by the proposed method has less distortion and rich texture details, and the resolution of the ground objects is greatly improved, which is far better than the reconstruction result of the bicubic interpolation method. Compared with the deep learning method, it has better performance at the edges of objects and detailed textures, and is closest to the real reference image.

[0057] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.

[0058] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral image super-resolution reconstruction method based on differential features and cross-domain features, characterized in that: The steps include: (1) Sampling and normalizing the images in the original data set to obtain low-resolution image blocks, and using the low-resolution image blocks to generate a training data set; (2) Build a feature extraction sub-network based on differential features and cross-domain features: (2.1) Construct a cross-domain feature extraction layer, whose structure is: the first convolution layer, the first sampling layer, the second convolution layer, the third convolution layer, the second sampling layer, the first mathematical operation layer, the fourth convolution layer, the second mathematical operation layer, the fifth convolution layer, the third sampling layer, the sixth convolution layer, the seventh convolution layer, the fourth sampling layer, the third mathematical operation layer, the eighth convolution layer, and the ninth convolution layer; wherein the first and second sampling layers are the spectral domain downsampling layer and the upsampling layer, respectively, the third and fourth sampling layers are the spatial domain downsampling layer and the upsampling layer, respectively, and all sampling layers use trilinear interpolation functions as sampling functions; the first to third mathematical operation layers are all point-by-point addition operations; (2.2) constructing a spectral differential feature extraction layer, whose structure is: differential layer, first convolution layer, second convolution layer, first mathematical operation layer, second mathematical operation layer, first integration layer, third convolution layer, fourth convolution layer, first activation function layer, second activation function layer; wherein, the first mathematical operation layer is a point-by-point multiplication operation, the second mathematical operation layer is a point-by-point addition operation, the first activation function layer is implemented by using the exp(-|x|) exponential function; the second activation function layer is implemented by using the softmax function; (2.3) Construct a spatial differential feature extraction layer, whose structure is: the first convolution layer, the second convolution layer, the second-order differential operator layer, the first mathematical operation layer, the first differential layer, the second differential layer, the second mathematical operation layer, the first activation function layer, the third mathematical operation layer, the fourth mathematical operation layer, the fifth mathematical operation layer, the sixth mathematical operation layer, the combination layer, the second activation function layer, and the third activation function layer. Among them, the second-order differential operator layer uses the Laplace operator as the second-order differential feature extraction operator; the first mathematical operation layer is a point-by-point subtraction operation, the second mathematical operation layer is a point-by-point Hessian matrix determinant operation, and the third to sixth mathematical operation layers use point-by-point multiplication operations; the first differential layer is applied in the spatial domain and uses a first-order difference operation; the second differential layer is applied in the spatial domain and uses a second-order difference operation; the combination layer uses a parallel method to combine features; the first activation function layer is implemented using the relu function, and the second and third activation function layers are implemented using the exp(-|x|) exponential function; (2.4) Using the feature extraction layer constructed in steps (2.1)-(2.3), a feature extraction subnetwork based on differential features and cross-domain features is constructed, and its structure is as follows: cross-domain feature extraction layer, spectral differential feature extraction layer, spatial differential feature extraction layer, first combination layer, second combination layer, third combination layer, and mathematical operation layer; (3) Using the shallow feature extraction layer and five feature extraction sub-networks based on differential features and cross-domain features, a hyperspectral image super-resolution reconstruction network based on differential features and cross-domain features is constructed; (4) inputting the training data set into the hyperspectral image super-resolution reconstruction network built in step (3), iteratively updating the network parameters through back propagation and gradient descent until the loss function converges, and obtaining a trained super-resolution network based on differential features and cross-domain features; (5) Use the trained super-resolution network based on differential features and cross-domain features to perform super-resolution reconstruction of real hyperspectral images.

2. The method according to claim 1, characterized in that: The steps for generating a training set in step (1) are as follows: (1.1) Downsampling the original data set by 1, 3 / 4, and 1 / 2, respectively, and using the downsampled images as the initial high-resolution images at 2, 4, and 8 magnification factors, respectively; (1.2) Using the maximum normalization method, normalize the initial high-resolution images at magnification factors of 2, 4, and 8 respectively to obtain high-resolution reference images at different magnification factors; (1.3) The high-resolution reference images of different magnifications are randomly cropped with overlaps, and the image blocks obtained after cropping are randomly rotated by angles of 90°, 180°, or 270°. The rotated image blocks are used as the true reference labels, and then downsampled by 2, 4, and 8 times to obtain low-resolution image blocks of size 32*32. All low-resolution image blocks and their corresponding true reference labels constitute the training dataset.

3. The method according to claim 2, characterized in that: The normalization process described in step (1.2) is to limit the pixel values ​​of the initial high-resolution image to the interval [0, 1] to complete the uniform normalization of the image.

4. The method according to claim 3, characterized in that: In step (1.3), high-resolution reference images with different magnifications are randomly cropped with overlaps. Specifically, the high-resolution reference images with different magnifications are cropped separately, and the high-resolution reference images with magnification factors of 2, 4, and 8 are cropped into image blocks of sizes 64*64, 128*128, and 256*256, respectively.

5. The method according to claim 1, characterized in that: The cross-domain feature extraction layer described in step (2.1) is set to have a size of 3×1×1 and a step size of 1 for the first to fourth convolutional layers, a size of 1×3×3 and a step size of 1 for the fifth to eighth convolutional layers, and a size of 1×1×1 and a step size of 1 for the ninth convolutional layer.

6. The method according to claim 1, characterized in that: The spectral differential feature extraction layer constructed in step (2.2), wherein the differential layer is a differential operation along the spectral dimension, the integral layer is an accumulation operation along the spectral dimension, and the size of the first and third convolutional layers are both 3×1×1 and the step size is 1; The size of the second and fourth convolutional layers is 1×3×3 and the stride is 1.

7. The method according to claim 1, characterized in that: The spatial differential feature extraction layer constructed in step (2.3), wherein the size of the first convolutional layer is 3×1×1 and the step size is 1; the size of the second convolutional layer is 1×3×1 and the step size is 1.

8. The method according to claim 1, characterized in that: In step (2.4), the first to third combination layers are connected in parallel to perform feature combination, and the mathematical operation layer is a point-by-point addition operation.

9. The method according to claim 1, characterized in that: The shallow feature extraction layer in step (3) is composed of a convolutional layer and an activation layer, wherein the size of the convolutional layer is 3×3×3 and the step size is 1.

Images