A hyperspectral image super-resolution method and device based on spatial-spectral joint matching

Abstract

The present invention proposes a hyperspectral image super-resolution method based on space-spectrum joint matching, including: constructing a supervised hyperspectral image and color image paired data set according to a degradation model; constructing a hyperspectral super-resolution network, using the data set to train the hyperspectral super-resolution network; establishing supervision constraints, using a loss function to optimize the parameters of the hyperspectral super-resolution network; obtaining input data, generating a high-resolution hyperspectral image, and completing reasoning and index evaluation. The present invention can obtain high-quality and high-resolution hyperspectral images by aligning and fusing low-resolution hyperspectral images and high-resolution based on a deep learning network.

Description

Technical Field

[0001] The invention belongs to the technical field of computer imaging. Background Art

[0002] Hyperspectral images are images that contain rich spectral dimension information and are widely used in mineral classification, remote sensing detection, target identification and other fields. Hyperspectral images are obtained by combining hyperspectral cameras with imaging technology and spectral detection technology. It can continuously collect spectra of target spatial elements while imaging the target in space, forming dozens or hundreds of narrow bands. However, existing imaging systems need to balance the relationship between high spatial resolution and spectral resolution, so this problem needs to be solved.

[0003] To solve this problem, there are currently two types of methods. One method is to sacrifice spatial resolution during the imaging process and reconstruct a hyperspectral image with high spatial resolution through post-processing single image super-resolution technology. Although these single hyperspectral image super-resolution methods have certain effects, they are limited by the input of low-resolution information, especially for large resolution gaps.

[0004] Another type of method is to perform super-resolution on hyperspectral images based on the provided reference image, where the reference image includes a color image or a multispectral image. Compared with high-resolution hyperspectral images, color images with high spatial resolution are relatively easy to obtain. Therefore, super-resolution technology based on reference images can achieve a trade-off between imaging cost and image quality. However, most methods are based on the assumption of strict alignment. For example, Renwei Dian et al. and Ying Fu et al. respectively proposed a deep convolutional neural network to fuse the features of hyperspectral images and reference images, and both are based on consistent spatial positions. This limits the practical use of these reference-based methods.

[0005] In recent years, some methods have tried to relax the assumption of exact alignment and collect misaligned simulated data through geometrically rigid or non-rigid transformations. Although these methods have improved the quality of reconstructed hyperspectral images, they still face two major challenges. First, spatial matching is a challenge due to the resolution gap between the input images, i.e., it is difficult to produce an exact spatial match between a low-resolution hyperspectral image and a high-resolution reference image. Second, spectral matching is also a challenging problem, i.e., exploring the spectral correlation between hyperspectral images and color images, which has been largely ignored by previous methods. Summary of the invention

[0006] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0007] To this end, the purpose of the present invention is to propose a hyperspectral image super-resolution method based on spatial-spectral joint matching to obtain high-quality and high-resolution hyperspectral images.

[0008] To achieve the above object, the first embodiment of the present invention proposes a hyperspectral image super-resolution method based on spatial-spectral joint matching, comprising:

[0009] According to the degradation model, a supervised dataset of hyperspectral images and color images is constructed;

[0010] Constructing a hyperspectral super-resolution network, and using the data set to train the hyperspectral super-resolution network;

[0011] Establishing supervision constraints and optimizing parameters of the hyperspectral super-resolution network using a loss function;

[0012] Obtain input data, generate high-resolution hyperspectral images, and complete reasoning and indicator evaluation.

[0013] In addition, the hyperspectral image super-resolution method based on spatial-spectral joint matching according to the above embodiment of the present invention may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present invention, the method of constructing a supervised data set of hyperspectral images and color images in pairs further includes:

[0015] The dataset is preprocessed, including upsampling the low-resolution hyperspectral image using Bicubic to obtain a resolution consistent with the reference color image.

[0016] Furthermore, in one embodiment of the present invention, constructing a hyperspectral super-resolution network comprises:

[0017] Construct an encoder to extract multi-scale features of hyperspectral images and color images respectively;

[0018] Construct a spatial matching network to perform spatial feature matching and feature aggregation on the feature scale;

[0019] A spectral perception network is constructed to model the correlation between the features of hyperspectral images and color images to complete spectral matching.

[0020] Furthermore, in one embodiment of the present invention, the constructing the encoder comprises:

[0021] The multi-scale features of the input image are extracted by the constructed encoder, which is expressed as:

[0022] F l =E l (X),

[0023] Among them, X represents the input hyperspectral image or color image, F represents the extracted feature map, E represents the encoder, and l represents the scale level.

[0024] Further, in one embodiment of the present invention, the spatial matching network obtains a global similarity and a related position map based on global patch matching, and performs feature aggregation by guiding the global similarity and the related position map, wherein the similarity and the related position map calculation is expressed as:

[0025] M i，j =norm(H T I),

[0026]

[0027] Where H represents the fixed-size patches divided from the features extracted from the hyperspectral image, I represents the fixed-size patches divided from the features extracted from the color image, T represents the transposition operation, norm(·) represents normalization, and M represents the fixed-size patches divided from the features extracted from the color image. i,j represents the correlation matrix, S i represents the similarity matrix, C i represents the correlation position matrix, i and j represent the positions of low-scoring feature patches and high-scoring feature patches respectively;

[0028] According to the obtained spatial correlation, similar features are aggregated and expressed as follows:

[0029]

[0030] in, represents the features extracted from the reference color image, represents the feature aggregation network, A l represents the aggregated reference image feature, p represents the current position, and p ′ Indicates finding the position with the highest correlation from the correlation position matrix, p k ∈{(-1,-1),(-1,0),…,(1,1)},ω k represents the convolution parameter of the network, Δp k represents the learnable offset, m k represents a learnable mask.

[0031] Furthermore, in one embodiment of the present invention, the spectral perception network uses the attention mechanism to guide the fusion of features and reconstruct a high-resolution hyperspectral image based on the long-distance similarity of the spectral dimension, which is expressed as follows:

[0032]

[0033] Among them, F lrepresents the features extracted from the hyperspectral image, A l represents the features after aggregation, represents the fused l-th layer features, represents the l-1th layer feature of the previous stage, and Attention(·) represents the attention network module.

[0034] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a hyperspectral image super-resolution device based on spatial-spectral joint matching, comprising the following modules:

[0035] An acquisition module is used to construct a supervised dataset of hyperspectral images and color images according to a degradation model;

[0036] A construction module, used for constructing a hyperspectral super-resolution network, and using the data set to train the hyperspectral super-resolution network;

[0037] An optimization module, for establishing supervision constraints and optimizing parameters of the hyperspectral super-resolution network using a loss function;

[0038] The output module is used to obtain input data, generate high-resolution hyperspectral images, and complete reasoning and indicator evaluation.

[0039] Furthermore, in one embodiment of the present invention, the construction module is also used to:

[0040] Construct an encoder to extract multi-scale features of hyperspectral images and color images respectively;

[0041] Construct a spatial matching network to perform spatial feature matching and feature aggregation on the feature scale;

[0042] A spectral perception network is constructed to model the correlation between the features of hyperspectral images and color images to complete spectral matching.

[0043] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the above-mentioned high-spectral image super-resolution method based on spatial-spectral joint matching.

[0044] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, a hyperspectral image super-resolution method based on spatial-spectral joint matching as described above is implemented.

[0045] Compared with the prior art, the hyperspectral image super-resolution method based on spatial-spectral joint matching proposed in the present invention has the following advantages:

[0046] 1) The present invention can effectively perform super-resolution on low-resolution hyperspectral images based on reference high-resolution color images, and is applicable to different degradation problems, effectively solving the problems of spatial misalignment and spectral mismatch between input images;

[0047] 2) The present invention learns the spatial matching relationship between input images through a spatial matching network, and completes the alignment of reference features by aggregating features at multiple similar positions, thereby solving the problem of spatial misalignment;

[0048] 3) The present invention learns the correlation between hyperspectral images and color images through a spectral perception network, captures long-range spectral dependence, maintains the spectral consistency of the image after super-resolution, and solves the problem of spectral mismatch. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0050] Figure 1 A schematic flow chart of a hyperspectral image super-resolution method based on spatial-spectral joint matching provided in an embodiment of the present invention.

[0051] Figure 2 A schematic diagram of a neural network structure provided by an embodiment of the present invention.

[0052] Figure 3 A schematic diagram of the composition of a hyperspectral image super-resolution system based on spatial-spectral joint matching provided in an embodiment of the present invention.

[0053] Figure 4 A schematic diagram of a hyperspectral image super-resolution device based on spatial-spectral joint matching provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0055] The hyperspectral image super-resolution method based on spatial-spectral joint matching according to an embodiment of the present invention is described below with reference to the accompanying drawings.

[0056] Figure 1A schematic flow chart of a hyperspectral image super-resolution method based on spatial-spectral joint matching provided in an embodiment of the present invention.

[0057] like Figure 1 As shown, the hyperspectral image super-resolution method based on spatial-spectral joint matching includes the following steps:

[0058] S101: Based on the degradation model, construct a supervised dataset of hyperspectral images and color images.

[0059] Specifically, the present invention can construct a training data set according to the degradation model under different specific parameters, including Gaussian blur and downsampling. The training data set includes: paired low-resolution hyperspectral images (obtained based on the collected high-resolution images and the degradation model) and high-resolution color images. The degradation model is uniformly expressed as:

[0060] y=Φx,

[0061] Where x is the original hyperspectral image, y is the low-resolution hyperspectral image, and Φ is the degradation matrix.

[0062] Furthermore, in one embodiment of the present invention, constructing a supervised data set of hyperspectral images and color images in pairs also includes:

[0063] The dataset is preprocessed, including upsampling the low-resolution hyperspectral image using Bicubic to obtain a resolution consistent with the reference color image.

[0064] S102: construct a hyperspectral super-resolution network and train the hyperspectral super-resolution network using the dataset;

[0065] Furthermore, in one embodiment of the present invention, a hyperspectral super-resolution network is constructed, comprising:

[0066] Construct an encoder to extract multi-scale features of hyperspectral images and color images respectively;

[0067] Construct a spatial matching network to perform spatial feature matching and feature aggregation on the feature scale;

[0068] A spectral perception network is constructed to model the correlation between the features of hyperspectral images and color images to complete spectral matching.

[0069] Specifically, considering the resolution gap between low-resolution hyperspectral images and high-resolution color images, it greatly affects the corresponding spatial matching. Therefore, in order to eliminate its influence, the multi-scale features of the input image are first extracted through the constructed encoder.

[0070] Furthermore, in one embodiment of the present invention, an encoder is constructed, comprising:

[0071] The multi-scale features of the input image are extracted by the constructed encoder, which is expressed as:

[0072] F l =E l (X),

[0073] Among them, X represents the input hyperspectral image or color image, F represents the extracted feature map, E represents the encoder, and l represents the scale level.

[0074] Different encoders are used for inputting hyperspectral images and color images to extract features according to the characteristics of the images.

[0075] Specifically, a spatial matching network is used to obtain global similarity and related location maps based on global patch matching, which guides the dynamic aggregation of related features.

[0076] Further, in one embodiment of the present invention, the spatial matching network obtains the global similarity and the related position map based on the global patch matching, and performs feature aggregation by guiding the global similarity and the related position map, wherein the similarity and the related position map calculation is expressed as:

[0077] M i，j =norm(H T I),

[0078]

[0079] Where H represents the fixed-size patches divided from the features extracted from the hyperspectral image, I represents the fixed-size patches divided from the features extracted from the color image, T represents the transposition operation, norm(·) represents normalization, and M represents the fixed-size patches divided from the features extracted from the color image. i,j represents the correlation matrix, S i represents the similarity matrix, C i represents the correlation position matrix, i and j represent the positions of low-scoring feature patches and high-scoring feature patches respectively;

[0080] According to the obtained spatial correlation, similar features are aggregated and expressed as follows:

[0081]

[0082] in, represents the features extracted from the reference color image, represents the feature aggregation network, A l represents the aggregated reference image feature, p represents the current position, p′ represents the position with the highest correlation from the correlation position matrix, and p k ∈{(-1,-1),(-1,0),…,(1,1)},ωk represents the convolution parameter of the network, Δp k represents the learnable offset, m k represents a learnable mask.

[0083] Specifically, the spectral perception network uses the attention mechanism to guide the fusion of features and reconstruct high-resolution hyperspectral images based on the long-distance similarity of the spectral dimension.

[0084] Furthermore, in one embodiment of the present invention, the spectral perception network uses the attention mechanism to guide the fusion of features and reconstruct a high-resolution hyperspectral image based on the long-distance similarity of the spectral dimension, as shown below:

[0085]

[0086] Among them, F l represents the features extracted from the hyperspectral image, A l represents the features after aggregation, represents the fused l-th layer features, represents the l-1th layer feature of the previous stage, and Attention(·) represents the attention network module.

[0087] The fused features at the last scale are added pixel by pixel to the input image through residual connection to obtain the final hyperspectral image.

[0088] S103: Establish supervision constraints and use loss functions to optimize the parameters of the hyperspectral super-resolution network;

[0089] Specifically, the loss function of the supervised constraint is based on the input and output hyperspectral images, using L1 loss. It is expressed as follows:

[0090]

[0091] in, i represents the true value of the hyperspectral image, Y i Represents the output hyperspectral image, and N represents the number of data in the dataset.

[0092] S104: Obtain input data, generate high-resolution hyperspectral images, and complete reasoning and indicator evaluation.

[0093] Specifically, in order to objectively evaluate the effect of the generated hyperspectral image, objective evaluation indicators can be generated based on peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and spectral angle mapping (SAM).

[0094] Figure 2 This is a schematic diagram of the neural network structure used in the embodiment of the present application. The low-resolution hyperspectral image and high-resolution color image datasets are constructed through the set degradation model; the multi-scale features of the hyperspectral image and color image are extracted through the encoder; feature matching and feature aggregation are performed through the spatial matching network to solve the spatial misalignment problem of the data; the characteristic spectral correlation modeling of the hyperspectral image and color image is performed through the spectral perception network, and the long-distance spectral dependence is learned and captured to solve the spectral mismatch problem. Compared with other super-resolution methods, it can solve more complex data misalignment situations and obtain high-resolution hyperspectral image results.

[0095] Figure 3 A schematic diagram of the composition of a hyperspectral image super-resolution device based on space-spectrum joint matching provided in an embodiment of the present application includes a data preprocessing subsystem 10, an encoding subsystem 20, a spatial matching subsystem 30, a spectral sensing subsystem 40, and a supervised optimization and result evaluation subsystem 50:

[0096] The data preprocessing subsystem 10 is used to process the collected hyperspectral images and color images, generate low-resolution hyperspectral images according to a specific degradation model, and retain high-resolution color images to form paired data pairs to constitute a training data set.

[0097] Furthermore, the encoding subsystem 20 includes a hyperspectral image encoder and a color image encoder, which respectively extract multi-scale features for the image, which are based on basic convolutional layers and activation layers, etc.

[0098] Furthermore, the spatial matching subsystem 30 includes two modules, which respectively calculate the similarity and the related position map of the image patches, and aggregate the features of similar positions on the reference image features using the priors obtained by the previous calculation.

[0099] Furthermore, the spectral perception subsystem 40 includes an attention network, which integrates the hyperspectral features obtained from the encoding subsystem 20 and the aligned aggregated features of the reference color image obtained from the spatial matching subsystem 30 based on the spectral correlation between the input features and its own long-range spectral dependence, to complete the reconstruction of the high-resolution hyperspectral image.

[0100] Furthermore, the supervised optimization and result evaluation subsystem 50 is used to establish a loss function to optimize the network of the aforementioned system, further retain the trained network parameters and generate results, and use built-in evaluation indicators to evaluate the results.

[0101] The connection relationship between the above-mentioned component systems is:

[0102] The output of the data preprocessing subsystem is connected to the input of the encoding subsystem, the output of the encoding subsystem is connected to the input of the spatial matching subsystem and the spectral sensing subsystem, and the output of the spatial matching subsystem is connected to the input of the spectral sensing subsystem. Finally, the output of the spectral sensing subsystem is connected to the supervised optimization and result evaluation subsystem.

[0103] Compared with the prior art, the hyperspectral image super-resolution method based on spatial-spectral joint matching proposed in the present invention has the following advantages:

[0104] 1) The present invention can effectively perform super-resolution on low-resolution hyperspectral images based on reference high-resolution color images, and is applicable to different degradation problems, effectively solving the problems of spatial misalignment and spectral mismatch between input images;

[0105] 2) The present invention learns the spatial matching relationship between input images through a spatial matching network, and completes the alignment of reference features by aggregating features at multiple similar positions, thereby solving the problem of spatial misalignment;

[0106] 3) The present invention learns the correlation between hyperspectral images and color images through a spectral perception network, captures long-range spectral dependence, maintains the spectral consistency of the image after super-resolution, and solves the problem of spectral mismatch.

[0107] In order to implement the above embodiment, the present invention also proposes a hyperspectral image super-resolution device based on spatial-spectral joint matching.

[0108] Figure 4 A schematic structural diagram of a hyperspectral image super-resolution device based on spatial-spectral joint matching provided in an embodiment of the present invention.

[0109] like Figure 4 As shown, the hyperspectral image super-resolution device based on space-spectrum joint matching includes: an acquisition module 100, a construction module 200, an optimization module 300, and an output module 400, wherein:

[0110] An acquisition module is used to construct a supervised dataset of hyperspectral images and color images according to a degradation model;

[0111] A construction module is used to construct a hyperspectral super-resolution network and train the hyperspectral super-resolution network using a data set;

[0112] The optimization module is used to establish supervision constraints and optimize the parameters of the hyperspectral super-resolution network using a loss function;

[0113] The output module is used to obtain input data, generate high-resolution hyperspectral images, and complete reasoning and indicator evaluation.

[0114] Furthermore, in one embodiment of the present invention, the construction module is also used to:

[0115] Construct an encoder to extract multi-scale features of hyperspectral images and color images respectively;

[0116] Construct a spatial matching network to perform spatial feature matching and feature aggregation on the feature scale;

[0117] A spectral perception network is constructed to model the correlation between the features of hyperspectral images and color images to complete spectral matching.

[0118] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the hyperspectral image super-resolution method based on spatial-spectral joint matching as described above.

[0119] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the hyperspectral image super-resolution method based on spatial-spectral joint matching as described above is implemented.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0122] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A hyperspectral image super-resolution method based on spatial-spectral joint matching, characterized in that: The following steps are involved: According to the degradation model, a supervised dataset of hyperspectral images and color images is constructed; Constructing a hyperspectral super-resolution network, and using the data set to train the hyperspectral super-resolution network; Establishing supervision constraints and optimizing parameters of the hyperspectral super-resolution network using a loss function; Obtain input data, generate high-resolution hyperspectral images, and complete reasoning and indicator evaluation; Wherein, constructing a hyperspectral super-resolution network comprises: Construct an encoder to extract multi-scale features of hyperspectral images and color images respectively; Construct a spatial matching network to perform spatial feature matching and feature aggregation on the feature scale; Construct a spectral perception network to model the correlation between hyperspectral image and color image features and complete spectral matching; Wherein, the encoder is constructed, comprising: The multi-scale features of the input image are extracted by the constructed encoder, which is expressed as: F l =E l (X), Where X represents the input hyperspectral image or color image, F represents the extracted feature map, E represents the encoder, and l represents the scale level; The spatial matching network obtains a global similarity and a related position map based on global patch matching, and performs feature aggregation by guiding the global similarity and the related position map, wherein the similarity and the related position map calculation is expressed as: M i，j =norm(H T I), Where H represents the fixed-size patches divided from the features extracted from the hyperspectral image, I represents the fixed-size patches divided from the features extracted from the color image, T represents the transposition operation, norm(·) represents normalization, and M represents the fixed-size patches divided from the features extracted from the color image. i，j Represents the correlation moment Array, S i represents the similarity matrix, C i represents the correlation position matrix, i and j represent the positions of low-scoring feature patches and high-scoring feature patches respectively; According to the obtained spatial correlation, similar features are aggregated and expressed as follows: in, represents the features extracted from the reference color image, represents the feature aggregation network, A l represents the aggregated reference image feature, p represents the current position, p′ represents the position with the highest correlation from the correlation position matrix, and p k ∈{(-1,-1),(-1,0),…,(1,1)},ω k represents the convolution parameter of the network, Δp k represents the learnable offset, m k represents a learnable mask; the spectral perception network is based on the attention mechanism to guide the fusion of features and reconstruct a high-resolution hyperspectral image based on the long-distance similarity of the spectral dimension, which is expressed as follows: Among them, F l represents the features extracted from the hyperspectral image, A l represents the features after aggregation, represents the fused l-th layer features, represents the l-1th layer feature of the previous stage, and Attention(·) represents the attention network module.

2. The method according to claim 1, characterized in that: The method of constructing a supervised data set of hyperspectral images and color images as pairs also includes: The dataset is preprocessed, including upsampling the low-resolution hyperspectral image using Bicubic to obtain a resolution consistent with the reference color image.

3. A hyperspectral image super-resolution device based on spatial-spectral joint matching, characterized in that: Includes the following modules: An acquisition module is used to construct a supervised dataset of hyperspectral images and color images according to a degradation model; A construction module, used for constructing a hyperspectral super-resolution network, and using the data set to train the hyperspectral super-resolution network; An optimization module, for establishing supervision constraints and optimizing parameters of the hyperspectral super-resolution network using a loss function; Output module, used to obtain input data, generate high-resolution hyperspectral images, and complete reasoning and indicator evaluation; Wherein, the construction module is also used for: Construct an encoder to extract multi-scale features of hyperspectral images and color images respectively; Construct a spatial matching network to perform spatial feature matching and feature aggregation on the feature scale; Construct a spectral perception network to model the correlation between hyperspectral image and color image features and complete spectral matching; Wherein, the encoder is constructed, comprising: The multi-scale features of the input image are extracted by the constructed encoder, which is expressed as: F l =E l (X), Where X represents the input hyperspectral image or color image, F represents the extracted feature map, E represents the encoder, and l represents the scale level; The spatial matching network obtains a global similarity and a related position map based on global patch matching, and performs feature aggregation by guiding the global similarity and the related position map, wherein the similarity and the related position map calculation is expressed as: M i,j =norm(H T I), Where H represents the fixed-size patches divided from the features extracted from the hyperspectral image, I represents the fixed-size patches divided from the features extracted from the color image, T represents the transposition operation, norm(·) represents normalization, and M represents the fixed-size patches divided from the features extracted from the color image. i，j Represents the correlation moment Array, S i represents the similarity matrix, C i represents the correlation position matrix, i and j represent the positions of low-scoring feature patches and high-scoring feature patches respectively; According to the obtained spatial correlation, similar features are aggregated and expressed as follows: in, represents the features extracted from the reference color image, represents the feature aggregation network, A l represents the aggregated reference image feature, p represents the current position, p′ represents the position with the highest correlation from the correlation position matrix, and p k ∈{(-1,-1),(-1,0),…,(1,1)),ω k represents the convolution parameter of the network, Δp k represents the learnable offset, m k represents a learnable mask; the spectral perception network is based on the attention mechanism to guide the fusion of features and reconstruct a high-resolution hyperspectral image based on the long-distance similarity of the spectral dimension, which is expressed as follows: Among them, F l represents the features extracted from the hyperspectral image, A l represents the features after aggregation, represents the fused l-th layer features, represents the l-1th layer feature of the previous stage, and Attention(·) represents the attention network module.

4. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for super-resolution of hyperspectral images based on spatial-spectral joint matching as claimed in any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the hyperspectral image super-resolution method based on spatial-spectral joint matching as described in any one of claims 1 to 2 is implemented.

Images