A hyperspectral remote sensing image super-resolution reconstruction method based on self-attention full probability neural network

Abstract

The application discloses a hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full-probability neural network, acquires a hyperspectral image under a same sensor, carries out a low-resolution preprocessing on the hyperspectral image, and acquires a low-resolution hyperspectral image; the low-resolution hyperspectral image is input into a super-resolution network reconstruction network which is constructed and trained in advance, a reconstructed high-resolution image is acquired, and the training steps of the network comprise the following steps: coarse extraction is carried out on the low-resolution hyperspectral image, a coarse spectral feature map is acquired, a feature enhancement is carried out by using a global self-attention mechanism, a fine spectral feature map is acquired, a discrete intensity interval is set, a discrete intensity probability of the fine spectral feature map is calculated, the discrete intensity probability is combined with a low-resolution position priori, a full-probability feature under a high resolution is acquired, and a high-resolution image is reconstructed according to the full-probability feature. The application realizes efficient super-resolution reconstruction of the hyperspectral image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing image processing, and particularly relates to a hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full-probability neural network. BACKGROUND

[0002] Unlike traditional images (such as grayscale images, RGB images, etc.), which mainly save the spatial information of the observed scene, hyperspectral remote sensing images contain rich spatial-spectral information, which not only saves the spatial information of the scene, but also saves the highly distinguishable spectral information. Therefore, hyperspectral remote sensing images are widely used in remote sensing information acquisition and processing tasks, such as target detection, scene classification, and target tracking. However, due to the limitations of imaging mechanisms, the spatial resolution of the hyperspectral remote sensing images obtained by traditional hyperspectral imaging instruments is often limited, which greatly affects the actual application effect of the hyperspectral remote sensing images. The hyperspectral image super-resolution reconstruction technology can generate high-resolution hyperspectral images with high spatial resolution and complete spectral information by improving the spatial resolution of the hyperspectral images, thereby providing high-quality data support for the accurate implementation of subsequent tasks.

[0003] In recent years, various algorithms have appeared for the problem of hyperspectral remote sensing image super-resolution reconstruction, which can be roughly divided into model-driven hyperspectral image super-resolution technology and data-driven hyperspectral image super-resolution technology. The model-driven hyperspectral image super-resolution technology performs super-resolution reconstruction by simulating the imaging process of the sensor or by statistical modeling of the image. Typical methods include interpolation reconstruction methods based on physical models (bilinear interpolation, etc.), but the recovered high-resolution images often lack details and introduce serious artifacts; and model methods based on sparse representation, which represent the pixels of the hyperspectral image as a linear combination of a set of sparse bases to recover the details of the image, but are usually sensitive to noise and image scale changes, and have high computational complexity. With the rise of deep learning, data-driven hyperspectral image super-resolution methods have begun to be widely used, especially convolutional neural network architectures, which can automatically learn the spatial and spectral features of hyperspectral images from a large amount of training data through an end-to-end training process, thereby effectively improving the resolution of the image. The self-attention mechanism has a significant advantage in capturing global dependencies, so it has received widespread attention in the field of image processing. Methods based on the self-attention mechanism, such as VisionTransformer, can process spatial-spectral information in images through the self-attention mechanism, which is expected to further improve the effect of hyperspectral image super-resolution. However, these methods still have problems such as insufficient use of global spatial-spectral features and poor adaptability to different resolution images, which limit their performance in actual remote sensing applications.

[0004] Therefore, overcoming the aforementioned pain points is an urgent issue that needs to be addressed. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a method for super-resolution reconstruction of hyperspectral remote sensing images based on a self-attention full probability neural network. This method combines a global self-attention mechanism with a global probabilistic feature modeling method to achieve efficient super-resolution reconstruction of hyperspectral images.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for super-resolution reconstruction of hyperspectral remote sensing images based on a self-attention full probability neural network, comprising the following steps:

[0008] Acquire hyperspectral images from the same sensor, perform resolution down-processing on the hyperspectral images to obtain low-resolution hyperspectral images;

[0009] A low-resolution hyperspectral image is input into a pre-constructed and trained super-resolution network for reconstruction to obtain a reconstructed high-resolution image. The training steps of the super-resolution network for reconstruction include:

[0010] Coarse extraction is performed on low-resolution hyperspectral images to obtain coarse spatial-spectral feature maps. A global self-attention mechanism is used for feature enhancement to obtain fine spatial-spectral feature maps. Discrete feature intensity intervals are set, and the discrete probability distribution of low-resolution pixel location features is calculated based on the fine spatial-spectral feature maps. The discrete probability distribution is combined with the normalized weighted probabilities of high- and low-resolution location features to obtain the full probability features under high resolution. The high-resolution image is reconstructed based on the full probability features.

[0011] As a preferred technical solution, the spatial size of the hyperspectral image and the corresponding low-resolution hyperspectral image satisfy a direct proportional relationship.

[0012] As a preferred technical solution, the step of performing resolution reduction preprocessing on the hyperspectral image includes: first smoothing the hyperspectral image, and then downsampling it to obtain a low-resolution hyperspectral image.

[0013] As a preferred technical solution, the super-resolution network reconstruction network includes a feature coarse extraction module, a global self-attention enhancement module, a probability parameter prediction module, a full probability feature reconstruction module, and a spectral dimension restoration module.

[0014] The feature coarse extraction module is used to coarsely extract features from low-resolution training images to obtain coarse spatial spectral feature maps.

[0015] A global self-attention enhancement module is configured to utilize a global self-attention mechanism to perform feature enhancement and obtain a fine spectral feature map;

[0016] A probability parameter prediction module is configured to set a discrete feature intensity interval and calculate a discrete probability distribution of a low-resolution pixel position feature according to the fine spectral feature map;

[0017] A global probability feature reconstruction module is configured to combine the discrete probability distribution and a normalized weight probability of a high-low resolution position feature to obtain a global probability feature at a high resolution.

[0018] A spectral dimension restoration module is configured to reconstruct a high-resolution image according to the global probability feature.

[0019] As a preferred technical solution, the feature coarse extraction module comprises a feature extraction convolutional layer Conv1, a spectral attention module, and a feature extraction convolutional layer Conv2.

[0020] The feature extraction convolutional layer Conv1 comprises 64 groups of 3x3 size convolutional kernels, each group of convolutional kernels having c convolutional kernels, and outputs a 64-channel feature F1.

[0021] The spectral attention module utilizes a global average pooling operation to compress the spatial information of the feature F1, adjusts the spectral information through 64 groups of 1x1x64 size convolutional kernels, obtains a spectral attention S, multiplies the spectral attention S with the feature F1, and outputs a 64-channel spectral feature F2.

[0022] The feature extraction convolutional layer Conv2 comprises 64 groups of 3x3 size convolutional kernels, takes the sum of the features F1 and F2 as input, and outputs a 64-channel coarse spectral feature map F3.

[0023] As a preferred technical solution, the global self-attention enhancement module comprises a feature extraction convolutional layer Conv3 and a self-attention module.

[0024] The feature extraction convolutional layer Conv3 comprises 64 groups of 3x3 size convolutional kernels, takes the coarse spectral feature map F3 as input, and outputs a 64-channel feature map F4.

[0025] The self-attention module comprises three convolutional blocks, each convolutional block having 64 groups of 3x3 size convolutional kernels, the convolutional blocks respectively obtaining a query, a key, and a value, utilizing a Softmax function to multiply and normalize the product of the query and the key to obtain a self-attention A, multiplying the self-attention A with the value to obtain a fine spectral feature map F5.

[0026] As a preferred technical solution, the discrete feature intensity interval is set, and the discrete probability distribution of the low-resolution pixel position feature is calculated according to the fine spectral feature map, specifically as follows:

[0027] discretize the range of the imageable pixel value into a plurality of feature intensity intervals, and obtain corresponding interval centers and interval boundaries;

[0028] map the fine spectral feature map using a plurality of convolution kernels to obtain multi-dimensional features on low-resolution pixel points , the multi-dimensional features include features of the first low-resolution position coordinates , and the probability distribution of each dimension of the multi-dimensional features is calculated by Softmax normalization ;

[0029] with the high-resolution query position coordinates and the set of low-resolution position coordinates as inputs, the normalized weight probability of the features of the high-resolution position coordinates relative to the features of the low-resolution position coordinates is calculated , as follows:

[0030] ,

[0031] wherein the function is used to measure the direct similarity between the features and the features , is a constant to prevent numerical overflow at zero distance, N represents the total number of low-resolution pixels, represents the coordinates of the tth low-resolution pixel, t is an accumulated intermediate variable, and represents the tth low-resolution pixel point.

[0032] As a preferred technical solution, the normalized weight probability of the combination of the discrete probability distribution and the high and low resolution position features is used to obtain the full probability feature at high resolution, specifically:

[0033] with the probability distribution and the normalized weight probability as inputs, the probability distribution of the K-dimensional intermediate feature of the high-resolution position is predicted using the discrete full probability formula ;

[0034] construct the full probability feature on the high-resolution query position , as follows:

[0035] ,

[0036] wherein ​​​​For the first Each feature intensity range The central value of .

[0037] As a preferred technical solution, the spectral dimension restoration module, through... Group The convolution kernel, with full probability features As input, output the final predicted high-resolution image.

[0038] As a preferred technical solution, the training of the super-resolution network reconstruction network adopts the mean absolute error loss function to calculate the difference between the reconstructed high-resolution image and the original hyperspectral image, and feeds the difference back to the super-resolution network reconstruction network to optimize the parameters.

[0039] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0040] (1) This invention uses coarse extraction, that is, using convolution operation to capture the spatial information of the input image, and at the same time uses channel attention mechanism to extract spectral information, thereby effectively refining the spatial-spectral information of the image and providing high-quality preliminary features for the subsequent self-attention enhancement module.

[0041] (2) The present invention can fully extract global context features through the global self-attention enhancement module, thereby ensuring the stability of subsequent probability parameter prediction.

[0042] (3) The present invention uses the enhanced spatial spectral feature map as input, so that the module can accurately predict the probability distribution parameters of each position in the low-resolution image, thereby enhancing the robustness of subsequent full probability feature reconstruction.

[0043] (4) This invention utilizes the discrete total probability formula, combined with prior information of low-resolution location features and discrete intensity probability, to generate high-resolution features, thereby achieving fine reconstruction of high-resolution images. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network, as described in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the feature coarse extraction module according to an embodiment of the present invention;

[0047] Figure 3 A structural schematic diagram of a global self-attention enhancement module of an embodiment of the present application;

[0048] Figure 4 A structural diagram of a probability parameter prediction module of an embodiment of the present application;

[0049] Figure 5 A global probability feature reconstruction module and a spectral dimension restoration module of an embodiment of the present application;

[0050] Figure 6 A hyperspectral image is processed in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to enable persons skilled in the art to better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor fall within the scope of protection of the present application.

[0052] In the present application, the phrase "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments.

[0053] Please refer to Figure 1 The present embodiment provides a hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention global probability neural network, which comprises:

[0054] S1, acquiring a hyperspectral image under the same sensor, and performing a low-resolution preprocessing on the hyperspectral image to obtain a low-resolution hyperspectral image.

[0055] When the hyperspectral image is preprocessed, the hyperspectral image is reduced in resolution by downsampling, thereby obtaining a low-resolution hyperspectral image. The low-resolution hyperspectral image and the corresponding original training hyperspectral image constitute a training set. The low-resolution preprocessing comprises:

[0056] S11, preprocessing the original hyperspectral image data A low-pass filter based on a sensor modulation transfer function and a frequency response is constructed to perform a smoothing operation on the hyperspectral image;

[0057] S12, further reduce the image resolution by using a downsampling method to obtain a low-resolution hyperspectral image with a spatial resolution reduced by r times .

[0058] wherein the hyperspectral image and the low-resolution hyperspectral image registered therewith satisfy the following quantity relationship:

[0059] ,

[0060] ,

[0061] wherein r represents the ratio of the spatial resolution of the original hyperspectral image to the low-resolution hyperspectral image after downsampling, and are the height and width of the original hyperspectral image, respectively, and and represent the height and width of the low-resolution hyperspectral image, and c represents the number of spectral channels contained in the hyperspectral image.

[0062] S2, input the low-resolution hyperspectral image into the pre-constructed and trained super-resolution network reconstruction network to obtain a reconstructed high-resolution image, and the training steps of the super-resolution network reconstruction network include:

[0063] coarse extraction is performed on the low-resolution hyperspectral image to obtain a coarse spectral feature map, feature enhancement is performed by using a global self-attention mechanism to obtain a fine spectral feature map, a discrete feature intensity interval is set, a discrete probability distribution of the low-resolution pixel position feature is calculated according to the fine spectral feature map, a global probability feature under high resolution is obtained by combining the discrete probability distribution and the normalized weight probability of the high and low resolution position features, and a high-resolution image is reconstructed according to the global probability feature.

[0064] The embodiment constructs a super-resolution network reconstruction network based on self-attention global probability, which includes a feature coarse extraction module, a global self-attention enhancement module, a probability parameter prediction module, a global probability feature reconstruction module and a spectral dimension restoration module. After setting the hyperparameters of the network and initializing the weights and biases of the network, the low-resolution hyperspectral image processed above is input into the network.

[0065] wherein, as shown in Figure 2 , the structure of the feature coarse extraction module is:

[0066] Feature extraction convolutional layer Conv1: This layer uses 64 groups of 3x3 size convolution kernels, and the low-resolution hyperspectral image after downsampling is used as the input, each group of convolution kernels contains convolution kernels, and 64-channel features are output Spectral Attention Module (SPA): This layer uses global average pooling to compress spatial information while preserving spectral information, and then utilizes 64 groups of... The size of the convolutional kernel is further adjusted to refine the spectral information, thereby generating spectral attention. Its characteristics As input, output spectral attention. Then spectral attention With features Multiply to output 64-channel spectral features Feature extraction convolutional layer Conv2: This layer uses 64 sets of 3×3 convolutional kernels, which... and The summed feature map is used as input, and the output is a 64-channel low-resolution spatial-spectral fusion coarse feature map. .

[0067] like Figure 3 As shown, the structure of the global self-attention enhancement module is as follows:

[0068] Feature extraction convolutional layer Conv3: This layer uses 64 sets of 3×3 convolutional kernels, which fuse coarse features with low-resolution spatial-spectral fusion. As input, output a 64-channel feature map. Self-Attention Module (SAT): This layer uses three sets of 64 3×3 convolutional kernels to generate Query tensors, Key tensors, and Value tensors, respectively. The Softmax function is then used to normalize the product of the Query and Key tensors to obtain the self-attention layer. Finally, the self-attention is multiplied by the Value tensor. This is done using the feature map. As input, the output is the low-resolution spatial-spectral self-attention feature after feature enhancement. .

[0069] like Figure 4 As shown, the structure of the probability parameter prediction module is as follows:

[0070] Discrete Intensity Range Setting Module: Discretizes the range of possible pixel values ​​(normalized to the [0,1] interval) into... Given a feature intensity interval, the center of the interval is obtained. and interval boundary ; It is a positive integer, for example, take =10, the center of the interval can be set as... ;

[0071] Discrete Intensity Probability Prediction Module: This layer uses indivual Convolution kernel for low-resolution spatial spectrum self-attention features Map the K-dimensional feature of the low-resolution pixel point } where is the t-th low-resolution pixel point. Then, normalize the feature of the low-resolution position coordinate to obtain the probability distribution of each dimension;

[0072] ,

[0073] where .

[0074] Low-resolution position feature prior probability prediction module: This layer is based on the inverse of the spatial distance between the high-resolution query position and each low-resolution position to construct the prior (the closer the distance, the greater the weight). That is, to calculate the normalized weight probability of the feature of the high-resolution position coordinate relative to the feature of the low-resolution position coordinate :

[0075] ,

[0076] where the function is used to measure the direct similarity of the feature and the feature , is a constant to prevent numerical overflow of zero distance, N represents the total number of low-resolution pixels, represents the coordinate of the t-th low-resolution pixel, t is an accumulated intermediate variable, and represents the t-th low-resolution pixel point.

[0077] As shown in Figure 5 , the total probability feature reconstruction module uses a discrete total probability formula to predict the probability distribution of the high-resolution feature in discrete intensity intervals, taking the discrete probability distribution of the low-resolution pixel feature and the normalized weight probability of the high and low resolution position features as inputs, using a discrete total probability formula to predict the probability distribution of the K-dimensional intermediate feature of the high-resolution position , as follows:

[0078] ,

[0079] where is the calculated conditional probability of discrete intensity intervals on the low-resolution position, is the high-resolution query position ​prior probability of the low-resolution position of the high-resolution query position.

[0080]

[0081] wherein is the center value of the jth feature intensity interval.

[0082] Referring again to Figure 5 , the spectral dimension reduction module outputs the final predicted high-resolution image by taking the full probability feature as input through a set of convolution kernels.

[0083] In training the super-resolution network reconstruction network, the loss function adopts the mean absolute error loss function to calculate the difference between the reconstructed high-resolution image and the original hyperspectral image, as follows:

[0084]

[0085] wherein represents the mapping relationship between the input and output of the super-resolution network, represents the parameters of the network, represents the number of samples input in each iteration, represents the Frobenius norm.

[0086] The difference is fed back to the super-resolution network reconstruction network to optimize the network parameters, and when the set number of iterations is reached or the samples are exhausted, the training of the resolution network reconstruction network is completed.

[0087] In order to better reflect the beneficial effects of the present embodiment, a more specific embodiment is given as follows.

[0088] In this embodiment, the hyperspectral images in the Merced and Changping data sets captured by the Hyperion imaging spectrometer and the ALI imaging spectrometer are used to verify the effect of the super-resolution experiment. The Merced hyperspectral image covers a spectral range of 0.4-2.5 microns, and has a total of 134 bands, and the Changping hyperspectral image covers a spectral range of 0.4-2.5 microns, and has a total of 137 bands. In this experiment, the ratio of the spatial resolution of the original hyperspectral image to the low-resolution hyperspectral image in the training set is set to 4:1.

[0089] Figure 6 ​​​​​​​In the image, (a) and (d) are the original hyperspectral image data used for the Merced and Changping predictions, which serve as reference images for the experiment. Figure 6 Figures (b) and (e) show hyperspectral images obtained by upsampling using bicubic interpolation. As can be seen from the figures, the hyperspectral images processed by this method exhibit blurred detail and texture information, and spatial spectral information distortion. Figure 6 (c) and Figure 6 Figure (f) shows the hyperspectral images processed using the method described in this embodiment. As can be seen from the results, this method performs excellently in hyperspectral image super-resolution reconstruction, recovering most spatial details without significant spectral distortion. This is attributed to the super-resolution method design based on a global self-attention mechanism and probabilistic global modeling, employing a dual-global data model joint-driven architecture. This ensures the full extraction and fusion of spatial and spectral information, resulting in a significant improvement in super-resolution performance, achieving good recovery results in both spatial and spectral dimensions. These results confirm the superiority and effectiveness of this method in hyperspectral image super-resolution.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for super-resolution reconstruction of hyperspectral remote sensing images based on a self-attention full-probability neural network, characterized in that, Includes the following steps: Acquire hyperspectral images from the same sensor, perform resolution down-processing on the hyperspectral images, and obtain low-resolution hyperspectral images; A low-resolution hyperspectral image is input into a pre-constructed and trained super-resolution network for reconstruction to obtain a reconstructed high-resolution image. The training steps of the super-resolution network for reconstruction include: Coarse extraction is performed on low-resolution hyperspectral images to obtain coarse spatial-spectral feature maps. A global self-attention mechanism is used to enhance features and obtain fine spatial-spectral feature maps. Discrete feature intensity intervals are set, and the discrete probability distribution of low-resolution pixel position features is calculated based on the fine spatial-spectral feature maps. The discrete probability distribution is combined with the normalized weighted probabilities of high and low resolution position features to obtain the full probability features under high resolution. The high-resolution image is reconstructed based on the full probability features. The step of setting discrete feature intensity intervals and calculating the discrete probability distribution of low-resolution pixel location features based on the fine spatial spectral feature map is as follows: Discretize the range of possible pixel values ​​into multiple feature intensity intervals and obtain the corresponding interval center and interval boundary; Multiple convolutional kernels are used to map the fine spatial-spectral feature map to obtain multidimensional features at low-resolution pixels. The multidimensional features Including the Low-resolution location coordinates Features Multidimensional features are calculated using Softmax normalization. Probability distribution of each dimension ; Query location coordinates with high resolution With low-resolution location coordinate set Calculate the high-resolution query location coordinates using the input. Features Relative to low-resolution position coordinates Features Normalized weighted probabilities As shown in the following formula: ， Among them, the function Used to measure characteristics With features Direct similarity, To prevent zero-distance numerical overflow, N represents the total number of low-resolution pixels. This represents the coordinates of the t-th low-resolution pixel, where t is an accumulated intermediate variable representing the t-th low-resolution pixel.

2. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 1, characterized in that, The spatial size of the hyperspectral image and the corresponding low-resolution hyperspectral image satisfy a direct proportional relationship.

3. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 1, characterized in that, The process of preprocessing the hyperspectral image to reduce its resolution includes: first smoothing the hyperspectral image, then downsampling it to obtain a low-resolution hyperspectral image.

4. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 1, characterized in that, The super-resolution network reconstruction network includes a feature coarse extraction module, a global self-attention enhancement module, a probability parameter prediction module, a full probability feature reconstruction module, and a spectral dimension restoration module. The feature coarse extraction module is used to coarsely extract features from low-resolution training images to obtain coarse spatial spectral feature maps. The global self-attention enhancement module is used to enhance features using the global self-attention mechanism to obtain fine spatial-spectral feature maps. The probability parameter prediction module is used to set discrete feature intensity ranges and calculate the discrete probability distribution of low-resolution pixel position features based on the fine spatial spectral feature map. The full probability feature reconstruction module is used to combine the discrete probability distribution with the normalized weighted probabilities of high and low resolution location features to obtain full probability features at high resolution. The spectral dimension restoration module is used to reconstruct high-resolution images based on full probability features.

5. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 4, characterized in that, The coarse feature extraction module includes a feature extraction convolutional layer Conv1, a spectral attention module, and a feature extraction convolutional layer Conv2. The feature extraction convolutional layer Conv1 includes 64 groups of 3×3 convolutional kernels, each group of convolutional kernels has c convolutional kernels, and outputs 64-channel features F1; The spectral attention module uses global average pooling to compress the spatial information of feature F1, adjusts the spectral information through 64 sets of 1×1×64 convolution kernels to obtain spectral attention S, and multiplies spectral attention S with feature F1 to output 64-channel spectral feature F2. The feature extraction convolutional layer Conv2 includes 64 groups of 3×3 convolutional kernels. The input is the sum of features F1 and F2, and the output is a 64-channel coarse spatial spectral feature map F3.

6. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 4, characterized in that, The global self-attention enhancement module includes a feature extraction convolutional layer Conv3 and a self-attention module; The feature extraction convolutional layer Conv3 includes 64 groups of 3×3 convolutional kernels, with the input being a coarse spatial spectral feature map F3 and the output being a 64-channel feature map F4. The self-attention module includes three convolutional blocks, each with 64 groups of 3×3 convolutional kernels. The convolutional blocks respectively obtain the query, key, and value. The query and key are multiplied by the Softmax function and normalized to obtain the self-attention A. The self-attention A is multiplied by the value to obtain the fine spatial-spectral feature map F5.

7. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 1, characterized in that, The method of combining the discrete probability distribution with the normalized weighted probabilities of high and low resolution location features to obtain the full probability features at high resolution is as follows: With probability distribution and normalized weighted probabilities Using the discrete law of total probability as input, predict high-resolution query location coordinates. probability distribution of K-dimensional intermediate features ; Construct high-resolution query location coordinates Full probability features As shown in the following formula: ， in, For the first Each feature intensity range The central value of .

8. The hyperspectral remote sensing image super-resolution reconstruction method based on a self-attention full probability neural network according to claim 4, characterized in that, The spectral dimension restoration module, through Group The convolution kernel, with full probability features As input, output the final predicted high-resolution image.

9. The method for super-resolution reconstruction of hyperspectral remote sensing images based on a self-attention full probability neural network according to claim 1, characterized in that, The training of the super-resolution network reconstruction network uses the mean absolute error loss function to calculate the difference between the reconstructed high-resolution image and the original hyperspectral image. The difference is then fed back into the super-resolution network reconstruction network to optimize the parameters.

Images