Efficient hyperspectral remote sensing image generation method based on spatial-spectral information self-selection

Abstract

The invention provides a high-efficiency hyperspectral remote sensing image generation method based on spatial spectrum information self-selection. The method mainly comprises the following three core steps: 1, multi-level self-selection spatial feature coding; 2, multi-scale semantic fusion based on mixed pooling; and 3, spatial prior guidance and waveband relevance modeling. Starting from the characteristics of a hyperspectral remote sensing image generation task, a generation framework guided from spatial characteristics to spectral information driving is constructed, and a spatial-spectral information self-selection mechanism and a lightweight calculation strategy are introduced, so that the model can adaptively and dynamically pay attention to important spatial regions and spectral bands. The problems of insufficient physical consistency, redundant feature information, high video memory occupation and the like of a current hyperspectral remote sensing image generation method are effectively relieved, and the method has the advantages of high reconstruction precision, low calculation complexity, high training reasoning speed and the like; and high-quality hyperspectral remote sensing image data support can be provided for remote sensing downstream tasks such as image classification and change detection.

Description

Technical Field

[0001] This invention relates to an efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection, belonging to the fields of computer vision and remote sensing. Background Technology

[0002] (1) Hyperspectral Remote Sensing Image Generation refers to the technology of generating hyperspectral remote sensing images using certain model methods, either without conditions or with image, text, or other information as input. This technology can effectively alleviate the problems of high cost and technical difficulty in acquiring hyperspectral remote sensing images, and provide richer and more scalable hyperspectral data support for downstream remote sensing applications such as land cover classification, semantic segmentation, and target recognition.

[0003] Existing hyperspectral remote sensing image generation methods that use images as input can be mainly divided into three categories based on the different types of input data and generation principles: generation methods based on Spectral Super-Resolution (SSR), generation methods based on Hyperspectral Image Super-Resolution (HSR), and generation methods based on Image Fusion (IF).

[0004] Hyperspectral remote sensing image generation based on spectral super-resolution technology uses RGB or multispectral images with few bands and low spectral resolution as input. It leverages their spatial information and limited spectral features to achieve spectral super-resolution reconstruction through a model, thereby generating a hyperspectral remote sensing image. This type of generation method has low input data acquisition costs and wide availability. However, since RGB or multispectral images contain far less spectral information than hyperspectral images, efficiently and accurately reconstructing hyperspectral images that conform to the spectral characteristics of ground objects under conditions of insufficient spectral information becomes a key technical challenge.

[0005] Hyperspectral remote sensing image generation based on hyperspectral super-resolution technology works by using low-spatial-resolution hyperspectral images as input, improving the spatial resolution through a model, and outputting a hyperspectral image with both high spatial and spectral resolution. While this type of method focuses on enhancing spatial structural information, it still relies heavily on a large number of hyperspectral images as input, failing to fundamentally solve the problems of high acquisition costs and technical complexity associated with hyperspectral remote sensing images.

[0006] Hyperspectral remote sensing image generation based on image fusion works by simultaneously inputting a high spatial resolution RGB or multispectral image and a hyperspectral image. It learns detailed spatial information from the RGB (or multispectral) image and rich spectral information from the hyperspectral image to generate a high spatial resolution hyperspectral image. This type of generation method has advantages in joint spatial-spectral modeling, but its reliance on both hyperspectral and other modal images as input limits its scalability in large-scale hyperspectral remote sensing image generation scenarios.

[0007] In summary, existing hyperspectral remote sensing image generation methods still primarily rely on image condition input, and the three types of methods mentioned above each face their own technical challenges. Among them, hyperspectral remote sensing image generation methods based on spectral super-resolution techniques have received widespread attention in recent years due to the ease of data acquisition and significant application potential. However, these methods suffer from insufficient spectral information in the input images, making the reconstruction process essentially an ill-conditioned inverse problem. Therefore, how to generate hyperspectral remote sensing images with fine spatial structure and consistent spectral characteristics with real ground features while ensuring computational efficiency has become an urgent technical problem to be solved.

[0008] (2) The spectral super-resolution method based on deep learning transforms the traditional spectral super-resolution technology that relies on physical models and prior assumptions into a data-driven learning paradigm through deep neural networks. It can generate hyperspectral remote sensing images using modal images such as RGB remote sensing images that are readily available.

[0009] Currently, deep learning models for generating hyperspectral remote sensing images based on spectral super-resolution technology can be mainly divided into four categories: methods based on Convolutional Neural Networks (CNN), methods based on Transformer, methods based on generative models, and methods based on Mamba.

[0010] Convolutional neural network-based generative methods extract local spatial features from images layer by layer through convolution, and enhance the spectral reconstruction process by incorporating dense structures, attention mechanisms, and skip connections. While these methods effectively model local spatial features, their limited receptive field prevents them from effectively capturing long-distance spatial dependencies and band correlations, making them ill-suited for generating hyperspectral remote sensing images with complex ground cover distributions and severe spectral mixing.

[0011] Transformer-based generative methods utilize a self-attention mechanism to model global features of images. These methods, by introducing spatial and spectral attention modules, learn the spatial structural features and spectral band correlations of images, overcoming the limitation of CNN methods that only model local features. However, most current Transformer-based methods fail to address the specific characteristics of hyperspectral image reconstruction tasks, lacking in spatial information-guided encoding and spectral characteristic-driven modeling; furthermore, they suffer from high computational complexity, large memory consumption, and high computational resource requirements.

[0012] Generative methods based on generative models mainly include three categories: Generative Adversarial Networks (GANs), likelihood estimation-based generative models, and diffusion models. These methods possess the ability to sample from noise and generate diverse results, offering advantages in terms of generation quality and diversity. However, they generally suffer from problems such as unstable training processes, high computational costs, and slow inference speeds, making them difficult to directly apply to engineering tasks involving hyperspectral remote sensing image generation.

[0013] Mamba-based generative methods are a relatively new research approach. Their technical principle involves progressively modeling images using a state-space model, resulting in a global receptive field and near-linear computational complexity. Compared to Transformer and generative model methods, these methods offer significant advantages in computational efficiency. Currently, research based on the Mamba architecture is in the exploratory stage for spectral reconstruction, and its effectiveness in this area requires further validation.

[0014] Therefore, efficiently and accurately generating hyperspectral remote sensing images with high spatial resolution and conforming to the spectral characteristics of real ground objects remains a key challenge in the field.

[0015] This invention proposes an efficient hyperspectral remote sensing image generation method based on spatial-spectral information self-selection, alleviating the problems of insufficient model adaptability to hyperspectral remote sensing image reconstruction tasks, high computational complexity, and large memory consumption in existing methods. It achieves hyperspectral remote sensing image generation under the condition of a single RGB image input. This invention constructs a model framework from spatial feature guidance to spectral information driving, fully mining the spatial structure information in RGB images and effectively modeling spectral band autocorrelation during the reconstruction stage. Simultaneously, this invention proposes a spatial-spectral information self-selection mechanism, which can dynamically evaluate the importance of different spatial locations and spectral bands, enabling the model to selectively focus on key feature regions and reduce efficiency bottlenecks caused by noise interference and feature redundancy. Furthermore, this invention proposes a spectral band correlation evaluation method, combining spatial prior guidance and band correlation modeling to effectively learn the spectral characteristics of hyperspectral remote sensing images, thereby generating hyperspectral remote sensing images that conform to physical consistency. Summary of the Invention

[0016] 1. Objective: This invention aims to propose an efficient hyperspectral remote sensing image generation method based on self-selection of spatial spectral information. Using a single RGB image as input, it generates a hyperspectral remote sensing image that balances detailed spatial structure features with the spectral physical properties of real ground objects. This invention effectively alleviates the practical challenges of acquiring high-quality hyperspectral remote sensing images, which are technically difficult, costly, and time-consuming. The generated hyperspectral remote sensing images can be widely used in downstream tasks such as image classification, image segmentation, and target detection, fully leveraging the rich and continuous spectral information of hyperspectral remote sensing images to provide abundant data for applications such as geological exploration, environmental monitoring, and land cover classification.

[0017] 2. Technical Solution: This invention is achieved through the following technical solution:

[0018] This invention is an efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection, which is divided into three implementation steps: multi-level self-selection spatial feature encoding, multi-scale semantic fusion based on hybrid pooling, and spatial prior guidance and band correlation modeling.

[0019] Step 1: Multi-level self-selection spatial feature encoding

[0020] This invention employs an efficient Transformer coding structure in the remote sensing image input and encoding stages to extract spatial structure information at different levels of RGB remote sensing images and dynamically score the importance of features at different spatial locations, thereby reducing the impact of spatial redundancy features and noise interference.

[0021] Specifically, this invention first performs multi-level spatial feature encoding on the input RGB remote sensing image to simultaneously acquire shallow spatial structure information and deep semantic information. The image is then divided into blocks and mapped through feature fusion encoding, projecting them onto a multi-level feature space to obtain the i-th layer of multi-scale features. Where H i W i C i These represent the height, width, and number of feature channels of the i-th layer, respectively. Due to the use of overlapping windows, adjacent windows can share some image information, which helps maintain the continuity of the local structure. The feature maps are then expanded spatially and rearranged into a feature sequence of length H×W, so that each spatial location corresponds to a feature vector. Efficient self-attention computation is performed on the feature sequences of each layer, and the importance of features at different spatial locations is dynamically evaluated through a spatial feature self-selection module. This reduces the weight of redundant information, allowing the network to focus on regions containing important information before feeding them into the next layer for feature encoding, ultimately obtaining spatial features at different levels.

[0022] In the aforementioned encoder, this invention proposes a spatial feature self-selection module. The core objective of this module is to evaluate the importance of feature vectors at different spatial locations and weight the importance scores with feature information, enabling the network to prioritize the modeling of key information regions with limited computational overhead, thereby enhancing the information extraction capability during the encoding stage.

[0023] Step 2: Multi-scale semantic fusion based on hybrid pooling

[0024] In step one, the present invention completes the encoding of the spatial features of the input RGB remote sensing image, and obtains multi-level spatial feature information.

[0025] This invention addresses the problem of multi-scale modeling and multi-level fusion of deep semantic information in this step, proposing a multi-scale semantic fusion method based on hybrid pooling. It efficiently extracts features from deep semantic information under different receptive fields, and then uniformly aligns and fuses the obtained multi-level features.

[0026] Specifically, for the deepest feature x in the multi-level features output in step one... n Multi-scale hybrid pooling is performed, and the pooling results at each scale are upsampled to the original feature space size before concatenation. This process effectively expands the feature receptive field while maintaining near-linear computational complexity, alleviating the computational pressure brought by deeper self-attention mechanisms. In the feature fusion module, a multilayer perceptron (MLP) is used to integrate the features {x1, x2, ..., x...} n Linear mapping to the same spatial size and feature dimension, the aligned multi-level features are concatenated in the channel dimension and fused through multiple convolutional layers.

[0027] Step 3: Spatial Prior Guidance and Band Correlation Modeling

[0028] After obtaining the fusion features in step two, this invention further introduces a spatial prior guidance and band correlation modeling module to enhance the spatial feature constraints and band correlation learning during the spectral reconstruction process.

[0029] Specifically, this invention performs a linear transformation on the fused features to align them with the target output size H in both spatial and channel dimensions. out ×W out ×C out Based on the target output band range, the feature is fused with the input RGB remote sensing image using band-level matching to construct a feature representation with clear spatial prior guidance. This process fully utilizes the spatial structure information contained in the RGB image, providing stable and effective spatial guidance for subsequent spectral reconstruction. Subsequently, band correlation modeling is performed based on the above prior features, using a spectral-dimensional Transformer to learn the correlation between bands. Finally, the obtained features are passed through linear mapping and nonlinear activation layers to output the hyperspectral remote sensing image generation result.

[0030] 3. Advantages and Efficacy

[0031] This invention presents a highly efficient hyperspectral remote sensing image generation method based on spatial-spectral information self-selection. Its advantages include: constructing a hyperspectral remote sensing image generation framework guided by spatial features and driven by spectral information; employing multi-level self-selective spatial feature encoding; achieving efficient generation through multi-scale semantic fusion based on hybrid pooling; and using spatial prior guidance and band correlation modeling. This invention proposes a dynamic evaluation mechanism for spatial-spectral feature self-selection, which allows the model to prioritize key information regions, enhancing feature extraction capabilities. Furthermore, this invention combines global modeling of Transformer with local feature learning of CNN. By introducing efficient Transformer modules and multi-scale pooling strategies, it significantly reduces model computational complexity and memory usage, accelerates training and inference speed, and thus enhances the engineering applicability of this generation method. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall framework of the hyperspectral remote sensing image generation method.

[0033] Figure 2 It is a multi-level self-selection spatial feature encoding.

[0034] Figure 3 This is for multi-scale semantic fusion based on hybrid pooling.

[0035] Figure 4 Modeling spatial prior guidance and band correlation.

[0036] Figure 5 The result is generated from a hyperspectral remote sensing image. Specific implementation methods

[0037] To better understand the technical solution of the present invention, the implementation method of the present invention will be further described below:

[0038] This invention is implemented using Python within the PyTorch framework. First, an RGB remote sensing image is input into a multi-level self-selection spatial feature encoder, with a feature information self-selection module added between the self-attention modules. The resulting deep semantic features are then subjected to multi-scale hybrid pooling and concatenated. Next, spatial prior guidance and band correlation modeling are applied to the features. Finally, the features are passed through convolutional and activation layers to output the target hyperspectral remote sensing image.

[0039] The server is configured with 20GB of RAM, an NVIDIA GeForce GTX 4090 graphics card with 24.0GB of video memory, and a Linux operating system.

[0040] The hyperspectral remote sensing image generation method includes the following steps:

[0041] Step 1: Multi-level self-selection spatial feature encoding

[0042] The hyperspectral remote sensing image generation framework constructed in this invention is as follows: Figure 1 As shown, the model mainly consists of multi-level self-selected spatial feature encoding, multi-scale semantic fusion based on hybrid pooling, and spatial prior guidance and band correlation modeling.

[0043] This invention employs feature fusion encoding, a spatial self-attention layer, and a spatial feature self-selection module in the encoder section to obtain multi-level spatial features of the image, such as... Figure 2 As shown. In this stage, the image is mapped to the feature space through feature fusion encoding, and then the global features of the image are learned using an efficient Transformer. A spatial feature self-selection module is added between the self-attention layers.

[0044] The input image is encoded layer by layer to obtain multi-scale features {x}. i Its dimensions are This allows for the simultaneous acquisition of features at different fine-grained levels. Specifically, a feature fusion encoding module is used to perform convolutional downsampling and aggregation on the features to obtain features of different sizes. The specific implementation process is as follows:

[0045] For the i-th coding layer, its input is in For the input RGB remote sensing image, The convolution projection process is as follows, using the feature map output from the previous layer:

[0046]

[0047] In the above formula, x in x represents the feature map of the i-th layer. pi The feature map k represents the feature map after feature fusion encoding of the i-th layer. i s is the kernel size. i p is the step size. i For the fill size, s i <k i The convolutional windows overlap, resulting in a feature map size of the i-th layer after block encoding.

[0048] The feature map is then unfolded to obtain a feature sequence, which is then subjected to Layer Normalization (LN):

[0049]

[0050]

[0051] The feature sequence obtained from the above formula The data is fed into Transformer encoding blocks. Each module uses a multi-head self-attention (MHSA) module to model the long-range dependencies of the feature sequences. Then, it is connected to a feed-forward network (FFN) with residual connections. The process is as follows:

[0052]

[0053] In the above formula, To expand the resulting feature sequence, Feature map encoded by Transformer.

[0054] This task employs an efficient self-attention module that reduces the length of the feature sequence in self-attention computation by spatially downsampling Key(K) and Value(V) while keeping Query(Q) unchanged.

[0055] X sr =LN(Flatten(Conv) sr,sr (Reshape(X))))

[0056] Q = XW Q K = X sr W k V = X srW V

[0057]

[0058] head spa =Attention spa V

[0059] In the above formula, sr represents the downsampling factor during the self-attention calculation process. spa LN represents spatial attention head, and LN represents layer normalization.

[0060] The self-attention output described above is linearly mapped and then fed into a feedforward network. Spatial relationships can be modeled without explicit position encoding, as calculated below:

[0061] FFN(x (i) =MLP(GELU(Conv(MLP(x)) (i) ))))

[0062] In the above formula, x (i) The symbols represent the features after passing through the self-attention mechanism, GELU represents the activation function, MLP represents the linear layer, and FFN represents the feedforward network.

[0063] This invention introduces a spatial feature self-selection module between Transformer blocks of the encoder to dynamically score the importance of different spatial locations in the feature map. The specific implementation process is as follows:

[0064] By applying linear mapping and nonlinear activation to the feature sequence, intermediate feature representations are obtained:

[0065]

[0066] In the above formula, C represents the intermediate layer feature sequence. hidden "Linear" represents the feature dimension of the intermediate layer, while "Linear" represents the linear layer.

[0067] x (i) The data is divided into two parts along the channel dimension: local features and global features. Local features directly retain spatial information, while global features are averaged along the spatial dimension to extract global contextual information.

[0068]

[0069] The global context features are broadcast and concatenated with the local features along the channel dimension to obtain the fused feature representation:

[0070]

[0071] Based on this, the importance score of each feature vector is predicted through two layers of linear mapping and a non-linear activation layer, and then normalized to the [0,1] interval using the Sigmoid function:

[0072]

[0073] s (i) =Sigmoid(Linear(s (i) ))

[0074] Importance scores are used as self-selected weights on the original feature sequence to achieve adaptive weighting of the feature sequence:

[0075]

[0076] Where ⊙ represents element-wise multiplication, and the weighted feature is obtained by summing. This serves as the input to the next layer of Transformer encoding blocks. After multi-layer Transformer global modeling, a multi-level spatial feature representation is obtained.

[0077] Step 2: Multi-scale semantic fusion based on hybrid pooling

[0078] In step one, the encoder outputs multi-level spatial features. in This represents the deepest semantic features, possessing the strongest semantic expressive power but the lowest spatial resolution. For example... Figure 3 As shown, to enhance the contextual modeling capability of deep features in generation tasks while avoiding the reintroduction of the computationally complex Transformer module, this invention focuses on the deepest features. A low-complexity multi-scale pooling module is used, and the specific calculation process is as follows:

[0079]

[0080]

[0081]

[0082] In the above formula, x4 represents the deepest feature, s k Representing different pooling scales, This represents the deepest semantic features after multi-scale pooling; Pool represents adaptive pooling, GELU represents the activation function, Up represents the upsampling process, and Concat represents the multi-scale feature concatenation process.

[0083] This invention relates to After mapping and upsampling, the spatial dimensions are aligned with the feature dimensions, and then concatenated along the channel dimension to achieve multi-level feature fusion. Subsequently, the fused features are transformed through multi-level linear mapping and non-linear activation functions to generate initial output features, calculated as follows:

[0084]

[0085] x = RELU(Linear(x) concat ))

[0086] In the above formula, x represents the initial output feature result obtained after feature fusion, and its size is consistent with the target generated image, i.e. ReLU is the activation function.

[0087] Step 3: Spatial Prior Guidance and Band Correlation Modeling

[0088] In step three, this invention primarily employs a spatial prior guidance module to strengthen the constraint effect of RGB image spatial information, and uses a spectral dimension Transformer to model band correlations. The implementation process is as follows: Figure 4 As shown.

[0089] Based on the correspondence between the target hyperspectral band range and the RGB channels, this invention constructs spatial prior information for band-by-band replication:

[0090] x pri =Concat(Repeat(I B ,n B Repeat(I) G ,n G Repeat(I) R ,n R ))

[0091] In the above formula, I B ,I G ,I R This represents the three channels of the input RGB image, i.e., I rgb =[I B ,I G ,I R Repeat means repeating n times in the channel dimension.

[0092] To achieve band-by-band self-selection of spectral information and enable the model to adaptively balance between network-predicted features and RGB prior guidance information, this invention introduces a set of learnable fusion weight vectors α to perform weighted fusion of the two features by band. The calculation process is as follows:

[0093]

[0094] In the above formula, x fuse ⊙ represents a fusion feature with spatial prior constraints, ⊙ represents element-wise multiplication, and Sigmoid represents the activation function.

[0095] This invention uses a spectral-dimensional Transformer to model the correlation between different bands of a hyperspectral image, and employs residual connections to enhance generation stability. The correlation modeling process is as follows:

[0096] x spe =S-MSA(x fuse )+x fuse

[0097] In the above formula, x spe S-MSA represents the feature representation obtained by spectral self-attention computation. It is used to model the correlation between different spectral bands, thereby learning the true spectral features of ground objects.

[0098] The S-MSA in this step differs from the spatial self-attention in step one: it maintains the spatial location, models correlations in the spectral dimension, and characterizes the continuity of the spectral curve and the inter-band correlations. Given the fusion feature x... fuse Expanding it in spatial dimensions yields a feature sequence, resulting in... The unrolling and linear mapping process remains consistent with step one. Subsequently, linear mapping is performed only on the spectral dimension to obtain Query(Q), Key(K), and Value(V). The spectral dimension is then divided into multiple subspaces (i.e., multiple spectral attention heads), and self-attention calculation and linear mapping are performed on each spectral attention head. The calculation process for each spectral attention head is as follows:

[0099] Q = XW Q K = XW K V = XW V

[0100] Attention spec (Q,K,V)=Softmax(αQK T )

[0101] head spec =VAttention spec

[0102] S-MSA(x flatten = (Concat(head)) spec ))W+PE(V)

[0103] In the above equation, α represents the learnable spectral scale parameter, W represents the learnable projection matrix of the linear layer, and PE represents the function used to generate the positional encoding. The structure of the spectral-dimensional Transformer is as follows: Figure 4 As shown.

[0104] After the feature map is processed by a multi-layer spectral dimension Transformer module, this invention employs multi-layer linear mapping and a non-linear activation function to obtain the final hyperspectral remote sensing image generation result:

[0105] x conv =Linear(x) spe )

[0106] x output =Clamp(RELU(x) conv ))

[0107] Experimental Results: To verify the hyperspectral remote sensing image generation effect of this invention, the IEEE grss_dfc_2018 hyperspectral remote sensing dataset was selected as the experimental sample for comparative verification. During the testing phase, a single RGB image was used as input for inference generation, and the generated results were visualized and quantitatively evaluated against the corresponding real hyperspectral images (Ground Truth, GT) in the test set. Experimental results show that the hyperspectral images generated by this invention perform well in both spatial structure preservation and spectral characteristic reconstruction. The visualized generation results for the test set are shown below. Figure 5 As shown. Among them, Figure 5 (a) A comparison of the false-color image generation effects synthesized from bands 23, 12, and 5 is given; Figure 5 (b) Provides a comparison of the full image and magnified details of grayscale images of different bands.

[0108] As shown in Table 1, under the condition of having real reference images, the present invention achieved excellent levels in multiple quantitative evaluation indicators (RMSE, MRAE, SAM, MSSIM, and MPSNR) on the test set. Among them, RMSE and MRAE reflect the pixel-level error of the generated results; SAM reflects the spectral angle deviation and spectral reconstruction accuracy; MSSIM and MPSNR reflect the structural similarity of the generated images and the overall reconstruction quality.

[0109] Table 1

[0110]

Claims

1. A highly efficient hyperspectral remote sensing image generation method based on self-selection of spatial spectral information, characterized in that: The steps are as follows: Step 1: Multi-level self-selection spatial feature encoding In the RGB remote sensing image input and spatial feature encoding stage, a multi-level Transformer encoding structure is used to extract multi-level spatial feature representations, obtaining both shallow structural details and deep semantic information. Based on this, a spatial feature self-selection module is introduced to dynamically evaluate the importance of features at different spatial locations, guiding the network to focus on key information regions. Step 2: Multi-scale semantic fusion based on hybrid pooling A module for multi-scale modeling and multi-level fusion of deep semantic information is constructed, and a multi-scale semantic fusion method based on hybrid pooling is proposed. Efficient feature extraction is performed on deep semantic information under different receptive fields, and then the obtained multi-level features are uniformly aligned and fused. Step 3: Spatial Prior Guidance and Band Correlation Modeling First, spatial priors are used to guide the input RGB remote sensing image, and then adaptive weighted fusion is performed with the network's predicted features. Then, a spectral dimension self-attention mechanism is used to model the correlation of spectral bands. Finally, the hyperspectral remote sensing image generation result is obtained through the output layer.

2. The efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection according to claim 1, characterized in that: In step one, a multi-layer coding module employing feature fusion coding, spatial feature self-selection dynamic evaluation, and an efficient self-attention mechanism is used to obtain multi-scale coded features {x}. i Its dimensions are 3. The efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection according to claim 1 or 2, characterized in that: In step one, the feature fusion encoding module performs convolutional downsampling and aggregation on the features to obtain features of different sizes. The calculation process is as follows: In the above formula, x in x represents the feature map of the i-th layer. pi The feature map k represents the feature map after feature fusion encoding of the i-th layer. i s is the kernel size. i p is the step size. i For the fill size, s i <k i The convolution windows overlap, and then the feature maps are expanded to obtain feature sequences and layer normalization is performed to obtain feature sequences.

4. The efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection according to claim 1 or 2, characterized in that: In step one, a spatial feature self-selection module is used to dynamically score the importance of features at different spatial locations. The specific implementation process is as follows: First, the feature sequence is linearly mapped and nonlinearly activated to obtain intermediate feature representations. The features are then divided into local and global features along the channel dimension. Local features directly retain spatial information, while global features undergo average pooling along the spatial dimension to extract global contextual information. The calculation process is as follows: The global context features are broadcast and concatenated with the local features along the channel dimension to obtain the fused feature representation: Based on this, the importance score of each feature vector is predicted through two layers of linear mapping and nonlinear activation layer, and normalized to the [0,1] interval by the Sigmoid function. The importance score is then used as a self-selected weight to apply to the original feature sequence, thereby achieving adaptive weighting of the feature sequence. The resulting feature is used as the input of the next layer of coding block.

5. A highly efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection as described in claim 1 or 2, characterized in that: In step one, the Transformer module with an efficient self-attention mechanism is used to model the long-distance dependencies of the feature sequences. The calculation process is as follows: X sr =LN(Flatten(Conv sr,sr (Reshape(X)))) In the above formula, sr represents the downsampling factor in the self-attention calculation process. Spatial downsampling is performed on Key(K) and Value(V) while keeping Query(Q) unchanged to reduce the length of the feature sequence in the self-attention calculation. The calculation process is as follows. Q=XWQ Q ,K=X sr W k ,V=X sr W V head spa =Attention spa V In the above formula, sr represents the downsampling factor during the self-attention calculation process. spa LN represents the spatial attention head, and LN represents layer normalization. Features are linearly mapped and then fed into the feedforward network, enabling the modeling of spatial relationships without explicit location encoding. The computation process is as follows: FFN(x (i) )=MLP(GELU(Conv(MLP(x (i) )))) In the above formula, x (i) The symbols represent the features after passing through the self-attention mechanism, GELU represents the activation function, MLP represents the linear layer, and FFN represents the feedforward network.

6. The efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection according to claim 1, characterized in that: In step two, multi-scale semantic fusion based on hybrid pooling is employed. To enhance the contextual modeling capability of deep features in the generation task, while avoiding the reintroduction of the computationally complex Transformer module, this invention introduces a low-complexity multi-scale pooling module for the deepest features. The calculation process is as follows: In the above formula, x4 represents the deepest feature, s k Representing different pooling scales, Pool stands for adaptive pooling. Features obtained from pooling at each scale are upsampled to the same spatial size as the original features and then concatenated to form a deep semantic representation containing contextual information from multiple receptive fields. Simultaneously, multi-level features are linearly mapped and upsampled to ensure their spatial size matches the feature dimension, and concatenation is performed along the channel dimension to achieve multi-level feature fusion. Specifically, in the feature fusion module, a multilayer perceptron (MLP) is used to fuse multi-level features {x1, x2, ..., x...} n The linearly mapped features are aligned to the same spatial size and feature dimension. The aligned multi-level features are then concatenated along the channel dimension and fused through multiple convolutional layers. Subsequently, the fused features are transformed using multi-level linear mappings and non-linear activation functions to generate the initial output features.

7. The efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection according to claim 1, characterized in that: In step three, a spatial prior guidance module is used to strengthen the constraint effect of spatial information of RGB image, and band correlation modeling is performed through spectral dimension Transformer. Then, multi-layer linear mapping and nonlinear activation function are used to obtain the final hyperspectral remote sensing image generation result.

8. A highly efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection as described in claim 1 or 7, characterized in that: In step three, a spatial prior guidance module is used to perform band-level matching and fusion of the feature with the input RGB remote sensing image based on the target output band range, constructing a feature representation with clear spatial prior guidance. The calculation process is as follows: x pri =Concat(Repeat(I B ,n B ),Repeat(I G ,n G ),Repeat(I R ,n R )) In the above formula, I B ,I G ,I R This represents the three channels of the input RGB image, i.e., I rgb =[I B ,I G ,I R Repeat means repeating n times in the channel dimension. To achieve band-by-band self-selection of spectral information and enable the model to adaptively balance between network-predicted features and RGB prior guidance information, this invention introduces a set of learnable fusion weight vectors α to perform weighted fusion of the two features by band. The calculation process is as follows: In the above formula, xf use ⊙ represents a fusion feature with spatial prior constraints, ⊙ represents element-wise multiplication, and Sigmoid represents the activation function.

9. A highly efficient hyperspectral remote sensing image generation method based on spatial spectral information self-selection as described in claim 1 or 7, characterized in that: In step three, a spectral-dimensional Transformer is used to model band correlations, learn the spectral autocorrelation between different bands, and use residual connections to enhance generation stability. The specific process is as follows: x spe =S-MSA(x fuse )+x fuse In the above formula, x spe S-MSA represents the feature representation obtained by spectral self-attention computation. The S-MSA in this step differs from the spatial self-attention in step one: spatial location remains unchanged, while correlation is modeled in the spectral dimension. Specifically, given the fused feature x... fuse Expanding it in spatial dimensions yields a feature sequence, resulting in... The unrolling and linear mapping process remains consistent with step one. Subsequently, linear mapping is performed only on the spectral dimension to obtain Query(Q), Key(K), and Value(V). The spectral dimension is then divided into multiple subspaces (i.e., multiple spectral attention heads), and self-attention calculation and linear mapping are performed on each spectral attention head. The calculation process for each spectral attention head is as follows: Q=XW Q ,K=XW K ,V=XW V Attention spec (Q,K,V)=Softmax(αQK T ) head spec =VAttention spec S-MSA(x flatten )=(Concat(head spec ))W+PE(V) In the above equation, α represents the learnable spectral scale parameter, W represents the learnable projection matrix of the linear layer, and PE represents the function used to generate the positional encoding. After the feature map passes through a multi-layer spectral dimension Transformer module, this invention employs multi-layer linear mapping and non-linear activation functions to obtain the final hyperspectral remote sensing image generation result.

Images