Hyperspectral image reconstruction method and system based on frequency separation and physical perceptual attention

Abstract

The present application relates to the technical field of hyperspectral image reconstruction, and discloses a hyperspectral image reconstruction method and system based on frequency separation and physical perception attention, which comprises the following steps: acquiring a compressed snapshot observation image, and performing inverse dispersion operation to restore the initial three-dimensional signal; inputting the initial three-dimensional signal into a physical perception mask attention U-Net reconstruction network to output a reconstructed hyperspectral image; performing high-low frequency separation guidance on the reconstructed hyperspectral image, inputting the extracted high-frequency and low-frequency features into a selective multi-head spatial attention module respectively, inputting the obtained high-frequency attention feature map and low-frequency attention feature map into a mask attention module, generating a dynamic weight map of a high-fidelity area according to the physical light transmittance and position encoding of the modulation mask; performing mask enhancement on the high-frequency feature, moderately modulating the low-frequency feature, and obtaining a fusion feature map through fusion; and outputting after residual connection and inverse dispersion alignment operation.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image reconstruction technology, and more specifically to a hyperspectral image reconstruction method and system based on frequency separation and physical perception attention. Background Technology

[0002] With the development of intelligent sensing technology, the demand for real-time reconstruction of high-quality images is urgent for airborne hyperspectral target detection. Hyperspectral imaging, by acquiring rich spectral information across multiple bands, is an important tool for airborne target identification and scene analysis. Traditional point-by-point scanning imaging systems are complex and time-consuming to acquire data, making them difficult to meet the needs of dynamic scenes. Although coded aperture snapshot spectral imaging (CASSI) technology can acquire data in a single exposure, reconstructing high-fidelity images from compressed measurements in airborne environments remains a challenge.

[0003] Traditional reconstruction methods rely on manual priors such as sparsity, which have problems such as parameter sensitivity, weak generalization and slow speed. Although deep learning methods improve efficiency through neural networks, they suffer from imbalance between fine details and global structure reconstruction in complex scenes and insufficient accuracy due to fixed mask modulation and failure to utilize spatial transmissivity to guide feature learning.

[0004] Therefore, there is an urgent need for a high-precision hyperspectral image reconstruction method that fully utilizes mask guidance and balances details and structure. Summary of the Invention

[0005] In view of this, the present invention provides a hyperspectral image reconstruction method and system based on frequency separation and physical perception attention, which solves the problems of insufficient utilization of mask guidance, imbalance between fine spatial details and global structure reconstruction, and insufficient reconstruction accuracy in existing methods.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] Hyperspectral image reconstruction methods based on frequency separation and physical perception attention include:

[0008] Acquire compressed snapshot observation images and restore the two-dimensional compressed snapshot observation images to the initial three-dimensional signal through inverse dispersion operation;

[0009] The initial 3D signal is input into the Physically Aware Mask Attention U-Net reconstruction network, which outputs a reconstructed hyperspectral image, specifically including:

[0010] The initial three-dimensional signal is converted into a high-dimensional feature representation through an embedding layer;

[0011] The high-dimensional feature representation in the encoder stage is obtained by sequentially passing through 2 FMAB modules, downsampling, 4 FMAB modules, and downsampling operations to obtain deep features;

[0012] The bottleneck layer uses four FMAB modules to perform cross-channel interaction and multi-scale fusion of deep features to generate bottleneck features;

[0013] After upsampling, the bottleneck features are concatenated with the corresponding layer features processed by the four FMAB modules in the encoder, fusing deep semantics and mid-level details, and then optimized by convolution and the four FMAB modules.

[0014] After the optimized features are upsampled again, they are concatenated with the features initially processed by the two FMAB modules in the encoder to supplement the original spatial structure information, and then optimized by convolution and two FMAB modules.

[0015] The feature dimensions are restored by mapping layers and added to the residual of the initial three-dimensional signal to generate a reconstructed hyperspectral image.

[0016] Preferably, the specific processing steps of the FMAB module include:

[0017] The input features are separated into high and low frequencies to guide the output of high-frequency and low-frequency features.

[0018] High-frequency features and low-frequency features are input into the selective multi-head spatial attention module, respectively, to obtain high-frequency attention feature maps and low-frequency attention maps.

[0019] The high-frequency attention feature map and the low-frequency attention feature map are respectively input into the mask attention module, and a dynamic weight map of the high-fidelity region is generated based on the physical transmittance and position encoding of the modulation mask.

[0020] The high-frequency attention feature map is enhanced by masking based on the dynamic weight map, the low-frequency attention feature map is moderately modulated, and a fused feature map is obtained by fusion.

[0021] The fused feature map is output after being aligned with residual connections and anti-dispersion.

[0022] Preferably, the input features are guided by high- and low-frequency separation to output high-frequency and low-frequency features, including:

[0023] High-frequency components are extracted from the input features using 3×3 convolution, and high-frequency features are output.

[0024] Low-frequency components are extracted from the input features using 7×7 convolution, and low-frequency features are output.

[0025] The preferred selective multi-head spatial attention module processing procedure is as follows:

[0026] S101. Use S convolutional kernels of different scales to extract multi-scale features;

[0027] S102. Divide each scale feature into L heads for parallel processing, and perform a convolution operation on the j-th head at the i-th scale:

[0028] ;

[0029] Where Conv represents the convolution operation. , X is the input feature. This represents the kernel size at the i-th scale. This represents the output of the j-th head convolution operation at the i-th scale;

[0030] By concatenating L head outputs from each scale, an intermediate feature map is obtained. :

[0031] ;

[0032] Wherein, Concat represents the concatenation operation;

[0033] S103. The intermediate feature map is mapped through a convolutional layer to obtain feature maps of different scales. :

[0034] ;

[0035] The weights are calculated using Softmax to obtain the feature map. Corresponding weight matrix :

[0036] ;

[0037] Where exp represents the exponential function;

[0038] Weighted summation generates output feature map :

[0039] .

[0040] Preferably, the high-frequency attention feature map and the low-frequency attention feature map are respectively input into the mask attention module, and a dynamic weight map of the high-fidelity region is generated based on the physical transmittance and position encoding of the modulation mask, including:

[0041] S201. Using the high-frequency attention feature map and the low-frequency attention feature map as input feature maps, the input feature map modulation mask is subjected to dispersion operation to obtain the feature map. Feature maps are obtained through two convolutional layers. :

[0042] ;

[0043] Where Conv represents the convolution operation;

[0044] S202, Combine the learnable location code ε with the feature map The data is concatenated and then subjected to convolution, ReLU activation, and convolution operations to generate a weight map. :

[0045] ;

[0046] Where Sigmoid represents Sigmoid activation, and Concat represents concatenation operation;

[0047] S203, Weighted graph With feature map Fusion and with feature maps Residual connections yield the output feature map :

[0048] ;

[0049] Where λ is an adjustable parameter. This indicates a fusion operation, and + indicates a residual connection;

[0050] S204, Output feature map Invert the dispersion to obtain a dynamic weight map. .

[0051] Preferably, the high-frequency attention feature map is masked and enhanced based on a dynamic weight map, the low-frequency attention feature map is moderately modulated, and a fused feature map is obtained through fusion, including:

[0052] Masking enhancement of high-frequency attention feature maps based on dynamic weight maps yields enhanced high-frequency features. :

[0053] ;

[0054] in, Indicates a fusion operation. Represents a dynamic weighted graph. Represents high-frequency attention feature maps;

[0055] By appropriately modulating the low-frequency attention feature map based on the dynamic weight map, enhanced low-frequency features are obtained. :

[0056] ;

[0057] Where α represents the learnable parameter, Represents low-frequency attention feature maps;

[0058] Adding the enhanced high-frequency features and the enhanced low-frequency features yields the fused features. ;

[0059] ;

[0060] Fusion features The fused feature map is output through 1×1 convolution.

[0061] A hyperspectral image reconstruction system based on frequency separation and physical perception attention includes:

[0062] Preprocessing unit: acquires compressed snapshot observation images and restores the two-dimensional compressed snapshot observation images to the initial three-dimensional signal through inverse dispersion operation;

[0063] Frequency separation and physical perception attention unit: The initial 3D signal is input into the physical perception mask attention U-Net reconstruction network, and the output is a reconstructed hyperspectral image, specifically including:

[0064] The initial three-dimensional signal is converted into a high-dimensional feature representation through an embedding layer;

[0065] The high-dimensional feature representation in the encoder stage is obtained by sequentially passing through 2 FMAB modules, downsampling, 4 FMAB modules, and downsampling operations to obtain deep features;

[0066] The bottleneck layer uses four FMAB modules to perform cross-channel interaction and multi-scale fusion of deep features to generate bottleneck features;

[0067] After the bottleneck features are upsampled, they are concatenated with the corresponding layer features in the encoder that have been processed by 4 FMAB modules, fusing deep semantics and mid-level details, and then optimized by convolution and 4 FMAB modules;

[0068] After the optimized features are upsampled again, they are concatenated with the features initially processed by the encoder through two FMAB modules to supplement the original spatial structure information, and then optimized by convolution and two FMAB modules.

[0069] The feature dimensions are restored by mapping layers and added to the residual of the initial three-dimensional signal to generate a reconstructed hyperspectral image.

[0070] A computer device includes a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, it implements a hyperspectral image reconstruction method based on frequency separation and physical perception attention.

[0071] A computer-readable storage medium storing a computer program that, when executed by a processor, implements a hyperspectral image reconstruction method based on frequency separation and physical perception attention.

[0072] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a hyperspectral image reconstruction method and system based on frequency separation and physical perception attention, with the following beneficial effects:

[0073] (1) The high-fidelity mask attention mechanism of the present invention dynamically generates a weight map based on the difference in transmittance at different positions of the mask, and introduces learnable position encoding. This allows the model to accurately perceive high-fidelity areas, guide attention to focus on key positions, strengthen the preservation of detailed features such as edges and textures, and improve the accuracy of hyperspectral image reconstruction in complex scenes.

[0074] (2) The frequency separation guidance strategy of this invention uses small-scale convolution to extract high-frequency detail features and large-scale convolution to extract low-frequency global features. Masked attention is used to enhance the detail representation of high-frequency features, and learnable weighted modulation is used to maintain the consistency of global structure for low-frequency features. This solves the problem of imbalance between fine details and global structure reconstruction in traditional methods, and achieves the unity of detail restoration and structural stability.

[0075] (3) This invention constructs a deep network architecture based on U-Net, combining a frequency-aware mask attention module (FMAB) with skip connections to achieve efficient extraction and fusion of hierarchical features, improving feature representation capabilities while reducing information loss. Experiments show that this method significantly outperforms mainstream methods in PSNR (average 36.93dB) and SSIM (average 0.962), and balances algorithm efficiency and performance by optimizing the network structure. At the same time, it utilizes a physical perception mask guidance strategy to fit the characteristics of the CASSI system, effectively reducing reconstruction artifacts and providing high-precision, high-robust hyperspectral data support for airborne intelligent sensing tasks. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0077] Figure 1 A diagram of the network framework for reconstructing the U-Net network based on physical perception mask attention.

[0078] Figure 2 This is a structural diagram of the frequency-aware masked attention module.

[0079] Figure 3 This is a structural diagram of the selective multi-head spatial attention module.

[0080] Figure 4 This is a structural diagram of the masked attention module.

[0081] Figure 5 A visualization of the reconstruction results for Scene 7.

[0082] Figure 6 and Figure 7 The images show the spectral feature curves of the reconstructed hyperspectral images of blocks a and b in scenes 1 and 8, respectively.

[0083] Figure 8 A visualization of the reconstructed results from real data.

[0084] Figure 9 A block diagram of a hyperspectral image reconstruction system for frequency separation and physical perception attention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] This invention discloses a hyperspectral image reconstruction method based on frequency separation and physical perception attention, comprising:

[0087] A compressed snapshot observation image is acquired, and the initial three-dimensional signal is restored from the two-dimensional compressed snapshot observation image through an inverse dispersion operation. Specifically, a coded aperture snapshot spectral imaging (CASSI) system is used to acquire the compressed snapshot observation image. In this system, a modulation mask encodes the scene, a dispersion device disperses the spectral information, and a detector integrates to form a two-dimensional compressed image. An inverse dispersion operation is then performed on it to recover the initial three-dimensional signal.

[0088] The initial 3D signal is input into the Physically Perceptive Mask Attention U-Net reconstruction network, which outputs a reconstructed hyperspectral image, such as... Figure 1 As shown, it specifically includes:

[0089] S1, Embedding Layer: High-Dimensional Mapping of Initial 3D Signal

[0090] The initial 3D signal is transformed into a high-dimensional feature representation using an embedding layer (1×1 convolution), denoted as . This allows the input to fit into the feature processing space within the model.

[0091] S2, Encoder: Hierarchical Feature Extraction

[0092] By leveraging the Frequency Aware Mask Attention Module (FMAB) and a hierarchical downsampling approach, multi-scale hierarchical features are extracted:

[0093] Phase 1 (Shallow Feature Processing): Features First, the input features pass through two FMAB modules, followed by downsampling (e.g., 3×3 convolution with a stride of 2), compressing the spatial size to H / 2×W / 2 and expanding the number of channels to 2C, where H, W, and C are the input features, respectively. The height, width, and number of channels are used to obtain intermediate features. .

[0094] Second stage (deep feature processing): the above intermediate features The signal flows through four FMAB modules to deepen semantic information, further downsamples, compresses the spatial size to H / 4×W / 4, and expands the number of channels to 4C, thus obtaining deep features. .

[0095] S3, Bottleneck Layer: Deep Feature Fusion

[0096] Four FMAB modules are set up at the bottleneck layer to process deep features. (Dimensions H / 4×W / 4×4C) Conduct cross-channel interaction and multi-scale feature fusion to capture global dependencies and generate deeply refined bottleneck features.

[0097] S4, Decoder: Upsampling + Skip Connection Aggregation

[0098] The feature size is restored and cross-layer information is aggregated through "upsampling + skip connections (associated encoder FMAB output) + convolution + FMAB", which is carried out in two stages:

[0099] Phase 1: Mid-level Feature Recovery and Fusion

[0100] After the bottleneck feature undergoes an upsampling operation (transposed convolution with step 2), its spatial size is restored to H / 2×W / 2, and the number of channels is adjusted to 2C. At this point, the first skip connection is triggered, and the upsampled feature is compared with the feature in the encoder. The features (dimensions H / 2×W / 2×2C) output after processing by 4 FMAB modules are concatenated to integrate shallow semantic details and deep bottleneck features.

[0101] The concatenated features (dimensions H / 2×W / 2×4C) are processed through a 1×1 convolutional layer to compress the number of channels to 2C, and then through four FMAB modules to enhance feature consistency before outputting the features. .

[0102] Phase 2: Shallow Feature Recovery and Fusion

[0103] Output features After upsampling (transposed convolution with stride 2), the spatial dimensions are restored to H×W and the number of channels is adjusted to C;

[0104] At this point, the second skip connection is triggered, and the upsampled features are concatenated with the features (dimensions H×W×C) output by the encoder after processing by two FMAB modules in the first stage, to supplement the most original spatial structure information.

[0105] The concatenated features (dimensions H×W×2C) are processed through a 1×1 convolutional layer to compress the number of channels to C, and then flow through two FMAB modules to optimize the feature representation, resulting in the output. This is to prepare for subsequent processing.

[0106] S5, Mapping Layer: Output Feature Reduction

[0107] Using a 1×1 convolutional layer, the decoder finally outputs features The number of channels is converted back to the target spectral dimension, and then residual connection is performed with the original input features to generate a reconstructed hyperspectral image, ensuring spectral dimension matching and preserving the complementarity of the original information.

[0108] In this embodiment, as Figure 2 As shown, the specific processing procedure of the FMAB module is as follows:

[0109] The input features are separated into high and low frequencies to guide the output of high-frequency and low-frequency features.

[0110] High-frequency features and low-frequency features are input into the selective multi-head spatial attention module, respectively, to obtain high-frequency attention feature maps and low-frequency attention maps.

[0111] The high-frequency attention feature map and the low-frequency attention feature map are respectively input into the mask attention module, and a dynamic weight map of the high-fidelity region is generated based on the physical transmittance and position encoding of the modulation mask.

[0112] The high-frequency attention feature map is enhanced by masking based on the dynamic weight map, the low-frequency attention feature map is moderately modulated, and a fused feature map is obtained by fusion.

[0113] The fused feature map is output after being aligned with residual connections and anti-dispersion.

[0114] In this embodiment, a frequency separation guidance strategy is introduced, and the specific steps are as follows:

[0115] High-frequency components are extracted from the input features using 3×3 small-scale convolution. It captures edge and texture details.

[0116] Low-frequency components in the input features are extracted using 7×7 large-scale convolution. Model global structural information.

[0117] High- and low-frequency characteristics are input into subsequent modules as independent paths to achieve separate modeling, avoid mutual interference, and simulate the physical modulation process.

[0118] In this embodiment, high-frequency features and low-frequency features are input into the selective multi-head spatial attention module, respectively, to obtain high-frequency attention feature maps and low-frequency attention feature maps.

[0119] The specific processing steps of the selective multi-head spatial attention module are as follows:

[0120] S101. Use S convolutional kernels of different scales to extract multi-scale features to adapt to the diversity of target scales.

[0121] S102. Each scale feature is divided into L heads for parallel processing. The i-th scale and j-th head convolution operation is performed as follows:

[0122]

[0123] Where Conv represents the convolution operation. , X is the input feature. This represents the kernel size at the i-th scale;

[0124] By concatenating the outputs of all heads at each scale, an intermediate feature map is obtained. :

[0125]

[0126] S103. The intermediate feature map is mapped through a 1×1 convolutional layer to obtain feature maps of different scales. :

[0127]

[0128] Softmax calculates the weights and obtains Corresponding weight matrix :

[0129]

[0130] Weighted summation generates output feature map :

[0131]

[0132] By dynamically adjusting weights through multi-scale multi-head convolution and attention, feature representation is enhanced, such as... Figure 3 As shown.

[0133] Here This is the output of the selective multi-head spatial attention module, used here. It is a general expression, which can actually be derived from... Figure 2 In the middle view, the corresponding output after the above processing of high and low frequency features is the high-frequency attention feature map. and low-frequency attention feature map .

[0134] In this embodiment, the high-frequency attention feature map and the low-frequency attention feature map are respectively input into the mask attention module. A dynamic weight map of the high-fidelity region is generated based on the physical transmittance and position encoding of the modulation mask to guide feature enhancement. The specific steps are as follows:

[0135] S201. Using the high-frequency attention feature map and the low-frequency attention feature map as input feature maps, the modulation mask of the input feature maps is obtained through a dispersion operation. More expressive feature maps are obtained through two 1×1 convolutional layers. :

[0136]

[0137] S202, Introduce learnable positional encoding ε, and combine it with the feature map. The data is concatenated and then subjected to 3×3 convolution, ReLU activation, and another 3×3 convolution to generate a weight map. :

[0138]

[0139] This invention introduces learnable positional encoding and integrates it with feature maps. The features are concatenated to form a physical prior enhanced feature map; a high-fidelity weight map is generated using 3×3 convolution and sigmoid activation to represent the importance of different locations.

[0140] S203, Weighted graph With feature map Fusion and with feature maps Residual connections enhance expressive power:

[0141]

[0142] Wherein, λ is an adjustable parameter that ensures synergy between the original and enhanced features. This indicates a fusion operation, and + indicates a residual connection;

[0143] S204, Output feature map Invert the dispersion to obtain a dynamic weight map. ,like Figure 4 As shown.

[0144] In this embodiment, a mask enhancement is performed on the high-frequency attention feature map based on the dynamic weight map, and the low-frequency attention feature map is moderately modulated. A fused feature map is then obtained through fusion, including:

[0145] Masking enhancement of high-frequency attention feature maps based on dynamic weight maps yields enhanced high-frequency features. :

[0146]

[0147] in, Indicates a fusion operation. Represents a dynamic weighted graph. Represents high-frequency attention feature maps;

[0148] By appropriately modulating the low-frequency attention feature map based on the dynamic weight map, enhanced low-frequency features are obtained. :

[0149]

[0150] Where α represents the learnable parameter, with an initial value set to 0.5. Represents low-frequency attention feature maps;

[0151] Adding the enhanced high-frequency features and the enhanced low-frequency features yields the fused features. ;

[0152]

[0153] Finally, a frequency-aware mask attention module is formed by integrating 1×1 convolutional layers to simulate the adaptive modulation and selective attention of the physical system, ensuring both detailed and global information.

[0154] In this embodiment, the fused feature map is restored to the reconstructed hyperspectral image output through residual connection and anti-dispersion alignment operations.

[0155] The fused feature map is restored to the hyperspectral image dimension through convolutional mapping and residual connection. The spatial position is aligned by anti-dispersion operation to ensure that the output image is consistent with the geometric structure of the original scene. Finally, the reconstructed hyperspectral image H' is generated, completing the entire reconstruction process.

[0156] The experiment used 32 different CAVE datasets to randomly crop out 256×256 images and generated compressed images using the same size real mask to train the model. Ten 256×256 scenes were cropped from the KAIST dataset of 30 different scenes for testing, namely scene 1 to scene 10. This experiment compared 11 mainstream reconstruction algorithms, namely: Two-Step Iterative Shrinkage / Thresholding algorithm (TwIST), Generalized Alternating Projection based Total Variation Minimization algorithm (GAP-TV), Alternating Direction Method of Multipliers (ADMM-TV), U-Net, Dual-Stage Generative Network (λ-net), Spatial-Spectral Self-Attention Network (TSA-Net), Generalized Alternating Projection Network (GAP-Net), Alternating Direction Method of Multipliers Network (ADMM-Net), Deep Gaussian Scale Mixture Prior Model (DGSMP), and High-Resolution Dual-Domain Learning Network. Dual-Domain Learning Network (HDNet), Mask-guided Spectral-wise Transformer Network (MST-L), and the frequency separation and physical sensing network of this invention (Ours), such as Figure 5 As shown, a visual comparison of the reconstruction results for Scene 7 is presented. At wavelengths of 487.0, 536.5, 575.3, and 648.1 nm, the images reconstructed by this method, as seen in the magnified image within the yellow box, show that the rose stem is clearer and closer to the true value than other algorithms, with more complete image edges and significantly reduced visual artifacts. Figure 6 , Figure 7 The figures shown are the spectral feature curves of the reconstructed hyperspectral images of blocks a and b in scenes 1 and 8, respectively. Figure 6 As shown in Figure (a), in the spectral feature curve of the reconstructed hyperspectral image of scene 1 block a, the correlation coefficients are as follows: λ-net 0.9664, TSA-Net 0.9535, GAP-Net 0.9517, DGSMP 0.9588, ADMM-Net 0.9734, HDNet 0.9507, MST-L 0.9588, and the correlation coefficient of this invention 0.984. Figure 6 As shown in Figure (b), in the spectral feature curve of the reconstructed hyperspectral image of block b in scene 1, the correlation coefficients are as follows: λ-net 0.9799, TSA-Net 0.9709, GAP-Net 0.9716, DGSMP 0.9196, ADMM-Net 0.9618, HDNet 0.9595, MST-L 0.9719, and the correlation coefficient of this invention 0.9902. Figure 7 As shown in Figure (a), in the spectral feature curves of the reconstructed hyperspectral images of scene 8 blocks a, the correlation coefficients are as follows: λ-net 0.9171, TSA-Net 0.9915, GAP-Net 0.8967, DGSMP 0.873, ADMM-Net 0.9422, HDNet 0.9054, MST-L 0.9798, and the correlation coefficient of this invention 0.9863. Figure 7 As shown in (b), in the spectral feature curves of the reconstructed hyperspectral image of scene 8 b, the correlation coefficients are as follows: λ-net 0.6379, TSA-Net 0.7638, GAP-Net 0.8729, DGSMP 0.9738, ADMM-Net 0.9175, HDNet 0.9403, MST-L 0.9597, and the correlation coefficient of this invention 0.9792. The spectral curves of this method show the highest similarity to the original spectra, and the correlation coefficients in the labeled areas are superior to those of the comparison methods. Figure 8 As shown in the figure, the reconstruction results of real data show that the reconstruction effect of this method is better. The artifacts, blurring and distortion in complex target areas (such as the center of the pentagram) are significantly reduced, and the visual effect is closer to the actual scene.

[0157] Quantitative experimental results are shown in Tables 1 and 2. The proposed method achieves an average PSNR of 36.93 dB and an SSIM of 0.962 across 10 test scenarios, demonstrating significant improvements over traditional methods (TwIST, GAP-TV, ADMM-TV) and mainstream deep learning methods (λ-Net, TSA-Net, MST-L, etc.). For example, compared to the MST-L method, PSNR is improved by 1.75 dB and SSIM by 0.014. Table 3 shows a comparison of algorithm efficiency and performance. The proposed method has an inference time of 0.0201 s, which is superior to most of the compared methods, achieving a balance between efficiency and performance.

[0158] Table 1

[0159]

[0160] Table 2

[0161]

[0162] Table 3

[0163]

[0164] The effectiveness verification of key modules is shown in Tables 4 and 5: After removing the mask attention module, PSNR decreased by 1.63 dB and SSIM decreased by 0.013; after removing the selective multi-head spatial attention module, PSNR decreased by 1.52 dB and SSIM decreased by 0.019; if both are removed simultaneously, PSNR significantly decreased by 3.97 dB and SSIM decreased by 0.046. A comparison of different mask injection mechanisms is shown in Table 5. Compared with the traditional direct multiplication (H*M) and mask guidance (MM) methods, the mask attention strategy of this method improves PSNR by 1.29 dB and 0.29 dB respectively, demonstrating the superiority of physical perception mask attention.

[0165] Table 4

[0166]

[0167] Table 5

[0168]

[0169] This invention provides a hyperspectral image reconstruction system based on frequency separation and physical perception attention, such as... Figure 9 As shown, it includes:

[0170] Preprocessing unit: acquires compressed snapshot observation images and restores the two-dimensional compressed snapshot observation images to the initial three-dimensional signal through inverse dispersion operation;

[0171] Frequency separation and physical perception attention unit: The initial 3D signal is input into the physical perception mask attention U-Net reconstruction network, and the output is a reconstructed hyperspectral image, specifically including:

[0172] The initial three-dimensional signal is converted into a high-dimensional feature representation through an embedding layer;

[0173] The high-dimensional feature representation in the encoder stage is obtained by sequentially passing through 2 FMAB modules, downsampling, 4 FMAB modules, and downsampling operations to obtain deep features;

[0174] The bottleneck layer uses four FMAB modules to perform cross-channel interaction and multi-scale fusion of deep features to generate bottleneck features;

[0175] After upsampling, the bottleneck features are concatenated with the corresponding layer features processed by the four FMAB modules in the encoder, fusing deep semantics and mid-level details, and then optimized by convolution and the four FMAB modules.

[0176] After the optimized features are upsampled again, they are concatenated with the features initially processed by the two FMAB modules in the encoder to supplement the original spatial structure information, and then optimized by convolution and two FMAB modules.

[0177] The feature dimensions are restored by mapping layers and added to the residual of the initial three-dimensional signal to generate a reconstructed hyperspectral image.

[0178] The specific implementation process and method of the system of this invention are the same, please refer to the method section, and will not be repeated here.

[0179] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements a hyperspectral image reconstruction method based on frequency separation and physical perception attention.

[0180] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a hyperspectral image reconstruction method based on frequency separation and physical perception attention.

[0181] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0182] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0183] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hyperspectral image reconstruction method based on frequency separation and physical perception attention, characterized in that, include: Acquire compressed snapshot observation images and restore the two-dimensional compressed snapshot observation images to the initial three-dimensional signal through inverse dispersion operation; The initial 3D signal is input into the Physically Aware Mask Attention U-Net reconstruction network, which outputs a reconstructed hyperspectral image, specifically including: The initial three-dimensional signal is converted into a high-dimensional feature representation through an embedding layer; The high-dimensional feature representation in the encoder stage is obtained by sequentially passing through 2 FMAB modules, downsampling, 4 FMAB modules, and downsampling operations to obtain deep features; The bottleneck layer uses four FMAB modules to perform cross-channel interaction and multi-scale fusion of deep features to generate bottleneck features; After upsampling, the bottleneck features are concatenated with the corresponding layer features processed by the four FMAB modules in the encoder, fusing deep semantics and mid-level details, and then optimized by convolution and the four FMAB modules. After the optimized features are upsampled again, they are concatenated with the features initially processed by the two FMAB modules in the encoder to supplement the original spatial structure information, and then optimized by convolution and two FMAB modules. The feature dimensions are restored by mapping layers and added to the initial three-dimensional signal residual to generate a reconstructed hyperspectral image; The specific processing steps of the FMAB module include: The input features are separated into high and low frequencies to guide the output of high-frequency and low-frequency features. High-frequency features and low-frequency features are input into the selective multi-head spatial attention module, respectively, to obtain high-frequency attention feature maps and low-frequency attention maps. The high-frequency attention feature map and the low-frequency attention feature map are respectively input into the mask attention module, and a dynamic weight map of the high-fidelity region is generated based on the physical transmittance and position encoding of the modulation mask. The high-frequency attention feature map is enhanced by masking based on the dynamic weight map, the low-frequency attention feature map is moderately modulated, and a fused feature map is obtained by fusion. The fused feature map is output after being aligned with the inverse dispersion through residual connections. The high-frequency attention feature map and the low-frequency attention feature map are respectively input into the mask attention module. A dynamic weight map of the high-fidelity region is generated based on the physical transmittance and positional encoding of the modulation mask, including: S201. Using the high-frequency attention feature map and the low-frequency attention feature map as input feature maps, the input feature map modulation mask is subjected to dispersion operation to obtain the feature map. Feature maps are obtained through two convolutional layers. : ； Where Conv represents the convolution operation; S202, Combine the learnable location code ε with the feature map The data is concatenated and then subjected to convolution, ReLU activation, and convolution operations to generate a weight map. : ； Where Sigmoid represents Sigmoid activation, and Concat represents concatenation operation; S203, Weighted graph With feature map Fusion and with feature maps Residual connections yield the output feature map : ； Where λ is an adjustable parameter. This indicates a fusion operation, and + indicates a residual connection; S204, Output feature map Invert the dispersion to obtain a dynamic weight map. .

2. The hyperspectral image reconstruction method based on frequency separation and physical perception attention according to claim 1, characterized in that, The input features are separated into high-frequency and low-frequency features to guide the output, which include: High-frequency components are extracted from the input features using 3×3 convolution, and high-frequency features are output. Low-frequency components are extracted from the input features using 7×7 convolution, and low-frequency features are output.

3. The hyperspectral image reconstruction method based on frequency separation and physical perception attention according to claim 1, characterized in that, The selective multi-head spatial attention module processing procedure is as follows: S101, Use S Different scale convolutional kernels extract multi-scale features; S102. Divide each scale feature into... L Each head is processed in parallel, the first i The first scale j Individual convolution operations: ； Where Conv represents the convolution operation. , X is the input feature. Indicates the first i The size of the convolutional kernel on each scale, Indicates the first i The first scale j The output of each head convolution operation; splicing each scale L Output the head to obtain the intermediate feature map. : ； Wherein, Concat represents the concatenation operation; S103. The intermediate feature map is mapped through a convolutional layer to obtain feature maps of different scales. : ； The weights are calculated using Softmax to obtain the feature map. Corresponding weight matrix : ； Where exp represents the exponential function; Weighted summation generates output feature map : 。 4. The hyperspectral image reconstruction method based on frequency separation and physical perception attention according to claim 1, characterized in that, High-frequency attention feature maps are masked and enhanced based on dynamic weight maps, while low-frequency attention feature maps are moderately modulated. A fused feature map is then obtained through fusion, including: Masking enhancement of high-frequency attention feature maps based on dynamic weight maps yields enhanced high-frequency features. : ； in, Indicates a fusion operation. Represents a dynamic weighted graph. Represents high-frequency attention feature maps; By appropriately modulating the low-frequency attention feature map based on the dynamic weight map, enhanced low-frequency features are obtained. : ； in, α Represents learnable parameters. Represents low-frequency attention feature maps; Adding the enhanced high-frequency features and the enhanced low-frequency features yields the fused features. ; ； Fusion features The fused feature map is output through 1×1 convolution.

5. A hyperspectral image reconstruction system for frequency separation and physical perception attention, used to implement the method described in any one of claims 1-4, characterized in that, include: Preprocessing unit: acquires compressed snapshot observation images and restores the two-dimensional compressed snapshot observation images to the initial three-dimensional signal through inverse dispersion operation; Frequency separation and physical perception attention unit: The initial 3D signal is input into the physical perception mask attention U-Net reconstruction network, and the output is a reconstructed hyperspectral image, specifically including: The initial three-dimensional signal is converted into a high-dimensional feature representation through an embedding layer; The high-dimensional feature representation in the encoder stage is obtained by sequentially passing through 2 FMAB modules, downsampling, 4 FMAB modules, and downsampling operations to obtain deep features; The bottleneck layer uses four FMAB modules to perform cross-channel interaction and multi-scale fusion of deep features to generate bottleneck features; After upsampling, the bottleneck features are concatenated with the corresponding layer features processed by the four FMAB modules in the encoder, fusing deep semantics and mid-level details, and then optimized by convolution and the four FMAB modules. After the optimized features are upsampled again, they are concatenated with the features initially processed by the two FMAB modules in the encoder to supplement the original spatial structure information, and then optimized by convolution and two FMAB modules. The feature dimensions are restored by mapping layers and added to the residual of the initial three-dimensional signal to generate a reconstructed hyperspectral image.

6. A computer device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor, when executing the computer program, implements the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 4.

Images