A hyperspectral compressive imaging method, system, terminal and storage medium

Abstract

The application discloses a hyperspectral compression imaging method, system, terminal and storage medium in the technical field of hyperspectral image processing, and aims to solve the problem of low reconstruction accuracy caused by high-frequency detail loss, weak spectral correlation and poor noise resistance in the compression imaging process in the prior art. It comprises obtaining the compression measurement and the perception mask of the coded aperture snapshot spectral imaging system, obtaining the initial feature map according to the compression measurement and the perception mask; inputting the initial feature map into the pre-constructed U-Net model to obtain the decoding feature map; and performing convolution mapping on the decoding feature map to obtain the reconstructed hyperspectral image in combination with the initial feature map; the spectral-space state synchronizer efficiently processes long sequence data, and the wavelet variance modulation block can effectively capture the local and global features of the image through multi-scale decomposition, thereby solving the problem of low reconstruction accuracy caused by high-frequency detail loss, weak spectral correlation and poor noise resistance in the traditional compression imaging process.

Description

A hyperspectral compressed imaging method, system, terminal, and storage medium Technical Field

[0001] This invention relates to a hyperspectral compressed imaging method, system, terminal, and storage medium, belonging to the field of hyperspectral image processing technology. Background Technology

[0002] Hyperspectral imaging technology captures information about objects across multiple consecutive spectral bands, forming a high-dimensional data cube that provides detailed spectral features for fields such as remote sensing, agriculture, and medicine. However, traditional systems require band-by-band scanning, resulting in long imaging times, large data volumes, and high transmission and storage costs, making it difficult to meet the needs of dynamic scenes and real-time applications.

[0003] To address these issues, compressed sensing theory has been introduced into the field of hyperspectral imaging, reconstructing complete images with a small number of measurements. Among these, coded aperture snapshot spectral imaging (CASSI) is the mainstream hardware solution. It uses a coded mask to modulate the spatial light field and, in conjunction with a dispersive prism, superimposes multi-band information in a staggered manner, forming a compressed measurement on a single-frame detector, achieving snapshot acquisition and significantly reducing the amount of data.

[0004] Traditional reconstruction methods mainly rely on optimization algorithms (such as total variation minimization and sparse representation) or early convolutional networks. Although they can achieve a certain degree of recovery, they have significant drawbacks such as loss of high-frequency details, blurred edges, insufficient spectral-space interaction, sensitivity to noise, and low computational efficiency. Especially in low sampling rates or complex scenarios, the reconstruction quality is difficult to meet the requirements of high-fidelity applications.

[0005] In summary, existing hyperspectral compression imaging methods suffer from problems such as loss of high-frequency details, weak spectral correlation, and poor noise resistance during the compression imaging process, resulting in low reconstruction accuracy and affecting practical application effects. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a hyperspectral compressed imaging method, system, terminal, and storage medium. It efficiently processes long sequence data through a spectral-spatial state synchronizer, and the wavelet variance modulation block effectively captures local and global features of the image through multi-scale decomposition, achieving high-fidelity spectral-spatial reconstruction of compressed measurements in a coded aperture snapshot spectral imaging system. While improving reconstruction resolution, it fully preserves spectral information. By fully extracting spectral and spatial information through the wavelet variance modulation block and the spectral-spatial state synchronizer, it achieves efficient end-to-end reconstruction from compressed measurements to hyperspectral images, improving the accuracy of feature representation and obtaining higher-quality reconstructed hyperspectral images. This solves the problems of low reconstruction accuracy caused by high-frequency detail loss, weak spectral correlation, and poor noise resistance in traditional compressed imaging processes.

[0007] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0008] In a first aspect, the present invention provides a hyperspectral compressed imaging method, comprising:

[0009] Obtain compressed measurement and sensing masks from the coded aperture snapshot spectral imaging system, and acquire initial feature maps based on the compressed measurement and sensing masks;

[0010] The initial feature map is input into the pre-built U-Net model to obtain the decoded feature map;

[0011] The decoded feature map is convolved and combined with the initial feature map to obtain the reconstructed hyperspectral image;

[0012] The U-Net model includes:

[0013] The initial feature map is processed using a pre-constructed wavelet variance modulation block to obtain the initial coded feature map and statistical hints;

[0014] Based on wavelet variance modulation blocks and statistical cueing, the initial coded feature map is downsampled to obtain the final coded feature map and statistical bridging.

[0015] A pre-built spectral-spatial state synchronizer is used to process the final encoded feature map to obtain the bottleneck feature map;

[0016] Multi-stage decoding of the bottleneck feature map is performed to obtain the decoded feature map;

[0017] The pre-built spectral-spatial state synchronizer includes:

[0018] Based on the final encoded feature map, obtain the state selection mask;

[0019] Dimensionality reduction and adaptive pooling are performed on the final encoded feature map to obtain a low-resolution sequence;

[0020] Based on state selection mask and statistical bridging, a selective scan of the low-resolution sequence is performed to obtain the weighted state vector of the scan.

[0021] The weighted state vectors of the scan are reversed and sorted, and the original resolution is restored by combining layer normalization to obtain the bottleneck feature map.

[0022] Furthermore, the process of using a pre-constructed wavelet variance modulation block to process the initial feature map to obtain the initial coded feature map and statistical hints includes:

[0023] Perform discrete wavelet transform on the initial feature map to obtain low-frequency and high-frequency sub-bands;

[0024] Directional variance modulation is performed on the low-frequency subband to obtain the processed low-frequency subband, and high-frequency dense convolutional coding is performed on the high-frequency subband to obtain the processed high-frequency subband.

[0025] An initial encoded feature map is generated based on the processing of low-frequency and high-frequency subbands. Statistical hints are then obtained based on the processing of low-frequency and high-frequency subbands.

[0026] The step of obtaining statistical prompts based on processing low-frequency and high-frequency sub-bands includes:

[0027] Based on the processing of low-frequency sub-bands, obtain the channel variance of the low-frequency sub-bands;

[0028] The mean value of the high-frequency sub-band is obtained from the high-frequency sub-band.

[0029] Based on the channel variance of the low-frequency sub-band and the mean of the high-frequency sub-band, the statistical results are calculated as follows:

[0030]

[0031] In the formula: For statistical purposes, To handle the channel variance of the low-frequency subband, The mean of the high-frequency subband; Linear represents the linear projection layer; cat represents the splicing operation.

[0032] Furthermore, the step of performing directional variance modulation on the low-frequency sub-band to obtain the processed low-frequency sub-band includes:

[0033] Obtain the spatial mean and variance of the low-frequency sub-band, and calculate the variance coefficient based on the spatial mean and variance of the low-frequency sub-band. The specific expression is as follows:

[0034]

[0035] In the formula: The variance coefficient, For low-frequency sub-band, Spatial mean For variance;

[0036] The gradient magnitude is calculated using the Sobel operator, and the specific expression is as follows:

[0037]

[0038] In the formula: This represents the gradient magnitude. The horizontal gradient of the image; The vertical gradient of the image;

[0039] The modulation weights are calculated based on the variance coefficients and gradient magnitudes, as shown in the following expression:

[0040]

[0041] In the formula: For modulation weights, These are learnable parameters; Use the Sigmoid activation function;

[0042] The low-frequency sub-band is obtained based on the low-frequency sub-band and modulation weights, and the specific expression is as follows:

[0043]

[0044] In the formula: To handle the low-frequency subband.

[0045] Furthermore, the downsampling includes primary downsampling and secondary downsampling;

[0046] The process of downsampling the initial encoded feature map based on wavelet variance modulation blocks and statistical cues to obtain the final encoded feature map and statistical bridging includes:

[0047] Perform a downsampling on the initial encoded feature map to obtain a downsampling result map;

[0048] A pre-constructed wavelet variance modulation block is used to process the first downsampling result image to obtain a first coded feature map and a first statistical cue;

[0049] Perform a second downsampling on the first encoded feature map to obtain the second downsampling result map;

[0050] The pre-constructed wavelet variance modulation block is used to process the secondary downsampling result image to obtain the final encoded feature map and secondary statistical hints;

[0051] Obtain statistical bridging based on statistical prompts, primary statistical prompts, and secondary statistical prompts.

[0052] Furthermore, obtaining the state selection mask based on the final encoded feature map includes:

[0053] Perform based on the final encoded feature map The class label mapping is used to generate a tokenized routing policy, and a state selection mask is generated based on the tokenized routing policy. The specific expression is as follows:

[0054]

[0055]

[0056] In the formula: For tokenized routing strategies, Select a mask for the state; x is the final encoded feature map; B is the batch size; N is the sequence length; K is the preset number of discrete label categories; GumbelSample is a differentiable sampling operation based on the Gumbel distribution; LogSoftmax is a log-normalized exponential function; Linear represents a linear projection layer;

[0057] The final encoded feature map is subjected to dimensionality reduction and adaptive pooling to obtain a low-resolution sequence, as shown in the following expression:

[0058]

[0059] In the formula: For low-resolution sequences; AdaptiveAvgPool2d represents a two-dimensional adaptive average pooling operation.

[0060] Furthermore, the step of selectively scanning the low-resolution sequence based on state selection mask and statistical bridging to obtain the weighted state vector of the scan includes:

[0061] The state selection mask is used to address and extract the corresponding state embedding from a preset embedding library, and then fused with statistical bridging to generate an initial prompt. The specific expression is as follows:

[0062]

[0063] In the formula: This is the initial prompt. This is a pre-defined embedded library; For statistical bridging;

[0064] The initial cue is injected into the state-space model, the scan is initiated, and the original hidden state vector is generated, as shown in the following expression:

[0065]

[0066] In the formula: The original hidden state vector, This is the original state projection vector obtained by linearly projecting the encoded feature map;

[0067] A spectral-spatial attention model is constructed based on the low-resolution sequence, and weights are calculated. The original hidden state vector is then subjected to weighted modulation using these weights to obtain a weighted state vector. The specific expression is as follows:

[0068]

[0069]

[0070] In the formula: For weighted state vectors, As weight, and All of these are learnable equilibrium parameters. The sigmoid activation function is used; MLP represents a multilayer perceptron. and These represent average pooling along the spectral and spatial dimensions, respectively;

[0071] Selective scanning is performed based on the weighted state vector to obtain the scanned weighted state vector. The specific expression is as follows:

[0072]

[0073] In the formula: This is the weighted state vector for the scan. The joint variance of the state vectors, This is an indicator function.

[0074] Furthermore, the multi-stage decoding of the bottleneck feature map to obtain the decoded feature map includes:

[0075] The bottleneck feature map is upsampled using transposed convolution, and a pre-constructed wavelet variance modulation block is repeatedly applied. The skip connection features of the corresponding downsampled stage are fused by convolution to output the decoded feature map.

[0076] In a second aspect, the present invention provides a hyperspectral compressed imaging system, comprising:

[0077] Initial fusion module: used to acquire compressed measurement and sensing masks from the coded aperture snapshot spectral imaging system, and to obtain initial feature maps based on the compressed measurement and sensing masks;

[0078] Output module: Used to input the initial feature map into the pre-built U-Net model to obtain the decoded feature map;

[0079] The residual mapping module is used to perform convolutional mapping on the decoded feature map and combine it with the initial feature map to obtain the reconstructed hyperspectral image.

[0080] Thirdly, the present invention provides a terminal, including a processor and a storage medium;

[0081] The storage medium is used to store instructions;

[0082] The processor is configured to operate according to the instructions to perform the steps of the method according to the first aspect.

[0083] Fourthly, a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect.

[0084] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0085] This hyperspectral compressed imaging method efficiently processes long sequence data through a spectral-spatial state synchronizer. The wavelet variance modulation block, through multi-scale decomposition, effectively captures local and global features of the image, achieving high-fidelity spectral-spatial reconstruction of compressed measurements in the coded aperture snapshot spectral imaging system. While improving reconstruction resolution, it fully preserves spectral information. By fully extracting spectral and spatial information through the wavelet variance modulation block and the spectral-spatial state synchronizer, it achieves efficient end-to-end reconstruction from compressed measurement to hyperspectral image, improving the accuracy of feature representation and obtaining higher-quality reconstructed hyperspectral images. This method solves the problems of low reconstruction accuracy caused by loss of high-frequency details, weak spectral correlation, and poor noise resistance in traditional compressed imaging. Attached Figure Description

[0086] Figure 1 is a flowchart illustrating a hyperspectral compressed imaging method according to an embodiment of the present invention;

[0087] Figure 2 is a schematic diagram of the framework of the U-Net model provided according to an embodiment of the present invention;

[0088] Figure 3 is a schematic diagram of the framework of a spectrum-space state synchronizer provided according to an embodiment of the present invention;

[0089] Figure 4 is a schematic diagram of the frame of the wavelet variance modulation block provided in an embodiment of the present invention;

[0090] Figure 5 shows reconstructed hyperspectral images in three different bands in a KAIST dataset test scenario using a hyperspectral compression imaging method provided by an embodiment of the present invention. Detailed Implementation

[0091] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0092] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0093] Example 1:

[0094] As shown in Figures 1-4, the present invention provides a hyperspectral compressed imaging method, comprising:

[0095] Obtain compressed measurement and sensing masks from the coded aperture snapshot spectral imaging system, and acquire initial feature maps based on the compressed measurement and sensing masks;

[0096] The initial feature map is input into the pre-built U-Net model to obtain the decoded feature map;

[0097] The decoded feature map is convolved and combined with the initial feature map to obtain the reconstructed hyperspectral image;

[0098] The U-Net model includes:

[0099] The initial feature map is processed using a pre-constructed wavelet variance modulation block to obtain the initial coded feature map and statistical hints;

[0100] Based on wavelet variance modulation blocks and statistical cueing, the initial coded feature map is downsampled to obtain the final coded feature map and statistical bridging.

[0101] A pre-built spectral-spatial state synchronizer is used to process the final encoded feature map to obtain the bottleneck feature map;

[0102] Multi-stage decoding of the bottleneck feature map is performed to obtain the decoded feature map;

[0103] The pre-built spectral-spatial state synchronizer includes:

[0104] Based on the final encoded feature map, obtain the state selection mask;

[0105] Dimensionality reduction and adaptive pooling are performed on the final encoded feature map to obtain a low-resolution sequence;

[0106] Based on state selection mask and statistical bridging, a selective scan of the low-resolution sequence is performed to obtain the weighted state vector of the scan.

[0107] The weighted state vectors of the scan are reversed and sorted, and the original resolution is restored by combining layer normalization to obtain the bottleneck feature map.

[0108] Specifically, compressed measurement and sensing masks are obtained from the coded aperture snapshot spectral imaging system. The input compressed measurement and sensing masks are then concatenated along the channel axis using a 1×1 convolution to form an initial feature map. The initial feature map is then subjected to a 3×3 convolution to embed features, and reflection filling is used to make the spatial size divisible by 8.

[0109] As shown in Figure 2, C in the figure represents a connection. This invention adopts a multi-scale U-Net architecture. First, compressed measurement and sensing mask are mapped to an initial feature map. Then (encoding layer), the initial feature map is processed through multiple encoding layers. Each layer extracts frequency domain features through wavelet variance modulation blocks and reduces spatial resolution through downsampling. At the same time, statistical bridging extracts mean and variance information from features at each level to generate statistical cues. Subsequently (bottleneck layer), the features of the deepest layer are input to the spectrum-spatial state synchronizer. Statistical cues and discretization routing strategies are used to denoise and enhance the global context, generating a bottleneck feature map. Finally (decoding layer), the bottleneck feature map is processed through multiple decoding layers. Resolution is restored through upsampling and fused with the features corresponding to the encoding layer. Finally, the hyperspectral image is reconstructed by outputting residual mapping.

[0110] In this embodiment, the step of processing the initial feature map using a pre-constructed wavelet variance modulation block to obtain the initial coded feature map and statistical hints includes:

[0111] Perform discrete wavelet transform on the initial feature map to obtain low-frequency and high-frequency sub-bands;

[0112] Directional variance modulation is performed on the low-frequency subband to obtain the processed low-frequency subband, and high-frequency dense convolutional coding is performed on the high-frequency subband to obtain the processed high-frequency subband.

[0113] An initial encoded feature map is generated based on the processing of low-frequency and high-frequency subbands. Statistical hints are then obtained based on the processing of low-frequency and high-frequency subbands.

[0114] The step of obtaining statistical prompts based on processing low-frequency and high-frequency sub-bands includes:

[0115] Based on the processing of low-frequency sub-bands, obtain the channel variance of the low-frequency sub-bands;

[0116] The mean value of the high-frequency sub-band is obtained from the high-frequency sub-band.

[0117] Based on the channel variance of the low-frequency sub-band and the mean of the high-frequency sub-band, the statistical results are calculated as follows:

[0118]

[0119] In the formula: For statistical purposes, To handle the channel variance of the low-frequency subband, The mean of the high-frequency subband; Linear represents the linear projection layer; cat represents the splicing operation.

[0120] The Discrete Wavelet Transform (DWT) decomposes the initial feature map into low-frequency and high-frequency sub-bands. The low-frequency sub-band enhances edge texture through directional variance modulation, while the high-frequency sub-band is encoded using high-frequency dense convolution, i.e., multi-layer... Convolution and cumulative residual connections enhance detailed features; the decomposition and enhancement process separately processes low-frequency structural information and high-frequency texture information to avoid the loss of high-frequency details in traditional methods.

[0121] Specifically, the channel variance of the low-frequency subband is first extracted and processed. The extraction process involves obtaining the global texture intensity and then extracting the mean value of the high-frequency subbands. The extraction process involves obtaining detailed distribution information, concatenating it, and mapping it to the state dimension through a linear transformation to obtain statistical hints p. The generation process of statistical hints p provides contextual guidance for the overall texture and details of the image, which is used for decision optimization of the subsequent spectral-spatial state synchronizer.

[0122] The architecture of the wavelet variance modulation block is shown in Figure 4. The initial coding feature map is generated based on the processing of low-frequency sub-bands and high-frequency sub-bands. This includes fusing the processing of low-frequency sub-bands and high-frequency sub-bands, processing them through a feedforward neural network, and finally performing an inverse wavelet transform to obtain the initial coding feature map.

[0123] In this embodiment, performing directional variance modulation on the low-frequency sub-band to obtain the processed low-frequency sub-band includes:

[0124] Obtain the spatial mean and variance of the low-frequency sub-band, and calculate the variance coefficient based on the spatial mean and variance of the low-frequency sub-band. The specific expression is as follows:

[0125]

[0126] In the formula: The variance coefficient, For low-frequency sub-band, Spatial mean For variance;

[0127] The gradient magnitude is calculated using the Sobel operator, and the specific expression is as follows:

[0128]

[0129] In the formula: This represents the gradient magnitude. The horizontal gradient of the image; The vertical gradient of the image;

[0130] The modulation weights are calculated based on the variance coefficients and gradient magnitudes, as shown in the following expression:

[0131]

[0132] In the formula: For modulation weights, The parameters are learnable to achieve adaptive edge enhancement, which amplifies the signal in regions with rich texture or obvious edges and suppresses noise in flat regions. The activation function is Sigmoid; the fusion process dynamically balances texture and edge information.

[0133] The low-frequency sub-band is obtained based on the low-frequency sub-band and modulation weights, and the specific expression is as follows:

[0134]

[0135] In the formula: To handle the low-frequency subband.

[0136] As shown in Figure 2, in this embodiment, the downsampling includes a first downsampling and a second downsampling;

[0137] The process of downsampling the initial encoded feature map based on wavelet variance modulation blocks and statistical cues to obtain the final encoded feature map and statistical bridging includes:

[0138] Perform a downsampling on the initial encoded feature map to obtain a downsampling result map;

[0139] A pre-constructed wavelet variance modulation block is used to process the first downsampling result image to obtain a first coded feature map and a first statistical cue;

[0140] Perform a second downsampling on the first encoded feature map to obtain the second downsampling result map;

[0141] The pre-constructed wavelet variance modulation block is used to process the secondary downsampling result image to obtain the final encoded feature map and secondary statistical hints;

[0142] Calculate the mean of the statistical prompts, primary statistical prompts, and secondary statistical prompts to obtain the statistical bridge.

[0143] As shown in Figure 3, the encoded feature map is the final encoded feature map. In this embodiment, obtaining the state selection mask based on the final encoded feature map includes:

[0144] Perform based on the final encoded feature map (A differentiable approximation method for simulating discrete variable sampling operations) uses class label mapping to generate a tokenized routing policy, and generates a state selection mask based on the tokenized routing policy. The specific expression is as follows:

[0145]

[0146]

[0147] In the formula: For tokenized routing strategies, Select a mask for the state; x is the final encoded feature map; B is the batch size; N is the sequence length; K is the preset number of discrete label categories, K=64 is optional; GumbelSample is a differentiable sampling operation based on the Gumbel distribution; LogSoftmax is a log-normalized exponential function; this process reduces the high-dimensional features to 64 learnable labels to reduce computational complexity.

[0148] In this embodiment, the step of performing dimensionality reduction and adaptive pooling on the final encoded feature map to obtain a low-resolution sequence is specifically expressed as follows:

[0149]

[0150] In the formula: For low-resolution sequences; AdaptiveAvgPool2d represents a two-dimensional adaptive average pooling operation, which reshapes the features into a spectral-spatial sequence to optimize subsequent scans.

[0151] In this embodiment, the step of selectively scanning the low-resolution sequence based on state selection mask and statistical bridging to obtain the weighted state vector of the scan includes:

[0152] The state selection mask is used to address and extract the corresponding state embedding from a preset embedding library, and then fused with statistical bridging to generate an initial prompt. The specific expression is as follows:

[0153]

[0154] In the formula: This is the initial prompt. This is a pre-defined embedded library; For statistical bridging;

[0155] The initial cue is injected into the state-space model, the scan is initiated, and the original hidden state vector is generated, as shown in the following expression:

[0156]

[0157] In the formula: The original hidden state vector, This is the original state projection vector obtained by linearly projecting the encoded feature map;

[0158] A spectral-spatial attention model is constructed based on the low-resolution sequence, and weights are calculated. The original hidden state vector is then subjected to weighted modulation using these weights to obtain a weighted state vector. The specific expression is as follows:

[0159]

[0160]

[0161] In the formula: For weighted state vectors, As weight, and All of these are learnable equilibrium parameters. The sigmoid activation function is used; MLP represents a multilayer perceptron. and These represent average pooling along the spectral and spatial dimensions, respectively; the weight generation process dynamically weights spectral channels and spatial locations to indicate importance.

[0162] Selective scanning is performed based on the weighted state vector to obtain the scanned weighted state vector. The specific expression is as follows:

[0163]

[0164] In the formula: This is the weighted state vector for the scan. The joint variance of the state vectors, This is an indicator function, and the process is to achieve efficient noise-resistant spectral-spatial feature synchronization.

[0165] In this embodiment, the step of performing multi-stage decoding on the bottleneck feature map to obtain the decoded feature map includes:

[0166] The bottleneck feature map is upsampled using transposed convolution, and a pre-constructed wavelet variance modulation block is repeatedly applied. The skip connection features of the corresponding downsampled stage are fused by convolution to output the decoded feature map.

[0167] Specifically, the bottleneck feature map is upsampled using transposed convolution, the upsampling process being to progressively enlarge the spatial size; the encoder skip connection features of the corresponding stage are fused through 1×1 convolution, the fusion process being to supplement high-frequency details and prevent information loss; discrete wavelet transform, directional variance modulation, and high-frequency dense convolution are repeatedly applied, the repetition process being to enhance spectral-spatial details at each scale; and a decoded feature map is output, the output process providing a complete feature representation for subsequent residual mapping.

[0168] In this embodiment, the decoded feature map is subjected to 3×3 convolution mapping, combined with the residual connection of the initial feature map and cropped to the original size to output the reconstructed hyperspectral image.

[0169] In some possible embodiments, 10 scenes are extracted from the KAIST dataset, each with a spatial size of 256×256 and 31 spectral channels. The test scenes include typical indoor and outdoor objects (such as fruits and fabrics). During the experiment, the original hyperspectral images of the above dataset are used as the ground truth. Compressed measurement values ​​are generated by simulating the physical mask modulation of the coded aperture snapshot spectral imaging system. Subsequently, the compressed measurements are processed using the flowchart shown in Figure 1. During the processing, the wavelet variance modulation block shown in Figure 4 effectively extracts the frequency domain features of the image at multiple scales, while the spectral-spatial state synchronizer shown in Figure 3 successfully captures long-distance spectral-spatial correlations and filters out state noise by using discretization routing and attention mechanisms. Figure 5, from top to bottom, shows the reconstructed hyperspectral images of the test scenes in the KAIST dataset at three different bands: 420nm, 540nm, and 660nm, using this application.

[0170] Compared with existing technologies, this invention efficiently processes long sequence data through a spectral-spatial state synchronizer, and effectively captures local and global features of images through wavelet transform multi-scale decomposition, achieving high-fidelity spectral-spatial reconstruction of compressed measurements in an coded aperture snapshot spectral imaging system. While improving reconstruction resolution, it fully preserves spectral information. The method proposed in this invention achieves multi-scale encoding and decoding through wavelet transform, and fully extracts spectral and spatial information by combining spectral-spatial state synchronizer with directional variance modulation and selective scanning. This achieves efficient end-to-end reconstruction from compressed measurements to hyperspectral images, improves the accuracy of feature representation, obtains higher quality reconstructed hyperspectral images, and solves the problems of low reconstruction accuracy caused by loss of high-frequency details, weak spectral correlation, and poor noise resistance in traditional compressed imaging.

[0171] Example 2:

[0172] This invention provides a hyperspectral compressed imaging system, comprising:

[0173] Initial fusion module: used to acquire compressed measurement and sensing masks from the coded aperture snapshot spectral imaging system, and to obtain initial feature maps based on the compressed measurement and sensing masks;

[0174] Output module: Used to input the initial feature map into the pre-built U-Net model to obtain the decoded feature map;

[0175] The residual mapping module is used to perform convolutional mapping on the decoded feature map and combine it with the initial feature map to obtain the reconstructed hyperspectral image.

[0176] Example 3:

[0177] This invention also provides a terminal, including a processor and a storage medium;

[0178] The storage medium is used to store instructions;

[0179] The processor is configured to operate according to the instructions to execute the steps of the method described in Embodiment 1.

[0180] Example 4:

[0181] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0182] Since the storage medium provided in this embodiment of the invention can execute the method provided in Embodiment 1 of the invention, it has the corresponding functional modules and beneficial effects for executing the method.

[0183] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0184] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0186] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0187] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A hyperspectral compressed imaging method, characterized in that, include: Obtain the compressed measurement and sensing mask of the coded aperture snapshot spectral imaging system, and obtain the initial feature map based on the compressed measurement and sensing mask; input the initial feature map into the pre-built U-Net model to obtain the decoded feature map; The decoded feature map is convolved and combined with the initial feature map to obtain the reconstructed hyperspectral image; The U-Net model includes: processing the initial feature map using a pre-constructed wavelet variance modulation block to obtain an initial encoded feature map and statistical cue; downsampling the initial encoded feature map based on the wavelet variance modulation block and statistical cue to obtain a final encoded feature map and statistical bridging; processing the final encoded feature map using a pre-constructed spectral-spatial state synchronizer to obtain a bottleneck feature map; and performing multi-stage decoding on the bottleneck feature map to obtain a decoded feature map. The pre-constructed spectral-spatial state synchronizer includes: obtaining a state selection mask based on the final encoded feature map; performing dimensionality reduction and adaptive pooling on the final encoded feature map to obtain a low-resolution sequence; selectively scanning the low-resolution sequence based on the state selection mask and statistical bridging to obtain a scanned weighted state vector; and inverting the scanned weighted state vector and combining it with layer normalization to restore... The original resolution is restored to obtain the bottleneck feature map. The initial feature map is then processed using a pre-constructed wavelet variance modulation block to obtain an initial encoded feature map and statistical hints. This includes: performing discrete wavelet transform on the initial feature map to obtain low-frequency and high-frequency sub-bands; performing directional variance modulation on the low-frequency sub-bands to obtain processed low-frequency sub-bands; performing high-frequency dense convolutional coding on the high-frequency sub-bands to obtain processed high-frequency sub-bands; generating an initial encoded feature map based on the processed low-frequency and high-frequency sub-bands; and obtaining statistical hints based on the processed low-frequency and high-frequency sub-bands. Specifically, obtaining statistical hints based on the processed low-frequency and high-frequency sub-bands includes: obtaining the channel variance of the processed low-frequency sub-band; obtaining the mean of the high-frequency sub-band; and calculating statistical hints based on the channel variance of the processed low-frequency sub-band and the mean of the high-frequency sub-band, with the specific expression as follows: In the formula: For statistical purposes, To handle the channel variance of the low-frequency subband, The mean of the high-frequency subband; Linear represents the linear projection layer; cat represents the concatenation operation; the step of obtaining the state selection mask based on the final encoded feature map includes: performing a process based on the final encoded feature map. The class label mapping is used to generate a tokenized routing policy, and a state selection mask is generated based on the tokenized routing policy. The specific expression is as follows: In the formula: For tokenized routing strategies, A mask is selected for the state; x is the final encoded feature map; B is the batch size; N is the sequence length; K is the preset number of discrete label categories; GumbelSample is a differentiable sampling operation based on the Gumbel distribution; LogSoftmax is a log-normalized exponential function; Linear represents a linear projection layer; the dimensionality reduction and adaptive pooling of the final encoded feature map are performed to obtain a low-resolution sequence, as specifically expressed below: In the formula: For low-resolution sequences; AdaptiveAvgPool2d represents a two-dimensional adaptive average pooling operation.

2. The hyperspectral compressed imaging method according to claim 1, characterized in that, The step of performing directional variance modulation on the low-frequency sub-band to obtain the processed low-frequency sub-band includes: obtaining the spatial mean and variance of the low-frequency sub-band, and calculating the variance coefficient based on the spatial mean and variance of the low-frequency sub-band, with the specific expression as follows: In the formula: The variance coefficient, For low-frequency sub-band, Spatial mean The variance is used; the gradient magnitude is calculated using the Sobel operator, and the specific expression is as follows: In the formula: This represents the gradient magnitude. The horizontal gradient of the image; The vertical gradient of the image is given; the modulation weights are calculated based on the variance coefficient and gradient magnitude, as shown in the following expression: In the formula: For modulation weights, These are learnable parameters; The Sigmoid activation function is used; the low-frequency subband is obtained based on the low-frequency subband and modulation weights, and the specific expression is as follows: In the formula: To handle the low-frequency subband.

3. The hyperspectral compressed imaging method according to claim 1, characterized in that, The downsampling includes primary downsampling and secondary downsampling; the step of downsampling the initial coded feature map based on wavelet variance modulation blocks and statistical hints to obtain the final coded feature map and statistical bridging includes: performing primary downsampling on the initial coded feature map to obtain a primary downsampling result map; processing the primary downsampling result map using a pre-constructed wavelet variance modulation block to obtain a primary coded feature map and a primary statistical hint; performing secondary downsampling on the primary coded feature map to obtain a secondary downsampling result map; processing the secondary downsampling result map using a pre-constructed wavelet variance modulation block to obtain the final coded feature map and a secondary statistical hint; and obtaining a statistical bridging based on the statistical hints, primary statistical hints, and secondary statistical hints.

4. The hyperspectral compressed imaging method according to claim 1, characterized in that, The step of selectively scanning the low-resolution sequence based on the state selection mask and statistical bridging to obtain the scanned weighted state vector includes: using the state selection mask to address and extract the corresponding state embedding from a preset embedding library, and fusing it with the statistical bridging to generate an initial prompt, as shown in the following expression: In the formula: This is the initial prompt. This is a pre-defined embedded library; For statistical bridging, the initial cue is injected into the state-space model, the scan is initiated, and the original hidden state vector is generated, as shown in the following expression: In the formula: The original hidden state vector, The original hidden state projection vector is obtained by linearly projecting the encoded feature map. A spectral-spatial attention model is constructed based on the low-resolution sequence, and weights are calculated. The original hidden state vector is then weighted and modulated using these weights to obtain a weighted state vector. The specific expression is as follows: In the formula: This is a weighted state vector. As weight, and All of these are learnable equilibrium parameters. The sigmoid activation function is used; MLP represents a multilayer perceptron. and These represent average pooling along the spectral and spatial dimensions, respectively; selective scanning is performed based on the weighted state vector to obtain the scanned weighted state vector, as shown in the following expression: In the formula: This is the weighted state vector for the scan. The joint variance of the state vectors, This is an indicator function.

5. The hyperspectral compressed imaging method according to claim 1, characterized in that, The process of multi-stage decoding of the bottleneck feature map to obtain the decoded feature map includes: upsampling the bottleneck feature map using transposed convolution, repeatedly using a pre-constructed wavelet variance modulation block, fusing the skip connection features of the corresponding stage through convolution, and outputting the decoded feature map.

6. A hyperspectral compressed imaging system for performing the steps of the method according to any one of claims 1 to 5, characterized in that, include: Initial fusion module: used to acquire compressed measurement and sensing masks from the coded aperture snapshot spectral imaging system, and to obtain initial feature maps based on the compressed measurement and sensing masks; Output module: used to input the initial feature map into the pre-built U-Net model to obtain the decoded feature map; Residual mapping module: used to perform convolution mapping on the decoded feature map, and combine it with the initial feature map to obtain the reconstructed hyperspectral image.

7. A terminal, characterized in that, It includes a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 5.

Images