Masked uncertainty based on self-attention for hyperspectral image reconstruction

Abstract

The application discloses a kind of based on self-attention mask uncertainty hyperspectral image reconstruction method, specifically is: step 1: using CASSI optical system compressed sampling, obtain two-dimensional measurement Y;Step 2: using the self-optimization network of mask uncertainty automatically captures the uncertainty between different mask space structures, to obtain the optimized mask m';Step 3: two-dimensional measurement and optimized mask are input into the residual reconstruction network of transformer, to obtain the final high light spectrum.The application shows good reconstruction quality and reconstruction speed, and has the adaptability of different masks, meets the practical application of CASSI system, and further promotes the in-depth development and popularization and application of hyperspectral compressed snapshot imaging technology.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision image processing, and particularly relates to a mask uncertainty hyperspectral image reconstruction method based on self-attention. BACKGROUND

[0002] Hyperspectral imaging has received significant research attention in the computer vision community in recent years, but the high dimensionality of hyperspectral data has brought great challenges to the storage and transmission of hyperspectral data.

[0003] Compressive sensing (CS) is a new sampling method that reduces encoding complexity by simultaneously sampling and compressing through linear projection. It captures sparse or compressible signals as compressed signals through linear projection. The core problem of compressive sensing reconstruction is to recover the original high-dimensional signal from a small number of compressed measurements using a reconstruction algorithm. The coded aperture snapshot spectral imaging (CASSI) system is a typical compressive snapshot imaging system that modulates the signals of different wavelengths of hyperspectral data through a coded aperture and a disperser to obtain compressed two-dimensional snapshot measurements. Then, by solving an optimization problem of hyperspectral compressive snapshot reconstruction, the underlying hyperspectral image is reconstructed from the captured two-dimensional snapshot measurements. The CASSI system has the advantages of low hardware cost, low bandwidth requirement, and high data throughput, but the captured snapshot measurements are a projection transformation of the original image pixel values, not the hyperspectral data itself. Therefore, it is particularly important to design a reasonable hyperspectral compressive snapshot reconstruction algorithm.

[0004] In recent years, with the excellent learning ability of deep networks, scholars have begun to use deep networks to directly learn the nonlinear mapping from snapshot measurements to hyperspectral images in an end-to-end manner. This end-to-end learning method significantly reduces the reconstruction time.

[0005] This deep learning-based method mainly uses the inner product between the mask and the offset measurement as input. This method destroys the input hyperspectral information and does not fully explore the randomness of the mask, resulting in limited improvement. In recent years, the Transformer has been introduced into the field of computer vision. The multi-head self-attention module in the Transformer is good at capturing non-local similarities and long-term dependencies. This advantage can well solve the limitations of the above deep learning-based methods in hyperspectral image reconstruction. Although good reconstruction results have been achieved, there is still room for improvement in reconstruction performance, and further research is still needed.

[0006] The above prior art also has the following problems:

[0007] 1.Hyperspectral images have rich image information. The widely used deep network reconstruction algorithm currently has a complex structure, which increases the reconstruction time. At the same time, these reconstruction networks are pre-trained for specific masks, and need to retrain the network for different hardware masks and noise environments, thereby wasting a large amount of computing resources.

[0008] 2.Hyperspectral images have rich image information. Although the local window self-attention of the Transformer performs significantly in visual tasks, it still has the problems of limited receptive field and weak modeling ability. SUMMARY

[0009] The purpose of the application is to solve the problems existing in the prior art, and provide a mask uncertainty hyperspectral image reconstruction method based on self-attention.

[0010] Technical scheme: The application provides a mask uncertainty hyperspectral image reconstruction method based on self-attention, which specifically comprises the following steps:

[0011] Step 1: using a CASSI optical system for compression sampling to obtain a two-dimensional measurement value Y;

[0012] Step 2: using a self-optimization network of mask uncertainty to automatically capture the uncertainty between different mask space structures, thereby obtaining an optimized mask m';

[0013] Step 3: inputting the two-dimensional measurement value and the optimized mask into a residual reconstruction network of the transformer to obtain a final hyperspectral image.

[0014] Further, the step 1 is specifically: in the CASSI optical system, a two-dimensional coded aperture M * is used to code a preset three-dimensional cube F(:,:,n λ ) to obtain F'(:,:,n λ ), and a disperser is used to shear the signal F'(:,:,n λ ) along the y-axis, thereby obtaining a two-dimensional measurement Y:

[0015]

[0016] Wherein, (u,v) is a coordinate on a positioning detector plane coordinate system, n λ represents an n λ th spectral channel, N λ represents the total number of spectral channels, and G is a two-dimensional measurement noise in the imaging process on a sensing detector; the expression of F”(u,v,n λ ) is: F'(:,:,n λ ) = F'(x,y+d(λ n -λc ),n λ ), where d represents the step size of the spatial displacement, λ c As the reference wavelength, λ n Let f' be the wavelength of the nth channel, and x and y represent points in the 3D cube with x as the x-coordinate and y as the y-coordinate; F'(:,:,n λ The expression for ) is F'(:,:,n λ )=F'(:,:,n λ )⊙M * , ⊙ indicates element-wise multiplication.

[0017] Furthermore, step 2 specifically includes:

[0018] Step 2.1: The self-tuning network q φ (m) is a Gaussian distribution around a given mask m:

[0019] q φ (m)=N(m,g φ (m))

[0020] Among them, g φ (m) is a deep network:

[0021] g φ (m)=δ(Conv(A))

[0022] Where Conv(.) is the convolution operation, δ(.) is the SoftPlus function, and A is the attention map of mask m. The expression for A is:

[0023] A = t⊙(LT(t)+1)

[0024] Where t represents the feature map of the real mask m, LT(.) represents the LeWin Transformer network, and ⊙ represents element-wise multiplication;

[0025] Step 2.2: Input the mask m into the self-tuning network q φ (m), q φ (m) for m and g φ The output of (m) is distributed using a Gaussian distribution to obtain the tuning mask m':

[0026] m'=m+g φ (m)⊙ε

[0027] Where ε~N(0,1) represents a random distribution.

[0028] Furthermore, the residual reconstruction network of the transformer in step 3 includes a pseudo-inverse projection module, a Transformer residual block, and an output projection module connected in sequence.

[0029] Furthermore, the expression for the output Z of the pseudo-inverse projection module is:

[0030] Z = Conv 3×3 (M'⊙shift(Y))

[0031] Where M' represents the vectorized form of the tuning mask m' expanded along the channel dimension, shift(.) represents the size transformation operation, and Conv 3×3 (.) represents a 3×3 convolution.

[0032] Furthermore, the Transformer residual block includes a Lewin Transformer module and a residual block; the output Z of the pseudo-inverse projection module is input into the Lewin Transformer module to obtain the output Z”:

[0033] Z”=LeFF(LN(Z'))+Z'

[0034] Where LN(·) is the layer normalization operation, LeFF(.) is the local enhancement feedforward network, and the expression for Z' is:

[0035] Z' = W - MSA(LN(Z)) + Z

[0036] Where W-MSA(.) represents a non-overlapping local window multi-head attention module, and its expression is as follows:

[0037] W-MSA(Q,K,V)={Atten1,Atten2,...,Atten H}

[0038] Among them, Atten i It is the i-th self-attention module, where H represents the total number of self-attention modules. i The expression is:

[0039]

[0040] Where B is the position encoding matrix, d represents the query dimension in the window, T is the transpose, SoftMax(.) represents the SoftMax function, and the expressions for matrices Q, K, and V are as follows:

[0041] Q = RP q K = RP k V = RP v

[0042] Where R represents window features, P q ,P k and P v Represents the projection matrix;

[0043] Input the Z” output from the Transformer residual block into the residual block to obtain the output.

[0044]

[0045] Here, ReLU(·) represents the activation function.

[0046] Furthermore, the output projection adjusts the number of channels of the output features through convolution operations, and inputs the feature map into the output projection to obtain the final reconstructed hyperspectral image HSI.

[0047] Beneficial Effects: This invention combines the advantages of Transformer and Convolutional Neural Networks by designing high-performance modules and utilizing global and local spatial-spectral correlation features to capture more spatial and spectral feature information from two-dimensional snapshot measurements. It also proposes a method for estimating mask uncertainty by explicitly modeling the mask uncertainty as Gaussian noise around a given mask using full variational Bayesian learning on the mask set. The reconstruction algorithm maintains high-quality reconstruction even when the hardware mask of the hyperspectral compressed snapshot imaging system changes. Compared to other reconstruction algorithms, the proposed algorithm exhibits better reconstruction quality and speed, and is adaptable to different masks, meeting the practical application requirements of CASSI systems and further promoting the in-depth development and widespread application of hyperspectral compressed snapshot imaging technology. Attached Figure Description

[0048] Figure 1 : Flowchart of the method of the present invention;

[0049] Figure 2 This invention utilizes the CASSI optical system to compress sampling and obtain two-dimensional measurement values.

[0050] Figure 3 The network diagram for reconstructing mask uncertainty based on Transformer in this invention;

[0051] Figure 4 : A schematic diagram of the mask self-tuning network structure based on Transformer in this invention;

[0052] Figure 5 : A schematic diagram of the residual reconstruction network structure based on Transformer in this invention;

[0053] Figure 6 : Schematic diagram of the Transformer residual block structure in this invention;

[0054] Figure 7: A schematic diagram of the LeFF module structure in this invention. Detailed Implementation

[0055] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0056] like Figure 1 The present implementation includes the following steps:

[0057] (1) During the encoding stage, the CASSI optical system is used to compress the sampling to obtain two-dimensional measurement values;

[0058] (2) In the decoding stage, the uncertainty between different mask spatial structures is automatically captured by the self-tuning layer of mask uncertainty, and mask self-tuning is performed.

[0059] (3) Feature maps are obtained by extracting feature information using a transformer-based residual reconstruction network;

[0060] (4) Input the feature map into the output projection to obtain the reconstructed hyperspectral image.

[0061] like Figure 2 As shown, in step (1), the CASSI optical system is used to compress sampling to obtain two-dimensional measurement values. Specifically, given a three-dimensional HSI cube... This serves as the input to the CASSI imaging model. Here, H represents the height of the input HSI cube, W represents the width of the input HSI cube, and N... λ This represents the number of wavelengths in the input cube. First, an coded aperture (i.e., a physical mask) M is applied along the channel dimension. * ∈R H×W Modulation F:

[0062] F'(:,:,n λ )=F'(:,:,n λ )⊙M *

[0063] in Represents the modulated HSIs,n λ ∈[1,...,N λ ] represents the spectral channel, N λ represents the total number of spectral channels, and ⊙ represents element-wise multiplication. Passed through a single disperser (SD), F' becomes skewed after passing through the disperser and can be considered as shearing along the y-axis.

[0064] set up Let be a tilted data cube, where d represents the step size of spatial displacement, and λ c As the reference wavelength, then For an anchor point image without clipping, the dispersion can be expressed as:

[0065] F"(u,v,n) λ )=F'(x,y+d(λ n -λ c ),n λ )\

[0066] Where x and y represent points in the 3D cube with x as the x-coordinate and y as the y-coordinate, (u,v) is the coordinate system on the plane of the positioning detector, and λ n Let λ be the wavelength of the nth channel. c d(λ) represents the reference wavelength. n -λ c ) represents the spatial displacement offset of the nth channel on F”.

[0067] Then, by integrating all channels, the data cube is compressed into a two-dimensional measurement, resulting in the captured two-dimensional compressed measurement.

[0068]

[0069] in The two-dimensional measurements captured by CASSI This refers to the two-dimensional measurement noise during the imaging process on the sensor detector.

[0070] In step (2), the self-tuning layer of mask uncertainty is used in the decoding stage to automatically capture the uncertainty between different mask spatial structures and perform mask self-tuning, including:

[0071] like Figure 3 As shown in Figure 4, a Transformer-based mask self-tuning layer is used to automatically capture different masks M = {m1, ..., m}. N The uncertainty between spatial structures is addressed to achieve a smoother mask distribution on real samples. The self-adjusting layer for mask uncertainty is based on variational Bayesian learning, which explicitly models the mask uncertainty as Gaussian noise around a given mask and learns to model the mask uncertainty by adapting the self-adjusting variance to different mask spatial patterns.

[0072] The self-tuning layer directly explores the relationships between different mask structures, estimating mask uncertainty and enabling the single-trained deep reconstruction network to adapt to different real masks. First, it estimates the uncertainty of any mask m. Then, it uses a Transformer-based mask self-tuning network to capture the uncertainty around the mask, generating an attention map A to obtain g. φ (m) The output of the module.

[0073] Uncertainty estimation of the mask:

[0074] Mask decomposition models the uncertainty surrounding each mask. The distribution of the true mask, exhibiting two peaks at 0 and 1 and a Gaussian shape in the middle, allows the true mask *m* to be decomposed into unknown clean masks. Adding random noise z, which has a Gaussian distribution, can be expressed by the formula:

[0075]

[0076] Here, we assume that z follows a Gaussian distribution, and the noise prior is expressed as p(z) = N(μ,σ). N(μ,σ) is a Gaussian distribution, where μ is the mean of the Gaussian distribution and σ is the variance. The uncertainty of the mask can be estimated by directly approximating the mask posterior P(m|X,Q), where X = {X1,...,X...} N} and Q = {Y1,...,Y N}, where X represents the set of hyperspectral images, X N Let Y represent the Nth spectra, Q be the set of two-dimensional measurements, and Y be the Nth spectra. N Two-dimensional measurement of the Nth spectra.

[0077] Based on the core idea of ​​variational inference, a relatively simple and learnable variational distribution q is used. φ (m) (i.e., a mask self-tuning network based on Transformer) is used to approximate P(m|X,Y), such that q φ The KL divergence between (m) and P(m|X,Y) is minimized, i.e., min φ KL[q φ The expression [(m)||P(m|X,Y)] can be transformed into finding the maximum value of the lower bound of the evidence factor (ELBO), as shown in the following formula:

[0078]

[0079] Among them, in the above formula It is a measurement reconstruction, KL[q] φ The second term [(m)||P(m)] is a regularization term given the mask prior P(m). Clean mask It can be viewed as a two-dimensional constant with a specific structure; the true mask is the uncertainty caused by the noise z. Therefore, the variational distribution q φ (m) can be modeled as a Gaussian distribution around a given mask m.

[0080] q φ (m)=N(m,g φ (m))

[0081] Among them, g φ(m) is a deep network used to learn self-adjusting variance to simulate the uncertainty of sampling in the real mask set M.

[0082] Generate attention map A to obtain g φ Output of module (m):

[0083] g φ (m) After processing a single arbitrary ground truth mask through convolution, the LeWin Transformer is used to generate an attention score. Let t represent the input features of the LeWin Transformer, and t be the feature map obtained after convolution and ReLU operations on m. Then, the attention map A generated by the LeWin Transformer is:

[0084] A = t⊙(LT(t)+1)

[0085] Here, LT(.) represents the global mapping function of the LeWin Transformer module. Attention map A, after convolution and activation, yields g. φ The output of module (m) can be expressed mathematically as follows:

[0086] g φ (m)=δ(Conv(A))

[0087] Here, Conv(.) represents the convolution operation, and δ(.) is the SoftPlus function.

[0088] Obtain the tuning mask m':

[0089] The tuning mask m' (output of the self-tuning layer with mask uncertainty) used during network training is derived from the variational distribution q. φ The random variable m' drawn from (m) can be expressed mathematically as:

[0090] m'=m+g φ (m)⊙ε

[0091] Where m' (output of the self-tuning layer) represents the tuning mask used during network training, ⊙ represents element-wise multiplication, and ε~N(0,1) represents random distribution.

[0092] In step (3), feature information is extracted using a transformer-based residual reconstruction network to obtain a feature map, including:

[0093] The obtained optimized mask m' and two-dimensional measurement Y are input into a Transformer-based residual reconstruction network to extract feature information and obtain a feature map. For example... Figure 5The residual reconstruction network shown consists of three parts: pseudo-inverse projection, Transformer residual block, and output projection. The pseudo-inverse projection module calculates the initial reconstruction x'∈R based on the input mask and snapshot measurements. H ×W×C The Transformer residual block contains two LeWin Transformers (LT) and two residual blocks (RB). The LeWin Transformer treats each pixel of the hyperspectral image as a vector (Token) and captures the spatial spectral context information of each pixel within a local window through self-attention, thereby effectively improving the spectral fidelity of the reconstructed image. Simultaneously, the residual blocks utilize convolution operations to extract local feature information, resulting in a feature map.

[0094] The tuning mask m' and the two-dimensional measurement Y are input into the pseudo-inverse projection module to obtain the output result Z, specifically:

[0095] Based on the input tuning mask m' and snapshot measurement Y∈R H(W+C-1) The initial reconstruction result of pseudo-inverse projection x'∈R H ×W×C This can be expressed mathematically as:

[0096] x'=M'⊙shift(Y)

[0097] Here, ⊙ represents element-wise multiplication of two matrices, M'=copy(m'[:,:,1:C]) is the vectorized form of the mask m' after expansion along the channel dimension, and shift(·) represents the operation of converting the snapshot measurement into the required tensor size H×W×C. Finally, x' undergoes a 3×3 convolution operation to obtain the final output Z∈R of the pseudo-inverse projection module. H×W×64 ,Right now

[0098] Z = Conv 3×3 (x')

[0099] The output Z is input into the Transformer residual block to extract features and obtain the feature map:

[0100] Transformer-based residual reconstruction networks stack multiple Transformer residual blocks and deploy them as an encoding / decoding architecture for hyperspectral image reconstruction. Specifically, such as... Figure 6 As shown, the residual blocks in the encoder are deployed after the Lewin Transformer, while the residual blocks in the decoder are deployed before the Lewin Transformer. This embodiment will use the encoder as an example to describe the Transformer residual blocks in detail.

[0101] The Lewin Transformer employs non-overlapping local window multi-head attention (W-MSA). First, the input feature map of size H×W×C is divided into I non-overlapping local window blocks of size S×S, where I = HW / S. 2 Then, the correlation within each window is calculated using a multi-head self-attention mechanism. Representing the characteristics of a local window, a matrix of Query, Key, and Value. Defined as:

[0102] Q = ZP q K = ZP k V = ZP v

[0103] Among them, P q ,P k ,P v This is the projection matrix, whose parameters are shared across different windows. The self-attention of pixels within a local window is:

[0104]

[0105] Where B is the learnable positional encoding matrix, K T Let be the transpose of K, and d represent the dimension of the query or key in the window. Multi-head attention can be obtained by parallelizing multiple self-attention mechanisms; W-MSA is mathematically expressed as follows:

[0106] W-MSA(Q,K,V)={Atten1,Atten2,...,Atten H}

[0107] Among them, Atten i Let be the i-th self-attention module, and H represent the total number of self-attention modules. Compared to the traditional Transformer feedforward network, the LeWin Transformer replaces the multilayer perceptron with a Local Enhancement Feedforward Network (LeFF). Figure 7 As shown, LeFF mainly consists of a 3×3 convolutional operation and two fully connected layers. Compared with multilayer perceptrons, the Local Enhancement Feedforward Network reduces the number of weight parameters and is able to capture more local feature information.

[0108] The LeWin Transformer adds W-MSA and LeFF to the normalization operations of the two layers respectively, and performs a residual connection between the two modules. If Z is the input to the LeWin Transformer, then the output Z” is:

[0109] Z' = W - MSA(LN(Z)) + Z

[0110] Z”=LeFF(LN(Z'))+Z'

[0111] Here, LN(·) is the layer normalization operation. After two LeWin Transformers, two identical residual blocks are concatenated to further enhance feature learning and improve the network's convergence speed. Z” is the input to the residual block, and the output is the feature map. for:

[0112]

[0113] Among them, Conv 3×3 (·) represents a 3×3 convolution operation, and ReLU(·) represents an activation operation.

[0114] Step (4) involves inputting the feature map into the output projection to obtain the reconstructed hyperspectral image, including:

[0115] The output projection adjusts the number of channels in the output features using a simple 1×1 convolution operation. The feature map is then input into the output projection to obtain the final reconstructed hyperspectral image. The specific expression is as follows:

[0116]

[0117] in, denoted as Transformer-based residual reconstruction network, m' represents the tuning mask output by the self-tuning layer, Y represents two-dimensional measurement, and HSI represents the reconstructed hyperspectral image.

[0118] In summary, this invention provides a self-attention-based hyperspectral image reconstruction method for mask uncertainty. Compared with previous traditional compressed sensing reconstruction algorithms, it first utilizes an optical system (Coded Aperture Snapshot Spectroscopic Imaging (CASSI) system) to encode the aperture modulation HSI signal and compress it into a two-dimensional measurement. Subsequently, in the decoding stage, a self-tuning layer for mask uncertainty is used to automatically capture the uncertainty between different mask spatial structures, achieving a smoother mask distribution on real samples, thus performing mask self-tuning. Then, a Transformer-based residual reconstruction network is used to simultaneously extract local and global spatial spectral correlation features to obtain feature maps. Finally, the feature maps are input into the output projection to obtain the reconstructed hyperspectral image. This method combines the advantages of Transformer and convolutional neural networks by designing high-performance modules, and utilizes both global and local spatial spectral correlation features to capture more spatial and spectral feature information from two-dimensional snapshot measurements. Furthermore, a method for estimating mask uncertainty is proposed, which explicitly models the mask uncertainty as Gaussian noise around a given mask by using full variational Bayesian learning on the mask set. Even when the hardware mask of the hyperspectral compressed snapshot imaging system changes, the reconstruction algorithm can still maintain high-quality reconstruction. Compared with other reconstruction algorithms, the proposed algorithm exhibits better reconstruction quality and speed, and is adaptable to different masks, meeting the practical application requirements of the CASSI system and further promoting the in-depth development and widespread application of hyperspectral compressed snapshot imaging technology.

[0119] The average values ​​of the reconstruction results of the algorithms GSM, SRN, GST, and the algorithm of this embodiment under 100 unknown mask measurements are shown in Table 1.

[0120] Table 1

[0121] Algorithm GSM SRN GST Ours PSNR 28.20 32.24 33.02 34.22 SSIM 0.885 0.912 0.929 0.949

[0122] As shown in Table 1, compared with several excellent deep learning methods, the TMU-Net method in this embodiment exhibits the best reconstruction quality across all ten test scenarios. Clearly, the TMU-Net proposed in this embodiment improves the network's robustness to mask variations by exploring the relationships between different hardware mask structures on the mask set, enabling the reconstruction network to maintain high-quality reconstructions under different masks.

[0123] The average values ​​of the reconstruction results on a single real mask using the GSM algorithm, SRN algorithm, GST algorithm, and the algorithm of this embodiment are shown in Table 2.

[0124] Table 2

[0125] Algorithm TSA-Net GSM SRN Ours PSNR 27.47 26.34 30.17 31.61 SSIM 0.864 0.858 0.887 0.929

[0126] As shown in Table 2, the Transformer-based mask uncertainty reconstruction network proposed in this embodiment aims to maintain the robustness of the reconstruction network to mask changes. Even when the model is pre-trained under a single mask, the network still exhibits good adaptability to different masks, which fully demonstrates the algorithm's robustness and verifies the effectiveness of the mask uncertainty evaluation strategy in this embodiment.

[0127] The present invention has been described above with reference to the accompanying drawings, which are merely preferred embodiments of the present invention. However, the scope of protection of the present invention is not limited thereto. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements or equivalent substitutions made using the inventive concept and technical solution of the present invention and directly applied to other occasions should be covered within the scope of protection of the present invention.

Claims

1. A self-attention-based mask uncertainty hyperspectral image reconstruction method, characterized in that, Specifically, the steps include the following: Step 1: Use the CASSI optical system to compress the sample and obtain the two-dimensional measurement value Y; Step 2: Automatically capture the uncertainty between different mask spatial structures using a self-tuning network for mask uncertainty, thereby obtaining the tuned mask m′; Step 3: Input the two-dimensional measurement values ​​and the tuning mask into the transformer's residual reconstruction network to obtain the final spectrogram; Step 2 specifically involves: Step 2.1: The self-tuning network q φ (m) is a Gaussian distribution around a given mask m: q φ (m)=N(m,g φ (m)) Among them, g φ (m) is a deep network: g φ (m)=δ(Conv(A)) Where Conv(.) is the convolution operation, δ(.) is the SoftPlus function, and A is the attention map of mask m. The expression for A is: A = t⊙(LT(t)+1) Where t represents the feature map of the real mask m, LT(.) represents the LeWin Transformer network, and ⊙ represents element-wise multiplication; Step 2.2: Input the mask m into the self-tuning network q φ (m), q φ (m) for m and g φ The output of (m) is distributed using a Gaussian distribution to obtain the tuning mask m′: m′=m+g φ (m)⊙ε Where ε~N(0,1) represents a random distribution.

2. The self-attention-based mask uncertainty hyperspectral image reconstruction method according to claim 1, characterized in that, Step 1 specifically involves: in the CASSI optical system, using a two-dimensional coded aperture M * For the preset three-dimensional cube F(:,:,n) λ Encode to obtain F′(:,:,n) λ ), the signal F′(:,:,n) is sheared along the y-axis by a diffuser. λ Thus, the two-dimensional measurement value Y is obtained: Where (u,v) are the coordinates on the plane coordinate system of the positioning detector, and n λ Indicates the nth λ Spectral channels, N λ Represents the total number of spectral channels, G is the two-dimensional measurement noise during the imaging process on the sensor detector; F(u,v,n) λ The expression for F is: F(u,v,n) λ )=F′(x,y+d(λ n -λ c ),n λ ), where d represents the step size of the spatial displacement, λ c As the reference wavelength, λ n Let f' be the wavelength of the nth channel, and x and y represent points in the 3D cube with x as the x-coordinate and y as the y-coordinate; F′(:,:,n λ The expression for ) is F′(:,:,n λ )=F(:,:,n λ )⊙M * , ⊙ indicates element-wise multiplication.

3. The self-attention-based mask uncertainty hyperspectral image reconstruction method according to claim 1, characterized in that, The residual reconstruction network of the transformer in step 3 includes a pseudo-inverse projection module, a Transformer residual block, and an output projection module connected in sequence.

4. The self-attention-based mask uncertainty hyperspectral image reconstruction method according to claim 3, characterized in that, The expression for the output Z of the pseudo-inverse projection module is: Z=Conv 3×3 (M′⊙shift(Y)) Where M′ represents the vectorized form of the tuning mask m′ expanded along the channel dimension, shift(.) represents the size transformation operation, and Conv 3×3 (.) represents a 3×3 convolution.

5. The self-attention-based mask uncertainty hyperspectral image reconstruction method according to claim 3, characterized in that, The Transformer residual block includes a Lewin Transformer module and a residual block; the output Z of the pseudo-inverse projection module is input into the Lewin Transformer module to obtain the output Z”: Z″=LeFF(LN(Z′))+Z′ Where LN(·) is the layer normalization operation, LeFF(.) is the locally enhanced feedforward network, and the expression for Z′ is: Z′=W-MSA(LN(Z))+Z Where W-MSA(.) represents a non-overlapping local window multi-head attention module, and its expression is as follows: W-MSA(Q,K,V)={Atten1,Atten2,...,Atten H } Among them, Atten i It is the i-th self-attention module, where H represents the total number of self-attention modules. i The expression is: Where B is the position encoding matrix, d represents the query dimension in the window, T is the transpose, SoftMax(.) represents the SoftMax function, and the expressions for matrices Q, K, and V are as follows: Q=RP q ,K=RP k ,V=RP v Where R represents window features, P q ,P k and P v Represents the projection matrix; Input the Z” output from the Transformer residual block into the residual block to obtain the output. Here, ReLU(·) represents the activation function.

6. The self-attention-based mask uncertainty hyperspectral image reconstruction method according to claim 3, characterized in that, The output projection adjusts the number of channels of the output features through convolution operations, and the feature map is input into the output projection to obtain the final reconstructed hyperspectral image HSI.

Images