Calculation spectrum reconstruction method based on CS-Unet and spectral imaging system

Abstract

The invention discloses a computing spectrum reconstruction method based on CS-Unet and a spectral imaging system. The calculation spectrum reconstruction method comprises two parts, namely TwIST pre-reconstruction and Unet coupling reconstruction, wherein the TwIST pre-reconstruction is used for preprocessing an initial coding spectrum image to generate a pre-reconstructed spectrum image; and the Unet part couples the pre-reconstructed image and the initial coding spectral image in the spectral dimension and inputs the coupled image and the initial coding spectral image into a network for fine reconstruction, and finally a target space-spectral image is recovered. The imaging system is composed of a spectral image coding module and a CS-Unet reconstruction module, a metasurface spectral modulator applied to the spectral image coding module is composed of 36 silicon-based all-dielectric structures, and rich spectral response is achieved by adjusting the period, the size and the direction angle of three basic C4 symmetric units, namely a round unit, a cross unit and a square unit. According to the method, high-robustness and high-imaging-quality spectrum reconstruction is realized in a visible light wave band (400-700 nm), and the method has relatively high engineering applicability.

Description

Technical Field

[0001] This invention belongs to the field of spectral imaging technology, and relates to a computational spectral reconstruction method and spectral imaging system based on CS-Unet. Background Technology

[0002] Spectral imaging technology, by analyzing the spatial and spectral dimensions of objects and relying on the "fingerprint" spectral characteristics of materials, achieves precise identification and has been widely applied in remote sensing, medical, and industrial inspection fields. Traditional spectral imaging systems rely on optical dispersive elements or filters and require mechanical scanning to achieve spectral resolution, resulting in large system size, low efficiency, and difficulty in meeting the requirements of portability and real-time scenarios. In recent years, with the development of nanofabrication technology, computational spectral imaging technology based on structures such as metasurfaces and photonic crystals has emerged. By replacing scanning with encoding and single-snapshot acquisition, it significantly improves integration, speed, and sensitivity. However, the macro-pixel encoding structure also sacrifices spatial resolution, making efficient and robust reconstruction algorithms crucial to system performance.

[0003] In terms of algorithms, early compressed sensing methods utilized spectral sparsity for reconstruction, which, while robust to hardware errors, suffered from low accuracy, long processing time, and poor noise resistance. Deep learning methods, on the other hand, significantly improved reconstruction speed and accuracy through end-to-end mapping, but were limited by the quality and breadth of training data, resulting in insufficient generalization ability to actual disturbances such as processing bias and environmental drift, thus hindering their industrial application. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a computational spectral reconstruction method and a spectral imaging system based on CS-Unet.

[0005] A computational spectral reconstruction method based on CS-Unet includes:

[0006] The compressed spectral image encoded by the spectral modulator is input into the compressed sensing algorithm for pre-reconstruction to obtain the pre-reconstructed spectral image.

[0007] The pre-reconstructed spectral image and the compressed spectral image are fused along the spectral dimension to form a coupled spectral image;

[0008] The coupled spectral image is input into a trained deep learning network, which then fuses the coarse spectral features of the pre-reconstructed spectral image with the detailed spatial features of the compressed spectral image to output the target hyperspectral image.

[0009] A computational spectral imaging system based on CS-Unet includes:

[0010] The spectral encoding module is used to modulate the incident light spectrally and form a compressed spectral image;

[0011] The photoelectric detection module is used to acquire the compressed spectral image and output an electrical signal;

[0012] The computational reconstruction module, connected to the photoelectric detection module, is used for:

[0013] The compressed spectral image is subjected to compressed sensing pre-reconstruction to obtain a pre-reconstructed spectral image;

[0014] The pre-reconstructed spectral image and the compressed spectral image are fused along the spectral dimension to form a coupled spectral image;

[0015] The coupled spectral image is reconstructed end-to-end using a deep learning network to output the target hyperspectral image.

[0016] The beneficial effects of this invention are:

[0017] (1) By directly integrating the metasurface spectral modulator onto the imaging pixel of the detector, single snapshot data acquisition can be provided, and data encoding and acquisition can be realized at the same time. This effectively simplifies the data acquisition process, improves the data sampling rate, and helps to reduce the size and cost of the computational spectral imaging system.

[0018] (2) The spectral image is reconstructed by computational imaging reconstruction algorithm based on CS-Unet. The coarse features of the spectral image are obtained by TwIST pre-reconstruction. The initial encoded spectral image is then coupled with it to supplement the detailed texture features of the spectral image. Finally, higher quality spectral imaging is achieved by reconstructing the end-to-end learning mode of the Unet network.

[0019] (3) By using the CS-Unet computational imaging reconstruction algorithm to reconstruct the spectral image, the robustness of compressed sensing and the efficiency of deep learning are fully combined, which improves the algorithm's generalization ability and robustness to device consistency errors, and provides a feasible solution for the large-scale production of computational spectral imaging systems based on nanophotonic devices. Attached Figure Description

[0020] Figure 1 This is a diagram of the visible light computational spectral imaging system of the present invention;

[0021] Figure 2 This is a structural diagram of the CS-Unet spectral reconstruction algorithm in this invention;

[0022] Figure 3 These are schematic diagrams of three basic structures of the metasurface spectral modulator of the present invention;

[0023] Figure 4 This refers to 36 spectral transmittances that can be achieved by changing the period, size, and orientation angle of three basic structures in a metasurface spectral modulator. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and examples.

[0025] This invention proposes a computational spectral imaging reconstruction method based on a two-step iterative shrinkage / thresholding algorithm (TwIST) coupled with a U-Net network (CS-Unet), as well as a computational spectral imaging system using this algorithm for reconstruction, achieving high robustness and high quality reconstruction of visible spectral images.

[0026] In a first aspect, the present invention provides a CS-Unet computational spectral imaging reconstruction algorithm that can fully combine the robustness of compressed sensing with the efficiency of deep learning. It includes two parts: a pre-reconstruction based on a two-step iterative shrinkage / thresholding algorithm (TwIST) and a computational spectral imaging reconstruction based on U-Net network coupling (CS-Unet). The TwIST pre-reconstruction part is responsible for pre-reconstructing the encoded spectral image data and generating training datasets and simulation datasets; the Unet network coupling reconstruction part realizes the complete computational reconstruction of the target spectral image.

[0027] 1. TwIST pre-reconstruction

[0028] The TwIST pre-reconstruction part inputs the encoded spectral image data into the TwIST algorithm to pre-reconstruct the data, and divides the large number of spectral images obtained by the TwIST algorithm pre-reconstruction and the corresponding encoded spectral images into a training sample library and a simulation sample library, which are used for training and testing, respectively.

[0029] In a preferred embodiment, before the TwIST pre-reconstruction officially begins, a genetic algorithm is used to select the optimal TwIST parameters (including regularization parameters and number of iterations) suitable for the metasurface spectral modulator. Then, the metasurface spectral modulator matrix, the preferred TwIST parameters, and the encoded spectral image data are input into the TwIST algorithm to pre-reconstruct the data, resulting in a large number of pre-reconstructed spectral images.

[0030] 2. Unet Coupling Reconstruction

[0031] The Unet coupling reconstruction part takes the coupled spectral image data obtained by directly coupling the TwIST pre-reconstructed spectral image and the encoded spectral image in the spectral dimension as input, and reconstructs a spectral image with the same spectral channels as the target spectral image, thus realizing the complete computational reconstruction of the target spectral image.

[0032] TwIST pre-reconstruction can obtain coarse features of the target spectral image, realizing the initial three-dimensional reconstruction of the target spectral image; while the initial encoded image contains more image detail and texture features, which can supplement the details of the spatial and spectral dimensions of the reconstructed image.

[0033] The Unet coupling network in this invention fuses the coarse features of the TwIST pre-reconstructed image with the detailed texture features of the initial encoded image based on the end-to-end learning mode of deep learning, and deeply learns the feature information of the target spectral image, thus achieving high-quality spectral image reconstruction.

[0034] In a preferred embodiment, the Unet coupled network comprises an input layer, a double convolutional layer, four downsampling layers, four residual blocks, four upsampling layers, one convolutional layer, and one output layer. The double convolutional layer is a module containing two convolutional layers with a kernel size of 3×3, padding of 1, and stride of 1. The double convolutional layer does not change the spatial size of the feature map and is used for feature extraction and combination. Each downsampling layer consists of a pooling layer and a double convolutional layer. Each upsampling layer consists of a transposed convolutional layer and a double convolutional layer, with skip connections between the four downsampling and upsampling layers. Each residual block has a consistent structure, containing three convolutional layers, all using the ReLU activation function. The first and third convolutional layers have a kernel size of 1×1, a stride of 1, and no padding, and do not change the spatial size of the spectral feature map. The second convolutional layer has a kernel size of 3×3, padding of 1, and a stride of 1, and also does not change the spatial size of the spectral feature map.

[0035] The networks are introduced in the following order:

[0036] Input layer: The input data is the coupled spectral image obtained by directly coupling the TwIST pre-reconstructed image and the initial encoded spectral image in the spectral dimension. Here, to facilitate the network's processing of the image in the spectral dimension, the N×N×(M+1) spatial-spectral image matrix is ​​transformed into a (M+1)×N×N spectral-spatial image before being input into the network;

[0037] The first double convolutional layer performs two convolutions on the input data. Each convolutional layer has a 3×3 kernel size, padding of 1, a stride of 1, and uses the ReLU activation function. The activation function is further defined as follows:

[0038]

[0039] That is, when the input a > 0, the output is equal to the input; when the input a ≤ 0, the output is 0. The spatial size of the output spectral feature map of the double convolutional layer does not change, that is, the output size is 64×N×N spectral feature map;

[0040] The first downsampling layer uses max pooling with a size of 2×2 and a stride of 2. After pooling, the spatial size of the feature map is halved. The double convolutional layer does not change the spatial size of the feature map. The output of the first downsampling layer is a spectral feature map of size 128×(N / 2)×(N / 2).

[0041] The second downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. After pooling, the spatial size of the spectral feature map becomes half of the original size. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 256×(N / 4)×(N / 4).

[0042] The third downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. After pooling, the spatial size of the spectral feature map becomes half of the original size. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 512×(N / 8)×(N / 8).

[0043] The fourth downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. After pooling, the spatial size of the spectral feature map becomes half of the original size. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 1024×(N / 16)×(N / 16).

[0044] After the first residual block, the output is a spectral feature map of size 1024×(N / 16)×(N / 16);

[0045] The first upsampling layer uses a transposed convolution operation for upsampling, with a size of 2×2 and a stride of 2. After transposing the convolution, the spatial size of the spectral feature map becomes twice its original size. Then, the third downsampled spectral feature map is connected along the channel dimension, without changing its spatial size. The double convolutional layer does not change the spatial size of the spectral feature map. The output after the upsampling layer is a spectral feature map of size 512×(N / 8)×(N / 8).

[0046] After the second residual block, the output is a spectral feature map of size 512×(N / 8)×(N / 8);

[0047] The second upsampling layer uses a transposed convolution operation for upsampling, with a size of 2×2 and a stride of 2. After transposing the convolution, the spatial size of the spectral feature map becomes twice the original size. Then, the second downsampled spectral feature map is connected along the channel dimension, and the spatial size of the spectral feature map remains unchanged. The double convolutional layer does not change the spatial size of the spectral feature map. The output is a spectral feature map with a size of 256×(N / 4)×(N / 4).

[0048] After the third residual block, the output is a spectral feature map of size 256×(N / 4)×(N / 4);

[0049] The third upsampling layer uses a transposed convolution operation for upsampling, with a size of 2×2 and a stride of 2. After transposing the convolution, the spatial size of the spectral feature map becomes twice the original size. Then, it connects the first downsampled spectral feature map along the channel dimension, without changing the spatial size of the spectral feature map. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 128×(N / 2)×(N / 2).

[0050] After the fourth residual block, the output is a spectral feature map of size 128×(N / 2)×(N / 2);

[0051] The fourth upsampling layer uses transposed convolution to perform upsampling, with a size of 2×2 and a stride of 2. After transposed convolution, the spatial size of the spectral feature map becomes twice the original size. Then, it connects the spectral feature map after the first double convolutional layer along the channel dimension. The spatial size of the spectral feature map does not change. The double convolutional layer does not change the spatial size of the spectral feature map. The output is a spectral feature map with a size of 64×N×N.

[0052] The first convolutional layer has a kernel size of 3×3, padding of 1, stride of 1, no activation function, and the spatial size of the spectral feature map remains unchanged. The output is a spectral feature map of size L×N×N.

[0053] Output layer: This layer transforms the M×N×N spectral feature map back into a N×N×M spatial-spectral map, corresponding to the output of the reconstructed target spectral image, which facilitates the subsequent use of some quantifiable evaluation metrics to evaluate the quality of the reconstructed spectral image.

[0054] Secondly, this invention provides a visible light computational spectral imaging system based on the CS-Unet computational reconstruction algorithm, comprising two modules: spectral image encoding and the CS-Unet computational spectral reconstruction algorithm. The spectral image encoding module is responsible for encoding the target spectral image and acquiring the encoded spectral image data; the CS-Unet-based computational spectral imaging reconstruction algorithm is responsible for computationally reconstructing the initial encoded image acquired by the spectral image encoding module.

[0055] 1. Spectral Image Coding

[0056] The spectral image encoding part of this invention is achieved by encoding the target spectral image using a metasurface spectral modulator and acquiring the encoded spectral image data using a photodetector, thereby realizing the encoding of the target spectral image and the acquisition of the encoded spectral image data to obtain an initial encoded spectral image;

[0057] In a preferred embodiment, the metasurface spectral modulator comprises 36 metasurface structures, which are generated by varying the period, size, and orientation angle of three basic C4 symmetric periodic patterns (circular, cross-shaped, and square) while maintaining a consistent structure height. The metasurface spectral modulator is directly integrated into the imaging pixel of a photodetector, which detects the light modulated by the metasurface spectral modulator. The spectral image encoding section also includes a light source for illuminating the target object.

[0058] 2. CS-Unet computational imaging reconstruction algorithm

[0059] The CS-Unet computational imaging reconstruction algorithm consists of two parts: TwIST pre-reconstruction based on compressed sensing and Unet coupled reconstruction based on deep learning. The initial spectral image acquired by the photodetector is input into the TwIST algorithm for pre-reconstruction, resulting in a large number of pre-reconstructed images and generating the training and testing sample libraries required for subsequent algorithms. The coupled spectral image, obtained by directly stitching the pre-reconstructed image and its corresponding initial encoded image along the spectral dimension, is input into the Unet network for computational reconstruction, ultimately outputting the target spectral image.

[0060] Compared to traditional single TwIST algorithms or U-Net networks that directly reconstruct the initial coded image, TwIST can achieve a peak signal-to-noise ratio (PSNR) of 25.76 dB when the metasurface spectral modulator has no manufacturing errors, U-Net achieves a PSNR of 35.85 dB, and CS-Unet achieves a PSNR of 36.74 dB. However, with manufacturing errors, TwIST achieves a PSNR of 25.73 dB, U-Net a PSNR of 28.25 dB, and CS-Unet a PSNR of 32.25 dB.

[0061] The TwIST algorithm is insensitive to manufacturing errors and maintains high robustness, but its reconstruction quality is relatively low. The U-Net network has the worst robustness; under manufacturing errors, the reconstructed PSNR drops directly by 7.60 dB, resulting in a significant decrease in reconstruction quality. While the PSNR decreases during reconstruction using CS-Unet, the reconstruction quality remains at a high level. Therefore, the CS-Unet can effectively combine the high robustness of the compressed sensing-based TwIST algorithm with the efficiency and high quality of the deep learning-based U-Net network, achieving higher-quality spectral reconstruction while remaining robust to errors.

[0062] When the system is in operation, the light source illuminates the target. The light carrying the target's spectral information is modulated and encoded by a metasurface spectral modulator, then compressed and acquired by a photodetector. Finally, the spatial-spectral three-dimensional data cube is reconstructed using the CS-Unet algorithm. This system fully combines the robustness of compressed sensing with the efficiency of deep learning, enabling visible light computational spectral imaging that achieves both high robustness and high quality.

[0063] This application provides a CS-Unet-based computational spectral imaging system suitable for visible light (400-700 nm), including a light source, a metasurface spectral modulator, a photodetector, and a CS-Unet computational spectral reconstruction module.

[0064] The sampling and reconstruction process of this system includes the following steps:

[0065] 1) When a light source shines on a target object, it produces incident light carrying the target's spatial-spectral information;

[0066] 2) The incident light carrying the spatial point spectral information of the target object is imaged onto the metasurface spectral modulator array integrated on the imaging pixel of the photodetector according to the spatial position. The metasurface spectral modulator array performs single-exposure modulation encoding on the incident light carrying the spatial spectrum.

[0067] 3) The encoded light is directly detected by a photodetector;

[0068] 4) The detected signal is input into the CS-Unet algorithm for reconstruction to obtain the spectral information within the target space range, and finally obtain the spatial-spectral three-dimensional data cube of the target object.

[0069] Furthermore, in step 1), the discretized incident light carrying the spatial-spectral information of the target object can be written in vector form f, with a magnitude of Where M is the size of the spectral dimension of the image to be reconstructed. The dimension is the spatial dimension.

[0070] Furthermore, in step 2), the matrix form of a single metasurface spectral modulator is H, and the size is... Each modulator can be viewed as a unit of size 1. Macropixels, where L is the number of metasurface structures that constitute the metasurface spectral modulator. N 2 A metasurface spectral modulator performs a single exposure of the target spectrum.

[0071] Furthermore, in step 3), the signal detected by the photodetector is an initial coded spectral image, which can be written in vector form g, with a size of .

[0072] Furthermore, in step 4), the detected signal is first input into the TwIST algorithm for pre-reconstruction to obtain a large number of pre-reconstructed spectral images. The pre-reconstructed spectral images and the corresponding initial encoded images constitute the pre-reconstructed spectral image sample library and the initial encoded image sample library, respectively. 80% of each is extracted as the training sample library, and the remaining 20% ​​is used as the test sample library.

[0073] Next, the pre-reconstructed spectral image and its corresponding initial encoded image are concatenated along the spectral dimension and input into the Unet network for reconstruction. The final output is a spatial-spectral 3D data cube of the target object, with a size of [missing information]. .

[0074] Preferably, the spectral dimension of the incident light at the spatial point is . .

[0075] Preferably, the number of dynamic spectral modulator structures is: .

[0076] Preferably, the spatial-spectral three-dimensional data cube of the target object. .

[0077] Example:

[0078] like Figure 1 As shown, the short-wave infrared computational spectral imaging system of this embodiment includes a light source 1, a metasurface spectral modulator array 2, a photodetector 3, and a computational spectral reconstruction algorithm based on CS-Unet 4.

[0079] The system sampling and reconstruction process includes:

[0080] 1) Light emitted from light source 1 is projected onto the target object. The target object is discretized into a three-dimensional data cube with a spatial dimension of 64×64 and a spectral dimension of 31. The spectral range is 400 nm-700 nm, and the spectral interval is 10 nm.

[0081] 2) The image of each spatial point of the target object is spectrally modulated by a single exposure on the metasurface spectral modulator array 2 based on the quasi-bound state of the continuum;

[0082] 3) The target object is directly detected by photodetector 3 after spatial-spectral modulation;

[0083] 4) The CS-Unet-based computational spectral reconstruction algorithm 4 reconstructs the spatial-spectral three-dimensional data cube of the target object, which is the visible light spectrum image of the target object.

[0084] like Figure 2 As shown, the CS-Unet spectral reconstruction algorithm structure in this embodiment is designed as follows:

[0085] 1) The initial spectral encoded image acquired by the photodetector is input into the TwIST algorithm for pre-reconstruction, resulting in a large number of pre-reconstructed spectral images;

[0086] 2) The pre-reconstructed spectral image and its corresponding initial coded spectral image are input into the Unet coupled reconstruction module;

[0087] 3) The pre-reconstructed spectral image and its corresponding initial encoded spectral image are directly concatenated in the spectral dimension to form a coupled spectral image input to the Unet network;

[0088] 4) The Unet network architecture is as follows:

[0089] Input layer: Input data coupled with a spectral image. Here, to facilitate the network's processing of the image in the spectral dimension, the spatial-spectral image matrix of size 64×64×(31+1) is converted into a spectral-spatial image of size (31+1)×64×64 before being input into the network;

[0090] The first double convolutional layer has a kernel size of 3×3, padding of 1, stride of 1, and ReLU activation function. The output is a spectral feature map of size 64×64×64.

[0091] The first downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. The double convolutional layer does not change the spatial size of the feature map, and the output is a spectral feature map with a size of 128×32×32.

[0092] The second downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 256×16×16.

[0093] The third downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 512×8×8.

[0094] The fourth downsampling layer uses a max pooling layer with a size of 2×2 and a stride of 2. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 1024×4×4.

[0095] After the first residual block, the output is a spectral feature map of size 1024×4×4;

[0096] The first upsampling layer uses a transposed convolution operation to upsample the spectral feature map to a size of 2×2 with a stride of 2. Then, it connects the upsampled spectral feature map along the channel dimension to the third downsampled spectral feature map. The double convolutional layer does not change the spatial size of the spectral feature map. The output after the upsampling layer is a spectral feature map of size 512×8×8.

[0097] After the second residual block, the output is a spectral feature map of size 512×8×8;

[0098] The second upsampling layer uses a transposed convolution operation to upsample the spectral feature map with a size of 2×2 and a stride of 2. Then, it connects the second downsampled spectral feature map along the channel dimension. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 256×16×16.

[0099] After the third residual block, the output is a spectral feature map of size 256×16×16;

[0100] The third upsampling layer uses a transposed convolution operation for upsampling, with a size of 2×2 and a stride of 2. Then, it connects the spectral feature map after the first downsampling along the channel dimension. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 128×32×32.

[0101] After the fourth residual block, the output is a spectral feature map of size 128×32×32;

[0102] The fourth upsampling layer uses a transposed convolution operation for upsampling, with a size of 2×2 and a stride of 2. Then, it connects the spectral feature map after the first double convolutional layer along the channel dimension. The double convolutional layer does not change the spatial size of the spectral feature map, and the output is a spectral feature map with a size of 64×64×64.

[0103] The first convolutional layer has a kernel size of 3×3, padding of 1, stride of 1, no activation function, and outputs a spectral feature map of size 31×64×64.

[0104] Output layer: This layer transforms the spectral feature map of size 31×64×64 back into a spatial-spectral map of size 64×64×31, and outputs the reconstructed target spectral image, which is convenient for subsequent quality evaluation of the reconstructed spectral image using some quantifiable evaluation metrics.

[0105] like Figure 3 As shown, the metasurface spectral modulator in this embodiment is based on a silicon dioxide substrate 31. The height of three basic structures—circular (32), cross-shaped (33), and square (34)—is fixed at 255 nm. By changing the period, size, and orientation angle of these three basic structures, 36 metasurface structures with different periods, sizes, and orientation angles (excluding height) are generated. This allows for 36 different transmission spectra to be achieved within the visible light wavelength range of 400 nm to 700 nm. Among these 36 structures, there are 6 circular structures with different periods and sizes; 18 cross-shaped structures with different periods, sizes, and orientation angles; and 12 square structures with different periods, sizes, and orientation angles. The specific 36 transmission spectra are detailed below. Figure 4.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computational spectral imaging reconstruction method, characterized in that, include: The compressed spectral image encoded by the spectral modulator is input into the compressed sensing algorithm for pre-reconstruction to obtain the pre-reconstructed spectral image. The pre-reconstructed spectral image and the compressed spectral image are fused along the spectral dimension to form a coupled spectral image; The coupled spectral image is input into a trained deep learning network, which then fuses the coarse spectral features of the pre-reconstructed spectral image with the detailed spatial features of the compressed spectral image to output the target hyperspectral image.

2. The method according to claim 1, characterized in that: The compressed sensing algorithm is a two-step iterative shrinkage / thresholding algorithm called TwIST.

3. The method according to claim 1 or 2, characterized in that: Prior to the pre-reconstruction, a genetic algorithm is used to optimize the regularization parameters and number of iterations of the compressed sensing algorithm to obtain the optimal parameter combination suitable for the current spectral modulator.

4. The method according to claim 1, characterized in that: The deep learning network is a U-Net architecture, which includes double convolutional layers, downsampling layers, residual blocks, upsampling layers, and skip connections.

5. The method according to claim 4, characterized in that: The input to the U-Net is a (M+1)×N×N spectral-spatial tensor, where M is the number of pre-reconstructed spectral channels and +1 corresponds to compressed spectral image channels. The output is an N×N×M hyperspectral data cube.

6. The method according to claim 1, characterized in that: The spectral dimension fusion adopts a direct stitching method, which cascades the compressed spectral image and the pre-reconstructed spectral image in the channel dimension to form a coupled spectral image.

7. The method according to claim 2, characterized in that: The spectral modulator is a metasurface array, and each macropixel has an independent and known spectral transmission response matrix H, which is used to perform single-snapshot encoding of the target spectral image.

8. The method according to claim 1, characterized in that: The deep learning network is generated through training using the following samples: a large number of simulated or measured compressed spectral images are pre-reconstructed using a compressed sensing algorithm to obtain corresponding pre-reconstructed-original spectral image pairs, which serve as the training set and the test set.

9. A computational spectral imaging system, characterized in that, include: The spectral encoding module is used to modulate the incident light spectrally and form a compressed spectral image; The photoelectric detection module is used to acquire the compressed spectral image and output an electrical signal; The computational reconstruction module, connected to the photoelectric detection module, is used for: The compressed spectral image is subjected to compressed sensing pre-reconstruction to obtain a pre-reconstructed spectral image; The pre-reconstructed spectral image and the compressed spectral image are fused along the spectral dimension to form a coupled spectral image; The coupled spectral image is reconstructed end-to-end using a deep learning network to output the target hyperspectral image.

10. A computational spectral imaging system according to claim 9, characterized in that, The spectral encoding module is integrated into the photoelectric detection module. The metasurface spectral modulator in the spectral encoding module is formed by changing the period, size and / or orientation angle of three C4 symmetric periodic patterns while maintaining structural consistency. It is used to provide at least two independent spectral transmission codes in the visible light range of 400nm-700nm.

Images