A compact hyperspectral imaging method based on denoising network

Abstract

A compact hyperspectral imaging method based on a denoising network, the method comprising the following steps: constructing and training a spectral perception network, from which a denoising network is extracted; imaging and spectral encoding the target and the calibration plate respectively, and correcting the target light intensity image according to the light intensity distribution on the calibration plate; inputting the calibrated light intensity image into the denoising network in parallel, and reconstructing the target hyperspectral image. The present application can effectively reduce the influence of noise in the hyperspectral imaging process, significantly improve the imaging quality, and reduce the requirements of the imaging light source and the volume and cost of the equipment.

Description

Technical Field

[0001] The present invention relates to the fields of spectral sensing, optical imaging, and optoelectronic devices, and provides a compact hyperspectral imaging method based on a denoising network. This method can be widely applied to spectral analysis and hyperspectral imaging systems under non-parallel light sources in daily life, and can improve overall image performance during the imaging process. Background Art

[0002] Images play an indispensable role in humanity's exploration and understanding of the world, becoming our primary means of acquiring information. Traditional imaging techniques, such as the common RGB imaging, record scenes by capturing red, green, and blue light, focusing on visually reproducing the appearance of objects. However, this approach is limited to color reproduction within the visible light range and ignores the inherent spectral properties of objects. Spectral imaging technology has emerged to overcome the limitations of RGB imaging. By recording the spectral response of an object in a targeted wavelength band in detail, it can reveal not only its color information but also its chemical and physical properties. This technology is crucial for identifying and analyzing the composition of substances and has already played an important role in various fields, including astronomical remote sensing, biomedicine, and food testing. Driven by the continued development of artificial intelligence and machine learning, snapshot spectral imaging methods have emerged. By integrating software and hardware design through deep learning, they have successfully overcome the limitations of traditional spectral imaging equipment, such as its bulky size, slow imaging speed, and inability to be applied to large scenes.

[0003] However, after modulation by encoders such as metasurfaces, nanowires, and thin-film filters, the light intensity received by the detector is often constrained by the uniformity of the imaging system's light source, ultimately affecting the rendered color image. To achieve higher image quality, laboratories typically use a lens array consisting of multiple convex and concave lenses or a collimator to generate parallel light, further increasing the size and cost of the device, making it difficult to apply in real life.

[0004] Detector noise is particularly critical for hyperspectral imaging systems, as it directly impacts the accuracy and reliability of spectral data. In addition to thermal noise, readout noise, and quantization noise, detectors also suffer from other types of noise, such as shot noise and calibration noise. Shot noise is typically associated with the randomness of photons arriving at the detector, while calibration noise arises from inconsistent responses between different pixels. These noise types can impact image quality to varying degrees. In the field of hyperspectral imaging, due to the large amount of spectral information involved, the signal-to-noise ratio requirement is relatively high. Currently, most hyperspectral imaging system designs focus primarily on spectral data acquisition and processing, with insufficient consideration given to the impact of noise, especially in complex practical application environments. For example, several patents (a neural network-based narrowband spectral imaging method, patent application number: CN202211467375.X; a spectral camera based on wide-spectral encoding and deep learning, patent application number: CN202110090003.9) achieve different functions through their respective imaging methods, but none of them consider the impact of scene noise. Noise not only limits the dynamic range of the imaging system, but may also cause distortion of the imaging results. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this paper proposes a compact hyperspectral imaging method based on a denoising network. This method can effectively reduce the impact of noise during imaging, significantly improve image quality, and reduce imaging light source requirements and equipment volume costs.

[0006] The technical solutions of the present invention are as follows:

[0007] A compact hyperspectral imaging method based on a denoising network is characterized in that the method includes a simulation stage and an imaging stage. The specific steps are as follows:

[0008] Simulation phase:

[0009] Step 1: Determine the target wide-band spectral imaging range λ max ,λ min , and the spectral interval △λ of each broadband channel, the number of broadband spectral channels M, M=(λ max -λ min ) / △λ+1;

[0010] Step 2: Build a spectral perception network to obtain N spectral response values ​​of M wide-band spectral channels, where N≤M;

[0011] Step 3: Use the noise model and loss function to train the spectral perception network;

[0012] Imaging stage:

[0013] Step 4: Imaging spectrum encoding:

[0014] Placing the imaging target and the calibration plate in an imaging environment respectively, and encoding the imaging target and the calibration plate respectively using the spectral encoders with known N spectral response values ​​and low correlation coefficients;

[0015] Step 5: Collect 2×N groups of grayscale images through the detector:

[0016] The incident light is modulated by the N spectral encoders, and after being reflected by the target and the calibration plate, the light is imaged in front of the image detector, thereby obtaining N groups of grayscale images.

[0017] Step 6: Based on the light intensity distribution at different positions on the grayscale images of the N sets of calibration plates caused by the lighting system, normalize the grayscale images of the N sets of imaging targets respectively, and approximately convert the light intensity information under the non-parallel light illumination environment into the light intensity information under the parallel light illumination environment;

[0018] Step 7: Reconstruct the spectral image of the imaging target:

[0019] Using the grayscale images of the N groups of calibration plates and the N spectrum encoders to modulate the spectrum, the spectral range is reconstructed through the denoising network. min to λ max , a hyperspectral image with a wide-band spectral channel number of M.

[0020] The second step is to construct a spectrum perception network, including an encoding part and a decoding part;

[0021] Among them, the encoding part: the number of input units in the first layer is set to M, the number of output units is set to N, the middle layer is set to a fully connected layer (i.e., encoding layer), and the connection weight between the i-th (i=1, 2, ... M) input neuron and the j-th (j=1, 2, ... N) output neuron in the encoding layer is W ij , that is, the spectral response of the j-th coding filter in the i-th spectral channel;

[0022] The decoding part is used to extract the denoising network, where the number of input units is N, the number of output units is M, and there are several hidden layers in the middle.

[0023] The third step uses the noise model and loss function to train the spectral perception network, specifically including:

[0024] a) Select a known spectral dataset and input it into the encoding part of the spectral perception network;

[0025] b) Modeling the detector noise as a certain noise model or noise combination, adding a certain amount of noise perturbation to the light intensity output by the encoding part in space, and then using it as the input of the decoding part, i.e., the denoising network;

[0026] c) determining the loss function E(θ) used to train the spectral perception network, setting it as the optimization objective function, and continuously updating the network parameters;

[0027] The loss function E(θ) is calculated as: Among them, θ is the neural network parameter, and y k are the predicted value and actual value of the wide-band spectrum, respectively, and K is the number of training sets.

[0028] In the step 2, N>=4.

[0029] In the step 2, the parameters such as the structure and units of the hidden layer are determined according to the actual scenario.

[0030] In the second step, the weight of the fully connected layer is set to the transmittance of N existing spectral encoders with known spectral responses and low correlation coefficients, and the bias is set to 0.

[0031] In step 3, the training set used in the training process is spectral data from CAVE and ICVL Multispectral Image Database and several randomly synthesized Gaussian superposition curves.

[0032] Using deep neural network algorithm.

[0033] Compared with the prior art, the present invention has the following beneficial technical effects:

[0034] 1. The method of the present invention can improve the imaging quality of the spectral imaging system.

[0035] 2. The method of the present invention can reduce the imaging light source requirements and equipment volume cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 .Schematic diagram of the structure of the light intensity sensing network proposed in this invention.

[0037] Figure 2 . Schematic diagram of the compact hyperspectral imaging device under non-parallel light illumination conditions proposed in the present invention.

[0038] Figure 3 .Flowchart of the calibration of imaging targets by using a calibration plate in the present invention.

[0039] Figure 4 .Flowchart of the entire compact hyperspectral imaging method of the present invention.

[0040] Figure 5. Comparison of the reconstruction effects of the denoising network proposed in an embodiment of the present invention on color blocks of some color cards, including (a) the spectral imaging results of the SED network on some color cards, and (b) the spectral imaging results of the denoising network on some color cards. DETAILED DESCRIPTION

[0041] The present invention will be described in detail below with reference to the embodiments and drawings, but the protection scope of the present invention shall not be limited thereto.

[0042] Example

[0043] A compact hyperspectral imaging method based on a denoising network, comprising:

[0044] (1) Simulation stage

[0045] The spectrum sensing network established is as follows Figure 1 The specific plan is as follows:

[0046] Step 1: Determine the target wide-band spectral imaging range λ max =700nm,λ min =400nm, and the spectral interval of each broadband channel △λ=1nm, the number of broadband spectral channels M=301.

[0047] Step 2: Build a spectral perception network, including encoding and decoding parts, to obtain 16 spectral response values ​​of 301 wide spectral channels.

[0048] a) Encoding part: Set parameters. The number of input units in the first layer is set to 301, the number of output units is set to 16, and the middle layer is set to a fully connected layer (i.e., encoding layer). The weights of the fully connected layer are set to the spectral responses of the existing 16 known spectral encoders, and the bias is set to 0; the connection weight between the i-th (i=1, 2, ... 301) input neuron and the j-th (j=1, 2, ... 16) output neuron in the encoding layer is W ij , represents the spectral response of the j-th coding filter in the i-th spectral channel;

[0049] b) Decoding: Extract the denoising network. The number of input units is 16, the number of output units is 301, and there are two hidden layers in the middle. Each layer of neurons is processed using a leaky ReLu layer and a batch normalization layer.

[0050] Step 3: Use the Gaussian noise model and loss function to train the spectral perception network.

[0051] a) Select a dataset consisting of 700,000 known CAVE, ICVL Multispectral Image Database and the generated Gaussian stacked spectra and input them into the encoding part of the spectral perception network;

[0052] b) Modeling the detector noise as a Gaussian noise model, adding a 6% Gaussian noise perturbation to the light intensity output by the encoding part in space, and using it as the input of the decoding part, i.e., the denoising network;

[0053] c) Determine the loss function E(θ) used to train the spectral perception network and set it as the optimization objective function, namely the mean square error (MSE) between the spectral response output by the denoising network and the actual spectral response in the dataset. Continuously update the network parameters using the mini-batch stochastic gradient descent method.

[0054] The loss function E(θ) is calculated as: Among them, θ is the neural network parameter, and y k are the predicted and actual values ​​of the wide-band spectrum, K=700000.

[0055] (2) Experimental imaging stage

[0056] Establish as Figure 2 The hyperspectral imaging device shown in the figure has the following specific steps:

[0057] Step 4: Imaging spectrum encoding.

[0058] The imaging target and the calibration plate are placed in the imaging environment respectively, and the imaging target and the calibration plate at the same position are encoded respectively using the aforementioned 16 spectral encoders with known spectral responses and low correlation coefficients.

[0059] Step 5: Collect 2×16 sets of grayscale images through the detector.

[0060] Sixteen spectral encoders with different spectra are used to sequentially modulate the light emitted from the optical fiber. After being reflected by the target and calibration plate, the light is imaged in front of the image detector, and 16 sets of grayscale images are obtained respectively.

[0061] Step 6: Based on the intensity distribution of the 16 sets of calibration plates, which is bright in the center due to fiber optic illumination and becomes weaker towards the edge, normalize the grayscale images of the 16 sets of imaging targets respectively, and convert the intensity information of the non-parallel light illumination environment into the intensity information of the parallel light illumination environment. The specific processing flow chart is as follows: Figure 3 shown.

[0062] Step 7: Reconstruct the spectral image of the imaging target.

[0063] Using the 16 sets of grayscale images after calibration mentioned above and combining the modulation spectra of each known encoder, a hyperspectral image with a spectral range of 400 to 700 nm and 301 wide-band spectral channels was reconstructed through a denoising network.

[0064] The hyperspectral imaging device includes four parts: a spectral and fiber optic lighting device, an encoder wheel, a detector, and a processing system. The working principle is the above-mentioned hyperspectral imaging method, wherein the light source is a xenon lamp with a broad spectrum, the imaging target is a standard color card, and the calibration plate is a diffuse reflection plate.

[0065] The entire flow chart of the compact hyperspectral imaging method is as follows: Figure 4 By inputting the calibrated full-band encoded image into the trained denoising network, a hyperspectral image can be reconstructed.

[0066] In order to verify the accuracy of the proposed denoising network, some standard color cards were selected to test the reconstruction error of the denoising network obtained by the present invention, and compared with the reconstruction error of the existing direct reconstruction network (SED network) (Wen J, Hao L, Gao C, et al. Deep Learning-Based Miniaturized All-Dielectric Ultracompact Film Spectrometer[J]. ACS Photonics, 2022, 10(1): 225-233.). It was found that the denoising network has better reconstruction effects for both spectral and image evaluation indicators, and has strong universality. The specific results are shown in Figures 5(a) and 5(b). Among them, Figure 5(a) is the reconstruction effect diagram of the SED spectrum reconstruction network proposed previously for some color cards. The spectral reconstruction error is 7.29×10 -4 to 1.05×10 -2 Figure (b) shows the reconstruction effect of the denoising network on part of the color card. The spectral reconstruction error is 8.33×10 -4 to 8.23×10 -3 , which is further reduced compared with Figure (a).

[0067] PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index) are two widely used image evaluation indicators that can measure the differences or similarities between images from different angles. Therefore, these two indicators are selected to evaluate the image reconstruction results.

[0068] The PSNR and SSIM values ​​calculated based on the standard image and reconstructed image generated by the true spectral value are:

[0069] Table 1: SED spectrum reconstruction network and denoising network

[0070]

[0071] It can be seen that the use of denoising network also greatly improves the overall image reconstruction quality.

[0072] The above description is only an example of a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A compact hyperspectral imaging method based on a denoising network, characterized in that: The method includes a simulation stage and an imaging stage, and the specific steps are as follows: Simulation phase: Step 1: Determine the target wide-band spectral imaging range λ max ,λ min , and the spectral interval △λ of each broadband channel, the number of broadband spectral channels M, M=(λ max -λ min ) / △λ+1; Step 2: Build a spectral perception network to obtain N spectral response values ​​of M wide-band spectral channels, where N≤M; Step 3: Train the spectral perception network based on the noise model and loss function; Imaging stage: Step 4: Imaging spectrum encoding: Placing the imaging target and the calibration plate in an imaging environment respectively, and encoding the imaging target and the calibration plate respectively using the spectral encoders with known N spectral response values ​​and low correlation coefficients; Step 5: Collect 2×N groups of grayscale images through the detector: The incident light is modulated by the N spectral encoders, and after being reflected by the target and the calibration plate, the light is imaged in front of the image detector, thereby obtaining N groups of grayscale images. Step 6: Based on the light intensity distribution at different positions on the grayscale images of the N sets of calibration plates caused by the lighting system, normalize the grayscale images of the N sets of imaging targets respectively, and approximately convert the light intensity information under the non-parallel light illumination environment into the light intensity information under the parallel light illumination environment; Step 7: Reconstruct the spectral image of the imaging target: Using the grayscale images of the N groups of calibration plates and the N spectrum encoders to modulate the spectrum, the spectral range is reconstructed through the denoising network. min to λ max , a hyperspectral image with a wide-band spectral channel number of M.

2. A compact hyperspectral imaging method based on a denoising network according to claim 1, characterized in that: The second step is to construct a spectrum perception network, including an encoding part and a decoding part; Among them, the encoding part: the number of input units in the first layer is set to M, the number of output units is set to N, the middle layer is set to a fully connected layer (i.e., encoding layer), and the connection weight between the i-th, i=1, 2, ...M input neurons and the j-th, j=1, 2, ...N output neurons in the encoding layer is W ij , that is, the spectral response of the j-th coding filter in the i-th spectral channel; The decoding part is used to extract the denoising network, where the number of input units is N, the number of output units is M, and there are several hidden layers in the middle.

3. The compact hyperspectral imaging method based on a denoising network according to claim 1, characterized in that: The third step is to train the spectrum perception network according to the noise model and the loss function, specifically including: a) Select a known spectral dataset and input it into the encoding part of the spectral perception network; b) Modeling the detector noise as a certain noise model or noise combination, adding a certain amount of noise perturbation to the light intensity output by the encoding part in space, and then using it as the input of the decoding part, i.e., the denoising network; c) determining the loss function E(θ) used to train the spectral perception network, setting it as the optimization objective function, and continuously updating the network parameters; The loss function E(θ) is calculated as: Among them, θ is the neural network parameter, and y k are the predicted value and actual value of the wide-band spectrum, respectively, and K is the number of training sets.

4. The compact hyperspectral imaging method based on a denoising network according to claim 1, characterized in that: In the step 2, N>=4.

5. The compact hyperspectral imaging method based on a denoising network according to claim 2, characterized in that: In the step 2, the structure and unit parameters of the hidden layer are determined according to the actual scenario.

6. The compact hyperspectral imaging method based on a denoising network according to claim 2, characterized in that: In the second step, the weight of the fully connected layer is set to the transmittance of N existing spectral encoders with known spectral responses and low correlation coefficients, and the bias is set to 0.

7. The compact hyperspectral imaging method based on a denoising network according to claim 3, characterized in that: In step 3, the training set used in the training process is spectral data from CAVE and ICVL Multispectral Image Database and several randomly synthesized Gaussian superposition curves.

8. The compact hyperspectral imaging method based on a denoising network according to claim 7, characterized in that: Using deep neural network algorithm.

Images