High-detection-efficiency long-wave infrared spectrum imaging device and imaging method

Abstract

The invention discloses a high-detection-efficiency long-wave infrared spectrum imaging device and an imaging method, and belongs to the field of micro-nano optical design, the scheme is as follows: a metasurface spectrum modulation layer is composed of a square matrix formed by arranging a plurality of metasurface periodic units in sequence, and each metasurface periodic unit is composed of a plurality of metasurface structure units; each metasurface structure unit comprises a substrate and a plurality of nano columns, and the plurality of nano columns are distributed on the substrate in a C4 symmetrical structure; reverse design of the high-transmittance metasurface structure units is achieved through a particle swarm optimization algorithm, and structure parameters of the high-transmittance metasurface structure units are obtained; incident light passes through the metasurface spectrum modulation layer to realize spectrum coding, coding information is converted into an electric signal by a long-wave infrared focal plane detection pixel below, and a high-resolution long-wave infrared spectrum image can be reconstructed in cooperation with a deep neural network algorithm. The spectral imaging method provided by the invention is perfectly compatible with semiconductor and MEMS technologies, and long-wave infrared band spectral imaging with high detection efficiency is realized.

Description

Technical Field

[0001] The present invention belongs to the field of micro-nano optical design, and relates to a spectral imaging method, in particular to a long-wave infrared spectral imaging device and imaging method with high detection efficiency. Background Art

[0002] Artificial intelligence and machine vision play an increasingly important role in industrial production. Infrared imaging technology has better application effects than visible light imaging in extremely harsh weather environments such as night, haze, and sandstorms. As the third dimension independent of the two-dimensional intensity distribution, the spectrum contains rich information. Traditional long-wave infrared spectrometers are divided into grating dispersion type, Fourier interference type, and rotating filter wheel type. The spectroscopic optical path not only results in a relatively large device volume but also has high requirements for the stability of the platform. In terms of imaging, these spectrometers rely on spatial scanning and cannot perform real-time imaging. At the same time, the price of the entire device is also very expensive.

[0003] With the in-depth research on metamaterials, researchers have found that integrating a metamaterial absorber onto the thermosensitive material of an infrared detector can endow it with the ability to independently adjust the spectral response. However, the integrated microbolometer has fewer spectral channels, and specific metamaterial absorbers need to be customized for different bands to meet the spectral modulation requirements. There is still a long way to go to achieve broadband large focal plane real-time spectral imaging.

[0004] Metasurface is a two-dimensional form of metamaterial. Metasurfaces have the characteristics of broadband spectral modulation and are compatible with semiconductor and MEMS processes. The transmission spectrum of the metasurface obtained based on parameter scanning is a forward design, which is often random. Inverting the metasurface structural parameters from the target spectrum is a reverse design. In particular, a spectral modulation layer with high transmittance can greatly improve the sensitivity of the spectrometer. With the rapid development of micro-nano processing technology and computational optical imaging technology, the computational long-wave infrared spectral imaging technology based on metasurfaces provides a new idea for broadband spectral imaging in the long-wave infrared band. Summary of the Invention

[0005] Aiming at the deficiencies of current long-wave infrared spectral imaging devices, the present invention provides a long-wave infrared spectral imaging device and imaging method with high detection efficiency. This spectral imaging method uses a high-transmittance metasurface spectral modulation layer based on reverse design to replace the filter wheel or spectroscopic optical path in the traditional spectral imaging system. At the same time, it is perfectly compatible with semiconductor and MEMS processes, and the degree of integration is greatly improved, realizing efficient spectral imaging in the long-wave infrared band.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A long-wave infrared spectral imaging device with high detection efficiency, comprising a number of spectral detection pixels. Each spectral detection pixel includes a long-wave infrared focal plane detection pixel array at the bottom and a metasurface spectral modulation layer on its surface. The metasurface spectral modulation layer is composed of a square matrix formed by arranging multiple metasurface periodic units in sequence. Each metasurface periodic unit is composed of multiple metasurface structural units, and the multiple metasurface structural units are arranged in a square matrix. Each metasurface structural unit includes a substrate and a number of nanocolumns, and the number of nanocolumns is fixed on the substrate. The distribution of the number of nanocolumns on the substrate has a C4 symmetry structure.

[0008] Preferably, the material of the substrate is ZnS or ZnSe, and the material of the nanocolumns is Si or Ge. Preferably, the material of the substrate is ZnS, and the material of the nanocolumns is Ge.

[0009] Furthermore, in each metasurface structural unit, the thickness range of the substrate is 4 - 8 μm, its cross-section is square, and the side length is 8 - 12 μm. Square nanocolumns are etched above the substrate, with a column height of 4 - 6 μm and a bottom side length of 0.4 - 0.8 μm.

[0010] Furthermore, the bottom surface size of each metasurface periodic unit corresponds to one or more long-wave infrared focal plane detection pixels.

[0011] Furthermore, the composition of the metasurface structural unit is as follows: the substrate plane is evenly divided into four sub-squares, each sub-square is divided into an N×N grid, and then an N×N Boolean matrix called C2_initial is generated. The probability of each matrix element being 0 and 1 is equal, so that the spatial position distribution of the sub-square corresponds one-to-one with the matrix element distribution of the Boolean matrix. The C2_initial is rotated clockwise three times in sequence to obtain three matrices, and the C2_initial and the three matrices obtained by each rotation are spliced in sequence to obtain a 2N×2N matrix with a C4 symmetry structure. Nanocolumns are etched at the spatial positions where the matrix elements are 1, otherwise not.

[0012] An imaging method for the long-wave infrared spectral imaging device with high detection efficiency described above. The incident light is spectrally modulated by the metasurface spectral modulation layer through the resonance effect. The modulated light intensity is converted into an electrical signal by the long-wave infrared focal plane detection pixel array below, and the trained deep neural network is used to decode the electrical signal to reconstruct the incident spectral image.

[0013] Furthermore, the particle swarm optimization algorithm is used to realize the inverse design of the structural parameters of the metasurface structural unit. The specific steps are as follows:

[0014] Step 1: Generate a group of target spectra as the object of optimization;

[0015] Step 2: Initialize the relevant parameters of particle swarm optimization;

[0016] Step 3: Conduct simulation and calculate the fitness of the result and the target spectrum;

[0017] Step 4: Update the individual and global optimal solutions;

[0018] Step 5: Update the velocity and position of the particle swarm;

[0019] Step 6: At the end of one loop iteration, check whether the conditions for terminating the loop are met, including whether the maximum number of iterations is reached or the fitness converges to a preset value. If any of the conditions is satisfied, stop the loop and output the result. Otherwise, continue the next loop iteration until the global optimal solution corresponding to the lowest fitness is obtained.

[0020] Further, in the said Step 3, each group of target spectra consists of M data points. Define the mean square error between the i-th target spectrum and the simulated transmission spectrum as their fitness:

[0021]

[0022] where tt ij 、ts ij represent the j-th data of the transmission spectrum of the i-th metasurface structural unit and its target spectrum.

[0023] Further, in the said Step 5,

[0024] The update formulas for the velocity v(t) and position x(t) of the particle swarm in each loop are:

[0025] v i (t + 1) = w·v i (t) + c1·r1·(pbest i - x i (t)) + c2·r2·(gbest - x i (t)) (2)

[0026] x i (t + 1) = x i (t) + v i (t) (3)

[0027] where t is the number of the current loop iteration, w is the inertia weight factor, c1, c2 are the personal and social learning factors, r1, r2 are random numbers between 0 and 1, pbest i is the individual optimal solution of the i-th particle, and gbest is the global optimal solution of the particle swarm.

[0028] Further, before the deep neural network decodes the electrical signal, the neural network needs to be trained. During training, the encoded information is used as the input of the network, and the incident spectral information is used as the output of the network. The middle is a fully connected layer, and the trained network is saved for future use.

[0029] By changing the structural parameters of the metasurface structural unit, the transmission spectrum of the metasurface periodic unit can be changed, realizing the modulation of the long-wave infrared band spectrum. The detection signal is decoded by the deep neural network to achieve spectral imaging. The structural parameters of the metasurface structural unit are obtained by the particle swarm optimization algorithm. Different from the randomness of the transmission spectrum obtained by parameter scanning, the transmission spectrum of the metasurface obtained by inverse design is almost completely consistent with the overall trend of the preset target spectrum. A group of target spectra with a relatively high average transmittance is generated before optimization as the optimization target.

[0030] The particle swarm optimization algorithm runs in the electromagnetic simulation software. Here, the distribution of nanocolumns in the etching layer of the metasurface structural unit is transformed into an N×N Boolean matrix. A matrix element of 1 indicates that a nanocolumn is etched at that spatial position, and a matrix element of 0 indicates no etching.

[0031] The decoding of the detection signal is achieved by a deep neural network, which consists of an input layer, a fully connected layer, and an output layer. The neural network needs to be trained before decoding. Since the role of the network is to reconstruct the incident spectrum, the incident light intensity is used as the output of the network and the modulated detection light intensity is used as the input of the network during training. The trained network is reserved for future use.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. The spectral modulation layer composed of metasurface periodic units is used to replace the traditional filter wheel or spectroscopic optical path, reducing the volume and cost of the device.

[0034] 2. Compared with the push-broom spectral imaging scheme, this method adopts a spatial metasurface array architecture to modulate the spectra within the band simultaneously on the incident surface, realizing real-time imaging.

[0035] 3. The metasurface structural unit based on inverse design overcomes the randomness of the parameter-scanned metasurface spectrum, and the transmission spectrum has a relatively high average transmittance throughout the band, greatly improving the detection efficiency and detection sensitivity.

[0036] 4. Integrating the metamaterial absorber on the thermally sensitive detection pixel can achieve specific wavelength and polarization regulation, but the number of spectral channels is small, similar to a narrowband filter wheel, and it is not suitable for broadband spectral imaging.

[0037] 5. The deep neural network decoding framework is highly robust to the correlations between the transmission spectra of the metasurface structural units. Even when the average correlation coefficient is relatively large, the test set still maintains a good reconstruction effect.

[0038] 6. This invention relies on the existing long-wave infrared focal plane detection pixel array. The manufacturing process of the metasurface is compatible with semiconductor and MEMS processes, and the processing technology is mature, with the potential for large-scale preparation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic structural diagram of the long-wave infrared spectral imaging device of the present invention;

[0040] Figure 2 It is a schematic plan view and three-dimensional structural diagram of a single metasurface structural unit;

[0041] Figure 3 It is a flow chart of the particle swarm optimization algorithm;

[0042] Figure 4 It is a schematic diagram of the optimized metasurface structural unit and the corresponding transmission spectrum;

[0043] Figure 5 It is a schematic diagram of the complete detection and imaging mechanism of the present invention;

[0044] Figure 6 It is a schematic diagram of some input spectrum reconstruction results and reconstruction fidelity provided in Embodiment 1 of the present invention;

[0045] Figure 7 It is a schematic diagram of the original image and the reconstructed image of the long-wave infrared spectral database provided in Embodiment 1 of the present invention and shows the spectrum reconstruction results corresponding to two pixel points. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] Next, the technical solutions in the present invention will be clearly and completely described in conjunction with the drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1:

[0048] The present invention provides a long-wave infrared spectral imaging device and imaging method with high detection efficiency, as Figure 1As shown in the figure, the long-wave infrared spectral imaging device includes a long-wave infrared focal plane detection pixel array located at the bottom for detecting modulated spectra and a metasurface spectral modulation layer located on its surface. The imaging method is as follows: the incident light is spectrally modulated by the metasurface spectral modulation layer through the resonance effect, the modulated light intensity is converted into an electrical signal by the long-wave infrared focal plane detection pixel array below, and the trained deep neural network is used to decode the detection signal to reconstruct the incident spectral image.

[0049] As Figure 1 shown in the figure, the metasurface spectral modulation layer is constructed by arranging multiple metasurface periodic units in sequence to form a square matrix. Each metasurface periodic unit is a square matrix composed of multiple metasurface structural units, and these metasurface structural units can interact with light at specific optical wavelengths to achieve the modulation function of the incident light.

[0050] In the present invention, ZnS is selected as the substrate material for the metasurface structural unit, and Ge nanocolumns are etched on it. The distribution of Ge nanocolumns on the ZnS substrate has a C4 symmetric structure. As Figure 2 shown in the figure, during simulation modeling, the substrate plane is evenly divided into four sub-square matrices, and each sub-square matrix is divided into an N×N grid. Then a Boolean matrix of N×N called C2_initial is generated, and the probability of each matrix element being 0 and 1 is equal, so that the spatial position distribution of the sub-square matrix corresponds one-to-one with the matrix element distribution of the Boolean matrix. The C2_initial is flipped clockwise three times in sequence to obtain three matrices, and the C2_initial and the matrices after each flip are sequentially spliced to obtain a 2N×2N matrix with C4 symmetry. Ge nanocolumns are etched at the spatial positions where the matrix elements are 1, otherwise not.

[0051] The traditional parameter scanning method obtains a set of transmission spectrum databases by changing parameters such as the length, width, height, duty cycle, and period of the metasurface, and then selects a set of the smallest metasurface structural units according to the correlation. The transmission spectra obtained by this forward design method have the characteristics of random regulation and relatively low average transmittance in the entire wavelength range. In the present invention, the metasurface structural units constituting the spectral modulation layer are obtained by the particle swarm optimization algorithm. This reverse design method can make the transmission spectra of the optimized metasurface be consistent with the target spectra in the overall trend. As Figure 3 shown in the figure, the flow chart of the particle swarm optimization algorithm is specifically implemented according to the following steps:

[0052] Step 1: Initialize the parameters of the particle swarm, including generating a set of target spectra as the object of optimization; setting the geometric parameters of a single metasurface structural unit and the definitions of various parameters in the algorithm.

[0053] Step 2: Based on the parameters, model and run to obtain the transmission spectrum of the metasurface, and calculate their fitness with the target spectrum. Each group of target spectra consists of M data points. Define the mean square error between the i-th target spectrum and the simulated transmission spectrum as their fitness:

[0054]

[0055] where tt ij , ts ij are the j-th data points of the transmission spectrum of the i-th metasurface structural unit and the corresponding target spectrum.

[0056] Step 3: After each generation of the loop ends, update the optimal solution of the particle swarm according to the obtained fitness, and save the position of the particle corresponding to the optimal solution.

[0057] Step 4: Update the velocity v(t) and position x(t) of the particle swarm based on the current individual and global optimal solutions. The formulas for velocity and position are defined as follows:

[0058] v i (t + 1) = w·v i (t) + c1·r1·(pbest i - x i (t)) + c2·r2·(gbest - x i (t)) (2)

[0059] x i (t + 1) = x i (t) + v i (t) (3)

[0060] where t is the current iteration number, w is the inertia weight factor, c1, c2 are the personal and social learning factors, r1, r2 are random numbers between 0 and 1, pbest i is the individual optimal solution of the i-th particle, and gbest is the global optimal solution of the particle swarm.

[0061] Step 5: After one loop iteration ends, check whether the conditions for terminating the loop are met, including whether the maximum iteration number is reached or the fitness converges to a preset value. If any condition is satisfied, stop the loop and output the result; otherwise, continue the next loop iteration.

[0062] Figure 4 Shows the three-dimensional structures of three groups of optimized metasurface structural units and their transmission spectra and target spectral curves. The mean square error between the curves is less than 0.1%, indicating that the algorithm has well achieved the inverse design of the target spectrum.

[0063] Figure 5Shows the complete detection mechanism for reconstructing the incident spectral image based on this spectral imaging method. First, the incident light contains rich spectral information. When these thermal spectral data pass through the metasurface modulation layer, they are encoded into two-dimensional light intensity information by the transmission response of the metasurface. Then, the long-wave infrared focal plane detection pixel array below receives the light intensity signal, and the thermosensitive material and the readout circuit convert the light intensity distribution into the original detection signal. Finally, the detection signal is fed as input to the trained deep neural network, and the network acts as a decoder to reconstruct the incident spectrum. To quantitatively measure the reconstruction effect, the concept of fidelity is introduced, and the incident spectrum I O and the reconstructed spectrum I R The fidelity F(I O ,I R ) is defined as:

[0064]

[0065] Figure 6 Shows the long-wave infrared spectral curve reconstructed by the deep neural network. Whether it is the spectral curve in the database or the Gaussian weight curve with random peak distribution, the reconstruction result maintains a very high fidelity with the incident spectrum.

[0066] As Figure 7 shown, the original image of a certain spectral channel in the thermal spectral database is in the upper left corner. Taking the thermal spectral data as the real incident spectrum, the image of the same spectral channel photographed by the spectral camera and decoded and reconstructed by the deep neural network is in the upper right corner. It can be seen from the results that the reconstructed spectral image restores some details and texture features in the original image. The incident spectrum and the reconstructed spectral curve corresponding to two pixel points are shown below the picture. It can be seen that the reconstructed spectrum coincides well with the incident spectral curve, indicating that this method has well achieved long-wave infrared spectral imaging.

[0067] In addition, it should be understood that although this specification is described according to the embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A long-wave infrared spectral imaging device with high detection efficiency, characterized in that, It includes a number of spectral detection pixels. Each spectral detection pixel includes a long-wave infrared focal plane detection pixel array at the bottom and a metasurface spectral modulation layer on its surface. The metasurface spectral modulation layer is composed of a square matrix formed by arranging multiple metasurface periodic units in sequence. Each metasurface periodic unit is composed of multiple metasurface structural units. The multiple metasurface structural units are arranged in a square matrix. Each metasurface structural unit includes a substrate and a number of nanocolumns. The number of nanocolumns is fixed on the substrate, and the distribution of the number of nanocolumns on the substrate has a C4 symmetric structure.

2. The long-wave infrared spectral imaging device with high detection efficiency according to claim 1, characterized in that: The material of the substrate is ZnS or ZnSe, and the material of the nanocolumns is Si or Ge.

3. The long-wave infrared spectral imaging device with high detection efficiency according to claim 1, characterized in that: In each metasurface structural unit, the thickness range of the substrate is 4-8 μm, its cross-section is square, the side length is 8-12 μm, square nanocolumns are etched above the substrate, the column height is 4-6 μm, and the bottom side length is 0.4-0.8 μm.

4. The long-wave infrared spectral imaging device with high detection efficiency according to claim 1, wherein: The bottom surface size of each metasurface periodic unit corresponds to one or more long-wave infrared focal plane detection pixels.

5. The long-wave infrared spectral imaging device with high detection efficiency according to claim 1, characterized in that: The composition of the metasurface structural unit is as follows: the substrate plane is evenly divided into four sub-square matrices, each sub-square matrix is divided into an N×N grid, and then a Boolean matrix called C2_initial is generated. The probability of each matrix element being 0 and 1 is equal, so that the spatial position distribution of the sub-square matrix corresponds one-to-one with the matrix element distribution of the Boolean matrix. The C2_initial is rotated clockwise three times in sequence to obtain three matrices. The C2_initial and the three matrices obtained by each rotation are spliced in sequence to obtain a 2N×2N matrix with a C4 symmetric structure. Nanocolumns are etched at the spatial positions where the matrix elements are 1, otherwise they are not etched.

6. An imaging method for a long-wave infrared spectral imaging device with high detection efficiency according to any one of claims 1-5, characterized in that: The incident light is spectrally modulated by the metasurface spectral modulation layer through the resonance effect. The modulated light intensity is converted into an electrical signal by the long-wave infrared focal plane detection pixel array below. The trained deep neural network is used to decode the electrical signal to reconstruct the incident spectral image.

7. The imaging method of a long-wave infrared spectral imaging device with high detection efficiency according to claim 6, characterized in that: The particle swarm optimization algorithm is used to realize the inverse design of the structural parameters of the metasurface structural unit. The specific steps are as follows: Step 1: Generate a group of target spectra as the object of optimization; Step 2: Initialize the relevant parameters of the particle swarm optimization; Step 3: Conduct simulation and calculate the fitness of the result and the target spectrum; Step 4: Update the individual and global optimal solutions; Step 5: Update the velocity and position of the particle swarm; Step 6: At the end of one cycle iteration, check whether the conditions for terminating the loop are met, including whether the maximum number of iterations is reached or the fitness converges to a preset value. If any condition is met, stop the loop and output the result. Otherwise, continue the next cycle iteration until the global optimal solution corresponding to the lowest fitness is obtained.

8. The imaging method of a long-wave infrared spectral imaging device with high detection efficiency according to claim 7, characterized in that: In the said Step 3, each group of target spectra is composed of M data points. Define the mean square error between the i-th target spectrum and the simulated transmission spectrum as their fitness: where tt ij and ts ij represent the j-th data of the transmission spectrum of the i-th metasurface structural unit and its target spectrum, respectively.

9. The imaging method of a long-wave infrared spectral imaging device with high detection efficiency according to claim 7, characterized in that, In the said Step 5, The update formulas for the velocity v(t) and position x(t) of the particle swarm in each cycle are: v i (t + 1)= w·v i (t)+ c1·r1·(pbest i - x i (t))+ c2·r2·(gbest - x i (t)) (2) x i (t + 1)= x i (t)+ v i (t)(3) where t is the number of current loop iterations, w is the inertia weight factor, c1 and c2 are the personal and social learning factors, r1 and r2 are random numbers between 0 and 1, and pbest i is the individual best solution of the i-th particle, and gbest is the global best solution of the particle swarm.

10. The imaging method of a long-wave infrared spectral imaging device with high detection efficiency according to claim 6, characterized in that: Before the deep neural network decodes the electrical signal, the neural network needs to be trained. During training, the encoded information is used as the input of the network, and the incident spectral information is used as the output of the network. In the middle is a fully connected layer, and the trained network is saved for future use.

Images