A metasurface-based computational visible-near infrared spectral imaging chip

Abstract

The present invention discloses a computational visible-near infrared spectral imaging chip based on metasurface. The chip includes a plurality of spectral detection pixels, wherein: the spectral detection pixel includes a CMOS detection pixel array and a metasurface structure unit array; the metasurface structure unit array is composed of a plurality of metasurface structure units with different transmission spectra. The shape and size of the air holes of the metasurface structure unit are selected by a greedy algorithm. Several selected metasurface structure units are arranged in a square in the order given by the algorithm to form a period, and multiple periods are repeatedly arranged to form an array. One metasurface structure unit corresponds to one CMOS detection pixel, and multiple CMOS detection pixels within each period together form a spectral detection pixel. This spectral imaging chip can be perfectly compatible with semiconductor processes and CMOS integration, complete the function of modulating spectral information, and achieve high-resolution spectral imaging in the visible-near infrared band.

Description

Technical Field

[0001] The present invention belongs to the field of micro-nano optical design, and relates to a spectral imaging chip, specifically to a computational visible-near infrared spectral imaging chip based on metasurface. Background Art

[0002] Image sensors are the core components of smart phones, autonomous driving, and various complex imaging systems. With the vigorous development of artificial intelligence, traditional image sensors limited to two-dimensional intensity information detection can no longer meet the demand for imaging information volume. Spectral imaging technology can simultaneously obtain spatial information and spectral information of all points in the field of view, increasing the dimension of imaging information. While obtaining spatial image details, it can also detect the physical structure and chemical composition of the object to be measured. Traditional miniature spectrometers are divided into three types: dispersive type, filter type, and Fourier transform type. They all rely on spatial or temporal scanning for imaging, and their size, weight, and cost are very high. Computational spectral imaging devices use optical elements to modulate incident light and then detect it, and reconstruct the incident spectral information through algorithms, which can achieve single-shot imaging.

[0003] However, the spectral modulation structure of traditional computational spectral imaging devices and the CMOS detection pixel array adopt a discrete design, which leads to complex integration and alignment processes, increasing the difficulty of fabricating spectral imaging sensors. At the same time, due to the difficulty of completely aligning the modulation layer and the detection layer, spectral superposition is likely to occur at different wavelengths, bringing difficulties to spectral reconstruction.

[0004] Metasurface is a class of artificial two-dimensional structures with sub-wavelength structures, which can flexibly control the phase and frequency of the light field. At the same time, metasurface has the advantages of miniaturization and integration, which provides a new idea for solving the spectral modulation and integration problems of computational spectral imaging sensors. Summary of the Invention

[0005] In order to solve the problems of the existing spectral imaging systems, such as large volume, high price, low spectral resolution, difficulty in single-shot imaging, and poor imaging effect, the present invention provides a computational visible-near infrared spectral imaging chip based on metasurface. This spectral imaging chip uses a metasurface structure to replace the traditional filter layer or spectral modulation layer, can be perfectly compatible with semiconductor processes and CMOS integration, complete the function of modulating spectral information, and achieve high-resolution spectral imaging in the visible-near infrared band.

[0006] The object of the present invention is achieved through the following technical solutions:

[0007] A computational visible-near infrared spectral imaging chip based on metasurface, comprising a plurality of spectral detection pixels that can simultaneously achieve spectral modulation and light intensity detection, wherein:

[0008] The spectral detection pixel includes a CMOS detection pixel array located at the bottom for detecting optical signals and a metasurface structure unit array located on its upper layer for spectral modulation;

[0009] The metasurface structure unit is a silicon dielectric layer etched with air holes. The shape of the air holes is one of circular, cross-shaped, or swastika-shaped. A number of air holes with the same shape and size are periodically arranged on the silicon dielectric layer and form a square corresponding to the size of the CMOS pixel, forming the metasurface structure unit. By changing the shape and size parameters of the air holes, the transmission spectrum of the metasurface structure unit can be changed, realizing random modulation of spectral information in the visible-near infrared band, and then decoding the spectral information through a DNN neural network to achieve the function of spectral imaging;

[0010] The metasurface structure unit array is composed of a number of metasurface structure units with different transmission spectra. The shape and size of the air holes of the metasurface structure unit are selected by a greedy algorithm. The selected several metasurface structure units are arranged in a square in the order given by the greedy algorithm to form a period, and multiple periods are repeatedly arranged to form an array. One metasurface structure unit corresponds to one CMOS detection pixel, and multiple CMOS detection pixels within each period together form a spectral detection pixel.

[0011] The principle of the present invention lies in that the visible-near infrared spectral imaging chip based on the metasurface mainly consists of a CMOS detection pixel array at the bottom and a periodic metasurface structure unit array at the upper layer. The lateral structure of the metasurface structure unit is isotropic circular, cross-shaped, and swastika-shaped, which can provide polarization-insensitive phase modulation. By an optimization algorithm, the size parameters of the metasurface structure constituting the spectral detection pixel are selected to have the most uncorrelated spectral modulation curves in the visible-near infrared band. The integration method of the metasurface and the detection pixel is similar to that of a Bayer color filter, and the spectral information of the incident light in the visible-near infrared range can be reconstructed through a neural network algorithm.

[0012] Compared with the prior art, the present invention has the following advantages:

[0013] 1. Each superpixel unit of the periodic metasurface structure unit can independently modulate the light field. Compared with a computational spectrometer that uses diffractive optical elements to modulate the light field, the superpixel structure of this device can avoid spectral superposition at different wavelengths, reducing the reconstruction difficulty, and the energy utilization rate can theoretically approach 100%.

[0014] 2. Traditional computational spectral imaging sensor devices often adopt a structure where a top-layer photonic crystal flat plate and a bottom-layer detector array are designed separately and then aligned to achieve effective regulation of the light field and reduce the device volume. However, it is difficult to achieve perfect alignment in the process, resulting in a deteriorated imaging effect. The structure proposed in the present invention directly integrates the metasurface array on top of the detection pixels, featuring higher integration and imaging accuracy.

[0015] 3. The present invention relies on existing CMOS image sensors. The integration mode of the metasurface and the detection pixels refers to the Bayer color filter scheme, and the processing technology is compatible with the mature complementary metal semiconductor sensor manufacturing process, enabling processing and assembly on the same production line. Therefore, this structure has the potential for mass production and a strong application background.

[0016] 4. Current on-chip spectral imaging chips are all narrowband filtering types. Structures such as metasurfaces, nanowires, and quantum dots are introduced into the detectors to filter light of specific wavelengths, which is then absorbed and photoelectrically converted by the detectors. Each detector can only receive light of a single wavelength. In contrast, the computational spectral imaging chip based on metasurfaces designed in the present invention can regulate the spectra of the entire band, achieving broadband response and breaking the limitation in traditional filtering spectrometers that it is difficult to simultaneously achieve high spectral resolution and high spatial resolution.

[0017] 5. The present invention proposes a spectral imaging scheme with metasurface integration that has higher spatial resolution and higher spectral resolution. The device has a smaller volume and better imaging effect, and has the potential for pixel-level integration with mature CMOS image sensors. Through the DNN neural network algorithm, the spectral information of incident light in the visible-near infrared range can be reconstructed, capable of replacing traditional spectral cameras that use multiple discrete devices stacked to achieve spectral imaging. Therefore, the device proposed in the present invention can effectively increase the imaging dimension of current image sensors and promote the practical application process of metasurfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 FIG. is a schematic diagram of the structure and function of the computational visible-near infrared spectral imaging chip based on metasurfaces of the present invention.

[0019] Figure 2 FIG. is a schematic diagram of the structure of a common computational spectral imaging device at present.

[0020] Figure 3 FIG. is a schematic diagram of the metasurface unit structure, with cross-sections being circular, cross-shaped, and swastika-shaped respectively, and an example of the result obtained from parameter scanning is shown on the right.

[0021] Figure 4It is a schematic diagram of the process of computational spectral imaging and a schematic diagram of a spectral detection pixel. The metasurface structure units that make up the spectral detection pixel are determined by a greedy algorithm, and a DNN neural network algorithm is used to decode the detected light intensity signal and reconstruct the spectrum.

[0022] Figure 5 It is a result diagram obtained by using 16 kinds of metasurface unit structures to form a spectral modulation unit and reconstructing some random input spectra.

[0023] Figure 6 It is a result diagram obtained by reconstructing a real object scene, including a picture taken by an ordinary color camera and a spectral image obtained by converting the reconstructed spatial spectral data set into a pseudo-color, and spectral reconstructions are respectively performed on the selected red and blue feature pixel points. Specific implementation mode

[0024] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings, but it is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention shall be covered by the protection scope of the present invention.

[0025] Figure 2 As shown, it is the structure of a traditional computational spectral imaging device based on a photonic crystal slab, which is mainly composed of a top-layer photonic crystal slab and a bottom-layer CMOS image sensor array. The two layers are designed and manufactured independently and then combined. Precise processing is required during combination to align the metasurface units with the detector pixels one by one. This greatly increases the difficulty of device preparation and is difficult to achieve in actual process production. A slight deviation will cause energy loss and spectral information overlap, resulting in problems such as low energy utilization rate and decreased sensitivity, which is not conducive to subsequent detection and reconstruction of spectral information.

[0026] Based on the above problems existing in the structure of the traditional computational spectral imaging device based on a photonic crystal slab, the present invention provides a computational visible-near-infrared spectral imaging chip based on a metasurface, as Figure 1 shown. The chip includes a CMOS detection pixel array located at the bottom for detecting optical signals and a metasurface structure unit array located above it for spectral modulation, wherein: the metasurface structure unit array is composed of a plurality of metasurface structure units arranged periodically, one metasurface structure unit corresponds to one CMOS detection pixel, and the metasurface structure unit array is integrated with the CMOS detection pixel array to form an efficient spectral imaging device in the visible-near-infrared band.

[0027] In the present invention, the metasurface structural unit is a silicon dielectric layer, on which air holes with different shapes and sizes are etched. Based on its sub-wavelength structure, the metasurface can flexibly control the characteristics such as amplitude, phase, and polarization in the incident field. After passing through the metasurface, the incident light can generate a transmission response with rich spectral characteristics and output a transmission spectrum with random modulation characteristics. The metasurface structural unit array performs the spectral information encoding function and provides a complex spectral modulation function. The incident light field outputs an incident spectrum with a random line shape after passing through the metasurface layer, which is received by a detector and decoded by a subsequent neural network algorithm to obtain spectral information.

[0028] In the present invention, the metasurface structural unit uses silicon dielectric with excellent optical modulation ability in the visible-near infrared band of sub-wavelength size, with a thickness of th = 300 nm. Its manufacturing process is compatible with the mature complementary metal semiconductor sensor technology, having strong practicality and mass production potential. Silicon dielectric has advantages such as high absorption rate and high refractive index in the visible-near infrared band, can provide a complex spectral modulation function, and by adjusting the structure and size of the cross-section of the metasurface structural unit, effective modulation of the entire band can be achieved with fewer types of metasurface structural units, which can be used for subsequent overall structure optimization design.

[0029] In the present invention, the metasurface structural unit is a variety of geometric structures with isotropic cross-sections, including round holes, cross holes, and swastika holes. The metasurface composed of isotropic unit structures is polarization-insensitive in a wide band. Through optimized design, the metasurface structural unit can randomly modulate the spectral information in the visible-near infrared band, and the spectral information of the incident light can be reconstructed using a neural network algorithm to achieve the purpose of spectral imaging.

[0030] In the present invention, the parameter selection range of the metasurface structural unit is obtained by parameter sweeping. Figure 3The geometric structure of the metasurface designed in the present invention and the corresponding parametric scanning results are shown. For the convenience of subsequent processing, the thickness of the metasurface layer is unified and fixed at 300 nm. In order to exclude the influence of polarization information on the results, the shape of the nano-air holes on the metasurface structural unit is designed as a C4 symmetric pattern that is completely polarization-independent. By adjusting the parameters of the unit structure, different broadband spectral modulation characteristics can be generated. In the present invention, the shape of the metasurface structural unit is designed as a C4 symmetric circle, cross, and swastika. To meet the process growth conditions, the minimum side length of each metasurface structural unit shape should not be less than 100 nm; the adjustable parameters of the metasurface include: the period size p of the structure, the radius r of the circular air hole, the short side a of the cross-shaped air hole, and the short side a of the swastika-shaped air hole. When selecting the structural parameters of the metasurface structural unit, in order to make the transmission curve carry more information and meet the needs of the subsequent neural network for a large amount of uncorrelated data, the modulated random spectral response should be as complex as possible, and the response correlations obtained by different structures should be relatively low. At the same time, considering the absorption of the backend detector, the overall spectral transmittance should be as large as possible so that more light can be transmitted to the detector for detection. Based on the above selection criteria, the selection ranges of each structural parameter are determined as follows: Through the above selection criteria, a rough selection of the selection ranges of the metasurface structural unit parameters is made, and the selection ranges of each structural parameter are obtained: the radius of the circular structure is taken as 150 - 350 nm, the side length of the cross-shaped structure is taken as 100 - 300 nm, the side length of the swastika-shaped structure is taken as 100 - 200 nm, and the period of each structure is taken as 550 - 1050 nm. After the parameter ranges are selected, a fine parameter scan is performed on each metasurface structure, and three 3D data matrices of 41×101×501 are obtained through two-dimensional scanning. Several representative spectral response curves are selected and shown as Figure 3 shown.

[0031] In the present invention, the CMOS detection pixel can detect the light intensity information in the visible-near infrared band, has almost no influence on the optical field modulation, is used to receive the optical signal modulated by the metasurface and convert it into a digital signal and input it into the backend algorithm. The CMOS processing technology is mature and has the value of batch production.

[0032] In the present invention, the spectral detection pixel is jointly composed of a CMOS pixel and the metasurface structural unit located on it. The greedy algorithm is used to select several metasurface structural units with the least correlated transmission spectra and arrange them periodically and integrated on the top of the detection pixel. Multiple CMOS detection pixels within each period jointly form a spectral detection pixel. The optical signal is modulated by the metasurface structural unit and then received by the detector to realize the recording of spectral information.

[0033] Figure 4The following is a schematic diagram of the entire process of encoding and reconstructing the spectrum in the present invention. The incident light is modulated by the metasurface structural unit to generate a random spectrum, which is received by the CMOS detection pixel to generate an original image containing only two-dimensional light intensity information. Each spectrum modulation structural unit is composed of a variety of different periodic structures of the metasurface structural unit. Each metasurface structural unit independently regulates the spectrum. In order to make the modulated random spectrum carry more information, it should be ensured that the correlation of the spectrum modulation functions of different metasurfaces is small. Therefore, it is necessary to design the metasurface periodic structure and the spectrum modulation structural unit separately, and at the same time, it is necessary to build a neural network for spectrum reconstruction. The specific process of selecting the metasurface structural unit by using the greedy algorithm is as follows: First, randomly select the transmission spectrum t of a unit structure from the dataset of the transmission response sampling basis i , and calibrate it as the first matrix element t1 of the transmission response matrix T in the spectrum reconstruction process S = T·I; then, calculate the correlation coefficient between it and the transmission spectra t of the remaining other structures in the parameter scan dataset j , and find the structure with the smallest correlation coefficient with t1 as the second matrix element t2 calibrated in the transmission response matrix T; the correlation coefficient between the transmission spectra of two metasurface structural units is defined as:

[0034]

[0035] where t i , t j represent the transmission spectrum responses of the i-th and j-th metasurface units, cov(t i , t j ) represents the covariance between the two transmission spectrum responses of t i and t j , represents the mean value of t i , t j , represents the standard deviation of t i , t j .

[0036] For Figure 4 in the spectrum reconstruction part, the present invention uses the deep neural network algorithm to decode and reconstruct the spectrum information. The random spectrum response generated after passing through the metasurface is used as the input layer of the neural network, and the spectrum response curve carried by the incident light is used as the output layer. The database is trained with the fully connected layer, and the trained neural network is used to decode and reconstruct the spectrum information, so as to decode the spectrum information of the incident light from the light intensity information detected by the detector.

[0037] Figure 5The figure shows the result of reconstructing the incident light spectrum information calculated using a neural network. This result is calculated using a spectral modulation unit composed of 16 meta - surface unit structures with the least correlation of the transfer matrices calculated by the greedy algorithm. For some random input spectra, meta - surface encoding and neural network algorithm decoding are performed. The reconstructed spectral response curve is compared with the spectral response curve carried by the incident light. To quantitatively analyze the coincidence degree between them, the concept of fidelity is introduced. As Figure 5 The shown reconstruction results all have a large spectral reconstruction fidelity, which can prove that the spectral modulation unit and neural network algorithm designed by the present invention can effectively realize spectral reconstruction. I is the original spectrum, I' is the reconstructed spectrum, and the magnitude of the fidelity F(I, I') is defined as:

[0038]

[0039] Figure 6 The figure shows the pseudo - color map of spectral reconstruction of the physical scene calculated, as well as the result of reconstructing the spectral response curve of the characteristic pixel points in the calculated picture. This result is still calculated using the spectral modulation unit composed of 16 meta - surface unit structures. The spectral camera is used to take pictures and decoded through neural network calculation. The reconstructed spatial spectral data set is converted into a pseudo - color spectral image and compared with the physical picture taken by an ordinary camera. From the results, it can be seen that the details and textures in the picture are well restored in the reconstructed pseudo - color picture. By selecting two characteristic pixel points, red and blue, to plot the spectral response curves, it can be seen that the reconstructed curves coincide well with the original curves. The spectral response of the red pixel points is concentrated in the range of 650 - 750 nm, and the spectral response of the blue pixel points is concentrated in the range of 450 - 500 nm. The spectral response corresponds to the color, demonstrating the imaging effect of the spectral imaging chip of the present invention.

Claims

1. A computational visible-near infrared spectral imaging chip based on metasurface, characterized in that The chip includes several spectral detection pixel units that can simultaneously achieve spectral modulation and light intensity detection, where: The spectral detection pixel unit includes a CMOS detection pixel array located at the bottom for detecting optical signals and a metasurface structure unit array located above it for spectral modulation; The metasurface structure unit is a silicon dielectric layer etched with air holes. A number of air holes with the same shape and size are periodically arranged on the silicon dielectric layer and form a square corresponding to the size of the CMOS pixel, forming the metasurface structure unit. By changing the shape and size parameters of the air holes, the transmission spectrum of the metasurface structure unit can be changed to achieve random modulation of spectral information in the visible-near infrared band, and then the spectral information is decoded by a DNN neural network to achieve the function of spectral imaging; The metasurface structure unit array is composed of several metasurface structure units with different transmission spectra. The shapes and sizes of the air holes of the metasurface structure units are selected by the greedy algorithm. The selected several metasurface structure units are arranged in a square in the order given by the greedy algorithm to form a period, and multiple periods are repeatedly arranged to form an array. One metasurface structure unit corresponds to one CMOS detection pixel, and multiple CMOS detection pixels in each period jointly form a spectral detection pixel. The specific process of selecting the shapes and sizes of the air holes of the metasurface structure units by the greedy algorithm is as follows: First, randomly select the transmission spectrum t of a unit structure from the dataset of the transmission response sampling basis i , and calibrate it as the first matrix element t1 of the transmission response matrix T in the spectral reconstruction process S = T·I; then, calculate the correlation coefficient between it and the transmission spectra t of the remaining other structures in the parameter scan dataset j , and find the structure with the smallest correlation coefficient with t1 as the second matrix element t2 calibrated in the transmission response matrix T; the magnitude of the correlation coefficient between the transmission spectra of two metasurface structure units is defined as: where t i , t j represent the transmission spectrum responses of the i-th and j-th metasurface units, and cov(t i , t j ) represents the covariance between the two transmission spectrum responses of t i and t j . represents the mean of t i , t j , represents the standard deviation of t i , t j .

2. The computational visible-near infrared spectral imaging chip based on metasurface according to claim 1, wherein The thickness of the silicon dielectric layer is 300 nm.

3. The computational visible-near infrared spectral imaging chip based on metasurface according to claim 1, wherein The air hole is one of a circular structure, a cross-shaped structure, or a character-'wan'-shaped structure.

4. The computational visible-near infrared spectral imaging chip based on metasurface according to claim 3, wherein For the circular structure, the radius is 150 - 350 nm; for the cross-shaped structure, the side length is 100 - 300 nm; for the character-'wan'-shaped structure, the side length is 100 - 200 nm. The period of each structure is 550 - 1050 nm.

5. The computational visible-near infrared spectral imaging chip based on metasurface according to claim 1, wherein When decoding the spectral information by the DNN neural network, the random spectral response generated after passing through the metasurface is used as the input layer of the neural network, and the spectral response curve carried by the incident light is used as the output layer. The database is trained with a fully connected layer, and the trained neural network is used to decode and reconstruct the spectral information to decode the spectral information of the incident light from the light intensity information detected by the detector.

Images