On-chip calculation type hyperspectral imaging device

Abstract

The invention provides the on-chip calculation type hyperspectral imaging device which is simple to process and low in cost. The device is based on an on-chip calculation type hyperspectral imaging system of an F-P cavity, and 61-channel hyperspectral image reconstruction with the wavelength of 400-1000 nm and the interval of 10 nm can be achieved. Due to the fact that metasurfaces and multi-layer films need to be elaborately designed to achieve 400-1000 nm wide-spectrum random coding, the difference between actual machining and design is large, and the fluctuation of 700-1000 nm is small. Therefore, an F-P cavity is used for designing four optical filter sets with large volatility and small correlation, the optical filter sets are arranged according to a mosaic array mode (2 * 2), and an on-chip calculation type hyperspectral imaging system is achieved. According to the F-P cavity, amorphous silicon is sequentially plated with Ti-Ag-Ti-ZrO2-Ti-Ag-Ti-SiO2, wherein Ti is 5 nm, Ag is 15 nm, SiO2 is 20 nm, and four optical filters of ZrO2 in the middle are 1000 nm, 1050 nm, 1100 nm and 1150 nm respectively. Wherein ZrO2 only needs to move the same mask plate for three times, so that the processing difficulty of the optical filter is greatly reduced, and the processing time of the optical filter is greatly shortened. And finally, reconstructing a 61-channel hyperspectral image with the wavelength of 400-1000 nm and the interval of 10 nm through a spectrum reconstruction algorithm.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral imaging technology, and in particular to an on-chip computational hyperspectral imaging device based on a Fabry-Perot (FP) cavity, which is used to efficiently acquire hyperspectral data within a wide spectral range and is particularly suitable for fields such as biomedical imaging, environmental monitoring, industrial detection and remote sensing. Technical Background

[0002] Spectral imaging technology has a wide range of applications in remote sensing, medical diagnosis, archaeological and art conservation, and food inspection due to its ability to obtain unique information in the spatial and spectral domains. Traditional spectral imaging methods include broom scanning, push-broom scanning, and wavelength scanning. A widely recognized example of broom-type pixel-by-pixel scanning is the airborne visible / infrared imaging spectrometer, which implements broom method scanning for ground remote sensing on an aircraft. The push-broom device uses an entrance slit to generate images line by line. The hyperspectral digital image acquisition experimental instrument uses a prism spectrometer to implement push-broom imaging optics, providing good remote sensing capabilities. The wavelength scanning method captures the spectral image cube by exchanging narrow bandpass filters or using tunable filters in front of the camera lens. However, traditional scanning methods have slow spectral image acquisition speeds due to the time-consuming scanning mechanism. Therefore, they are not suitable for large scenes or dynamic recording. To solve this problem, researchers began to explore snapshot spectral imaging methods. Early attempts included integral field spectroscopy, multispectral beam splitting, and image replication imaging spectrometers. Although these methods can achieve multispectral imaging through light splitting, they cannot obtain massive spectral channels, and the optical devices are bulky.

[0003] In recent years, computational hyperspectral imaging technology has attracted much attention due to its compact structure and snapshot characteristics. Random coding design combined with metasurfaces or multilayer films has gradually become a hot topic. However, the spectral response of RGB is not suitable for real-world hyperspectral reconstruction. To this end, the academic community has proposed many filter design schemes with wide-spectrum random coding. However, most of these schemes have problems such as high design difficulty, complex processing process, and large difference between actual manufacturing and simulation. The existing metasurfaces and multilayer films require precise design to achieve broadband random coding, which leads to cumbersome processing. Taking a 4*4 filter array as an example, the actual production and manufacturing requires moving the mask 15 times and performing micro-nano processing in sequence. In addition, the actual manufacturing is limited by the accuracy of the equipment, resulting in significant differences in filters produced in different batches. Taking a multilayer film as an example, due to the use of the thin film interference principle, the change of one layer of the film will cause a large difference in the spectral curve, which is not conducive to the design of a universal spectral reconstruction network.

[0004] In view of the above problems, the present invention proposes an on-chip computational hyperspectral imaging device based on the FP cavity, which realizes low-cost and high-precision hyperspectral image reconstruction through simplified processing technology and optimized spectral design. Summary of the invention

[0005] The present invention aims to solve the problems of complex design, high processing difficulty and poor spectral reconstruction effect of existing hyperspectral imaging devices, and proposes an on-chip computing hyperspectral imaging device based on FP cavity (such as Figure 1 The device adopts the FP cavity structure with a similar overall design, and only adjusts the thickness of the filling medium between the Ag layers of the reflector to achieve different optical properties, which greatly simplifies the processing process (as shown in Figure 2 As shown). The film structure of the FP cavity is Ti-Ag-Ti-ZrO 2 -Ti-Ag-Ti-SiO 2 The thickness of the Ti layer is 5nm, the thickness of the Ag layer is 15nm, and the thickness of the SiO2 layer is 20nm. The main function of the Ti layer is to enhance the surface adhesion between the oxide and the metal, the Ag layer serves as a reflector for the FP cavity, and the SiO 2 The Ti layer is the protective layer on the top of the filter. Due to the good stability of Ti and Ag metals and the extremely high coating accuracy, the entire device has high reliability and stability.

[0006] Between the Ag layers of the reflector, ZrO with high refractive index and low absorption coefficient is filled. 2 By adjusting the ZrO 2 Different filling thicknesses can achieve different optical properties. Specifically, 1000nm, 1050nm, 1100nm and 1150nm of ZrO 2 medium, just move the mask three times to remove the intermediate ZrO 2 Medium fast manufacturing.

[0007] According to the array FP cavity, the present invention designs a specific hyperspectral image reconstruction network, which is a U-net framework, and the spectral dimension Transformer is embedded in the U-net framework. The network fully utilizes the sparsity of hyperspectral images in space and the correlation in spectral dimension in a data-driven manner to achieve high-precision hyperspectral image reconstruction.

[0008] This design not only simplifies the processing, but also improves the effect and accuracy of spectral reconstruction.

[0009] Its technical features include:

[0010] FP cavity (Fabry-Perot cavity): The working principle of FP cavity is based on the phenomenon of multi-beam interference. When light is reflected multiple times between two parallel mirrors, constructive interference occurs, allowing light of a specific wavelength to form a stable standing wave pattern in the cavity. The present invention makes full use of this principle and cleverly designs the intermediate filling medium ZrO 2 The thickness of the spectral curves is 400 nm, which makes the four spectral curves have good orthogonality and volatility.

[0011] Mosaic array layout: Arranged in a 2×2 mosaic array, it is highly similar to the existing RGGB processing process. This technology is very suitable for investment in existing production lines.

[0012] Simplified processing technology: Ti-Ag-Ti-ZrO is plated on an amorphous silicon substrate in sequence 2 -Ti-Ag-Ti-SiO 2 , where Ti is 5nm, Ag is 15nm, SiO 2 20nm, the ZrO in the middle 2 The mask only needs to be moved three times, which greatly reduces the difficulty of processing.

[0013] Hyperspectral reconstruction capability: Based on the filter of this patent, a specific hyperspectral image reconstruction network is designed, which can realize 61-channel hyperspectral image reconstruction with an interval of 10nm in the range of 400~1000nm to meet the needs of multiple scenarios.

[0014] The present invention has the advantages of simple processing, low cost, excellent reconstruction performance, etc., and provides a new approach for on-chip integration and practical application of hyperspectral imaging technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An on-chip computational hyperspectral imaging device

[0016] Figure 2 FP cavity structure and spectrum curve

[0017] Figure 3 Spectral reconstruction DETAILED DESCRIPTION

[0018] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0019] Since existing hyperspectral cameras often use a push-broom method, they have disadvantages such as being bulky, taking a long time to collect data, and having a small amount of light input. They are not suitable for dynamic and low-light scenarios, such as real-time detection of river water quality and material analysis at cultural relics sites.

[0020] This scheme adopts the FP cavity thin film array design scheme with the same number of RGGB channels, and achieves good orthogonality and volatility of the four spectral curves by cleverly changing the thickness of the filling medium in the middle of the reflector. Then, the spectral dimension Transformers are embedded into the U-net framework through design to achieve high-precision hyperspectral image reconstruction.

[0021] Real-time detection of river water quality:

[0022] 1.Device construction and deployment:

[0023] Integrate the FP cavity-based hyperspectral imaging device onto a UAV or fixed monitoring station to ensure that the imaging device can cover the target river area.

[0024] Configure data acquisition and transmission modules to achieve real-time acquisition and wireless transmission of hyperspectral data.

[0025] 2. Hyperspectral data collection:

[0026] An integrated hyperspectral imaging device is used to scan the river channel and obtain hyperspectral image data of the water body.

[0027] During data collection, ensure that the drone or monitoring station remains stable to obtain high-quality hyperspectral images.

[0028] 3. Data processing and analysis:

[0029] The collected hyperspectral data is transmitted to the ground station or cloud server for processing.

[0030] The designed hyperspectral image reconstruction network (U-net framework combined with spectral dimension Transformer) is used to reconstruct and analyze hyperspectral data.

[0031] By analyzing the reconstructed hyperspectral images, the types and concentrations of pollutants in the water, such as chlorophyll and suspended matter, can be identified.

[0032] 4. Result display and early warning:

[0033] The analysis results are presented to users in the form of charts or reports, including pollutant distribution maps, concentration change trends, etc.

[0034] Based on the preset thresholds, early warnings are issued for pollutants that exceed the standard, and relevant departments are notified to take control measures.

[0035] Material analysis at the cultural relics site:

[0036] 1. Device preparation and calibration:

[0037] A hyperspectral imaging device based on the FP cavity is deployed at the cultural relic site, and necessary calibration is performed to ensure imaging accuracy.

[0038] Adjust the parameters of the imaging device, such as exposure time, gain, etc., according to the characteristics and requirements of the cultural relics.

[0039] 2. Hyperspectral data collection:

[0040] The device can scan the surface of cultural relics through hyperspectral scanning to obtain their reflectance spectrum data. The device can capture the tiny differences and features of the surface of cultural relics.

[0041] During the collection process, care was taken to avoid any damage or disturbance to the cultural relics.

[0042] 3. Data processing and material analysis:

[0043] The collected hyperspectral data are transmitted to a computer for processing.

[0044] The designed hyperspectral image reconstruction network is used to reconstruct and enhance hyperspectral data to improve image quality.

[0045] By analyzing the reconstructed hyperspectral images, the material composition and structural characteristics of the surface of the cultural relics can be identified. For example, different materials such as stone, wood, and metal can be distinguished.

[0046] 4. Interpretation and application of results:

[0047] Based on the results of hyperspectral analysis, the material of the cultural relics is interpreted and identified, which helps to understand the production process, historical background and protection status of the cultural relics.

[0048] The analysis results will be applied to the protection and restoration of cultural relics, providing a basis for formulating scientific and reasonable protection plans.

[0049] Through the above specific implementation, the on-chip computational hyperspectral imaging device provided by the present invention can play an important role in the fields of real-time detection of river water quality and material analysis of cultural relics. The device has the advantages of simple processing, low cost, and excellent reconstruction performance, and provides a new way for on-chip integration and practical application of hyperspectral imaging technology.

Claims

1. An on-chip computational hyperspectral imaging device based on a Fabry-Perot (FP) cavity, characterized in that: include: A FP cavity structure, the FP cavity structure comprising a Ti layer, an Ag layer, a Ti layer, a ZrO2 layer, a Ti layer, an Ag layer, a Ti layer and a SiO2 layer plated in sequence, wherein the thickness of the Ti layer is 5 nm, the thickness of the Ag layer is 15 nm, the thickness of the SiO2 layer is 20 nm, and the thickness of the ZrO2 layer is 1000 nm, 1050 nm, 1100 nm and 1150 nm respectively; The FP cavity structure utilizes the multi-beam interference phenomenon and realizes different optical properties by adjusting the filling thickness of the ZrO2 layer to obtain four spectral curves with orthogonality and wave properties.

2. The on-chip computational hyperspectral imaging device according to claim 1, characterized in that: The device also includes a hyperspectral image reconstruction network, which is a U-net framework in which a spectral dimension Transformer is embedded to achieve 61-channel hyperspectral image reconstruction with an interval of 10 nm in the range of 400 to 1000 nm.

3. The on-chip computational hyperspectral imaging device according to claim 1, characterized in that: The device is arranged in a 2×2 mosaic array, which is highly similar to the existing RGGB processing process and is suitable for investment in existing production lines.

4. The on-chip computational hyperspectral imaging device according to claim 1, characterized in that: The processing technology of the device comprises the following steps: sequentially plating Ti-Ag-Ti-ZrO2-Ti-Ag-Ti-SiO2 layers on an amorphous silicon substrate, wherein the filling thickness of the ZrO2 layer is rapidly manufactured by moving the mask three times.

5. A method for using the on-chip computational hyperspectral imaging device according to any one of claims 1 to 4, characterized in that: The following steps are involved: By adjusting the filling thickness of the ZrO2 layer, different optical properties can be achieved to obtain spectral curves with orthogonality and fluctuation; The collected hyperspectral data are reconstructed and analyzed using a hyperspectral image reconstruction network; The reconstructed hyperspectral images are applied to biomedical imaging, environmental monitoring, industrial detection and remote sensing.

6. The method according to claim 5, characterized in that The hyperspectral image reconstruction network is a U-net framework, in which a spectral dimension Transformer is embedded to achieve high-precision hyperspectral image reconstruction.

7. The method according to claim 5, characterized in that The method also includes using the reconstructed hyperspectral image for real-time detection of river water quality or material analysis at a cultural relic site.

Images