Sub-focal-plane high-flux bionic multi-spectral polarization imaging detector

Abstract

The present invention relates to a split focal plane type high luminous flux bionic multi-spectral polarization imaging detector, which comprises a photoelectric conversion unit, an image processing unit and an input / output interface connected in sequence; the photoelectric conversion unit comprises a spectral modulation layer, a polarization modulation layer and a photoelectric conversion layer arranged successively from top to bottom; the spectral modulation layer is composed of a micro-notch filter array; and the polarization modulation layer is composed of micro-polarizers. The present invention can realize synchronous detection imaging of target multi-spectral information and polarization information.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical imaging, and particularly relates to a split focal plane high-flux bionic multi-spectral polarization imaging detector. Background Art

[0002] With the continuous development of imaging technology, traditional light intensity imaging can no longer meet human needs. People have begun to pursue the developed compound eye imaging visual systems of some organisms. For example, the compound eye visual system of mantis shrimp can well detect polarized light that is imperceptible to the human eye. While distinguishing 4 linear polarization directions and 2 circular polarization states, it can also simultaneously sense spectral information of 16 different bands. Compared with traditional light intensity imaging, polarization imaging and spectral imaging can not only obtain light intensity information, but also have unique imaging capabilities for target features. Among them, polarization imaging technology can not only obtain the light intensity information of the target, but also calculate information such as the degree of polarization and polarization angle of the target, which can reflect the structural characteristics, surface roughness and other characteristics of the target, and can enhance the contrast between the target and the background, and remove the reflection of the water surface or the glass surface. Spectral information can reflect the absorption, reflection or refraction characteristics of the target to light waves. Combining the advantages of spectral imaging and polarization imaging technologies, high-quality multi-spectral polarization imaging technology has become a frontier research direction. At present, multi-spectral polarization imaging technology is mainly applied in the field of satellite remote sensing. In addition, there are also fields such as medical diagnosis, three-dimensional imaging, and target recognition. Studying multi-spectral polarization imaging technology can be used to better measure the information transmitted by light in the target scene, analyze target features, and serve various fields. However, there are still problems with the theory of high-flux multi-spectral polarization imaging technology and the feasibility of manufacturing high-flux multi-spectral polarization imaging. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a split focal plane high-flux bionic multi-spectral polarization imaging detector, aiming to solve the above problems.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] A split focal plane high-flux bionic multi-spectral polarization imaging detector includes a photoelectric conversion unit, an image processing unit and an input / output interface connected in sequence; the photoelectric conversion unit includes a spectral modulation layer, a polarization modulation layer and a photoelectric conversion layer arranged from top to bottom in sequence; the spectral modulation layer is composed of a micro-notch filter array; the polarization modulation layer is composed of micro-polarizers.

[0006] Further, each micro-notch filter in the micro-notch filter array is in the same plane.

[0007] Furthermore, each micro-polarizer of the polarization modulation layer is in the same plane and includes micro-polarizers with four different directions of 0°, 45°, 90°, and 135°.

[0008] Furthermore, a superpixel is formed by using four notch filters with different bands, or a superpixel is formed by using three notch filters with different bands and a pixel without a filter covering.

[0009] Furthermore, a demosaicing algorithm is used to improve the pixel resolution of a certain polarization state in a certain band. The demosaicing algorithm, when only considering the superpixels of micro-polarization units, can be solved according to the following equation:

[0010]

[0011] where ||·|| F represents the Frobenius norm, ||·||1 represents the 1-norm, S represents the mosaic data captured by the detector, T represents the downsampling matrix, D represents the dictionary, B represents the corresponding sparse matrix, and η represents the regularization coefficient. The downsampling matrices corresponding to the polarization state information in different directions are solved. Similarly, this demosaicing algorithm is applicable to the superpixel case where only the notch filter unit is considered, and also suitable for the superpixel case where both micro-polarizers and micro-notch filters are considered.

[0012] Furthermore, the photoelectric conversion layer uses a CCD, CMOS, visible light detector, or infrared band detector.

[0013] Furthermore, the image processing unit acquires the spectral resolution of the image data, and performs spectral super-resolution calculation on the acquired image by using the compressed sensing principle combined with the prior information of sparsity and smoothness. The equation to be solved is as follows

[0014]

[0015] where ||·|| F represents the Frobenius norm, ||·||1 represents the 1-norm, Y represents the data captured by the detector, H represents the system acquisition matrix, D represents the dictionary, B represents the corresponding sparse matrix, λ represents the wavelength independent variable, μ represents the coefficient of the sparse constraint term, and γ represents the coefficient of the spectral smooth constraint term;

[0016] D is obtained through dictionary training by K-SVD, and then the augmented Lagrangian method is used to solve by the alternating direction multiplier method, and finally the polarization hyperspectral data F = DB under the constraint conditions is solved.

[0017] The present invention has the following beneficial effects compared with the prior art:

[0018] 1. The present invention can achieve synchronous detection and imaging of target multi-spectral information and polarization information, and balance the contradictions among time resolution, spatial resolution, and spectral resolution. The four-dimensional information of space, multi-spectral, and polarization obtained by the detector is more conducive to target recognition and classification;

[0019] 2. The present invention can change the spectral range of the detection target by replacing the notch filter according to actual requirements, and realize flexible data acquisition for different scenarios.

[0020] 3. The present invention can use spectral reconstruction or super-resolution algorithms to achieve higher-resolution images according to actual requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic structural diagram of the detector of the present invention;

[0022] Figure 2 is a schematic diagram of the image processing unit and the input / output interface in an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the distribution of micro-notch filters in the spectral modulation layer in an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of the distribution of micro-polarizers in the polarization modulation layer in an embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of a 4×4 superpixel of the detector in an embodiment of the present invention;

[0026] Figure 6 is the transmittance curves of 488nm, 532nm, 632nm, and the all-transmissive filter (PAN) in an embodiment of the present invention;

[0027] In the figure: 1 - spectral modulation layer, 2 - polarization modulation layer, 3 - photoelectric conversion layer, 4 - image processing unit, 5 - input / output interface. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] Please refer to Figure 1 , the present invention provides a split focal plane high-flux bionic multi-spectral polarization imaging detector, which includes a photoelectric conversion unit, an image processing unit 4, and an input / output interface 5 connected in sequence; the photoelectric conversion unit includes a spectral modulation layer 1, a polarization modulation layer 2, and a photoelectric conversion layer 3 arranged from top to bottom in sequence.

[0030] Refer to Figure 3, in this embodiment, preferably, the spectral modulation layer consists of a micro-notch filter array; each micro-notch filter in the micro-notch filter array is in the same plane, and the notch center wavelength depends on the usage scenario.

[0031] Reference Figure 4 , in this embodiment, preferably, each micro-polarizer in the polarization modulation layer is in the same plane, and the micro-polarizers in different directions are arranged according to a certain rule, such as 90°, 45°, 135°, 0°, and different arrangement orders will have a certain impact on the demosaicing algorithm., including four different directions of micro-polarizers: 0°, 45°, 90°, and 135°.

[0032] Preferably, a demosaicing algorithm is used to improve the pixel resolution of a certain polarization state in a certain wavelength band.

[0033] For the demosaicing algorithm, when only considering the superpixels of micro-polarization units, it can be solved according to the following equation:

[0034]

[0035] where ||·|| F represents the F-norm, ||·||1 represents the 1-norm, S represents the mosaic data captured by the detector, T represents the downsampling matrix, D represents the dictionary, B represents the corresponding sparse matrix, and η represents the regularization coefficient. Solving the polarization state information in different directions corresponds to different downsampling matrices. Similarly, this demosaicing algorithm is applicable to the superpixel case where only the notch filter unit is considered, and also suitable for the superpixel case where both micro-polarizers and micro-notch filters are considered.

[0036] Preferably, the spatial plane coordinates of the spectral modulation layer and the polarization modulation layer are consistent with the pixel coordinates of the photoelectric conversion layer. The micro-notch filter array and the micro-polarizer array are distributed in a rectangular array.

[0037] In this embodiment, light rays with different incident or reflected angles are transmitted through the pre-optical system, then spectrally modulated by the micro-notch filter, and then polarization-modulated by the micro-polarizer, and finally the conversion from optical signal to electrical signal is completed in the photoelectric conversion layer. The characteristics of the finally obtained image are: the same polarization state has different spectral information; different polarization states have the same spectral information.

[0038] In this embodiment, a superpixel is composed of four notch filters with different wavelength bands, or a superpixel is composed of three notch filters with different wavelength bands and a pixel without a filter.

[0039] In this embodiment, the photoelectric conversion layer uses a CCD, CMOS, visible light detector, or infrared band detector.

[0040] In this embodiment, there are four polarization states on each camera chip, namely the four polarization states of 0°, 45°, 90°, and 135°. As Figure 5 shown, it is the distribution law of polarization state pixels on the polarization camera chip of the split focal plane high-flux bionic multi-spectral polarization imaging detector. If the polarization camera acquires a picture as matrix A with a pixel size of M×N, where M and N are even numbers, then the pixel size of each polarization state picture is M / 2×N / 2. According to the MATLAB software syntax, the 0° polarization state image P0 = A(2:2:end, 2:2:end), the 45° polarization state image P 45 = A(1:2:end, 2:2:end), the 90° polarization state image P 90 = A(1:2:end, 1:2:end), and the 135° polarization state image P 135 = A(2:2:end, 1:2:end).

[0041] When the optoelectronic conversion layer outputs digital signals to the processor unit, corresponding image processing operations can be performed. Taking spectral super-resolution as an example:

[0042] In the above system, only 4 polarized spectral band images are collected in one exposure. To improve the spectral resolution of the collected image data, the compressed sensing principle is used to combine the prior information of sparsity and smoothness to perform spectral super-resolution calculation on the collected image, and the equation to be solved is as follows

[0043]

[0044] where ||·|| F represents the F-norm, ||·||1 represents the 1-norm, Y represents the data captured by the detector, H represents the system acquisition matrix, D represents the dictionary, B represents the corresponding sparse matrix, λ represents the wavelength independent variable, μ represents the coefficient of the sparse constraint term, and γ represents the coefficient of the spectral smooth constraint term;

[0045] D is obtained through dictionary training by K-SVD, and then by using the augmented Lagrangian method to solve the alternating direction multiplier method, the polarized hyperspectral data F = DB under the constraint conditions is finally solved.

[0046] In the present invention, starting from the perspectives of imaging quality, simplified structure, synchronous multi-dimensional information acquisition, and system feasibility, a split focal plane high-flux bionic multi-spectral polarization imaging detector is designed and manufactured. The array distance should not be too large, otherwise the spatial resolution of the image will be reduced. According to the actual wavelength range of the target, cameras with appropriate quantum effects and corresponding filters are selected to obtain a more appropriate design result.

[0047] The above are only the preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

Claims

1. A focal plane type high light flux biomimetic multi-spectral polarization imaging detector, characterized in that: It includes a photoelectric conversion unit, an image processing unit and an input and output interface connected in sequence; the photoelectric conversion unit includes a spectral modulation layer, a polarization modulation layer and a photoelectric conversion layer arranged in sequence from top to bottom; the spectral modulation layer is composed of a micro-notch filter array; the polarization modulation layer is composed of a micro-polarizer; Use notch filters of four different bands to form a superpixel, or use notch filters of three different bands and a pixel without covering filters to form a superpixel; The image processing unit collects the spectral resolution of the image data and uses the compressed sensing principle combined with sparsity and smoothness prior information to perform spectral super-resolution calculation on the collected image to solve the following equation: where ||·|| F represents the F norm, ||·||1 represents the 1 norm, Y represents the data captured by the detector, H represents the system acquisition matrix, D represents the dictionary, B represents the corresponding sparse matrix, λ represents the wavelength independent variable, μ represents the coefficient of the sparse constraint term, and γ represents the coefficient of the spectral smoothness constraint term; D is obtained by dictionary training through K-SVD, and then the alternating direction multiplier method is used to solve the problem by using the augmented Lagrangian method, and finally the polarization hyperspectral data F=DB under the constraints is solved.

2. The focal plane type high light flux biomimetic multi-spectral polarization imaging detector according to claim 1, characterized in that: Each micro-notch filter in the micro-notch filter array is on the same plane.

3. The focal plane type high light flux biomimetic multi-spectral polarization imaging detector according to claim 1, characterized in that: Each micro-polarizer of the polarization modulation layer is in the same plane.

4. The split focal plane high luminous flux biomimetic multi-spectral polarization imaging detector according to claim 1, characterized in that: The photoelectric conversion layer adopts CCD, CMOS, visible light detector or infrared band detector.

Images