Infrared integrated polarization intelligent computing imaging system

Abstract

The present disclosure relates to the field of polarization computation imaging, and provides an infrared integrated polarization intelligent computation imaging system, which comprises an incident optical lens, an infrared polarization detection chip and an intelligent processing electric chip. The incident optical lens is arranged on the incident surface of the infrared integrated polarization intelligent computation imaging system, and is used for converging incident infrared light to the infrared polarization detection chip. The infrared polarization detection chip is used for polarization modulation of the incident infrared light, completion of light convolution operation, and output of the obtained light signal after being converted into an electric signal. The intelligent processing electric chip is used for processing the electric signal output by the infrared polarization detection chip, so as to obtain a polarization feature image of a detection target. The present disclosure reduces the convolution calculation amount of the back end, reduces the data transmission pressure and the back end intelligent processing pressure, can identify a target in a complex background through polarization information, further improves the image processing speed, reduces the volume, power consumption and weight of the imaging system, and realizes high-integration intelligent image processing on the end side.

Description

Technical Field

[0001] This disclosure relates to the field of polarization computational imaging technology, and in particular to an infrared integrated polarization intelligent computational imaging system. Background Technology

[0002] Polarization imaging systems overcome the dimensional limitations of traditional light intensity detection. Besides detecting light intensity information, they can simultaneously extract characteristic parameters such as the polarization degree and polarization angle of the target as polarization information. In complex backgrounds (such as fog or underwater), they can effectively suppress scattering interference, improving target detection accuracy. They also possess the unique ability to penetrate spectral camouflage coatings. Infrared polarization detection, due to its high sensitivity and strong anti-interference characteristics, has significant application value in military reconnaissance and industrial inspection. The focal plane polarization imaging detection system, through the monolithic integration of a miniaturized polarization array and detector, can simultaneously obtain information from multiple polarization components, offering advantages of high integration and high stability. Chinese patent document CN111947789B discloses a dual-color polarization uncooled infrared detector and its fabrication method. This document discloses an integrated polarization infrared detector structure, which directly integrates gratings and detector pixels in four polarization directions (0°, 45°, 90°, and 135°) to form a super-pixel structure for polarization detection.

[0003] A single detection of a focal plane polarization imaging detection system can simultaneously acquire multi-angle polarization information, offering significant advantages such as compact structure and strong anti-interference capabilities. It also exhibits high-efficiency response in dynamic scenarios. Its highly integrated design effectively adapts to the stringent limitations of payload size and weight imposed by micro-platforms, making it promising for applications in lightweight fields such as UAV reconnaissance and microsatellite payloads. However, in practical applications, the following problems still exist: First, after acquiring information, the detector chip with integrated micro-nano structures needs to transmit the acquired information to a backend processor for analysis to further obtain polarization degree and polarization angle images. This results in front-end and back-end separation, high data transmission pressure, and low efficiency. Second, traditional polarization image processing can only obtain two-dimensional images of polarization degree and polarization angle; target recognition still relies on backend intelligent processing algorithms and hardware, which cannot meet the real-time decision-making needs of dynamic scenarios. Summary of the Invention

[0004] This disclosure aims to address at least one of the problems existing in the prior art by providing an infrared integrated polarization intelligent computational imaging system.

[0005] This disclosure provides an infrared integrated polarization intelligent computational imaging system, including an incident optical lens, an infrared polarization detection chip, and an intelligent processing chip; wherein...

[0006] The incident optical lens is disposed on the incident surface of the infrared integrated polarization intelligent computing imaging system and is used to converge the incident infrared light onto the infrared polarization detection chip.

[0007] The infrared polarization detection chip is used to perform polarization modulation on the incident infrared light, complete the optical convolution operation, and output the optical signal obtained by the optical convolution operation as an electrical signal.

[0008] The intelligent processing chip is used to process the electrical signal output by the infrared polarization detection chip to obtain a polarization feature image of the target.

[0009] Optionally, the infrared polarization detection chip includes a polarization metasurface structure layer and a detection layer; the polarization metasurface structure layer is disposed on the incident surface of the detection layer through heterogeneous integration.

[0010] Optionally, the polarization metasurface structure layer is used to polarize and modulate the collected incident infrared light to complete the optical convolution operation.

[0011] Optionally, the detection layer includes an infrared photosensitive pixel array layer and a readout circuit layer, wherein the infrared photosensitive pixel array layer and the readout circuit layer are connected by indium pillars via a flip-chip bonding process; wherein,

[0012] The infrared photosensitive pixel array layer is used to detect light signals transmitted through the polarization metasurface structure layer and convert the detected light signals into electrical signals.

[0013] The readout circuit layer is used to output the electrical signal obtained from the infrared photosensitive pixel array layer.

[0014] Optionally, the polarization metasurface structure layer is a pixel-level polarization micro / nano structure, comprising multiple polarization metasurface structure units, each of which corresponds one-to-one with each infrared photosensitive pixel in the infrared photosensitive pixel array layer.

[0015] Optionally, the polarization metasurface structure unit includes a periodic metal strip grating, the period and deflection direction of which are determined according to the characteristics of the target being detected.

[0016] Optionally, the intelligent processing chip is specifically used to perform electrical convolution operation on the electrical signal output by the infrared polarization detection chip, and output the polarization feature image of the detection target.

[0017] Optionally, the intelligent processing chip is vertically integrated into the output surface of the infrared polarization detection chip through a three-dimensional stacking method.

[0018] Optionally, the incident optical lens is a transmission type.

[0019] Optionally, the incident optical lens includes multiple lenses, the material of which is determined according to its operating wavelength.

[0020] To address the issues of low efficiency and reliance on backend algorithms and hardware in existing polarization imaging detection systems, which require transmitting detected light information to the backend for processing, this disclosure proposes an infrared integrated polarization intelligent computational imaging system. This system offers the following advantages: Firstly, by integrating the optical convolutional layer onto the infrared focal plane array detector, preliminary optical convolution operations are performed within the detector, reducing the amount of backend convolutional computation and lowering the pressure on data transmission and backend intelligent processing. It can also identify targets in complex backgrounds using polarization information. Secondly, by vertically integrating the intelligent processing chip and the detector chip, image processing speed is further improved, while the size, power consumption, and weight of the imaging system are reduced, achieving highly integrated intelligent image processing at the edge. Attached Figure Description

[0021] One or more embodiments are illustrated by way of example with the corresponding pictures in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0022] Figure 1 A schematic diagram of the structure of an infrared integrated polarization intelligent computational imaging system provided in one embodiment of this disclosure;

[0023] Figure 2 A schematic diagram of the workflow of an infrared integrated polarization intelligent computational imaging system provided for another embodiment of this disclosure;

[0024] Figure 3 A schematic diagram illustrating the working principle of an infrared integrated polarization intelligent computational imaging system provided for another embodiment of this disclosure;

[0025] Figure 4 A schematic diagram of a macropixel unit and grating structure provided for another embodiment of this disclosure;

[0026] Figure 5 This is a schematic diagram of the structure of an optical convolutional layer provided for another embodiment of this disclosure. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the various embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this disclosure to facilitate a better understanding of the disclosure. However, the technical solutions claimed in this disclosure can be implemented even without these technical details and with various variations and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of this disclosure. The various embodiments can be combined with and referenced by each other without contradiction.

[0028] One embodiment of this disclosure relates to an infrared integrated polarization intelligent computational imaging system, the structure of which is as follows: Figure 1 As shown, it includes an incident optical lens 100, an infrared polarization detection chip 200, and an intelligent processing chip 300.

[0029] An incident optical lens 100 is positioned on the incident surface of the infrared integrated polarization intelligent computing imaging system to converge the incident infrared light 400 onto the infrared polarization detection chip 200.

[0030] The infrared polarization detection chip 200 is used to polarize the incident infrared light 400, complete the optical convolution operation, and output the optical signal obtained by the optical convolution operation as an electrical signal.

[0031] The intelligent processing chip 300 is used to process the electrical signal output by the infrared polarization detection chip to obtain the polarization feature image of the target.

[0032] Combined Figure 2 The specific workflow of the infrared integrated polarization intelligent computational imaging system includes: the incident optical lens 100 collects incident light, focuses the incident infrared light 400 onto the surface of the infrared polarization detection chip 200, the infrared polarization detection chip 200 extracts the polarization information of the incident infrared light 400 to complete polarization modulation, and uses it as the first convolutional layer of the convolutional neural network to perform preliminary convolution calculations on the light signal to complete the optical convolution operation and output the corresponding electrical signal, and the intelligent processing electrical chip 300 performs further convolution calculations on the electrical signal output by the infrared polarization detection chip 200 to complete the electrical convolution operation, thereby obtaining the polarization feature image of the detected target, and finally realizing polarization imaging and target recognition.

[0033] The incident optical lens 100, the infrared polarization detection chip 200, and the intelligent processing chip 300 will be described in detail below.

[0034] Combined Figure 1 and Figure 3An incident optical lens 100 is disposed on the incident surface of the infrared integrated polarization intelligent computing imaging system and includes multiple lenses for focusing the incident infrared light 400 onto the surface of the underlying infrared polarization detection chip 200. The incident optical lens 100 follows the laws of refraction and reflection of light. The lens material of the incident optical lens is determined according to the working wavelength of the lens; different lens materials can be selected for different working wavelengths, including but not limited to MgF2, CaF2, ZnS, ZnSe, Si, and chalcogenide glasses. The incident optical lens is preferably a transmissive type to avoid introducing additional polarization error interference.

[0035] Combined Figure 1 and Figure 3 The infrared polarization detection chip 200 includes a polarization metasurface structure layer 201 and a detection layer (not shown in the figure). The polarization metasurface structure layer 201 is disposed on the incident surface of the detection layer through heterogeneous integration. The incident infrared light 400, after being focused by the incident optical lens 100, first reaches the polarization metasurface structure layer 201.

[0036] The polarization metasurface structure layer 201 is used to polarize and modulate the collected incident infrared light, thus performing optical convolution operations. The polarization metasurface structure layer 201 is a pixel-level polarization micro / nano structure, comprising multiple periodically arranged metasurface macropixel units. Figure 4 This demonstrates one type of metasurface macropixel unit. Each metasurface macropixel unit can include multiple polarization metasurface structure units, and the specific number of polarization metasurface structure units in each unit can be determined based on the polarization directions required for imaging. For example, when nine polarization directions are required for imaging, the metasurface macropixel unit can contain nine polarization metasurface structure units in a 3×3 configuration, so that each polarization metasurface structure unit corresponds to one polarization direction. Of course, depending on the polarization directions required for imaging, the metasurface macropixel unit can also contain 16 polarization metasurface structure units in a 4×4 configuration or four polarization metasurface structure units in a 2×2 configuration. For example... Figure 4 A metasurface macropixel unit is shown, which contains 16 4×4 polarization metasurface structure units, each corresponding to a pixel. Figure 4 The demonstrated metasurface macropixel unit is a macropixel containing 16 pixels. For each polarization metasurface structure unit within the metasurface macropixel unit, its structure is a periodic metal strip grating distributed on the substrate; that is, the polarization metasurface structure unit includes a periodic metal strip grating. For example, Figure 4This paper demonstrates a periodic metallic strip grating for a polarization metasurface structure unit within a metasurface macropixel cell. This polarization metasurface structure unit corresponds to a single pixel, and its grating forms a 33.75° angle with the vertical direction. The grating material can be aluminum, and the deflection direction can be set according to actual detection requirements. All gratings within the metasurface macropixel cell maintain consistent thickness and duty cycle, while the grating period and orientation angle are adjustable variables. The period and deflection direction of the periodic metallic strip grating are determined based on the characteristics of the target being detected. When incident infrared light with different polarization directions illuminates the polarization metasurface structure layer, the metallic gratings in different directions modulate the polarization of the incident light.

[0037] Combined Figure 1 and Figure 3 The detection layer includes an infrared photosensitive pixel array layer 202 and a readout circuit layer 204, which are connected by indium pillars 203 via flip-chip bonding. A polarization metasurface structure layer 201 is heterogeneously integrated into the infrared photosensitive pixel array layer 202. Each polarization metasurface structure unit in the polarization metasurface structure layer 201 corresponds one-to-one with each infrared photosensitive pixel in the infrared photosensitive pixel array layer 202, achieving pixel-level alignment. The size of the infrared photosensitive pixel is greater than or equal to the size of its corresponding polarization metasurface structure unit. A metasurface macropixel unit, comprising its various polarization metasurface structure units and their corresponding infrared photosensitive pixels, collectively constitutes a macropixel detection unit. When incident infrared light 400 irradiates the polarization metasurface structure layer 201, only light with the corresponding polarization direction can pass through the polarization metasurface structure layer 201 to reach the corresponding infrared photosensitive pixel array layer 202. The infrared photosensitive pixel array layer 202 detects the light intensity at different polarization directions and spatial positions, converts the corresponding optical signals into electrical signals, and reads and transmits these electrical signals through the readout circuit layer 204. In other words, the infrared photosensitive pixel array layer 202 is used to detect the optical signals passing through the polarization metasurface structure layer 201 and convert the detected optical signals into electrical signals. The readout circuit layer 204 is used to output the electrical signals obtained by the infrared photosensitive pixel array layer 202.

[0038] The infrared polarization detection chip 200 extracts the polarization information of the incident infrared light 400 while simultaneously completing the first layer of convolution calculation, i.e., optical convolution. The specific process is as follows: The infrared polarization detection chip 200, particularly the polarization metasurface structure layer 201 within it, detects the light intensity at different spatial locations and sums these intensity values ​​according to the polarization angle, completing the first layer of the convolutional neural network calculation and achieving preliminary convolution calculation functionality. Subsequent electrical convolution operations are then completed by the intelligent processing chip 300. For example, such as... Figure 5As shown, the size of the convolution kernel used for optical convolution operations can be set to 2×2, and the corresponding single optical convolution unit contains 2×2 macropixel detection units and 3×3 convolution kernels (p, p ... 11 p 12 p 13 p 14 p 21 p 22 p 23 p 24 p 31 p 32 p 33 p 34 p 41 p 42 p 43 p 44 p 51 p 52 p 53 p 54 p 61 p 62 p 63 p 64 p 71 p 72 p 73 p 74 p 81 p 82 p 83 p 84 p 91 p 92 p 93 p 94 A complete optical convolutional layer contains H×W identical optical convolutional units. Since each 2×2 convolutional kernel corresponds to a predefined polarization angle, the weight vector of the convolutional kernel can be adjusted by setting the polarization transmittance of the infrared polarization detection unit corresponding to that specific kernel. The weight vector of the i-th (1≤i≤9) convolutional kernel can be represented as w. i1 (θ),w i2 (θ),w i3 (θ),w i4 (θ). Accordingly, the output value of the infrared polarization detection chip 200 is a matrix of size H×W×9. The polarization transmittance can be set by changing the period and orientation angle of the corresponding grating.

[0039] Traditional polarization imaging systems acquire polarization images using polarization filters and detectors. These images then need to be exported to a backend processor for post-processing operations such as polarization reconstruction and convolution calculations to achieve edge detection and feature extraction. A drawback of this type of system is that the data transmission from the sensor to the backend processor requires significant bandwidth, resulting in time lag and spatial discontinuities in image processing. To address this issue, an innovation of this disclosure is to use an infrared polarization detection chip as the first convolutional layer of a convolutional neural network. Preliminary optical convolution operations are performed within the infrared polarization detection chip, thereby reducing the amount of backend convolution calculations and lowering the data transmission burden.

[0040] The intelligent processing chip 300 is specifically used to perform electrical convolution operations on the electrical signal output by the infrared polarization detection chip 200, outputting a polarization feature image of the detected target. This is combined with... Figure 1 The intelligent processing chip 300 is vertically integrated on the output surface of the infrared polarization detection chip 200 through a three-dimensional stacking method. Specifically, the intelligent processing chip 300 and the infrared polarization detection chip 200 can be integrated through a silicon via 205 using three-dimensional vertical stacking technology.

[0041] The intelligent processing chip 300 reads and processes the output value of the readout circuit layer 204 in the infrared polarization detection chip 200, and performs subsequent convolution calculations of the convolutional neural network, i.e., electrical convolution operation. The difference between this and other intelligent computational imaging systems lies in the fact that the infrared integrated polarization intelligent computational imaging system provided in this embodiment uses three-dimensional vertical stacking technology to integrate the intelligent processing chip 300 and the infrared polarization detection chip 200, forming a highly integrated detection microsystem. Compared to transmitting data to a backend processor, this significantly reduces bandwidth usage and computational overhead. Simultaneously, the introduction of three-dimensional vertical stacking technology greatly reduces the size, weight, and power consumption of the imaging system, making it suitable as a payload for individual soldier equipment, remote sensing devices, border and coastal defense monitoring equipment, unmanned vehicles, drones, and other micro-sized unmanned devices.

[0042] Combined Figure 3 The working principle of the aforementioned infrared integrated polarization intelligent computational imaging system is as follows: Multiple incident infrared rays in the incident infrared light 400, namely incident infrared rays x1, x2, ..., x... N With different polarization angles, i.e. θ1, θ2, ..., θ NInfrared light 400 is incident on the incident optical lens 100 and converges onto the polarization metasurface structure layer 201 on the surface of the infrared polarization detection chip 200. The polarization metasurface structure layer 201 modulates the collected incident infrared light 400 and performs optical convolution. The infrared photosensitive pixel array layer 202 detects the intensity of light transmitted through the polarization metasurface structure layer 201 and converts the corresponding optical signal into an electrical signal, which is output to the readout circuit layer 204. The readout circuit layer 204 outputs the electrical signal from the infrared photosensitive pixel array layer 202 to the intelligent processing chip 300. The intelligent processing chip 300 uses a convolutional neural network to perform electrical convolution on the electrical signal output from the readout circuit layer 204 to obtain the polarization feature image of the detected target. The infrared polarization detection chip 200 and the intelligent processing chip 300 together form a complete convolutional neural network.

[0043] The infrared integrated polarization intelligent computational imaging system provided in this disclosure, compared with the prior art, integrates the optical convolutional layer onto the infrared focal plane array detector, and performs the initial optical convolution operation within the detector, reducing the amount of convolutional calculation at the back end, reducing the data transmission pressure and the back end intelligent processing pressure. It can identify targets in complex backgrounds through polarization information. At the same time, the intelligent processing chip and the detector chip are vertically integrated, which further improves the image processing speed and reduces the size, power consumption and weight of the imaging system, realizing highly integrated intelligent image processing at the edge.

[0044] Those skilled in the art will understand that the above embodiments are specific implementations of this disclosure, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this disclosure.

Claims

1. An infrared integrated polarization intelligent computational imaging system, characterized in that, This includes an incident optical lens, an infrared polarization detection chip, and an intelligent processing chip; among which, The incident optical lens is disposed on the incident surface of the infrared integrated polarization intelligent computing imaging system and is used to converge the incident infrared light onto the infrared polarization detection chip. The infrared polarization detection chip is used to perform polarization modulation on the incident infrared light, complete the optical convolution operation, and output the optical signal obtained by the optical convolution operation as an electrical signal. The intelligent processing chip is used to process the electrical signal output by the infrared polarization detection chip to obtain a polarization feature image of the target.

2. The infrared integrated polarization intelligent computational imaging system according to claim 1, characterized in that, The infrared polarization detection chip includes a polarization metasurface structure layer and a detection layer; the polarization metasurface structure layer is disposed on the incident surface of the detection layer through heterogeneous integration.

3. The infrared integrated polarization intelligent computational imaging system according to claim 2, characterized in that, The polarization metasurface structure layer is used to polarize and modulate the collected incident infrared light to complete the optical convolution operation.

4. The infrared integrated polarization intelligent computational imaging system according to claim 2, characterized in that, The detection layer includes an infrared photosensitive pixel array layer and a readout circuit layer, wherein the infrared photosensitive pixel array layer and the readout circuit layer are connected by indium pillars via a flip-chip bonding process; wherein... The infrared photosensitive pixel array layer is used to detect light signals transmitted through the polarization metasurface structure layer and convert the detected light signals into electrical signals. The readout circuit layer is used to output the electrical signal obtained from the infrared photosensitive pixel array layer.

5. The infrared integrated polarization intelligent computational imaging system according to claim 4, characterized in that, The polarization metasurface structure layer is a pixel-level polarization micro / nano structure, comprising multiple polarization metasurface structure units, each of which corresponds one-to-one with an infrared photosensitive pixel in the infrared photosensitive pixel array layer.

6. The infrared integrated polarization intelligent computational imaging system according to claim 5, characterized in that, The polarization metasurface structure unit includes a periodic metal strip grating, the period and deflection direction of which are determined according to the characteristics of the target being detected.

7. The infrared integrated polarization intelligent computational imaging system according to claim 1, characterized in that, The intelligent processing chip is specifically used to perform electrical convolution operation on the electrical signal output by the infrared polarization detection chip, and output the polarization feature image of the detection target.

8. The infrared integrated polarization intelligent computational imaging system according to claim 1, characterized in that, The intelligent processing chip is vertically integrated into the output surface of the infrared polarization detection chip through a three-dimensional stacking method.

9. The infrared integrated polarization intelligent computational imaging system according to any one of claims 1 to 8, characterized in that, The incident optical lens is a transmission type.

10. The infrared integrated polarization intelligent computational imaging system according to any one of claims 1 to 8, characterized in that, The incident optical lens includes multiple lenses, the material of which is determined according to its operating wavelength.

Images