A pixel-level metasurface device and system for realizing polarization high dynamic imaging
By introducing pixel-level metasurface devices at the front end of the image sensor and using topology optimization algorithms to generate structured pixels, a high dynamic range for polarization imaging is achieved, solving the problem of quality degradation in polarization imaging in existing technologies and improving the flexibility and imaging quality of the imaging system.
Patent Information
- Application Number
- CN202610518687.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-20
- Publication Date
- 2026-07-21
AI Technical Summary
Existing polarization imaging techniques struggle to achieve high-quality imaging within a high dynamic range, especially in complex scenarios, where traditional methods can lead to a decline in polarization imaging quality.
A pixel-level metasurface device is designed. By introducing a metasurface structure layer at the front end of an image sensor and using a topology optimization algorithm to generate structured pixels, the polarization-intensity joint mapping of incident light is realized. Incident light with different polarization states is projected onto different polarization regions of the corresponding photosensitive pixel array to form a target intensity gradient distribution.
It achieves high dynamic range polarization imaging, improves system flexibility and imaging quality, simplifies hardware requirements, reduces dynamic artifacts, and enhances the ability to capture details in both bright and dark areas.
Smart Images
Figure CN122438007A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of optical imaging technology, and more specifically, relates to a pixel-level metasurface device and system for realizing polarization high dynamic range imaging. Background Technology
[0002] Traditional imaging methods relying solely on light intensity and color information are insufficient to meet the demands of target recognition, material differentiation, surface defect detection, and glare reduction in complex scenarios. Polarization imaging, capable of acquiring modulation information of the target's light wave vibration direction, holds significant application value in transparent target recognition, metal surface detection, scratch and crack identification, and scattering medium observation, thus its importance continues to grow. In current industrial polarization imaging, the mainstream approach includes on-chip micro-polarization arrays based on focal plane segmentation. However, as scene complexity increases, imaging systems not only need to acquire polarization information but also require high dynamic range (HDR) to simultaneously preserve effective details in both bright and dark areas.
[0003] To achieve HDR imaging, existing technologies mainly fall into three categories: The first is a time-series multi-exposure method using a single device, which performs well in static scenes but is unsuitable for dynamic scenes; the second is a parallel imaging method using multiple devices or multiple optical paths, capable of simultaneously acquiring images from different exposures, but resulting in a large system size and complex optical paths; the third involves updating sensor circuitry or pixel structure to introduce large / small pixels, dual conversion gain, etc., which can expand the dynamic range but easily leads to problems such as increased dark current noise, decreased aperture ratio, and increased process complexity. It is evident that combining polarization imaging methods with the above-mentioned existing HDR imaging technologies generally reduces the quality of polarization imaging. Summary of the Invention
[0004] In view of the shortcomings of the prior art, the purpose of this application is to provide a pixel-level metasurface device and system for realizing polarization high dynamic range imaging, aiming to solve the technical problem that existing high dynamic range technology is difficult to combine with polarization imaging technology to achieve high-quality polarization imaging.
[0005] The first aspect of this application relates to a pixel-level metasurface device for realizing polarization high dynamic range imaging, used in the front end of an image sensor, comprising: At least one metasurface structure layer, the metasurface structure layer comprising a plurality of periodically arranged structure pixels; Each of the structural pixels corresponds to a photosensitive pixel array in the image sensor, and the photosensitive pixel array contains at least [number of pixels]. 1 photosensitive pixel, of which For the first The order of the target energy gradient distribution within each target polarization state; The number of target polarization states; When the metasurface structure layer receives incident light containing multiple polarization states, the structure pixels perform a joint polarization-intensity mapping on the incident light, projecting the incident light of various target polarization states onto different polarization regions in the corresponding photosensitive pixel array, so that the pixel regions marked with different intensity levels in the polarization regions receive incident light intensities of different intensity levels, forming the target intensity gradient distribution in the polarization regions.
[0006] Preferably, an objective function is designed based on the distribution of the target polarization state and the distribution of the target intensity gradient, and a topology optimization algorithm is used to generate the structural arrangement of the structural pixels with the objective function as the optimization objective.
[0007] Preferably, the objective function is: ; in, Let be the objective function. Let be the index of the target polarization state. The number of the target polarization states. For the first Spectroscopic efficiency of each target polarization state For the first The similarity between the actual intensity gradient distribution in each target polarization state and the target intensity gradient distribution. For the first The ratio of crosstalk intensity received by the polarization region corresponding to the target polarization state from other polarization states. , and For the first The weighting coefficients of each target polarization state.
[0008] Preferably, the spatial size of the structural pixel is The corresponding spatial size of the photosensitive pixel array is ,satisfy , .
[0009] Preferably, the target polarization state includes, but is not limited to, one or more combinations of 0° linear polarization state, 45° linear polarization state, 60° linear polarization state, 90° linear polarization state, 120° linear polarization state, 135° linear polarization light, left-handed circular polarization state, right-handed circular polarization state, left-handed elliptically polarization state, and right-handed elliptically polarization state.
[0010] Preferably, the order of the target intensity gradient distribution is equal to 2, such that the pixel regions marked with different intensity levels in the polarization region receive high-intensity incident light intensity and low-intensity incident light intensity respectively, forming high-intensity pixel regions and low-intensity pixel regions; the high-intensity pixel regions are responsible for capturing dark details, and the low-intensity pixel regions are responsible for capturing highlight details.
[0011] Preferably, the order of the target intensity gradient distribution is greater than or equal to 3, such that the pixel regions marked with different intensity levels in the polarization region respectively receive high-intensity incident light intensity, low-intensity incident light intensity, and at least one intermediate-intensity incident light intensity, forming high-intensity pixel regions, low-intensity pixel regions, and at least one intermediate-intensity pixel region; the high-intensity pixel regions are responsible for capturing dark details, the low-intensity pixel regions are responsible for capturing highlight details, and the intermediate-intensity pixel regions are responsible for capturing intermediate brightness details.
[0012] Preferably, the intensity ratio between any two adjacent intensity gradients in the target intensity gradient distribution is equal.
[0013] Preferably, the target intensity gradient distribution is 1:4:16:64.
[0014] The second aspect of this application relates to a pixel-level metasurface imaging system for realizing polarization high dynamic range imaging, comprising: any of the metasurface devices, photosensitive pixel arrays, and imaging reconstruction units described in the first aspect; The image sensor is used to convert incident light into electrical signals and transmit them to the imaging reconstruction unit; The imaging reconstruction unit is used to perform data filtering and weighted fusion on the received electrical signals according to the distribution of the target intensity gradient and the distribution of the target polarization state, and restore them into a high dynamic range image containing polarization information through spatial rearrangement.
[0015] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application achieves wavefront phase modulation of incident light through optical design of metasurface devices, realizing joint mapping of polarization and intensity of incident light in the image sensor. Incident light with different polarization states is projected onto different polarization regions in the corresponding photosensitive pixel array, thereby completing polarization imaging. Simultaneously, pixel regions marked with different intensity levels in the polarization region receive incident light intensities of different intensity levels, thus enabling the polarization region to capture both highlight details and shadow details, improving the dynamic range of polarization imaging. In summary, the technical solution of this application can achieve polarization imaging with high dynamic range.
[0016] (2) In this application, the structural pixels in the metasurface device can be designed with objective functions according to the required imaging indicators (target intensity gradient distribution and target polarization state distribution), and then the topology optimization algorithm is used to generate the structural arrangement of the structural pixels with the objective function as the evaluation index, so as to finally achieve the required polarization imaging performance indicators (polarization channel and dynamic range gain). Therefore, the scheme of this application is highly customizable and flexible.
[0017] (3) This application implements a polarization imaging system. In the system, the image sensor converts the incident light into an electrical signal and transmits it to the imaging reconstruction unit. The imaging reconstruction unit at the back end only needs to read out these preset strong and weak signals and filter the data according to the distribution of polarization regions before performing simple synthesis to complete high dynamic range polarization imaging. The imaging reconstruction unit has low hardware requirements and can be completed with simple circuits and algorithms. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the pixel-level metasurface system provided in the embodiments of this application.
[0019] Figure 2 This is a schematic diagram of the joint mapping of polarization and intensity provided in the embodiments of this application.
[0020] Figure 3 This is one of the schematic diagrams of the distribution of the target polarization state provided in the embodiments of this application.
[0021] Figure 4 This is the second schematic diagram of the distribution of the target polarization state provided in the embodiments of this application.
[0022] Figure 5 This is a schematic diagram of three structural distributions of metasurface structure pixels provided in the embodiments of this application. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0024] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.
[0025] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.
[0026] In this application, the term "electrical connection" can refer to a direct circuit connection or a signal transmission via a communication protocol.
[0027] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0028] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.
[0029] like Figure 1 As shown, this application presents a pixel-level metasurface system for achieving high dynamic range polarization imaging, which includes a metasurface device, a spacer layer 3, a photosensitive pixel array 5, and an imaging reconstruction unit.
[0030] The metasurface device includes at least one metasurface structure layer 1, which includes a plurality of periodically arranged structure pixels 2.
[0031] The metasurface structure layer 1 is located at the front end of the image sensor, where each structure pixel 2 corresponds to a photosensitive pixel array 5 in the image sensor.
[0032] In this embodiment, CMOS is selected as the image sensor.
[0033] The metasurface structure layer 1 is disposed at the front end of the image sensor. Each structured pixel 2 in the metasurface structure layer 1 corresponds to a photosensitive pixel array 5 in the back end image sensor. The photosensitive pixel array contains at least one of the following: 1 photosensitive pixel, of which For the first The order of the target energy gradient distribution within each target polarization state; The number of target polarization states.
[0034] In this embodiment, the target polarization states of the incident light are set to 0° linearly polarized light, 45° linearly polarized light, 90° linearly polarized light, and 135° linearly polarized light, and the number of target polarization states is 4. The value is set to 4. The target intensity gradient distribution within each target polarization state is designed as 1:4:16:64, and the order of the target intensity gradient distribution within each target polarization state is 4. Therefore, in this embodiment, the photosensitive pixel array 5 needs to contain at least 16 photosensitive pixels.
[0035] Therefore, in this embodiment, the photosensitive pixel array 5 is designed as a 4×4 pixel array. The spatial size of the photosensitive pixel array 5 is 4×4, and the side length is 4×P, where P is the spatial size of a single pixel. In this embodiment, the spatial size of the structure pixel 2 needs to be greater than or equal to the spatial size of the photosensitive pixel array; therefore, in this embodiment, the spatial size of the structure pixel 2 is designed to be 4×4, and the side length of the structure pixel 2 is designed to be 4×P.
[0036] When light is incident on structure pixel 2, structure pixel 2 will project 0° linearly polarized light, 45° linearly polarized light, 90° linearly polarized light and 135° linearly polarized light into the corresponding polarization regions in the 4×4 pixel array.
[0037] The 4×4 pixel array is divided into four polarization regions (A1, A2, A3, A4). Structure pixel 2 projects 0° linearly polarized light into polarization region A1, 45° linearly polarized light into polarization region A2, 90° linearly polarized light into polarization region A3, and 135° linearly polarized light into polarization region A4. See details... Figure 2 The left side.
[0038] Simultaneously, the pixel regions 4 marked with different intensity levels in the polarization region receive incident light intensities of different intensity levels, forming the target intensity gradient distribution in the polarization region.
[0039] See Figure 2 In the right-hand section, the polarization region is divided into four pixel regions (E1, E2, E3, E4). Structure pixel 2 ensures that each of these four regions receives incident light of different intensities. The different gray levels in the figure represent the intensity levels received by each pixel region. Pixel region E2 receives the highest incident light intensity, while pixel region E4 receives the lowest. This results in an intensity gradient distribution of 1:4:16:64 across the polarization regions E2, E1, E3, and E4. Simulation results show that the metasurface device proposed in this embodiment can provide approximately 36 dB of additional dynamic range gain at the optical level.
[0040] In other embodiments, the target polarization state of the incident light can also be preset as 0° linearly polarized light, 45° linearly polarized light, and 90° linearly polarized light. See also Figure 3The 4×4 photosensitive pixel array is divided into three polarization regions (A1, A2, and A3) and one normal region (A4). Structured pixel 2 projects 0° linearly polarized light into polarization region A1, 45° linearly polarized light into polarization region A2, and 90° linearly polarized light into polarization region A3. Unprocessed light enters the normal region A4. Alternatively, the normal region (A4) can be omitted. Or, a fourth pixel region can be configured as needed.
[0041] In other embodiments, the target polarization state of the incident light can be preset to include 0° linearly polarized light, 60° linearly polarized light, and 120° linearly polarized light. See also Figure 4 The 4×4 photosensitive pixel array is divided into three polarization regions (A1, A2, and A3) and one normal region (A4). Structured pixel 2 projects 0° linearly polarized light into polarization region A1, 60° linearly polarized light into polarization region A2, and 120° linearly polarized light into polarization region A3. Unprocessed light enters the normal region A4. Alternatively, the normal region (A4) can be omitted. Or, a fourth pixel region can be configured as needed.
[0042] In this application, the target intensity gradient distribution is not fixed and can be set according to the required dynamic range, such as setting 3 gradients: 1:10:10:100 or 1:10:100.
[0043] Alternatively, you can set only two gradients: 1:10 or 1:100.
[0044] The higher the number of gradients, the clearer the imaging details. The larger the gradient range, the higher the imaging dynamic range.
[0045] The objective function is designed based on the distribution of the target polarization state and the distribution of the target intensity gradient. The topology optimization algorithm is used to generate the structural arrangement of structural pixel 2 with the objective function as the evaluation index.
[0046] The material of the metasurface structure layer 1 can be one or more of silicon nitride, titanium dioxide, or amorphous silicon. To improve compatibility with existing semiconductor processing technologies, silicon nitride is used as the structural material of the metasurface structure layer 1 in this embodiment.
[0047] The photosensitive pixel array 5 is used to convert incident light into electrical signals and transmit them to the imaging reconstruction unit; The imaging reconstruction unit is used to perform weighted fusion of the received electrical signals based on the target intensity gradient distribution and the target polarization state distribution, and restore them into a high dynamic range image containing polarization information through spatial rearrangement.
[0048] The metasurface structure layer comprises multiple periodically arranged structural pixels; therefore, fabricating the metasurface structure layer by designing the structural arrangement of these structural pixels includes the following steps: Step (1): Determine the objective function required for the design based on the required target intensity gradient distribution and target polarization state distribution.
[0049] For example, in this embodiment, the distribution of the target polarization state is as follows: 0° linearly polarized light is projected into polarization region A1, 45° linearly polarized light is projected into polarization region A2, 90° linearly polarized light is projected into polarization region A4, and 135° linearly polarized light is projected into polarization region A3.
[0050] The target intensity gradient distribution is a 1:4:16:64 intensity gradient distribution formed in the E2 pixel region, E1 pixel region, E3 pixel region, and E4 pixel region.
[0051] Step (2): Select optimization algorithms such as topology optimization and genetic algorithm, and use the objective function in step (1) as the evaluation index to optimize the structural distribution of structural pixels. The shape can be a free shape or a regular shape such as cylinder, toroidal cylinder, cuboid, regular polygonal prism, ellipsoid, X-shaped cube, C-shaped cube, cross cube and one or more of the above-mentioned Babienne structures; the material can be one or more of silicon nitride, amorphous silicon, single crystal silicon, polycrystalline silicon, titanium dioxide, etc.
[0052] Step (3): Based on the structural distribution generated by the optimization algorithm in step (2), generate a model file of the periodic arrangement required for processing.
[0053] Step (4): Based on the model file generated in step (3), process it using micro / nano manufacturing technology. For example... Figure 5 The diagram shows the structural distribution of the three types of structural pixels obtained from the processing. From left to right, they are: free topological shape, periodically arranged nanopillars, and checkerboard-type structure filling.
[0054] Therefore, the intensity gradient distribution in the light field generated by the metasurface structure layer can be adjusted as needed; the distribution of the target polarization state can also be adjusted as needed.
[0055] The system structure of this application is simple, eliminating the need for complex circuits such as large and small pixels and multi-frame exposure synthesis algorithms, effectively solving problems such as system complexity and dynamic artifacts.
[0056] It should be understood that expressions such as “comprising” and “may include” used in this application indicate the existence of the disclosed functions, operations, or constituent elements, and do not limit one or more additional functions, operations, and constituent elements. In this application, terms such as “comprising” and / or “having” are to be interpreted as indicating a particular characteristic, number, operation, constituent element, component, or combination thereof, but not to exclude the existence or possibility of adding one or more other characteristics, numbers, operations, constituent elements, components, or combinations thereof.
[0057] Furthermore, in this application, the expression "and / or" includes any and all combinations of the associated listed words. For example, the expression "A and / or B" may include A, may include B, or may include both A and B.
[0058] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a detachable connection or a non-detachable connection; it can be a direct connection or an indirect connection through an intermediate medium. "Fixed connection" refers to a connection where the relative positional relationship remains unchanged after connection. "Rotary connection" refers to a connection where the components can rotate relative to each other after connection. "Sliding connection" refers to a connection where the components can slide relative to each other after connection. The directional terms mentioned in the embodiments of this application, such as "top," "bottom," "inner," "outer," "left," and "right," are only for reference to the directions in the accompanying drawings. Therefore, the directional terms used are for better and clearer explanation and understanding of the embodiments of this application, and are not intended to indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.
[0059] Furthermore, the mathematical concepts mentioned in the embodiments of this application, such as symmetry, equality, parallelism, and perpendicularity, are limitations specific to the current technological level, rather than absolute and strict mathematical definitions. Slight deviations are permissible; approximations of symmetry, equality, parallelism, and perpendicularity are all acceptable. For example, "A and B are parallel" means that A and B are parallel or approximately parallel, and the angle between A and B can be between 0 and 10 degrees. "A and B are perpendicular" means that A and B are perpendicular or approximately perpendicular, and the angle between A and B can be between 80 and 100 degrees.
[0060] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A pixel-level metasurface device for realizing polarization high dynamic range imaging, characterized in that, For use in the front end of an image sensor, including: At least one metasurface structure layer, the metasurface structure layer comprising a plurality of periodically arranged structure pixels; Each of the structural pixels corresponds to a photosensitive pixel array in the image sensor, and the photosensitive pixel array contains at least [number of pixels]. 1 photosensitive pixel, of which For the first The order of the target energy gradient distribution within each target polarization state; The number of target polarization states; When the metasurface structure layer receives incident light containing multiple polarization states, the structure pixels perform a joint polarization-intensity mapping on the incident light, projecting the incident light of various target polarization states onto different polarization regions in the corresponding photosensitive pixel array, so that the pixel regions marked with different intensity levels in the polarization regions receive incident light intensities of different intensity levels, forming the target intensity gradient distribution in the polarization regions.
2. The pixel-level metasurface device according to claim 1, characterized in that, Based on the distribution of the target polarization state and the distribution of the target intensity gradient, an objective function is designed, and a topology optimization algorithm is used to generate the structural arrangement of the structural pixels with the objective function as the optimization objective.
3. The pixel-level metasurface device according to claim 2, characterized in that, The objective function is specifically: ； in, Let be the objective function. Let be the index of the target polarization state. The number of the target polarization states. For the first Spectroscopic efficiency of each target polarization state For the first The similarity between the actual intensity gradient distribution in each target polarization state and the target intensity gradient distribution. For the first The ratio of crosstalk intensity received by the polarization region corresponding to the target polarization state from other polarization states. , and For the first The weighting coefficients of each target polarization state.
4. The metasurface device according to claim 1, characterized in that, The spatial dimensions of the structural pixel are: The corresponding spatial size of the photosensitive pixel array is ,satisfy , .
5. The pixel-level metasurface device according to claim 1, characterized in that, The target polarization state includes, but is not limited to, one or more combinations of the following: 0° linear polarization state, 45° linear polarization state, 60° linear polarization state, 90° linear polarization state, 120° linear polarization state, 135° linear polarization light, left-hand circular polarization state, right-hand circular polarization state, left-hand elliptically polarization state, and right-hand elliptically polarization state.
6. The pixel-level metasurface device according to claim 1, characterized in that, The order of the target intensity gradient distribution is equal to 2, so that the pixel regions marked with different intensity levels in the polarization region receive high intensity level incident light intensity and low intensity level incident light intensity respectively, forming high intensity level pixel regions and low intensity level pixel regions; the high intensity level pixel regions are responsible for capturing dark details, and the low intensity level pixel regions are responsible for capturing highlight details.
7. The pixel-level metasurface device according to claim 1, characterized in that, The order of the target intensity gradient distribution is greater than or equal to 3, such that the pixel regions marked with different intensity levels in the polarization region respectively receive high intensity level incident light intensity, low intensity level incident light intensity and at least one intermediate intensity level incident light intensity, forming high intensity level pixel regions, low intensity level pixel regions and at least one intermediate intensity level pixel regions; the high intensity level pixel regions are responsible for capturing dark details, the low intensity level pixel regions are responsible for capturing highlight details, and the intermediate intensity level pixel regions are responsible for capturing intermediate brightness details.
8. The pixel-level metasurface device according to claim 7, characterized in that, The intensity ratio between any two adjacent intensity gradients in the target intensity gradient distribution is equal.
9. The pixel-level metasurface device according to claim 7, characterized in that, The target intensity gradient distribution is specifically 1:4:16:
64.
10. A pixel-level metasurface imaging system for achieving high dynamic range polarization imaging, characterized in that, include: The metasurface device, image sensor, and imaging reconstruction unit according to any one of claims 1-9; The image sensor is used to convert incident light into electrical signals and transmit them to the imaging reconstruction unit; The imaging reconstruction unit is used to perform data filtering and weighted fusion on the received electrical signals according to the distribution of the target intensity gradient and the distribution of the target polarization state, and restore them into a high dynamic range image containing polarization information through spatial rearrangement.