Metasurface device for snapshot type high dynamic range imaging and imaging system and method thereof

Abstract

According to the metasurface device for snapshot type high dynamic range imaging and the imaging system and method thereof provided by the invention, the dynamic range of a camera can be improved by integrating a plurality of superlenses with different transmittances on a single metasurface; the polarization-independent metasurface is designed by using a pure phase modulation principle, so that the system stability is greatly improved; compared with other backlight imaging methods, the method has the advantages of high integration level, single measurement and space and time cost saving, and has a certain reference effect on development of ultra-compact imaging equipment and improvement of dynamic range imaging capability.

Description

Technical Field

[0001] This invention relates to the field of backlight imaging technology, and in particular to a metasurface device and its imaging system and method for snapshot-type high dynamic range imaging, which is a metasurface with complex amplitude response and polarization-independent response designed using the two-atom interference principle. Background Technology

[0002] Due to the limited dynamic range of detectors, imaging systems often encounter overexposure or underexposure issues when facing environments with large changes in intensity. Details in both bright and dark areas of the image are often not effectively preserved in a single frame. This problem is common in applications such as smartphone photography, microscopy, advanced manufacturing, and autonomous driving. Developing real-time high dynamic range (HDR) imaging devices is crucial for these applications. A direct solution is to sequentially capture multiple images of the same scene with different exposure settings. The exposure of each image can be controlled by adjusting the aperture of the optical system or the exposure time of the image sensor. The complementary characteristics of these images can be combined to generate a HDR image. While this method works well in static scenes, it is not suitable for video recording because different shooting times and exposure lengths lead to varying degrees of motion blur and other time-related effects.

[0003] To overcome the limitations of sequential imaging in static scenes, multipath imaging systems have lower requirements for detectors and can achieve real-time imaging. However, imaging devices based on this method are often bulky. Metasurface technology, as an emerging optical technology, can precisely control the propagation, reflection, and refraction of light at the subwavelength scale, achieving superior optical effects compared to traditional optical components. Metasurfaces have already demonstrated powerful modulation capabilities and high integration advantages in expanding imaging capabilities.

[0004] Some studies have proposed specialized high dynamic range sensors with unique light response characteristics, such as adjusting or measuring pixel saturation time, or using staggered pixels with non-uniform intensity responses. A major challenge with these methods is the need to fabricate entirely new detectors. While these detectors may be commercially viable in the future, their current high manufacturing costs make these methods inaccessible to most researchers. Previous work proposed a spatial interpolation-based superlens that can capture and fuse nine images of varying intensities within a single frame, effectively improving the dynamic range of the image and the system's integration. However, the incomplete sublens functionality used in the proposed solution inevitably degrades image quality. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides a metasurface device, imaging system, and method for snapshot-type high dynamic range imaging. This device can simultaneously capture multiple images with different attenuation levels under a single exposure condition and expand the dynamic range of the imaging system through image fusion using a fusion algorithm. The system employs a complex amplitude metasurface designed based on the principle of dual superatomic interference. This complex amplitude metasurface possesses polarization-independent characteristics and can simultaneously modulate the amplitude and phase of the light field on a single-layer structure, thus replacing the complex multi-component structure of traditional HDR imaging systems. The complex amplitude metasurface integrates the functions of multiple components in a traditional optical system, enabling the simultaneous capture of multiple images with different attenuation levels. By fusing these images, the dynamic range of the imaging system can be effectively expanded without significantly increasing the weight or volume of the optical system.

[0006] The technical solution of the present invention is as follows:

[0007] A metasurface device for snapshot-type high dynamic range imaging is characterized by comprising a complex amplitude metasurface structure, wherein the complex amplitude metasurface structure adopts the dual superatomic interference design principle and contains at least two superlens phases with different transmittances, for simultaneously generating images with different exposure levels in a single exposure; the superlens units are arranged at subwavelength intervals, and the amplitude and phase are simultaneously modulated on a single-layer metasurface through interference effects.

[0008] Furthermore, the complex amplitude metasurface structure has polarization-independent properties, making it suitable for incident light of any polarization state.

[0009] The present invention also provides a snapshot-type high dynamic range imaging system, characterized in that it includes:

[0010] Such as the metasurface devices mentioned above;

[0011] The image fusion module is used to fuse multiple images with different intensity attenuation levels captured by a single exposure of the complex amplitude metasurface structure to generate a high dynamic range image.

[0012] This invention also provides a high dynamic range imaging method based on metasurfaces, characterized by comprising the following steps:

[0013] Using the aforementioned metasurface device, multiple images with different intensity attenuation levels can be captured simultaneously in a single exposure;

[0014] Images with different attenuation levels are separated from a single frame image. The low-exposure image is then subjected to nonlinear weighting. The processed image is then fused with the high-exposure image to generate the final high dynamic range image.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] By employing a metasurface design technique that integrates multiple superlenses on a single piece, the system can replace traditional multi-component HDR systems, reducing the size of the imaging system by more than 80% and achieving a high degree of integration.

[0017] By employing a single-exposure dual-channel imaging technique, high / low exposure images can be captured simultaneously, and the imaging speed is more than twice that of traditional multi-exposure methods.

[0018] A nonlinear fusion algorithm based on the hyperbolic tangent function is adopted to retain the effective information of each exposure image, achieving a dynamic range improvement of at least 17.5dB. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating the working principle of the superlens designed in this invention.

[0020] Figure 2 This is a flowchart of the superlens parameter design for this invention.

[0021] Figure 3 This is a schematic diagram of the superlens experimental device designed for this invention.

[0022] Figure 4 (a) is a photograph of the detector taken during the experiment of this invention. Figure 4 (b) is the true value pattern of the experimental target. Figure 4 (c) is from Figure 4 The low-exposure pattern captured in (a) Figure 4 (d) is from Figure 4 The high-exposure pattern captured in (a) Figure 4 (e) is the merged pattern. Figure 4 (fh) are respectively from Figure 4 (ce) Extracts the intensity distribution at the dashed line. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the various embodiments of the present invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0024] The metasurface designed in this invention adopts the principle of dual superatomic interference to convert two independent transmission phases into amplitude and phase channels, thereby achieving robust and complete complex amplitude modulation on a single-layer metasurface.

[0025] The metasurface used in this invention integrates multiple superlens phases with different transmission coefficients, and its transmission function needs to satisfy the following conditions:

[0026]

[0027] Where α and β are the off-axis angles in the horizontal and vertical directions, respectively, λ is the wavelength, and f is the focal length of the superlens. m denoted as the transmission coefficient, and m is the number of imaging lenses.

[0028] The layout design process of the metasurface designed in this invention is as follows: First, a complete database is established by scanning the width of the dielectric pillars. Each value in the database represents the electromagnetic response of a set of geometric parameters (W1, W2), and the electromagnetic response is a complex number. Then, imaging parameters such as focal length, off-axis angle, and intensity ratio are substituted into formula (1) to obtain the spatial distribution of the metasurface transmission coefficient. This coefficient is a complex matrix function with respect to spatial coordinates. For a point (i, j) in the matrix, the closest electromagnetic response in the database is found by looking up a table, satisfying Min{abs[t(i, j)-t(W1, W2)]}. The corresponding (W1, W2) value is the geometric dimension of the current point. The geometric parameters of the entire metasurface are determined by traversing all points.

[0029] To further illustrate the technical effects of the embodiments of the present invention, the inventors selected the number of imaging lenses m=2 in formula (1) and conducted experimental verification of dynamic range enhancement imaging. Two images were obtained in a single measurement, and the images were fused. The fusion process includes applying nonlinear intensity conversion to the low-intensity mode and then summing the intensities with the high-intensity mode. Specifically, for the acquired image, the high-intensity mode I is first cropped out. H and low-intensity mode I L The regions are fixed and invariant in the designed optical system. Then, normalization is performed on both images separately. For the low-intensity image, the goal is to preserve details in bright areas while discarding details in dark areas; this work focuses on weakening low-intensity pixels and enhancing high-intensity pixels. To ensure the smoothness of the intensity transformation, the hyperbolic tangent function (tanh) is used in the weight calculation. This weight satisfies:

[0030]

[0031] Among them, I L(m,n) represents the original intensity of pixel (m,n), c(m,n) is the weight value of that pixel, β is the smoothing coefficient (taken as 2 here), and ε is the mean of the image. Multiplying each pixel of the image by its corresponding weight yields the transformed image I′. L (m,n):

[0032] I' L (m,n)=c(m,n)·I L (m,n) (3)

[0033] For the captured high-intensity pattern, theoretically, only saturated pixels need to be processed. Considering that the intensity values ​​near saturated pixels are close to 1, to ensure the smoothness of the fused image, the intensity values ​​of the saturated pixels are directly set to 1. This means that these pixels do not require additional processing, and the values ​​of the high-intensity pattern remain unchanged. Finally, the transformed low-intensity pattern is added to the high-intensity pattern to obtain the fused intensity image J(m,n):

[0034] J(m,n)=I′ L (m,n)+I H (4)

[0035] Through experimental verification and analysis, this invention achieves a dynamic range enhancement of up to 17.5 dB.

[0036] In this invention, the number of imaging lenses in formula (1) is m = 2, k1 = 1, k2 = 0.3, the horizontal off-axis angles are α1 = 0, α2 = 0, and the vertical off-axis angles are β1 = 3.5°, β2 = -3.5°. The focal length is f = 8mm, the working wavelength is λ = 532nm, the designed superlens diameter is 2mm, and the corresponding numerical aperture is approximately 0.124. Figure 1 This is a schematic diagram illustrating the working principle of the complex amplitude superlens proposed in this invention for acquiring dual images.

[0037] Figure 2 This is a schematic diagram of the metasurface layout process. The metasurface layout process is as follows:

[0038] 1. Scan the width of the dielectric column to establish a complete database, where each value represents the electromagnetic response of a set of geometric parameters (W1, W2), and the electromagnetic response is a complex number.

[0039] 2. Substitute imaging parameters such as focal length, off-axis angle, and intensity ratio into formula (1) to obtain the spatial distribution of the metasurface transmission coefficient.

[0040] 3. For a point (i,j) in the matrix, find the closest electromagnetic response in the database by looking up a table, satisfying Min{abs[t(i,j)-t(W1,W2)]}. The corresponding (W1,W2) value is the geometric dimension of the current point. The geometric parameters of the entire metasurface are determined by traversing all points. In the simulation, the values ​​of W1 and W2 range from 80-300nm.

[0041] Figure 3 This is a schematic diagram of the experimental setup. A mobile phone screen was selected as the self-illuminating object. The image was first filtered through a 532nm filter, and then projected onto the working area of ​​the superlens using a 10x magnified microscope objective, thus ensuring high-resolution imaging of complex targets. Subsequently, the superlens formed two sub-images, which were then received by the detector after passing through a 4× magnification system.

[0042] Figure 4 (a) is the entire image captured by the detector, with the corresponding real image of the object as shown below. Figure 4 As shown in (b). From Figure 4 The low-intensity and high-intensity images extracted in (a) are as follows: Figure 4 As shown in (c,d). In Figure 4 In (c), the image is severely affected by noise, with some areas being submerged. The noise mainly originates from metasurface manufacturing errors, and partly from the image sensor. Here, the image sensor is a PCO camera (PCO.edge4.2mHQ), with a uniform bit depth of 12 bits and a corresponding maximum grayscale range of 0–4095. Figure 4 In (d), due to overexposure, texture information is lost in some areas. For a more intuitive comparison, the low-intensity and high-intensity modes are considered as imaging results of a common superlens under different exposure settings. It can be seen that no single image can simultaneously present the bright and dark details of an object. Through fusion... Figure 4 (c,d), we get Figure 4 The image in (e) restores both highlight and shadow details.

[0043] Figure 4 The PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) values ​​provided in (be) indicate that the quality of the fused image is higher than that of any single image. Figure 4 (fh) are respectively from Figure 4 (ce) Extracts the intensity distribution at the dashed line.

Claims

1. A metasurface device for snapshot-type high dynamic range imaging, characterized in that, It includes a complex amplitude metasurface structure, which adopts the design principle of dual superatomic interference and contains at least two superlenses with different transmittances to capture multiple images with different intensity attenuation levels.

2. The metasurface device for snapshot-type high dynamic range imaging according to claim 1, characterized in that, The complex amplitude metasurface structure has polarization-independent properties and is suitable for incident light of any polarization state.

3. A snapshot-type high dynamic range imaging system, characterized in that, include: The metasurface device as described in claim 1; The image fusion module is used to fuse multiple images with different intensity attenuation levels captured by a single exposure of the complex amplitude metasurface structure to generate a high dynamic range image.

4. A high dynamic range imaging method based on metasurfaces, characterized in that, Includes the following steps: Using the metasurface device as described in claim 1 or 2, multiple images with different intensity attenuation levels can be captured simultaneously in a single exposure; Images with different attenuation levels are separated from a single frame image. The low-exposure image is then subjected to nonlinear weighting. The processed image is then fused with the high-exposure image to generate the final high dynamic range image.

Images