Compact near-infrared depth camera based on metasurface and depth estimation method thereof
By using single helical phase and diffraction-free phase superposition technology of the metasurface layer in near-infrared depth cameras, the problem of high volume and cost of traditional 3D imaging systems is solved, and a compact depth camera in natural environment is realized and the distance measurement is improved Accuracy and depth of field range reduce power consumption and external environmental impact.
Patent Information
- Application Number
- CN202310784999.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2024-12-31
AI Technical Summary
Traditional 3D imaging systems such as TOF and structured light require active illumination, which increases the volume and cost of the optical system. The performance of the binocular vision system is subject to the baseline length, making it difficult to achieve a compact depth camera system in a natural environment.
A compact near-infrared depth camera based on metasurface is designed, including lens and image sensor. The lens is composed of a stop layer, a filter layer and a metasurface layer. The metasurface layer is composed of a micro-nano structure array, and its phase variation range is 0~2π, the depth range and accuracy are improved through single-helical phase addition and diffraction-free phase superposition, and depth estimation is used using depth deconvolution algorithm.
It realizes a compact depth camera without active lighting, reduces system power consumption and external environmental impact, improves ranging accuracy and depth of field range, and reduces the size and cost of the optical system.
Smart Images
Figure BDA0004311995870000021 
Figure BDA0004311995870000022 
Figure BDA0004311995870000031
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of optical devices, and in particular relates to a compact near-infrared depth camera based on a metasurface and a depth estimation method thereof. Background Art
[0002] A depth camera is a camera that can sense the depth of a scene. Traditional cameras can only obtain two-dimensional image information, but cannot sense the distance and depth information of objects in the scene. A depth camera can obtain the distance information of different objects in the scene through a built-in depth sensor or passive optical methods, thereby obtaining a three-dimensional scene model containing depth information. Depth cameras are widely used in computer vision, human-computer interaction, virtual reality and other fields, and are one of the important basic technologies for realizing technologies such as three-dimensional reconstruction, stereoscopic vision, and gesture recognition.
[0003] However, traditional 3D imaging systems such as TOF (Time-of-flight) and structured light require active illumination, which increases the size and cost of the optical system. The performance of the binocular vision system is limited by its own baseline length, so a compact depth camera system that can work in natural environments is needed. Summary of the invention
[0004] The present application provides a compact near-infrared depth camera based on a metasurface and a depth estimation method thereof to at least solve the above technical problems existing in the prior art.
[0005] On the one hand, the embodiment of the present application provides a compact near-infrared depth camera based on a metasurface, including a lens and an image sensor, wherein the lens includes an aperture layer, a filter layer and a metasurface layer; the aperture layer is composed of an opaque film; the filter layer is used for filtering; the metasurface layer is composed of a micro-nano structure array, and its phase variation range is 0 to 2π, and the phase distribution of the metasurface layer is obtained according to the following calculation formula:
[0006]
[0007] where x, y are the metaatom coordinates, λ is the incident wavelength, and f is the focal length of the metasurface. is the vortex phase distribution of the ring zone.
[0008] In one embodiment, according to the compact near-infrared depth camera based on a metasurface according to claim 1, it is characterized in that the vortex phase distribution formula of the sub-annular band is as follows:
[0009]
[0010] Where l is the topological charge, θ = atan (y / x) is the azimuth angle, is the radial length.
[0011] In one possible implementation manner, the micro-nano structure array is composed of nano-pillars or nano-holes, and the arrangement of the micro-nano structure array includes a tetragonal arrangement or a hexagonal arrangement; the central wavelength of the filter layer is between 800-1800nm, and the half-wave width is between 10-50nm.
[0012] In one possible implementation manner, the central wavelength of the filter layer is 940 nm and the half-wave width is 50 nm.
[0013] In one embodiment, a structural support is further included, wherein the structural support fixes the lens and the image sensor.
[0014] Another aspect of the present application provides a depth estimation method for a compact near-infrared depth camera, comprising the following steps:
[0015] S1, the target object obtains a modulated single helical point spread function through the metasurface layer and is imaged on the image sensor;
[0016] S2. Obtain a two-dimensional image and a depth map of the reconstructed target object through digital decoding.
[0017] In one embodiment, the depth information of the target image is reconstructed by using a deep deconvolution algorithm with the target image modulated by the single-PSF and the image modulated by the standard PSF.
[0018] In one embodiment, the point spread function modulated by the single-PSF can be restored by the following formula:
[0019]
[0020] where F -1 represents the inverse Fourier transform, I rot =F{in rot},I ref =F{in ref}.
[0021] In one possible implementation, superimposing a diffraction-free phase on a metasurface improves the depth of field range and depth of field accuracy of a camera system.
[0022] In one possible implementation, the third-order phase is superimposed on the metasurface to improve the depth of field range and depth of field accuracy of the camera system.
[0023] Compared with the prior art, this application has the following advantages:
[0024] 1. This application adds a single-helix phase to the metasurface so that the metasurface forms an image with different rotation angles at different object distances on the camera imaging chip. The depth information of the object can be restored by the rotation angle, which is beneficial to improving the ranging accuracy.
[0025] 2. This application adopts semiconductor processing technology, which greatly reduces processing costs;
[0026] 3. This application uses a metasurface to replace the traditional lens group, effectively reducing the size and quality of the optical system;
[0027] 4. The present application does not require active lighting equipment, which reduces the system power consumption while also eliminating the impact of the external environment on active lighting, making the depth camera system of the present application also robust to the impact of the external environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 is a schematic diagram of the structure of a compact near-infrared depth camera in an embodiment of the present application;
[0029] Figure 2 is a flow chart of a depth estimation method in an embodiment of the present application;
[0030] Figure 3 is a schematic diagram of the structure of the super surface layer of an embodiment of the present application;
[0031] Figure 4 It is a schematic diagram of the quadrilateral arrangement of the embodiment of the present application;
[0032] Figure 5 is a schematic diagram of the simulation results of the super surface of the embodiment of the present application;
[0033] Description of reference numerals:
[0034] 1. Aperture layer; 2. Filter layer; 3. Super surface layer; 4. Image sensor; 5. Structural support. DETAILED DESCRIPTION
[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0036] In the description of the present application, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0037] The present application embodiment discloses a compact near-infrared depth camera based on a metasurface. Figure 1 The compact near-infrared depth camera includes a lens, an image sensor 4 and a structural support 5. The lens and the image sensor 4 are both fixedly installed in the structural support 5. The lens is fixed to one side of the structural support 5, and the image sensor 5 is installed to the other side of the structural support 5.
[0038] The lens includes an aperture layer 1, a filter layer 2 and a super surface layer 3 from left to right. The aperture layer 1 is composed of an opaque film, and the filter layer 2 is used for filtering. The filter layer 2 is evaporated onto the back of the super surface layer 3 by thermal evaporation. The aperture layer 1 is evaporated onto the filter layer 2 by thermal evaporation, so that the aperture layer 1, the filter layer 2 and the super surface layer 3 are integrated into an integral lens.
[0039] During detection, the light beam reflected back by the detected object will be blocked by the aperture layer 1 to remove stray light, the light beam passing through the aperture layer 1 will be filtered out of other bands by the filter layer 2, and the light beam will be focused onto the receiving end face of the image sensor 5 through the metasurface layer 3. The receiving end face will perform analog-to-electric conversion, and the infrared information will be converted into an image on the display.
[0040] The super surface layer 3 is composed of a micro-nano structure array, and its phase variation range is 0 to 2π. The phase distribution of the super surface layer can be obtained according to the following calculation formula:
[0041]
[0042] where x, y are the metaatom coordinates, λ is the incident wavelength, and f is the focal length of the metasurface. is the vortex phase distribution of the ring zone.
[0043] The vortex phase distribution formula of the sub-ring belt is:
[0044]
[0045] Where l is the topological charge, θ = atan (y / x) is the azimuth angle, is the radial length.
[0046] The micro-nano structure array is composed of nano columns or nano holes, and the arrangement of the micro-nano structure array is a tetragonal arrangement or a hexagonal arrangement. The central wavelength of the filter layer is between 800-1800nm, and the half-wave width is between 10-50nm. Preferably, the central wavelength of the filter layer is 850nm, 940nm or 1550nm, and the half-wave width is 10nm, 15nm, 20nm or 50nm. Preferably, the central wavelength of the filter layer is 940nm, and the half-wave width is 50nm.
[0047] The depth estimation method for any of the above compact near-infrared depth cameras is as follows:
[0048] S1, the target object obtains a modulated single-helix point spread function (single-PSF) through the metasurface layer and is imaged on the image sensor;
[0049] S2. Obtain a two-dimensional image and a depth map of the reconstructed target object through digital decoding.
[0050] Among them, the point spread function modulated by single-PSF can be restored by the following formula:
[0051]
[0052] where F -1 represents the inverse Fourier transform, I rot =F{in rot},I ref =F{in ref}.
[0053] The depth information of the target image is reconstructed by using a deep deconvolution algorithm on the target image modulated by single-PSF and the image modulated by standard PSF.
[0054] The non-diffraction phase can be superimposed on the metasurface to improve the depth of field range and depth of field accuracy of the camera system. Preferably, the three-time superposition of the phase on the metasurface is performed to improve the depth of field range and depth of field accuracy of the camera system.
[0055] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A compact near-infrared depth camera based on a metasurface, characterized in that: The lens comprises a lens and an image sensor, wherein the lens comprises an aperture layer, a filter layer and a super surface layer; the aperture layer is composed of a light-proof film; the filter layer is used for filtering; the super surface layer is composed of a micro-nano structure array, and its phase variation range is 0 to 2π, and the phase distribution of the super surface layer is obtained according to the following calculation formula: where x, y are the metaatom coordinates, λ is the incident wavelength, and f is the focal length of the metasurface. is the vortex phase distribution of the ring zone.
2. The compact near-infrared depth camera based on a metasurface according to claim 1, characterized in that: The vortex phase distribution formula of the sub-ring belt is as follows: Where l is the topological charge, θ = atany / x is the azimuth angle, is the radial length.
3. A compact near-infrared depth camera based on a metasurface according to claim 1 or 2, characterized in that: The micro-nano structure array is composed of nano columns or nano holes, and the arrangement of the micro-nano structure array includes a tetragonal arrangement or a hexagonal arrangement; the central wavelength of the filter layer is between 800-1800nm, and the half-wave width is between 10-50nm.
4. The compact near-infrared depth camera based on a metasurface according to claim 3, characterized in that: The central wavelength of the filter layer is 850nm, 940nm or 1550nm, and the half-wave width is 10nm, 15nm, 20nm or 50nm.
5. The compact near-infrared depth camera based on a metasurface according to claim 1, characterized in that: Also included is a structural support that secures the lens and the image sensor.
6. A depth estimation method for a compact near-infrared depth camera, characterized in that: The steps include: S1. The target object obtains a modulated single helical point spread function through the metasurface layer and is imaged on the image sensor; S2. Obtain a two-dimensional image and a depth map of the reconstructed target object through digital decoding.
7. A depth estimation method according to claim 6, characterized in that: The point spread function modulated by single-PSF can be restored by the following formula: where F -1 represents the inverse Fourier transform, I rot =F{in rot },I ref =F{in ref }.
8. A depth estimation method according to claim 6 or 7, characterized in that: The depth information of the target image is reconstructed by using a deep deconvolution algorithm on the target image modulated by single-PSF and the image modulated by standard PSF.
9. A depth estimation method according to claim 8, characterized in that: The diffraction-free phase is superimposed on the metasurface to improve the depth of field range and depth of field accuracy of the camera system.
10. A depth estimation method according to claim 9, characterized in that: The three-phase superposition on the metasurface improves the depth of field range and depth of field accuracy of the camera system.