An optical acquisition system based on material structure and its application

Abstract

The present invention relates to the field of optics, and discloses an optical acquisition system based on material structure and its application. An optical acquisition system based on material structure includes a three-dimensional object, a metasurface, a camera array, a filter, and a slide rail. The metasurface is parallel to the camera array plane, and the distance between the metasurface and the camera array cannot exceed 1 cm. It is divided into different sub-regions according to the camera position. The nanoantennas in each sub-region are designed for phase distribution according to the camera position. The area of ​​the region should be greater than or equal to the camera lens, and the nanoantennas in different sub-regions cannot intersect. The advantage of the present invention is that a layer of two-dimensional metasurface is introduced in front of the camera array plane. It is small and thin, which not only realizes the projection of orthogonal perspectives at corresponding positions, but also does not significantly increase the complexity of the optical system. It can be freely designed according to the complexity of the acquisition scene, and has great flexibility.

Description

Technical Field

[0001] The present invention relates to the field of optics, and mainly relates to an optical acquisition system based on material structure and applications thereof. Background Art

[0002] Holography is a true 3D display technology that uses the principles of light diffraction to generate and reconstruct the wavefront distribution of a complete three-dimensional object. Compared to other 3D display technologies, holographic display can record the amplitude and phase information of light waves emitted from every point in a three-dimensional scene, and when reproducing it, it can perfectly restore the actual light field of the three-dimensional scene. Therefore, it is considered the most ideal 3D display technology.

[0003] Computer-generated holography (CGH) uses computers to generate the complex-valued distribution of an object's light field. Commonly used methods include the point cloud method, polygon method, multi-plane method, and light field method. Among them, the light field model can capture real-life scenes and objects and can be applied to the recording and reproduction of real 3D scenes. It has great practical significance and has therefore attracted widespread attention from researchers. However, in the actual acquisition process, the following problems must be solved first: First, light field data can be regarded as a collection of orthogonal perspectives at different observation positions. The algorithm used to convert light fields into holograms often requires orthogonal images of the object along orthogonal perspectives. However, in reality, cameras cannot perfectly meet this acquisition condition. The light field data used to generate holograms is mainly obtained by computer simulation. Even if it is approximated by a perspective camera or a light field camera under certain conditions, the spatial resolution and directional resolution of the acquisition are easily restricted by the hardware of the camera itself. Secondly, for collecting light in different orthogonal directions of the object, although the propagation direction of light can be changed by refractive media or lenses, the lenses or media are large and thick. For light field acquisition systems that need to collect light in different directions at different positions, using refractive media or lenses is not particularly convenient. If the above problems cannot be solved, then the holographic technology based on light field conversion can only remain at the virtual object level, and the acquisition and 3D reconstruction in real scenes will still be greatly limited. Summary of the Invention

[0004] In order to address the shortcomings and deficiencies of the prior art, the purpose of the present invention is to provide an optical acquisition system based on material structure. The invention can use the light field data collected from real scenes for the generation of computational holography. A pre-designed metasurface is introduced into the optical acquisition system, and the metasurface is divided into different sub-regions according to the different positions of the camera. Each region is designed with a different phase distribution according to the angle θ of the camera along the main axis. According to the phase distribution of different regions on the metasurface, light in different orthogonal directions can be projected into the camera behind the metasurface, ultimately realizing the acquisition of the light field.

[0005] Specifically, the present invention is achieved through the following technical solutions:

[0006] An optical acquisition system based on material structure, comprising: a three-dimensional object, a metasurface, a camera array, a filter, and a slide rail.

[0007] Preferably, the metasurface is parallel to the camera array plane, and the distance between the metasurface and the camera array cannot exceed 1 cm.

[0008] Preferably, the metasurface is divided into different sub-regions according to the camera position, the position of the sub-region corresponds to the position or movement position of the camera, the area of ​​the sub-region is larger than the camera lens, and different sub-regions cannot intersect.

[0009] Preferably, the metasurface is composed of a transparent substrate and a nanoantenna array, the material type, structure, and spacing of the nanoantennas in the same sub-region are the same, and the material type, structure, and spacing of the nanoantennas in different sub-regions are the same or different.

[0010] Preferably, the material of the nanoantenna is at least one of titanium dioxide, silicon nitride, single crystal silicon or aluminum oxide;

[0011] The nanoantenna structure is at least one of a V-shape, a U-shape, a nanocolumn shape, a nanofin shape or a nanohole shape.

[0012] Preferably, the nano-antenna structure is a nano-fin type, the nano-fin width is set to 85nm, the length is 410nm, the height is 600nm, and the adjacent interval is set to 430nm.

[0013] When the filter is used to collect multi-color objects or scenes, the filter is introduced to collect light field data of three channels, red, green and blue, respectively. Different nanoantenna materials can be selected for different colors.

[0014] The slide rail is used to move the camera to collect image data from different perspectives, and its accuracy needs to be maintained at at least millimeter level.

[0015] Preferably, the optical phase distribution of the metasurface within the sub-region should satisfy

[0016] in,

[0017] k0: free space wave vector

[0018] n i : Refractive index of the incident space medium

[0019] θ i : Angle between incident light and optical axis

[0020] r: Projection coordinates of the line connecting the camera center and the object point on the hypersurface

[0021] c: constant.

[0022] Preferably, the camera's acquisition angle θ covers at least Where λ is the wavelength of light, dx is the pixel size of the hologram,

[0023] The angular interval dθ of cameras at different sampling positions remains consistent, that is, N u is the number of cameras, The cameras face the same direction, that is, towards the metasurface.

[0024] Preferably, when using an optical acquisition system to acquire light fields, the depth of the acquired three-dimensional objects should be within within the scope;

[0025] Alternatively, when collecting scenes beyond this depth range, the objects in the scene can be divided according to depth, and the light field data of different objects can be collected separately. The individual depth range of each object should meet the above range requirements, and the hologram corresponding to the light field of each object is calculated separately, and then the different holograms are added together through the mask-based diffraction calculation propagation method.

[0026] The application of the above-mentioned material structure-based optical acquisition system in light field acquisition.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects:

[0028] (1) The present invention introduces a two-dimensional metasurface in front of the camera plane, which is small and thin, and can achieve projection in different orthogonal directions at corresponding positions without significantly increasing the complexity of the optical system.

[0029] (2) The present invention divides the metasurface into different sub-regions, each of which corresponds to a camera at a different position. Although different camera positions require projections of light in different directions, different phase distributions can be designed within the sub-regions. Different phase distributions can cause light to be deflected at corresponding positions, greatly saving costs while improving efficiency. Therefore, when collecting the light field, it is only necessary to move the camera to the corresponding position.

[0030] (4) The nanomaterial arrays in each sub-region are independent of each other and can be freely designed according to the complexity of the acquisition scenario, which has great flexibility.

[0031] (3) This acquisition method is based on holography generated by light fields. The angular resolution of the reconstructed object is determined by the number of cameras (or the number of camera movements) rather than the cameras themselves. In this way, under special conditions, especially in scenarios requiring higher angular resolution, it is not affected by the camera hardware system and has higher flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above general description and the following detailed description of the drawings are only exemplary and explanatory and do not limit the present invention. The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the description, serve to explain the principles of the present invention.

[0033] Figure 1A This is a schematic diagram of a light field acquisition method for direct computationally generated holography in the present invention. The symbols in the figure are as follows: 11 - 3D object, 12 - metasurface plane, 13 - metasurface subregion, 14 - camera;

[0034] Figure 1B Schematic diagram of the camera structure used in the present invention for collecting images; the symbols in the figure are as follows: 15-object plane, 16-image plane, the size ratio between the actual object plane and the image plane is the orthogonal ratio M, 17-optical system of the camera.

[0035] Figure 2A This is a surface image of a nanoarray in a sub-region of the metasurface in Example 1 of the present invention;

[0036] Figure 2B This is a schematic diagram of a single nanofin in a subregion of the metasurface in Example 1 of the present invention; the markings in the figure are as follows: 21-nanofin length, 22-nanofin width, 23-center distance between two adjacent nanofins in the same subregion.

[0037] Figure 3A It is the light field data at different depth planes under their orthogonal viewing angles, and the data size is 40x40.

[0038] Figure 3B It is a hologram generated by collecting light field data.

[0039] Figure 3C are views of the hologram reconstructed at different depths.

[0040] Figure 4 Schematic diagram of image acquisition by a light field camera in the second embodiment of the present invention; the symbols in the figure are as follows: 31 - main lens, 32 - microlens array, 33 - sensor. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] Example 1

[0043] refer to Figures 1A-2BThis embodiment provides a light field acquisition method based on a camera array and a metasurface. The light field data collected by this method can be directly used to generate a computational hologram, avoiding the inherent contradiction between angular resolution and spatial resolution in traditional light field acquisition methods (see the comparative example for details). The operation is simple and fast, and can be used for light field acquisition in real scenes. In this embodiment, a sub-region of the metasurface is taken as an example. The metasurface is composed of a light-transmitting substrate and an array of nanoantennas (i.e., metamaterials). The nanoantenna material is amorphous titanium dioxide, and the structure adopts nanofins. The nanoantenna structure belongs to the geometric phase type. The nanofin width is set to 85nm, the length is 410nm, the height is 600nm, and the adjacent spacing is set to 430nm ( Figure 2B When left-handed (right-handed) circularly polarized light is incident vertically on the metasurface, the transmitted right-handed (left-handed) circularly polarized light will carry an additional phase of 2Ф, where Ф is the angle between the main axis of the nanofin and its rotation direction ( Figure 2B ). According to the generalized Snell's law:

[0044]

[0045] where n t is the refractive index of the transmission medium layer, θ t is the angle between the refracted light and the principal axis, n i is the refractive index of the incident medium layer, θ i is the angle between the center of the ith camera and the principal axis, For the required phase, it can be seen that the refraction of light is related to the phase distribution on the surface of the medium. Therefore, according to the orthogonal projection direction θ of the light field i ( Figure 1A ), first set the position of the camera, and then divide the corresponding sub-areas on the hypersurface according to the size of the camera lens. The distance between the hypersurface and the camera cannot be too far. In the sub-area, we need to orthogonal projection direction θ i The light is converted into vertical incidence on the camera lens to realize the collection of the orthogonal view of the object, so let θ t =0 and substituting it into the generalized Snell's law formula, we can get the phase gradient required in this sub-region. Then, based on the relationship between the metasurface light phase and the geometric properties of the metamaterial, the corresponding metamaterial is designed to project the light from the three-dimensional object in the orthogonal direction onto the camera surface. In this embodiment, we use a geometric phase metamaterial, whose transmitted light phase is That is, the metamaterial distribution should satisfy The final distribution of metamaterials in this sub-region is as follows: Figure 2A As shown in the figure. In different sub-areas, the distribution of metamaterials is determined by the camera position θ. i And change.

[0046] Optionally, the metamaterials in different sub-regions can be different. If it is a geometric phase, the angle between the main axis and the rotation direction is adjusted; if it is a resonant phase, the phase is adjusted by changing the geometric dimensions of the metamaterial, and so on.

[0047] Example 2

[0048] refer to Figure 3A-3C , this embodiment provides a way to directly use the light field collected by this method to calculate and generate holograms. Since the light field generation holographic algorithm based on Wigner transform needs to collect light field data from orthogonal projections in different directions, the traditional camera array cannot meet the collection needs, and the optical collection system based on the material structure proposed above can meet the requirements of the algorithm and realize holographic display in real scenes. This embodiment provides the generation of holograms from orthogonal projection light field data and the effect of its reconstruction. Figure 3A , This image set is a light field data set collected from a simulated object in different orthogonal directions. The six grid rectangles shown in the figure are in different depth layers [-6mm, 6mm]. According to the hologram diffraction angle calculation formula dx is the pixel size of the hologram. When collecting light field data, the angular range of camera movement should be greater than or equal to 2θ. In order to ensure that the spatial frequency intervals of the resulting holograms are equal, the angular intervals of camera movement should be equal. At this time, the number of camera samples is equivalent to the angular resolution of the holographically reconstructed object. The more cameras there are (or the number of times the same camera is moved), the smoother the transition of the image observed from different perspectives. The spatial resolution of the object corresponds to the resolution of the camera. The greater the image resolution, the clearer the reproduced object. At this point, the angular resolution and spatial resolution of the hologram are decoupled, and the angular resolution is no longer associated with the hardware of the camera itself. In this embodiment, a light field generation holographic algorithm based on Wigner transform is adopted, and the resulting hologram is as follows: Figure 3B As shown in the figure, the effects of its reproduction at different depths are as follows Figure 3C shown.

[0049] Comparative Example 1

[0050] like Figure 4As shown, this comparative example provides a commonly used light field camera structure for light field acquisition. The structure consists of a main lens (31), a microlens array (32) and a corresponding sensor (33) behind it. The light emitted by the object point is refracted by the main lens and then passes through a certain microlens. The pixels on the sensor behind the microlens can capture the light from a specific direction. When usually used for computational holography, it is necessary to extract the pixels at the same local position behind different microlenses as sub-images. The number of pixels in the sub-image is determined by the number of microlenses in the camera, and the number of sub-images is determined by the number of pixels behind the microlenses. Therefore, each sub-image represents an image at a different orthogonal perspective, that is, the number of microlenses in the camera determines the spatial resolution of the reconstructed object, and the number of sub-images represents the number of images collected at different perspectives, that is, the number of pixels behind the microlenses determines the angular resolution of the reconstructed object. The final hologram is composed of the sub-images after the Fourier transform. When the spatial resolution of the reconstructed object is very high, the number of microlenses increases, but the number of pixels behind them decreases, that is, the angular resolution decreases, and vice versa. There is an inherent contradiction between spatial resolution and angular resolution. In addition, the high cost of the light field camera also limits its application.

[0051] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. An optical acquisition system based on material structure, characterized in that: The optical acquisition system includes a three-dimensional object, a metasurface, and a camera array arranged in sequence; The metasurface is parallel to the camera array plane, and the distance between the metasurface and the camera array cannot exceed 1 cm; The metasurface is divided into different sub-regions according to the position of the camera. The position of the sub-region corresponds to the position or movement position of the camera. The area of ​​the sub-region is larger than the camera lens, and different sub-regions cannot intersect. The optical collection system also includes a filter and a slide rail; The slide rail is used to move the camera to collect image data from different perspectives, and its accuracy needs to be maintained at at least millimeter level.

2. The optical acquisition system based on material structure according to claim 1, characterized in that: The metasurface is composed of a transparent substrate and a nanoantenna array. The material type, structure and spacing of the nanoantennas in the same sub-region are the same, and the material type, structure and spacing of the nanoantennas in different sub-regions are the same or different.

3. The optical acquisition system based on material structure according to claim 2, characterized in that: The material of the nano-antenna is at least one of titanium dioxide, silicon nitride, single crystal silicon or aluminum oxide; The structure of the nanoantenna is at least one of a V-shape, a U-shape, a nanocolumn shape, a nanofin shape or a nanohole shape.

4. The optical acquisition system based on material structure according to claim 2, characterized in that: Optical phase distribution of the metasurface within the sub-region Should meet in, k0: free space wave vector n i : Refractive index of the incident space medium θ i : Angle between incident light and optical axis r: Projection coordinates of the line connecting the camera center and the object point on the hypersurface c: constant.

5. The optical acquisition system based on material structure according to claim 1, characterized in that: The camera's acquisition angle θ covers at least Where λ is the wavelength of light, dx is the pixel size of the hologram, The angular interval dθ of cameras at different sampling positions remains consistent, that is, N u is the number of cameras, The cameras face the same direction, that is, towards the metasurface.

6. The optical acquisition system based on material structure according to claim 1, characterized in that: When using an optical acquisition system to collect light fields, the depth of the collected three-dimensional objects should be within within the scope; Alternatively, when collecting scenes beyond this depth range, the objects in the scene are divided according to depth, and the light field data of different objects are collected separately. The individual depth range of each object should meet the above range requirements, and the hologram corresponding to the light field of each object is calculated separately, and then the different holograms are added together through the mask-based diffraction calculation propagation method.

7. Application of the material structure-based optical acquisition system according to any one of claims 1 to 6 in light field acquisition.

Images