Light field imaging device and method

Abstract

The invention relates to the technical field of imaging devices, and particularly discloses a light field imaging device and method, and the device comprises a main lens which is configured to be used for receiving light in a scene and refracting the light to form first imaging; the diffraction elements are arranged on the side, away from the scene, of the main lens, the multiple diffraction elements are distributed in an array mode, and the diffraction elements are configured to perform equal-focal-length diffraction focusing on monochromatic light of the target wavelength in first imaging so as to form focused light; the light filtering element is arranged on the side, away from the main lens, of the diffraction element, and the light filtering element is configured to receive the focusing light and enable the monochromatic light to penetrate through the light filtering element; the image sensor is arranged on the side, away from the diffraction element, of the light filtering element, and the image sensor is configured to receive the monochromatic light penetrating through the light filtering element for secondary imaging and form an element image. The light field imaging device and the light field imaging method can effectively eliminate imaging chromatic aberration and reduce cost.

Description

Technical Field

[0001] The present invention relates to the technical field of imaging devices, and in particular to a light field imaging device and method. Background Art

[0002] In order to eliminate the chromatic aberration of the light field imaging system based on diffractive optical elements, the following two aspects are taken into consideration: (1) Designing a single achromatic diffractive lens: for example, using aluminum, silver, and gold micro-nanoparticles embedded in silicon dioxide sheets to design single-layer metasurfaces with the same focusing capabilities for wavelengths of 30-600nm, 550nm, and 650nm, and then arranging these metasurfaces in coaxial concentric rings in the structure of Fresnel zone plates to achieve achromatic aberration of a single diffractive lens. The achromatic diffractive lenses are formed into an array to build a light field imaging system; (2) Post-correction of chromatic aberration images is implemented based on algorithms. At present, the design and manufacture of achromatic diffractive lenses are very difficult (especially for centimeter-level aperture devices required for some imaging systems), the cost is very high, and the scope of application is limited. Image restoration methods based on deep learning and the like are limited by the generalization and computational complexity of the algorithm and cannot obtain completely achromatic images. Therefore, the technical solution of the present application is urgently needed to solve the above problems. Summary of the invention

[0003] The object of the present invention is to provide a light field imaging device which can effectively eliminate imaging chromatic aberration and reduce costs.

[0004] In order to achieve the above object, the present invention provides a light field imaging device, comprising:

[0005] A main lens is configured to receive light from a scene and refract it to form a first image;

[0006] a diffraction element, disposed on a side of the main lens away from the scene, wherein a plurality of the diffraction elements are provided and arranged in an array, and wherein the diffraction element is configured to perform diffraction focusing with equal focal length on the monochromatic light of the target wavelength in the first imaging to form focused light;

[0007] a filter element, disposed on a side of the diffraction element away from the main lens, the filter element being configured to receive the focused light and allow the monochromatic light to pass through the filter element; and

[0008] The image sensor is arranged on a side of the filter element away from the diffraction element, and is configured to receive the monochromatic light passing through the filter element for secondary imaging and form an element image.

[0009] In some embodiments, the first imaging is located in front of the front focal plane of the diffraction element.

[0010] In some embodiments, the diffraction element is one of a Fresnel lens, a Pancharatnam-Berry (PB) phase liquid crystal lens, and a meta-lens.

[0011] In some embodiments, the diffraction element has three types, which are respectively denoted as the first diffraction element, the second diffraction element and the third diffraction element. The focal lengths of the red, green and blue light corresponding to the first diffraction element are respectively denoted as f 1R 、f 1G and f 1B , the focal lengths of the red, green and blue lights corresponding to the second diffraction element are represented by f 2R 、f 2G and f 2B , the focal lengths of the red, green and blue lights corresponding to the third diffraction element are represented by f 3R 、f 3G and f 3B , then the relationship is satisfied: f=f 1R =f 2G =f 3B .

[0012] In some embodiments, the first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along a longitudinal straight line.

[0013] In some embodiments, the first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along a transverse straight line.

[0014] In some embodiments, the first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along straight lines inclined at 30-60 degrees.

[0015] In some embodiments, the filter element is a narrow-band filter, a plurality of narrow-band filters are provided, and the narrow-band filters are arranged in a one-to-one correspondence with the diffraction elements.

[0016] In some embodiments, the image sensor is a two-dimensional image sensor.

[0017] On the other hand, the present invention provides a light field imaging method, comprising:

[0018] The main lens receives light in the scene, and the main lens refracts the light to form a first image;

[0019] The light formed by the first imaging is received by a plurality of diffraction elements distributed in an array, and a single diffraction element performs diffraction focusing with equal focal length on the monochromatic light of the target wavelength to form a focused light;

[0020] receiving the focused light through a filter element, and allowing the monochromatic light to pass through the filter element;

[0021] The monochromatic light transmitted through the filter element is received by an image sensor, and a second imaging is performed on the image sensor to form an element image.

[0022] The present invention provides a light field imaging device and method, which have the following beneficial effects compared with the prior art:

[0023] The diffraction element is arranged on the side of the main lens away from the scene. The diffraction element is provided in plurality and distributed in an array. The diffraction element is configured to perform equal focal length diffraction focusing on the monochromatic light of the target wavelength in the first imaging to form focused light. The filter element is configured to receive the focused light and allow the monochromatic light to pass through the filter element. The image sensor is configured to receive the monochromatic light passing through the filter element for second imaging and form an elemental image. The image finally formed can eliminate chromatic aberration and does not need to be configured with an achromatic diffraction lens, thereby reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the principle of a light field imaging device provided for some embodiments of the present invention.

[0025] Figure 2 A schematic diagram of the imaging principle of a light field imaging device provided in some embodiments of the present invention, in which the diffraction elements are arranged horizontally along a longitudinal straight line.

[0026] Figure 3 A schematic diagram of the imaging principle of a light field imaging device provided in some embodiments of the present invention, in which the diffraction elements are arranged horizontally along a transverse straight line.

[0027] Figure 4 Schematic diagram of the imaging principle of a light field imaging device provided by some embodiments of the present invention, in which the diffraction elements are arranged horizontally along an inclined straight line

[0028] Figure 5 A flow chart of a light field imaging method provided for some other embodiments of the present invention.

[0029] In the figure: 1. Main lens; 2. Diffraction element; 3. Filter element; 4. Image sensor; 5. Scene; 6. First imaging. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0031] It should be understood that, in the description of the present application, the terms "upper", "lower", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. The terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, that is, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, unless otherwise specified, "multiple" means two or more.

[0032] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0033] like Figure 1-5 As shown, some embodiments of the present invention provide a light field imaging device, comprising a main lens 1, a diffraction element 2, a filter element 3 and an image sensor 4. Light in the scene is refracted by the main lens 1 for the first imaging, and is diffracted by the diffraction element 2 and filtered by the filter element 3 before being imaged for the second time on the image sensor 4 to form an elemental image with chromatic aberration eliminated.

[0034] The main lens 1 is configured to receive light from a scene 5 and refract it to form a first image 6. Specifically, the main lens adopts an existing common imaging convex lens.

[0035] The diffraction element 2 is arranged on the side of the main lens 1 away from the scene 5. There are multiple diffraction elements 2 and they are arranged in an array. The diffraction element 2 is configured to perform equal focal length diffraction focusing on the monochromatic light of the target wavelength in the first imaging 6 to form a focused light. In addition, the light of other wavelengths except the target wavelength also has a diffraction focusing effect, but the focal length does not need to be specifically set, and can be used as a source of chromatic aberration. In order to achieve three-dimensional image reconstruction, the position of the formed element image can be found on the image sensor 4 according to the projection relationship of the main light passing through the center of each diffraction element 2 through similar triangles, which is recorded as (x, y). Therefore, each corresponding element image has a certain parallax relationship with respect to the element image imaged by other diffraction elements 2. All element images with certain parallax are arranged in an array on the image sensor 4 in a certain regularity, and the image array formed is called an element image array. In light field imaging, a large number of parallax relationships recorded in the element image array contain the spatial information of the three-dimensional object. The translation amount (Δx) required for each element image to overlap and combine can be obtained by inferring the imaging light path propagation process of the array-distributed diffraction elements 2 through the parallax relationship on the plane of the image sensor 4. i ,Δyj ), and the corresponding translation amount is added to the element image using a post-processing algorithm to complete the three-dimensional image reconstruction with spatial information.

[0036] The filter element 3 is arranged on the side of the diffraction element 2 away from the main lens 1, and the filter element 3 is configured to receive the focused light and allow the monochromatic light to pass through the filter element 3. In the same plane, the arrangement of the diffraction elements 2 is the first diffraction element, the second diffraction element and the third diffraction element from left to right. At the preset plane position of the image sensor 4, the R channel element image of the first diffraction element, the G channel element image of the second diffraction element, and the B channel element image of the third diffraction element are not defocused, but the image sensor will still record the G and B channel element images of the first diffraction element, the R and B channel element images of the second diffraction element, and the R and G channel element images of the third diffraction element with different defocus information without screening. A wavelength selection device is added between the diffraction lens array and the image sensor to only screen the R channel element image of the first diffraction element, the G channel element image of the second diffraction element, and the B channel element image of the third diffraction element, so as to obtain an element image array consisting of single channel element images without defocus information. For this purpose, the design of the wavelength selection device also needs to be arranged periodically, and the arrangement of the selected wavelengths corresponds to the R wavelength of the first diffraction element, the G wavelength of the second diffraction element, and the B wavelength of the third diffraction element, respectively.

[0037] The image sensor 4 is disposed on the side of the filter element 3 away from the diffraction element 2, and is configured to receive the monochromatic light passing through the filter element 3 for secondary imaging and form an elemental image. During installation, the position of the image sensor 4 needs to be set according to the imaging relationship of the diffraction element 2 and the primary imaging position of the main lens 1.

[0038] Based on the above structural setting, image chromatic aberration can be eliminated, and there is no need to configure an achromatic diffraction lens, which can reduce costs.

[0039] In some embodiments, the first imaging is located in front of the front focal plane of the diffraction element 2. This is beneficial for the diffraction element 2 to perform diffraction transmission imaging. Specifically, the distance between the first imaging position and the diffraction element 2 (i.e., the imaging object distance) is greater than the focal length of the diffraction element 2 (real image type).

[0040] In some embodiments, the diffraction element 2 is one of a Fresnel lens, a PB phase liquid crystal lens and a meta-lens.

[0041] In some embodiments, the diffraction element 2 has three types divided according to the focal length, which are respectively denoted as the first diffraction element, the second diffraction element and the third diffraction element. The focal lengths of the red, green and blue light corresponding to the first diffraction element are respectively denoted as f 1R 、f 1G and f1B The focal lengths of the red, green and blue lights corresponding to the second diffraction element are expressed as f 2R 、f 2G and f 2B The focal lengths of the red, green and blue lights corresponding to the third diffraction element are expressed as f 3R 、f 3G and f 3B , then the relationship is satisfied: f=f 1R =f 2G =f 3B This ensures that image information of one color channel, R, G, and B, under each diffraction element 2 is recorded on the two-dimensional image array without defocusing.

[0042] like Figure 2 As shown, in some embodiments, the first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along a longitudinal straight line to achieve the function of eliminating chromatic aberration along the horizontal and longitudinal directions.

[0043] like Figure 3 As shown, in some embodiments, the first diffraction element, the second diffraction element and the third diffraction element are respectively arranged horizontally along a horizontal straight line in sequence to achieve the function of eliminating chromatic aberration along the horizontal direction.

[0044] like Figure 4 As shown, in some embodiments, the first diffraction element, the second diffraction element and the third diffraction element are respectively arranged horizontally along a straight line inclined at 30-60 degrees. Compared with the above two arrangements, in this embodiment, the first diffraction element, the second diffraction element and the third diffraction element have one more degree of freedom for controlling the color channel of the element image in one direction, which improves the upper limit of the achromatic ability in the post-processing element image synthesis algorithm, and has a better chromatic aberration elimination effect with the robustness of the achromatic light field imaging system.

[0045] In some embodiments, the filter element 3 is a narrow-band filter, and a plurality of narrow-band filters are provided, and the narrow-band filters are arranged one-to-one corresponding to the diffraction elements. The narrow-band filter is used to allow monochromatic light to pass.

[0046] In some embodiments, the image sensor 4 is a two-dimensional image sensor, and illustratively, a CCD sensor is used.

[0047] like Figure 5 As shown, some other embodiments of the present invention provide a light field imaging method, comprising the following steps:

[0048] S1, receiving light in the scene through the main lens, and refracting the light to form the first image;

[0049] S2, receiving the light formed by the first imaging through a plurality of diffraction elements distributed in an array, and a single diffraction element performs diffraction focusing of the monochromatic light of the target wavelength with equal focal length to form a focused light;

[0050] S3, receiving the focused light through the filter element, allowing the monochromatic light to pass through the filter element;

[0051] S4. The monochromatic light transmitted through the filter element is received by the image sensor, and a second imaging is performed on the image sensor to form an element image.

[0052] Based on the above method, the obtained image can eliminate chromatic aberration and reduce the cost of use.

[0053] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention. These improvements and substitutions should also be regarded as the scope of protection of the present invention.

Claims

1. A light field imaging device, characterized in that: include: A main lens is configured to receive light from a scene and refract it to form a first image; a diffraction element, disposed on a side of the main lens away from the scene, wherein a plurality of the diffraction elements are provided and arranged in an array, and wherein the diffraction element is configured to perform diffraction focusing with equal focal length on the monochromatic light of the target wavelength in the first imaging to form focused light; A filter element, disposed on a side of the diffraction element away from the main lens, the filter element being configured to receive the focused light and allow the monochromatic light to pass through the filter element; and The image sensor is arranged on a side of the filter element away from the diffraction element, and is configured to receive the monochromatic light passing through the filter element for secondary imaging and form an element image.

2. The light field imaging device according to claim 1, characterized in that: The first imaging is located in front of the front focal plane of the diffraction element.

3. The light field imaging device according to claim 1, characterized in that: The diffraction element is one of a Fresnel lens, a PB phase liquid crystal lens and a meta-lens.

4. The light field imaging device according to claim 1, characterized in that: The diffraction element has three types, which are respectively denoted as the first diffraction element, the second diffraction element and the third diffraction element. The focal lengths of the red, green and blue light corresponding to the first diffraction element are respectively denoted as f 1R 、f 1G and f 1B , the focal lengths of the red, green and blue lights corresponding to the second diffraction element are represented by f 2R 、f 2G and f 2B , the focal lengths of the red, green and blue lights corresponding to the third diffraction element are represented by f 3R 、f 3G and f 3B , then the relationship is satisfied: f=f 1R =f 2G =f 3B .

5. The light field imaging device according to claim 4, characterized in that: The first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along a longitudinal straight line.

6. The light field imaging device according to claim 4, characterized in that: The first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along a transverse straight line.

7. The light field imaging device according to claim 4, characterized in that: The first diffraction element, the second diffraction element and the third diffraction element are horizontally arranged in sequence along straight lines inclined at 30-60 degrees.

8. The light field imaging device according to claim 1, characterized in that: The filter element is a narrow-band filter, a plurality of narrow-band filters are provided, and the narrow-band filters are arranged in a one-to-one correspondence with the diffraction elements.

9. The light field imaging device according to claim 1, characterized in that: The image sensor is a two-dimensional image sensor.

10. A light field imaging method, characterized in that: include: The main lens receives light in the scene, and the main lens refracts the light to form a first image; The light formed by the first imaging is received by a plurality of diffraction elements distributed in an array, and a single diffraction element performs diffraction focusing with equal focal length on the monochromatic light of the target wavelength to form a focused light; receiving the focused light through a filter element, and allowing the monochromatic light to pass through the filter element; The monochromatic light transmitted through the filter element is received by an image sensor, and a second imaging is performed on the image sensor to form an element image.

Images