Improved cage type light field imaging system and debugging method

Abstract

The application relates to an improved cage type light field imaging system and a debugging method, and belongs to the technical field of light field imaging. The system comprises a main lens for collecting light spots on an object plane and imaging light emitted by the light spots onto a microlens; the microlens is used for separating the light collected by the main lens into light beams in different directions and imaging the light beams onto a virtual image plane; a relay lens is used for twice imaging the light beams in different directions onto different pixels of a CCD detector of a camera; the relay lens comprises a first lens and a second lens, the focal lengths of the first lens and the second lens are both positive, the focal length of the first lens is smaller than that of the second lens, and the magnification of the relay lens is greater than 1:1. During the debugging process of the system, parallel light beams are arranged in front of the microlens, and the parallel light beams are vertically injected into the microlens to determine the position of the microlens. The system is not affected by the vignetting and distortion generated by the relay lens, and the debugging method improves the positioning accuracy of the microlens.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of light field imaging, and particularly relates to an improved cage type light field imaging system and a debugging method. BACKGROUND

[0002] Light field imaging technology is a new technology emerging in recent years, which is different from the traditional imaging mode of "what you see is what you get". A light field imaging system collects the light field of a three-dimensional scene in a material space, and then presents the optical information of the scene in the three-dimensional space by using mathematical methods or data processing methods. The light field collection device is typically a light field camera based on a microlens array. Compared with a traditional camera, the light field camera installs a microlens array at a certain distance (usually several hundred microns to several millimeters) in front of the CCD detector of the traditional camera. The microlens array is arranged in a rectangular or honeycomb arrangement by a series of small lenses, each of which has a size of several tens of microns to several hundred microns and a focal length of several hundred microns to several millimeters. The microlens array can subdivide and image the light collected by the main lens to different pixels on the CCD detector. Compared with a single traditional camera, a single light field camera can record the intensity, direction and position information of the light field in a three-dimensional space at one time. The light field camera can realize the function of focusing after shooting, and calculate the focused image of the three-dimensional scene at an arbitrary depth position by using digital refocusing technology. Different perspective images of the three-dimensional scene can also be extracted by using synthetic aperture technology. However, the traditional camera can only focus to one depth plane of the three-dimensional scene at one time, and can only shoot one perspective image of the three-dimensional scene.

[0003] Currently, there are three types of light field cameras based on microlens array, which are commercial light field cameras, self-assembled light field cameras and cage light field cameras. Commercial light field cameras include Lytro, Lytro Illum2 and Raytrix series light field cameras. In Lytro light field camera, the microlens array is placed at one focal length of the microlens array in front of the CCD detector, and the direction information of the light field in three-dimensional space is obtained by sacrificing the spatial resolution of the CCD detector. Therefore, the position resolution of the Lytro light field camera is relatively low. In order to solve this problem, Lytro Illum2 light field camera is proposed to improve the position resolution of the light field camera. Then Raytrix launched Raytrix series light field cameras, and the typical representative camera is R29. The microlens array in R29 is composed of three microlenses with different focal lengths, which can further improve the depth resolution of the light field camera. However, the optical parameters (internal and external parameters) of these commercial light field cameras are confidential. Therefore, before using these commercial light field cameras for flow field and combustion diagnosis, the optical parameters of the light field camera need to be calibrated to obtain its internal and external parameters. However, due to the introduction of the microlens array, it is difficult to obtain high-precision optical parameters for these commercial light field cameras. This hinders the secondary development of commercial light field cameras, which makes it complex to obtain high-precision experimental results. In addition, the microlens array in Lytro, Lytro Illum and Raytrix series light field cameras is fixed inside the camera, and the microlens array cannot be easily disassembled or replaced. Therefore, commercial light field cameras sometimes cannot meet the requirements of high-precision, high-resolution three-dimensional reconstruction in some scenarios. In order to obtain high-precision, high-resolution reconstruction of three-dimensional scenes, the trade-off between position resolution and direction resolution needs to be considered, so as to optimize the optical parameters of the light field camera. Therefore, the microlens array inside the light field camera needs to be replaced according to different measurement objects.

[0004] Self-assembled light field cameras are a modification of traditional cameras, which tightly attach the microlens array in front of the CCD detector of the traditional camera. However, this assembly method usually needs to remove the protective glass cover in front of the CCD detector, and the CCD detector and the protective glass are filled with gas to protect the CCD detector. Once the protective glass is removed, the CCD detector may be damaged or the life of the CCD detector may be affected. Therefore, this light field camera assembly method has the disadvantages of high price and easy damage to the CCD detector.

[0005] Cage-type light field cameras are an assembly technology developed in recent years. They use cage rods and plates to fix a microlens array at a specific distance in front of a relay mirror. The relay mirror then projects the image below the microlens onto a CCD detector. Cage-type light field cameras have the advantages of low cost and no damage to the CCD detector. They also allow for rapid assembly and replacement of the microlens array and lenses depending on the measurement object. Currently, in existing cage-type light field cameras, the relay mirror consists of two lenses with identical optical parameters, a fixed magnification ratio of 1:1, and is mounted on the camera. The microlens array is mounted in front of the relay mirror, and another lens is mounted in front of the microlens array. The disadvantage of this assembly method is severe vignetting and distortion at the edges of the acquired light field image, resulting in a small usable light field image area. Excluding the light field image area affected by vignetting and distortion, the usable light field image area acquired by current cage-type light field cameras for digital refocusing, synthetic aperture technology, and 3D reconstruction is typically only about 7.5mm × 7.5mm. Furthermore, existing techniques for positioning microlens arrays rely on camera observation of the clear location of dust particles on the microlens array surface, followed by moving the array to a position equivalent to one focal length. A drawback of this method is its inability to determine whether the dust is on the front or rear surface of the microlens, leading to significant assembly errors due to the thickness of the microlens array, such as 1 mm. The vignetting effect is caused by the excessively high F-number (the ratio of the lens's focal length to its pupil diameter) of the relay lens used. Summary of the Invention

[0006] The purpose of this invention is to provide an improved cage-type light field imaging system and debugging method to solve problems such as severe vignetting and distortion at the edges of light field images acquired by traditional cage-type light field cameras, and large assembly errors that can occur during microlens debugging.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] This invention relates to an improved cage-type light field imaging system, comprising:

[0009] The main lens is used to capture light spots on the object surface and image the light emitted by the light spots onto the microlens.

[0010] Microlenses are used to separate the light collected by the main lens into light beams in different directions and image them onto a virtual image plane;

[0011] A relay mirror is used to re-image light beams from different directions onto different pixels of a camera's CCD detector.

[0012] The relay lens includes a first lens and a second lens. Both the first and second lenses have positive focal lengths, and the focal length of the first lens is less than that of the second lens. Both the first and second lenses are focused to infinity, and the F-numbers are set to the minimum, so that the magnification of the relay lens is greater than 1:1.

[0013] Preferably, the magnification ratio of the relay mirror is 2.1:1.

[0014] Preferably, the inverse magnification of the main lens is expressed as:

[0015] ,

[0016] in, M mainlens The reverse magnification of the main lens, l 1 represents the distance from the object surface to the main lens. l 2 is the distance from the main lens to the microlens;

[0017] The range of the inverse magnification of the main lens is 0.5:1 to ∞:1.

[0018] Preferably, the magnification of the relay mirror is expressed as:

[0019] ,

[0020] in, M relay The magnification of the relay lens. l 3 represents the distance from the virtual image plane to the first camera. l 4 represents the distance from the second lens to the CCD detector.

[0021] The present invention also relates to a debugging method for the above-mentioned improved cage-type light field imaging system, which includes the following steps:

[0022] S1. Connect the repeater to the camera and place a microlens in front of the repeater;

[0023] S2. A parallel beam is placed in front of the microlens, and the parallel beam enters the microlens perpendicularly;

[0024] S3. Use a CCD detector to observe the size of the parallel beam, and move the microlens laterally left and right along the direction of the parallel beam until the size of the parallel beam observed by the CCD detector is the smallest, and determine the position of the microlens.

[0025] S4. Remove the parallel beam, install the main lens in front of the microlens, so that the microlens is one flange distance behind the main lens, cover the main lens with a piece of white paper, illuminate the white paper perpendicularly with white light, and adjust the aperture of the main lens so that each sub-image in the white image captured by the CCD detector is tangent.

[0026] Preferably, in step S3, before the parallel beam enters the microlens perpendicularly, two positive lenses are used to magnify the parallel beam so that the cross-section of the parallel beam is large enough to fully illuminate the microlens.

[0027] Compared with the prior art, the technical solution provided by this invention has the following advantages:

[0028] 1. The relay mirror of the improved cage-type light field imaging system of this invention includes a first lens and a second lens. Both the first and second lenses have positive focal lengths, with the first lens having a shorter focal length than the second lens. This allows the relay mirror to achieve a magnification ratio greater than 1:1, enabling magnification of the image below the microlens. The image formed by the microlens is then transferred to a CCD detector. Unlike traditional relay mirror technology using a dual-telecentric Hartmann sensor with one positive and one negative lens, these two lenses are equivalent to thin positive lenses, both focused to infinity with their F-number set to the minimum, maximizing the lens aperture. Under this magnification, the white image captured by the light field imaging system exhibits a more uniform pixel grayscale distribution, and all sub-images within the white image are nearly circular in shape. This indicates that the white image is no longer affected by vignetting and distortion caused by the relay mirror, which is beneficial for extracting the center of the sub-images. The area of ​​the light field image that can be used for secondary development technologies such as digital refocusing, synthetic aperture technology, and 3D reconstruction is increased.

[0029] 2. The debugging method of the improved cage-type light field imaging system involved in this invention first sets a parallel beam in front of the microlens; then, the size of the parallel beam is observed with a CCD detector, and the microlens is moved laterally left and right along the direction of the parallel beam until the size of the parallel beam observed by the CCD detector is minimized, thus determining the position of the microlens; finally, the parallel beam is removed, and a main lens is installed in front of the microlens, so that the microlens is at one flange distance behind the main lens. A piece of white paper is covered in front of the main lens, and white light is used to illuminate the white paper perpendicularly. The aperture of the main lens is adjusted so that each sub-image in the white image captured by the CCD detector is tangent. At this time, the light collected by the microlens array is exactly imaged at one focal length behind the microlens. This microlens positioning method avoids the problem of whether the dust is located on the front or back surface of the microlens, improves the positioning accuracy of the microlens, and has small assembly errors. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the improved cage-type light field imaging system involved in this invention;

[0031] Figure 2 This is a schematic diagram of the imaging principle of a relay mirror;

[0032] Figure 3 This is a schematic diagram of the microlens adjustment process;

[0033] Figure 4 This is a schematic diagram of microlens imaging during the debugging process;

[0034] Figure 5 A schematic diagram of a white image for an improved cage-type light field camera;

[0035] Figure 6(a) is M mainlens =1:1, M relay A schematic diagram of the pixel grayscale distribution of the white image when the grayscale ratio is 2.1:1;

[0036] Figure 6(b) is M mainlens =2.1:1, M relay A schematic diagram of the pixel grayscale distribution of the white image when the grayscale ratio is 2.1:1;

[0037] Figure 7(a) is M mainlens =1:1, M relay The usable light field image area when the ratio is 2.1:1;

[0038] Figure 7(b) is M mainlens =2.1:1, M relay The usable light field image area when the ratio is 2.1:1;

[0039] Figure labels: 1-Main lens, 2-Microlens, 3-Relay lens, 31-First lens, 32-Second lens, 4-Camera, 41-CCD detector. Detailed Implementation

[0040] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments. The following embodiments are used to illustrate the invention, but are not intended to limit the scope of the invention.

[0041] See attached document Figure 1 As shown, this invention relates to an improved cage-type light field imaging system, comprising a main lens 1, a microlens 2, and a relay mirror 3. One end of the relay mirror 3 is connected to a conventional camera 4, and a microlens 2 is installed at a certain distance in front of the other end. A main lens 1 is installed at the front end of the microlens 2. The relay mirror 3 includes a first lens 31 and a second lens 32. Both the first lens 31 and the second lens 32 have positive focal lengths, and the focal length of the first lens 31 is less than that of the second lens 32, resulting in a magnification ratio greater than 1:1 for the relay mirror.

[0042] The imaging principle of the improved cage-type light field imaging system described above is as follows: the main lens 1 is used to collect light spots on the object surface and image the light emitted from the light spots onto a microlens; the microlens 2 is used to separate the light collected by the main lens into light beams in different directions and image them onto a virtual image plane; the relay lens 3 is used to re-image the light beams in different directions onto different pixels of the camera's CCD detector. The first lens 31 and the second lens 32 can be equivalent to two thin positive lenses, and their imaging principle is as follows: Figure 2 As shown, light emitted from a point at a focal length in front of the first lens 31 passes through the first lens 31 and is incident parallel to the second lens 32. The parallel light then converges at a focal length behind the second lens 32.

[0043] The inverse magnification of the main lens is expressed as:

[0044] ,

[0045] in, M mainlens The reverse magnification of the main lens, l 1 represents the distance from the object surface to the main lens. l 2 is the distance from the main lens to the microlens;

[0046] The range of the inverse magnification of the main lens is 0.5:1 to ∞:1.

[0047] The magnification ratio of the relay mirror is expressed as: greater than 1:1

[0048] ,

[0049] in, M relay The magnification of the relay lens. l 3 represents the distance from the virtual image plane to the first camera. l 4 represents the distance from the second lens to the CCD detector;

[0050] The magnification ratio of the relay mirror is greater than 1:1.

[0051] In this embodiment, a Nikon lens with a focal length of 50mm and a maximum aperture of F1.2 is used as the first lens 31; a Nikon lens with a focal length of 105mm and a maximum aperture of F2.5 is used as the second lens 32. Both the first lens 31 and the second lens 32 are focused to infinity, and their F-numbers are set to the minimum to maximize the aperture of both lenses. The magnification of the relay lens 3... M relay for f 3: f 2= l 4:l 3 = 2.1:1, this magnification ratio can achieve the effect of magnifying the object.

[0052] The above-mentioned debugging method based on the improved cage-type light field imaging system includes the following steps:

[0053] S1. In a cage-type light field imaging system, firstly... Figure 2 The relay mirror 3 shown is connected to the camera, and the microlens 2 is mounted at a specific distance in front of the first lens 31 of the relay mirror 3 using cage rods and cage plates;

[0054] S2. A parallel beam is placed in front of the microlens 2, and two positive lenses are used to magnify the parallel beam to make it wider, so that the parallel beam is perpendicularly incident into the microlens 2.

[0055] S3. Observe the size of the parallel beam using CCD detector 41. To ensure that the distance between microlens 2 and the virtual image plane is one focal length of microlens 2, based on the principle that parallel light incident on microlens 2 converges at one focal length behind microlens 2, move microlens 2 laterally left and right along the direction of the parallel beam until the size of the parallel beam observed by CCD detector 41 is minimized. Figure 4 As shown, determine the position of microlens 2;

[0056] S4. After the microlens 2 is accurately positioned, remove the parallel beam and the two positive lenses. Install the main lens 1 in front of the microlens 2, positioning the microlens 2 at one flange distance behind the main lens 1. Cover the main lens 1 with a piece of white paper and illuminate it with white light perpendicularly. Adjust the aperture of the main lens 1 so that each sub-image in the white image captured by the CCD detector 41 is tangent. Figure 5 As shown.

[0057] Figures 6(a) and 6(b) are schematic diagrams of the pixel grayscale distribution of white images based on the improved cage light field camera under different inverse magnification of the main lens. As can be seen from the figures, there is no obvious vignetting at the four corners of the white image. The shape of each sub-image in regions (1), (2) and (3) of the white image is circular, and the pixel grayscale distribution of the sub-images in the white image is almost uniform.

[0058] Figures 7(a) and 7(b) show the usable light field image area of ​​the improved cage-type light field camera under different inverse magnification of the main lens. As can be seen from the figures, for... M mainlens =1:1 and M relay = 2.1:1, the usable light field image area captured by the improved cage-type light field camera is approximately 7.5 mm × 11.5 mm. For M mainlens =2:1 andM relay =2.1:1, the usable light field image area taken by the improved cage-type light field camera is approximately 15.5mm × 23.5mm.

[0059] The present invention has been described in detail above with reference to the embodiments, but the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. All equivalent changes and improvements made in accordance with the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. An improved cage-type light field imaging system, characterized in that: It includes: The main lens is used to capture light spots on the object surface and image the light emitted by the light spots onto the microlens. Microlenses are used to separate the light collected by the main lens into light beams in different directions and image them onto a virtual image plane; A relay mirror is used to re-image light beams from different directions onto different pixels of a camera's CCD detector. The relay lens includes a first lens and a second lens. Both the first lens and the second lens have positive focal lengths, and the focal length of the first lens is less than that of the second lens. Both the first lens and the second lens are focused to infinity, and the F-numbers are both set to the minimum, so that the magnification of the relay lens is greater than 1:

1. The light emitted from a point at a focal length in front of the first lens passes through the first lens and is incident parallel to the second lens. The parallel light then converges at a focal length behind the second lens.

2. The improved cage-type light field imaging system according to claim 1, characterized in that: The magnification ratio of the relay mirror is 2.1:

1.

3. The improved cage-type light field imaging system according to claim 1, characterized in that: The inverse magnification of the main lens is expressed as: ， in, M mainlens The reverse magnification of the main lens, l 1 represents the distance from the object surface to the main lens. l 2 is the distance from the main lens to the microlens; The range of the inverse magnification of the main lens is 0.5:1 to ∞:

1.

4. The improved cage-type light field imaging system according to claim 1, characterized in that: The magnification of the relay mirror is expressed as follows: ， in, M relay The magnification of the relay lens. l 3 represents the distance from the virtual image plane to the first camera. l 4 represents the distance from the second lens to the CCD detector.

5. A debugging method for the improved cage-type light field imaging system as described in claim 1, characterized in that, It includes the following steps: S1. Connect the repeater to the camera and place a microlens in front of the repeater; S2. A parallel beam is placed in front of the microlens, and the parallel beam enters the microlens perpendicularly; S3. Use a CCD detector to observe the size of the parallel beam, and move the microlens laterally left and right along the direction of the parallel beam until the size of the parallel beam observed by the CCD detector is the smallest, and determine the position of the microlens. S4. Remove the parallel beam, install the main lens in front of the microlens, so that the microlens is one flange distance behind the main lens, cover the main lens with a piece of white paper, illuminate the white paper perpendicularly with white light, and adjust the aperture of the main lens so that each sub-image in the white image captured by the CCD detector is tangent.

6. The debugging method for the improved cage-type light field imaging system according to claim 5, characterized in that: In S3, the parallel beam is magnified by two positive lenses before it is perpendicularly incident on the microlens.

Images