A long-distance passive three-dimensional imaging system

Abstract

This invention discloses a long-distance passive 3D imaging system, belonging to the field of applied optics technology, including a lens, a magnification system, a polarizer, a light field modulation system, and an image processor. Based on the principle of frequency-domain light field modulation, this invention obtains a series of projected images by loading a varying modulation pattern onto the spatial frequency spectrum. The longitudinal distance of the target object is determined based on the lateral position of the projection. A reconstruction algorithm is used to reconstruct these projected images, thereby obtaining the 3D information of the original object. This invention achieves long-distance passive 3D imaging and obtains high longitudinal resolution without sacrificing lateral resolution. This invention solves the problem that multiplexing imaging techniques sacrifice original lateral resolution while achieving longitudinal resolution. The imaging method of this invention uses a camera with a long working distance and a short shooting time, achieving high longitudinal resolution without reducing lateral resolution.

Description

Technical Field

[0001] The invention belongs to the field of applied optical technology, and specifically relates to a long-distance passive three-dimensional imaging system. Background Art

[0002] Machine vision technology allows machines to receive target scene information to replace human vision. It is widely used in target recognition, terrain remote sensing detection, digital city construction, unmanned driving, industrial manufacturing, life medicine and other fields. Machine vision technology can be divided into two types: two-dimensional imaging and three-dimensional imaging. Two-dimensional imaging technology reflects the intensity distribution of the target light field through the grayscale distribution of pixels in the image. However, due to the lack of depth information, two-dimensional imaging technology cannot obtain the three-dimensional information of the target in one imaging process. Three-dimensional imaging technology realized by technology or algorithm can achieve rapid three-dimensional positioning of the target and has great application prospects. At present, three-dimensional imaging technology can be mainly divided into three types: (1) Time of flight (TOF). This technology is an active detection three-dimensional imaging technology that uses the time difference of light propagation to record the depth information of the target. This technology can quickly and accurately measure the longitudinal position information of any target, but the realization of three-dimensional imaging requires multiple measurements, the imaging time is long, and active lighting is required. (2) Structured light illumination three-dimensional imaging technology. The basic working principle of structured light three-dimensional imaging technology is to illuminate the target object in different ways, record and infer the three-dimensional structure of the target. Structured light technology can achieve fast, high-resolution three-dimensional imaging of the target, but due to the limited propagation distance of the illumination beam of the special structure required for active excitation, the longitudinal working distance of this technology is relatively small. (3) Light field imaging technology. This technology is based on the full light function, calculates the trajectory of light propagation, and infers the three-dimensional structure of the object. This technology can be achieved through continuous camera or multiplexed imaging technology. However, the image acquisition time of continuous camera technology is too long, and the multiplexed imaging technology sacrifices the original lateral resolution while obtaining the longitudinal resolution. (4) Holographic imaging technology. This technology is a technology that achieves three-dimensional imaging by recording the angle information of the object through light beam interference. However, this technology still requires active illumination of the target object, which limits the working distance of the holographic technology. Summary of the invention

[0003] In view of the above shortcomings, the present invention provides a long-distance passive three-dimensional imaging system, which is based on the principle of frequency-domain light field modulation, combines the self-bending propagation characteristics of light beams and the non-diffraction characteristics, and adopts a projection image reconstruction algorithm to achieve long-distance passive three-dimensional imaging, and obtain high longitudinal resolution without reducing the lateral resolution. The technical solution adopted by the present invention is as follows: a long-distance passive three-dimensional imaging system, including a lens, an amplification system, a polarizer, a light field control system and an image processor. After the light field information of the long-distance target is collected by the lens, it is input into the amplification system. The amplification system is composed of a group of lenses, which are used to adjust the magnification of the imaging system and control the longitudinal resolution of the three-dimensional imaging system. The light field control system includes two lenses, a spatial light modulator and a camera. The light field output by the amplification system passes through the polarizer, lens L1, spatial light modulator, lens L2 in sequence to the camera. After the imaging result is recorded by the camera, the target is reconstructed in three dimensions by using an algorithm through the image processor, wherein the spatial light modulator is located between lens L1 and lens L2, the distance between the spatial light modulator and lens L1 is the focal length f1 of lens L1, and the distance between the spatial light modulator and lens L2 is the focal length f2 of lens L2;

[0004] The modulation pattern on the spatial light modulator needs to be processed to generate a modulation pattern M(f x ,f y ), the formula is as follows:

[0005] M(f x ,f y )=exp{iβ[(f x / f x0 ) 3 +(f y / f y0 ) 3 ]} (1)

[0006] In formula (1), f x ,f y is the spatial frequency coordinate, f x =X / (λf1), f y =Y / (λf1), X, Y are the plane coordinates of the image formed by the magnification system, λ is the wavelength of the incident light, β is the modulation coefficient, and the value is in the interval (0, 1) according to the imaging depth; f x0 、f y0 is the spatial frequency scaling constant, which is used to control the size of the modulation pattern. After modulation, the target light source can generate an Airy beam with non-diffraction and self-acceleration characteristics. x -f y|>C modulation area to suppress the side lobe of the Airy beam and avoid crosstalk caused by the side lobe artifact of the beam during imaging, wherein C is a cutoff frequency constant, and C is taken as 1 / 6 of the spot diameter of the incident light on the spatial light modulator. If C is too large, the Airy light will have obvious side lobes, affecting the imaging effect. If C is too small, the intensity of the Airy light will be weakened, and the propagation distance without diffraction will be reduced, and then a processed modulation pattern is obtained, and then the processed modulation pattern is rotated at a certain interval angle for one week to obtain multiple side views of the target light field rotated at a certain interval angle, which are used to perform three-dimensional reconstruction of the target;

[0007] The image processor needs to reconstruct the data of the target light field side view obtained after the modulation pattern is rotated at different angles, so as to obtain a three-dimensional image of the original object, and the reconstruction steps are as follows:

[0008] (1) Conjugate gradient method to solve two-dimensional linear equations: First, the two-dimensional light field is discretized, and a separate subscript m is used to mark the light field coordinates at different positions (m = 1, 2, 3, 4, 5...), and a separate subscript n is used to represent the recording matrix of different angles and different pixels of the camera (n = 1, 2, 3, 4, 5...). The projection matrix of the target light field is as follows:

[0009] GL=P (2)

[0010] Where L is the image light field matrix of the target, L = [l1,...,l m ] T ; P is the projection matrix recorded by the camera, P = [p1, ..., p n ] T ; G is the coefficient matrix of the equation group, describing the influence of the light field at different positions on the recorded projection matrix;

[0011] The back-projected reconstructed image can be written as:

[0012] L=G -1 P (3)

[0013] To solve the equation using the conjugate gradient method, we first need to ensure that the coefficient matrix of the equation system is at least a symmetric semi-positive definite matrix, and then multiply both sides of equation (3) by G T get:

[0014] G T GL=G T P (4)

[0015] Let H = G T G, Q = G T P, then:

[0016] HL=Q (5)

[0017] Assume the initial iteration coordinate matrix L(0) =0, the initial residual vector R0 and the initial direction V0 are:

[0018] R0=I0=Q-HL (0) (6)

[0019] For the i-th iteration (i=1,2,3,4,5...), the step size γ i for:

[0020]

[0021] The approximate solution of the equation system is:

[0022]

[0023] Residual vector R i for:

[0024] R i =Q-γ i L (i) (9)

[0025] Calculate the directional constant σ i Make the two consecutive forward directions V i-1 、V i Mutually conjugated:

[0026]

[0027] Conjugate direction V i can be expressed as:

[0028] V i =R i +σ i V i-1 (11)

[0029] After multiple iterations, the reconstructed light field matrix L' is obtained;

[0030] (2) The three-dimensional coordinates (x T ,y T 、z T ) is corrected to obtain the real coordinate distribution (x, y, z) of the object space:

[0031]

[0032] Where x0 is the lateral scaling factor.

[0033] Furthermore, the light field control system is a reflective light field control system or a transmissive light field control system.

[0034] Furthermore, the rotation interval angle is less than or equal to 60° and can be divided by 360°, that is, at least 6 target light field side views are required to reconstruct the three-dimensional data.

[0035] Furthermore, in order to make the longitudinal resolution of the three-dimensional imaging system reach 0.5m near a target of 10m, a magnification system is used to magnify the image behind the lens by 5 times, and the focal length of lens L1 is f1=40cm and the focal length of lens L2 is f2=10cm.

[0036] Advantages and beneficial effects of the present invention: The camera of the imaging method of the present invention has a long working distance, a short shooting time, and can obtain a high longitudinal resolution while the lateral resolution is not reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a diagram of the optical path structure of reflective imaging;

[0038] Figure 2 This is a diagram of the optical path structure of transmission imaging;

[0039] Figure 3 Modulation patterns, (a) Airy beam modulation pattern; (b) modulation pattern with sidelobe suppression; (c) modulation pattern rotated 30°;

[0040] Figure 4 For the reconstructed images of multiple LEDs, (a) three views of the reconstructed multiple LEDs; (b) summation of the x and z axes of the reconstructed matrix; (c) summation of the x and y axes of the reconstructed matrix;

[0041] Figure 5 (a) The three-dimensional images of two street lamps at u=90 and 67m; (b) The three-dimensional images of the two street lamps taken at the same position; (c) The summation and fitting results of the xy axes of the reconstructed three-dimensional matrix.

[0042] Among them, 1. lens, 2. magnification system, 3. polarizer, 4. lens L1, 5. spatial light modulator, 6. lens L2, 7. camera, 8. target. DETAILED DESCRIPTION

[0043] The present invention is further described below with reference to the accompanying drawings:

[0044] Example 1

[0045] 1. Optical path structure

[0046] The optical path of this embodiment can be realized by two methods, corresponding to the reflective imaging optical path structure as follows: Figure 1 The optical path structure of transmission imaging is shown in Figure 2As shown. The working principle of the two types of optical paths is: use a lens or other lens groups with similar effects to collect the light field information of a distant target and then input it into an amplification system. The amplification system is composed of a group of lenses, which are used to adjust the system magnification and control the longitudinal resolution of the system. The light field output by the amplification system is input into the light field control system after passing through a polarizer. The light field control system includes two lenses L1 and L2 with focal lengths f1 and f2, respectively, a spatial light modulator and a camera, wherein the spatial light modulator is located between lens L1 and lens L2, the distance between the spatial light modulator and lens L1 is the focal length f1 of lens L1, and the distance between the spatial light modulator and lens L2 is the focal length f2 of lens L2. The size of the modulation area on the spatial light modulator is adjusted by lens L1. When f1 is larger, the modulation area is larger and the modulation accuracy is higher, but the size of the entire system is elongated. The deflection displacement size and change rate of the Airy light are changed by lens L2. When f2 is larger, the prepared Airy light has a large deflection displacement and an obvious phenomenon, but the self-acceleration rate is slow and the longitudinal resolution is low. In order to make the longitudinal resolution of the three-dimensional imaging system reach 0.5m near the target of 10m, the image behind the lens is magnified 5 times by a magnification system, and the focal length of lens L1 is f1=40cm and the focal length of lens L2 is f2=10cm. After the imaging result is recorded by the camera, the algorithm is used to reconstruct the target in three dimensions. The difference between the two types of systems is that the spatial light modulators used are reflective modulation and transmissive modulation respectively.

[0047] 2. Light field modulation pattern

[0048] In order to achieve the ultimate goal of three-dimensional imaging, this embodiment needs to process the modulation pattern on the spatial light modulator to generate a modulation pattern M (f x ,f y ), the formula is as follows:

[0049] M(f x ,f y )=exp{iβ[(f x / f x0 ) 3 +(f y / f y0 ) 3 ]} (13)

[0050] In formula (1), f x ,f y is the spatial frequency coordinate, f x =X / (λf1), f y =Y / (λf1), X, Y are the plane coordinates of the image formed by the magnification system, λ is the wavelength of the incident light, β is the modulation coefficient, and the value is in the interval (0, 1) according to the imaging depth; f x0 、f y0is the spatial frequency scaling constant, which is used to control the size of the modulation pattern. Figure 3 a. After modulation, the target light source can generate an Airy beam with non-diffraction and self-acceleration characteristics. x -f y |>C modulation area, C is the cutoff frequency constant, C is 1 / 6 of the spot diameter of the incident light on the spatial light modulator. If C is too large, the Airy light will have obvious side lobes, affecting the imaging effect. If C is too small, the intensity of the Airy light will be weakened, and the propagation distance without diffraction will be reduced. Removing some areas of the modulation pattern, such as Figure 3 As shown in b, the side lobes of the Airy beam are suppressed to avoid crosstalk caused by the beam side lobe artifacts during imaging. Then the processed modulation pattern is obtained, and the modulation pattern is rotated for one week at 60° intervals to obtain 6 side views of the target light field with different rotation angles for 3D reconstruction. At this time, the shooting time is the shortest.

[0051] The accuracy of three-dimensional imaging can also be improved by reducing the rotation angle. In order to obtain a higher-precision image while ensuring a shorter shooting time, the rotation angle is set to 30°. Figure 3 c is a modulation pattern rotated by 30°. In this case, the images used for reconstruction are 12 side views of the target light field at different rotation angles.

[0052] 3. Image reconstruction algorithm

[0053] In order to obtain an ideal three-dimensional reproduced image in this embodiment, the image processor needs to reconstruct data of the target light field side view obtained after the modulation pattern is rotated at different angles, so as to obtain a three-dimensional image of the original object. The reconstruction steps are as follows:

[0054] (1) Conjugate gradient method to solve two-dimensional linear equations. First, discretize the two-dimensional light field, use a separate subscript m to mark the light field coordinates at different positions (m = 1, 2, 3, 4, 5...), use a separate subscript n to represent the recording matrix of different angles and different pixels of the camera (n = 1, 2, 3, 4, 5...), and the projection matrix of the target light field is as follows:

[0055] GL=P (14)

[0056] Where L is the image light field matrix of the target, L = [l1,...,l m ] T ; P is the projection matrix recorded by the camera, P = [p1, ..., p n ] T ; G is the coefficient matrix of the equation group, which describes the influence of light fields at different positions on the recorded projection matrix.

[0057] The back-projected image can be written as:

[0058] L=G -1 P (15)

[0059] In the actual reconstruction process, G is often irreversible, and the equations can only be solved iteratively. The conjugate gradient method can be used to solve the equations, which can speed up the solution of the equations. The conjugate gradient method first needs to ensure that the coefficient matrix of the equations is at least a symmetric semi-positive definite matrix, and at the same time, multiply both sides of the equation (3) by G. T get:

[0060] G T GL=G T P (16)

[0061] Let H = G T G, Q = G T P, then:

[0062] HL=Q (17)

[0063] Assume the initial iteration coordinate matrix L (0) =0, the initial residual vector R0 and the initial direction V0 are:

[0064] R0=I0=Q-HL (0) (18)

[0065] For the i-th iteration (i=1,2,3,4,5...), the step size γ i for:

[0066]

[0067] The approximate solution of the equation system is:

[0068]

[0069] Residual vector R i for:

[0070] R i =Q-γ i L (i) (twenty one)

[0071] Calculate the directional constant σ i Make the two consecutive forward directions V i-1 、V i Mutually conjugated:

[0072]

[0073] Conjugate direction V i can be expressed as:

[0074] V i =R i +σi V i-1 (twenty three)

[0075] Multiple iterations can result in the reconstructed light field matrix L'.

[0076] The conjugate gradient method is simple to calculate and does not require solving eigenvalues. It also has a small amount of calculation and only requires one matrix multiplication (HV i-1 ), which can reduce the amount of calculation during reconstruction and save calculation time. The conjugate gradient method takes up little storage space during calculation. Each iteration only needs to record the approximate solution, residual vector, conjugate direction, HV i-1 Four sets of data have low computer memory requirements during reconstruction. Using the conjugate gradient method to reconstruct images can reduce the hardware requirements for reconstruction.

[0077] (2) The three-dimensional coordinates (x T ,y T 、z T ) is corrected to obtain the real coordinate distribution (x, y, z) of the object space:

[0078]

[0079] Where x0 is the lateral scaling factor.

[0080] Example 2

[0081] The experimental results obtained by applying the system provided in Example 1 are as follows:

[0082] The three-dimensional imaging system of Example 1 is used to image and reconstruct the green light band LED near the object distance of 10m. The optical path used is as follows Figure 1 , the spatial frequency scaling constant of the cubic phase is 0.296mm -1 , the cut-off frequency of the bandpass filter is 0.48mm -1 The modulation phase rotation angle interval corresponding to the recorded projections at different angles is θ = 30°. In order to reduce the interference of the zero-order light spot, the camera is located at Z = 20cm to make the zero-order Gaussian light spot diverge. The LED is reconstructed in three dimensions using projections at 12 different angles. The projection angle is set. According to the above image reconstruction algorithm, the reconstructed image is obtained as follows: Figure 4 As shown in a. Figure 4 b and c are the two-dimensional summation results of the x-axis and the x-axis and the y-axis of the three-dimensional LED matrix reconstructed in the experiment. It can be concluded that this three-dimensional imaging system can image multiple targets at the same time, and the three-dimensional reconstruction results can simultaneously distinguish the front and back positions of LEDs at different distances.

[0083] The three-dimensional imaging system is used to simultaneously image and reconstruct two white light LED street lamps. The object distances u of the two street lamps are approximately 67m and 90m respectively. The reconstructed images are as follows: Figure 5 ,The highlighted area is the 3D structure of the two street lamps reconstructed by tomography, Figure 5 (a) is a photo of the street lamp under test taken by an ordinary camera. The result and the Gaussian fitting result for the summation result are shown in Figure 5 (c). The Gaussian peak values ​​of the 3D reconstructed images of the two street lamps at object distances u = 90m and 67m are 1.04 and 0.51 respectively. The values ​​of the two fitting curves at the center of the other fitting curve are 0.02 and 0.01 respectively, which are much smaller than the peak values. According to the judgment conditions of the Rayleigh criterion, the longitudinal positions of the two street lamps in the 3D reconstruction can be distinguished. It can be concluded that the 3D imaging system can simultaneously image two real objects within 100m. The 3D reconstruction results can distinguish their front and back positions.

[0084] The present invention is based on the basic idea of ​​frequency domain light field control. By loading a changing modulation pattern on the spatial spectrum plane, a series of projection images are obtained, and the longitudinal distance of the target object is resolved according to the lateral position of the projection; these projection images are reconstructed using a reconstruction algorithm to obtain the three-dimensional information of the original object. The present invention is suitable for long-distance target recognition, astronomy, remote sensing and other fields. In the imaging results of the present invention, the longitudinal resolution of the object space changes with the object distance, and the longitudinal resolution of the same object distance can be achieved by changing the design parameters of the light field modulation pattern, the focal length of the lens, etc.

Claims

1. A long-distance passive three-dimensional imaging system, comprising a lens, an amplification system, a polarizer, a light field control system and an image processor. After the light field information of a long-distance target is collected by the lens, it is input into the amplification system. The amplification system is composed of a group of lenses, which are used to adjust the magnification of the imaging system and control the longitudinal resolution of the three-dimensional imaging system. The light field control system comprises two lenses, a spatial light modulator and a camera. The light field output by the amplification system passes through the polarizer, lens L1, spatial light modulator, lens L2 in sequence to the camera. After the imaging result is recorded by the camera, the target is reconstructed in three dimensions by the image processor using an algorithm, wherein: The spatial light modulator is located between lens L1 and lens L2, the distance between the spatial light modulator and lens L1 is the focal length f1 of lens L1, and the distance between the spatial light modulator and lens L2 is the focal length f2 of lens L2; The modulation pattern on the spatial light modulator needs to be processed to generate a modulation pattern M (f x ,f y ), the formula is as follows: M(f x ,f y )=exp{iβ[(f x / f x0 ) 3 +(f y / f y0 ) 3 ]} (1) In formula (1), f x ,f y is the spatial frequency coordinate, f x =X / (λf1), f y =Y / (λf1), X, Y are the plane coordinates of the image formed by the magnification system, λ is the wavelength of the incident light, β is the modulation coefficient, and the value is in the interval (0, 1) according to the imaging depth; f x0 、f y0 is the spatial frequency scaling constant, used to control the size of the modulation pattern; Remove the modulation pattern |f x -f y |>C modulation area, C is 1 / 6 of the spot diameter of the incident light on the spatial light modulator, and then a processed modulation pattern is obtained, and then the processed modulation pattern is rotated at a certain interval angle for one circle to obtain multiple side views of the target light field rotated at a certain interval angle, which are used for three-dimensional reconstruction of the target; The image processor needs to reconstruct the data of the target light field side view obtained after the modulation pattern is rotated at different angles, so as to obtain a three-dimensional image of the original object, and the reconstruction steps are as follows: (1) Conjugate gradient method to solve two-dimensional linear equations: First, discretize the two-dimensional light field, use a separate subscript m to mark the light field coordinates at different positions, and use a separate subscript n to represent the recording matrix of different angles and different pixels of the camera. The projection matrix of the target light field is as follows: GL=P (2) Where L is the image light field matrix of the target, L = [l1,...,l m ] T ; P is the projection matrix recorded by the camera, P = [p1, ..., p n ] T ; G is the coefficient matrix of the equation group, describing the influence of the light field at different positions on the recorded projection matrix; The back-projected reconstructed image can be written as: L=G -1 P (3) To solve the equation using the conjugate gradient method, we first need to ensure that the coefficient matrix of the equation system is at least a symmetric semi-positive definite matrix, and then multiply both sides of equation (3) by G T get: G T GL=G T P (4) Let H = G T G, Q = G T P, then: HL=Q (5) Assume the initial iteration coordinate matrix L (0) =0, the initial residual vector R0 and the initial direction V0 are: R0=I0=Q-HL (0) (6) For the i-th iteration, the step size γ i for: The approximate solution of the equation system is: Residual vector R i for: R i =Q-γ i L (i) (9) Calculate the directional constant σ i Make the two consecutive forward directions V i-1 、V i Mutually conjugated: Conjugate direction V i can be expressed as: V i =R i +σ i V i-1 (11) After multiple iterations, the reconstructed light field matrix L' is obtained; (2) The three-dimensional coordinates (x T ,y T 、z T ) is corrected to obtain the real coordinate distribution (x, y, z) of the object space: Where x0 is the lateral scaling factor.

2. A long-distance passive three-dimensional imaging system according to claim 1, characterized in that: The light field control system is a reflective light field control system or a transmissive light field control system.

3. The long-distance passive three-dimensional imaging system according to claim 1, characterized in that: The rotation interval angle is less than or equal to 60° and can be divided by 360°.

4. A long-distance passive three-dimensional imaging system according to any one of claims 1 to 3, characterized in that: In order to make the longitudinal resolution of the three-dimensional imaging system reach 0.5m near a target of 10m, a magnification system is used to magnify the image behind the lens by 5 times, and the focal length of lens L1 is f1=40cm and the focal length of lens L2 is f2=10cm.

Images