Amplitude coded modulation-based incoherent synthesis imaging system

Abstract

An incoherent synthesis imaging system based on amplitude coded modulation is composed of an imaging main lens group 1, a programmable LCD panel 2 and a camera 3, after a series of Fresnel ZoneAperture (FZA) amplitude coded images are shot, convex set projection iteration is carried out in a Fourier domain until convergence is achieved, and a super-resolution image can be obtained. Compared with an existing multi-aperture imaging system, the device does not need any mechanical phase scanning device, and is simple in structure, rapid in measurement, easy to operate and capable of achieving stable and accurate measurement. Compared with a coherent synthetic aperture imaging system, the imaging system adopting the programmable LCD panel to carry out incoherent coding does not need active illumination, can directly carry out super-resolution imaging on an object under natural light illumination, and reduces the exposure risk during imaging. According to the imaging system and the super-resolution method thereof, long-distance super-resolution imaging can be successfully realized, and the target imaging resolution can be improved by more than two times.

Description

Technical Field

[0001] The present invention belongs to the field of computational optical imaging technology, and in particular to an imaging system based on incoherent synthesis of amplitude coded modulation. Background Art

[0002] For centuries, the basic structure of the camera has remained unchanged, using a set of lenses to project the observed object "point by point" onto the imaging plane. As early as 1702, in his book "Oculus artificialis", Johann Zahn mentioned such a camera structure: a closed darkroom with a focusing lens on one side and a device for recording images on the other side (Zahn J. Oculus artificialis teledioptricus sivetelescopium [M]. Heyl, 1702.). It seems no coincidence that this design is similar to the structure of the human eye. Over the years, although the technologies used to realize lenses (such as crystal lenses, glass lenses, 3D printing) and image recording devices (such as painters' hand-painted, photosensitive films, complementary metal oxide semiconductors (CMOS), charge-coupled devices (CCD)) have been continuously updated, this basic design structure has never changed. Over the past few centuries, lenses have been widely used, from observing cells with microscopes in laboratories to observing the movement of celestial bodies with astronomical telescopes. They support production and scientific research activities in many fields such as biology, medicine, physics, metallurgy, photography, food and chemical engineering, and have made great contributions to human development.

[0003] However, lenses do have some inherent limitations. First, lenses require that the imaging plane be on the focal plane. Although today's digital sensors can be very thin, the distance limit from the lens to the focal plane means that the entire camera will eventually be very thick, which is very unfavorable for some miniaturized use scenarios. At the same time, in scenarios where the camera has the ability to focus, a precise mechanical structure needs to be introduced to adjust the distance between the lens groups, which increases the complexity of the camera structure and the manufacturing cost. Secondly, for visible light lenses, materials such as glass or plastic can be used to produce them, but when it comes to infrared and ultraviolet lenses in the invisible light range, they become either extremely expensive or have serious aberration problems that make them difficult to use, especially under high-energy rays such as X-rays and gamma rays. Traditional lens imaging methods are simply not feasible. Third, cameras using lenses usually need to be assembled after manufacturing, resulting in low manufacturing efficiency. Nowadays, many emerging technologies such as wearable devices, implantable devices, self-driving cars, robots, the Internet of Things (IoT), virtual reality / augmented reality (Geiger A, Lenz P, Stiller C, et al. Vision meets robotics: The KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.) are promoting the miniaturization and lightweight of lenses, while the heavy traditional cameras can no longer meet such requirements. Therefore, it is necessary to completely redesign the traditional camera structure to meet the needs of miniaturization. Summary of the invention

[0004] The purpose of the present invention is to provide an imaging system based on incoherent synthesis of amplitude coded modulation, which can not only reduce the exposure time required by the camera and increase the image acquisition speed of the system, but also improve the resolution of the imaging system and optimize the image reconstruction quality.

[0005] The technical solution for achieving the purpose of the present invention is: an imaging system based on incoherent synthesis of amplitude coded modulation, comprising:

[0006] An imaging main lens group 1, a programmable LCD panel 2, a camera 3 and a super-resolution module, wherein the imaging main lens group 1, the programmable LCD panel 2 and the camera 3 are respectively arranged on an optical platform, the camera 3 is located on the back focal plane of the imaging main lens group 1, the programmable LCD panel 2 is arranged between the imaging main lens group 1 and the camera 3, the position of the programmable LCD panel 2 relative to the camera 3 remains unchanged, the FZA amplitude coding pattern is displayed on the programmable LCD panel 2, the camera takes four FZA amplitude coding images with different phases for each sub-aperture, and the super-resolution module is used to reconstruct and process the four FZA amplitude coding images with different phases for each sub-aperture to obtain a super-resolution image.

[0007] Preferably, the FZA amplitude coding pattern is specifically:

[0008]

[0009] Where β is the Fresnel parameter, φ F is the initial phase factor of FZA, r is the radial distance of the FZA plane, r k is the offset vector of different sub-apertures, k=1…N, N is the number of sub-apertures.

[0010] Preferably, the specific method for the super-resolution module to reconstruct and process four FZA amplitude coded images of different phases of each sub-aperture to obtain a super-resolution image is:

[0011] Step 1: For each sub-aperture A k Four FZA amplitude coded images taken at different phases I B , B=1,2,3,4 to reconstruct and obtain the intensity value I of N sub-apertures k , k = 1…N;

[0012] Step 2: Set the reconstructed intensity value I of the central sub-aperture o Perform Fourier transform to obtain frequency domain image, and use the frequency domain image as the initial high-resolution spectrum iter represents the number of iterations, iter=0 when initialized;

[0013] Step 3: High-resolution spectrum Select the subaperture A under the support domain k , perform Fourier transform on it to the spatial domain to obtain a low-resolution image The amplitude of the image captured under the corresponding sub-aperture I k Mapping to low resolution image Get the updated low-resolution image

[0014] Step 4: Update the low-resolution image Transform back to frequency domain, replacing the high-resolution spectrum Corresponding subaperture A k The spectrum below;

[0015] Step 5: Repeat steps 1 to 4 until all sub-apertures capture images I k All iterations are completed, which is regarded as one iteration process;

[0016] Step 6: Set iter = iter + 1 and repeat steps 3 to 5 for the next round of iteration until the error converges.

[0017] Preferably, for each sub-aperture A k Four FZA amplitude coded images taken at different phases I B , B=1,2,3,4 for reconstruction, specifically:

[0018] Each subaperture A k Four FZA amplitude coded images taken at different phases I B ,B=1,2,3,4Specifically:

[0019]

[0020] Among them, r k is the offset vector (x k ,y k ), corresponding to the offset of the subaperture relative to the central aperture, β is the Fresnel parameter, φ B is the FZA phase factor;

[0021] Each coded image I B Multiply Then add the results of the four phase shifts to get:

[0022]

[0023] The FZA amplitude modulation is converted into phase modulation by four-step phase shifting, and the following equation is obtained:

[0024] exp[iβr 2 ]*exp[-iβr 2 ]=δ(r)

[0025] Wherein, the “*” symbol represents convolution, r is the radial distance of the FZA plane, and δ(r) is the impulse response function;

[0026] The intensity image I and exp[-iβr 2 ] to perform convolution and obtain the reconstructed image:

[0027]

[0028] Where f(x,y) is the reconstructed image, β is the Fresnel parameter, (x k ,y k ) is the two-dimensional coordinate of the intensity image I, and (x, y) is the two-dimensional coordinate of the reconstructed image f(x, y).

[0029] Preferably, a high-resolution spectrum The update formula is:

[0030]

[0031] Wi =Diag(β i (|P i | 2 +γ) -1 )

[0032]

[0033] Among them, α iter is the step size, i is the sub-aperture number, is the error function ε i For high resolution spectrum Derivative, Ψ i is the subaperture spectrum, γ is a fixed constant, β i is the scaling factor, P i is the pupil function, A i is the subaperture support domain.

[0034] Preferably, the convergence criterion is that all generated target low-resolution images Low-resolution image reconstructed with sub-aperture I k The sum of the mean square errors is less than a threshold T.

[0035] Compared with the prior art, the present invention has the following significant advantages: (1) Compared with the existing multi-aperture imaging system, the present invention does not require any mechanical phase scanning device, has a simple structure, fast measurement, easy operation, and can measure stably and accurately. (2) Compared with the coherent synthetic aperture imaging system, the imaging system using a programmable LCD panel for incoherent encoding does not require active illumination, and can directly perform super-resolution imaging of objects under natural light, reducing the exposure risk during imaging. The imaging system and its super-resolution method can successfully achieve long-distance super-resolution imaging and can increase the target imaging resolution by more than 2 times.

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a ray tracing diagram of the imaging system based on incoherent synthesis of amplitude coded modulation of the present invention.

[0038] Figure 2 (a) is the central aperture FZA amplitude coding pattern (without spatial offset) used in the present invention, Figure 2 (b) is the surrounding sub-aperture FZA amplitude coding pattern (with spatial offset).

[0039] Figure 3 It is a schematic diagram of the sub-aperture reconstruction process of the present invention.

[0040] Figure 4 Schematic diagram of the super-resolution method of the present invention.

[0041] Figure 5 The simulation results of incoherent synthesis reconstruction of the resolution plate are shown in Figure 2. Figure 5 (a) is the low-resolution original image of the central aperture and its spectrum before super-resolution reconstruction. Figure 5 (b) is the low-resolution original image and its spectrum of the surrounding offset aperture before super-resolution reconstruction. Figure 5 (c) is a super-resolution image and spectrum thereof using the imaging system based on incoherent synthesis of amplitude coded modulation of the present invention, Figure 5 (d) is the comparison of point spread function before and after super-resolution reconstruction.

[0042] Figure 6 The experimental results of incoherent synthesis reconstruction of the resolution plate are shown in Figure 2. Figure 6 (a) is the low-resolution original image of the central aperture and its spectrum before super-resolution reconstruction. Figure 6 (b) is the amplitude profile of the resolution plate Group-1 before super-resolution reconstruction. Figure 6 (c) is a super-resolution image and spectrum thereof using the imaging system based on incoherent synthesis of amplitude coded modulation of the present invention, Figure 6 (d) is the amplitude profile of resolution plate Group-1 after super-resolution reconstruction. DETAILED DESCRIPTION

[0043] like Figure 1 As shown, the imaging system of the present invention based on incoherent synthesis of amplitude coded modulation includes:

[0044] An imaging main lens group 1, a programmable LCD panel 2, a camera 3 and a super-resolution module, wherein the imaging main lens group 1, the programmable LCD panel 2 and the camera 3 are respectively arranged on an optical platform, the camera 3 is located on the back focal plane of the imaging main lens group 1, the programmable LCD panel 2 is arranged between the imaging main lens group 1 and the camera 3, the position of the programmable LCD panel 2 relative to the camera 3 remains unchanged, the FZA amplitude coding pattern is displayed on the programmable LCD panel 2, the camera takes four FZA amplitude coding images with different phases for each sub-aperture, and the super-resolution module is used to reconstruct and process the four FZA amplitude coding images with different phases for each sub-aperture to obtain a super-resolution image.

[0045] As an embodiment, a programmable LCD panel 1 is placed in front of a camera 3 for incoherently encoding wavefront information. The camera 3 is located at the rear focal plane of the imaging main lens group 1. The imaging main lens group 1, the programmable LCD panel 2, and the camera 3 are respectively fixedly mounted on an optical platform. When the focal length of the imaging main lens group is adjusted, the position of the programmable LCD panel 2 relative to the camera 3 remains unchanged. The imaging main lens group 1 is a CANON (100-400mm), and the focal length is adjusted to 400mm. The camera 3 has a pixel size of 2.4×2.4μm.

[0046] In a further embodiment, Figure 2 (a) is the central aperture FZA amplitude coding pattern (without spatial offset) used in the present invention, Figure 2 (b) is the FZA amplitude coding pattern of the surrounding sub-aperture (with spatial offset). The FZA amplitude coding pattern used in the present invention has a Fresnel parameter β of 18 rad / mm 2 The radial distance r of FZA amplitude coding is 38.5 mm. The FZA amplitude coding pattern P is generated by computer MATLAB software. k The formula is:

[0047]

[0048] where β is the Fresnel parameter, φ F is the initial phase factor of FZA, r is the radial distance of the FZA plane, r k is the offset vector of different sub-apertures, k=1…N.

[0049] The FZA amplitude code P displayed by the programmable LCD panel 2 in front of the subaperture k Compared with the central aperture, it has a spatial offset characteristic. k , there is no offset, so the minimum spacing is the edge stripe width q, calculated as q = π / 2βR, R is the FZA amplitude code P k The radius of the stripes. The spatial period of the stripes at the edge is about 2q, that is, one black stripe and one white stripe, and the corresponding spatial frequency f = 1 / 2q = βR / π. When the FZA amplitude encoding space is offset by x0, and the position displayed by the programmable LCD panel 2 does not move, the minimum spacing is q = π / 2β(R+x0), and the corresponding frequency is f = β(R+x0) / π. The maximum frequency of the two is offset by βx0 / π. The spectrum after the offset is:

[0050]

[0051] Therefore, when the FZA amplitude encoding spatial domain is shifted by x0, the corresponding reconstructed image is shifted by βx0 / π in the spectrum, while the spectrum width remains unchanged and is always 2βR / π.

[0052] In a further embodiment, the specific method for the super-resolution module to reconstruct and process four FZA amplitude coded images of different phases of each sub-aperture to obtain a super-resolution image is:

[0053] Step 1: For each sub-aperture A k Four FZA amplitude coded images taken at different phases I B , B=1,2,3,4 to reconstruct and obtain the intensity value I of N sub-apertures k , k = 1…N;

[0054] Step 2: Set the reconstructed intensity value I of the central sub-aperture o Perform Fourier transform to obtain frequency domain image, and use the frequency domain image as the initial high-resolution spectrum iter represents the number of iterations, iter=0 when initialized;

[0055] Step 3: High-resolution spectrum Select the subaperture A under the support domain k , perform Fourier transform on it to the spatial domain to obtain a low-resolution image The amplitude of the image captured under the corresponding sub-aperture I k Mapping to low resolution image Get the updated low-resolution image

[0056] Step 4: Update the low-resolution image Transform back to frequency domain, replacing the high-resolution spectrum Corresponding subaperture A k The spectrum below;

[0057] Step 5: Repeat steps 1 to 4 until all sub-apertures capture images I k All iterations are completed, which is regarded as one iteration process;

[0058] Step 6: Set iter = iter + 1 and repeat steps 3 to 5 for the next round of iteration until the error converges.

[0059] In a further embodiment, for each sub-aperture A k Four FZA amplitude coded images taken at different phases I B , B=1,2,3,4 for reconstruction, specifically:

[0060] Each subaperture A k Four FZA amplitude coded images taken at different phases I B ,B=1,2,3,4Specifically:

[0061]

[0062] Among them, r k is the offset vector (x k ,y k ), corresponding to the offset of the subaperture relative to the central aperture, β is the Fresnel parameter, φ B is the FZA phase factor;

[0063] Each coded image I B Multiply Then add the results of the four phase shifts to get:

[0064]

[0065] The FZA amplitude modulation is converted into phase modulation by four-step phase shifting, and the following equation is obtained:

[0066] exp[iβr 2 ]*exp[-iβr 2 ]=δ(r)

[0067] Wherein, the “*” symbol represents convolution, r is the radial distance of the FZA plane, and δ(r) is the impulse response function;

[0068] The intensity image I and exp[-iβr 2 ] to perform convolution and obtain the reconstructed image:

[0069]

[0070] Where f(x,y) is the reconstructed image, β is the Fresnel parameter, (x k ,y k ) is the two-dimensional coordinate of the intensity image I, and (x, y) is the two-dimensional coordinate of the reconstructed image f(x, y).

[0071] In a further embodiment, the high resolution spectrum The update formula is:

[0072]

[0073] Wi=Diag(β i (|P i | 2 +γ) -1 )

[0074]

[0075] Among them, α iter is the step size, i is the sub-aperture number, is the error function εi For high resolution spectrum Derivative, Ψ i is the subaperture spectrum, γ is a fixed constant, β i is the scaling factor, P i is the pupil function, A i is the subaperture support domain.

[0076] Combination Figure 4 , a super-resolution method for an imaging system based on incoherent synthesis of amplitude coded modulation, the steps are as follows:

[0077] 1. Adoption Figure 3 The four-step phase shift method and its reconstruction method in each sub-aperture A k Four FZA amplitude coded images I taken with different phases (k=1…N) B , B=1,2,3,4 to reconstruct and obtain the intensity value I of N sub-apertures k , k=1…N.

[0078] 2. Take the reconstruction intensity value I of the central sub-aperture o , perform Fourier transform to the image frequency domain as the initialization high-resolution spectrum iter represents the number of iterations, and iter=0 during initialization.

[0079] 3. In high-resolution spectrum Select the subaperture A under a specific support domain k (k=1…N) Fourier transform it to the spatial domain to obtain a low-resolution image The amplitude of the image captured under the corresponding sub-aperture I k Mapped to this low-resolution image superior.

[0080] 4. Update the low-resolution image Transform back to frequency domain, replacing the high-resolution spectrum Corresponding subaperture A k The spectrum below

[0081] 5. Repeat the above process until all sub-apertures capture images I k All iterations are completed and regarded as one iteration process.

[0082] 6. Let iter = iter + 1, and repeat steps (3) to (5) for the next iteration until the error converges to a suitable level. The convergence criterion is that all generated target low-resolution images Low-resolution image reconstructed with sub-aperture I k The sum of the mean square errors is less than a threshold T, and the value of T is generally 0.001.

[0083] In order to test the effectiveness of the imaging system based on incoherent synthesis of amplitude coded modulation and the iterative super-resolution method of the present invention, two groups of experiments are selected for illustration.

[0084] Figure 5 The simulation results of incoherent synthesis reconstruction of the resolution plate are shown in Figure 2. Figure 5 (a) is the low-resolution original image of the central aperture and its spectrum before super-resolution reconstruction. Figure 5 (b) is the low-resolution original image and its spectrum of the surrounding offset aperture before super-resolution reconstruction. Figure 5 (c) is a super-resolution image and spectrum thereof using the imaging system based on incoherent synthesis of amplitude coded modulation of the present invention, Figure 5 (d) is the comparison of point spread function before and after super-resolution reconstruction. Figure 6 The experimental results of incoherent synthesis reconstruction of the resolution plate are shown in Figure 2. Figure 6 (a) is the low-resolution original image of the central aperture and its spectrum before super-resolution reconstruction. Figure 6 (b) is the amplitude profile of the resolution plate Group-1 before super-resolution reconstruction. Figure 6 (c) is a super-resolution image and spectrum thereof using the imaging system based on incoherent synthesis of amplitude coded modulation of the present invention, Figure 6 (d) is the amplitude profile of the resolution plate Group-1 after super-resolution reconstruction. It can be seen that the low-resolution original image of the central aperture before super-resolution reconstruction has obvious diffraction limitation problems, such as Figure 5 As shown in (a), the details of the resolution plate logo cannot be discerned, and there is an obvious diffraction limitation problem. Figure 6 In (a), the highest resolvable line pair is Group-1Element 3. The super-resolution reconstruction of the imaging system based on incoherent synthesis of amplitude coded modulation of the present invention is Figure 5 In (c), we can see from the marking line that the super-resolution effect has reached 2 times, which corresponds to the super-resolution reconstruction result of the real scene. Figure 6 In (c), it can be seen that the details of the resolution board are greatly improved, and the highest resolvable line pair is Group 0Element 5, and the super-resolution reconstruction effect is obvious. Figure 5 (c) and Figure 6 The results in (c) show that the imaging system based on incoherent synthesis of amplitude coded modulation and its super-resolution method can achieve a super-resolution effect that is more than twice the diffraction limit resolution.

Claims

1. An imaging system based on incoherent synthesis of amplitude coded modulation, characterized in that: include: An imaging main lens group 1, a programmable LCD panel 2, a camera 3 and a super-resolution module, wherein the imaging main lens group 1, the programmable LCD panel 2 and the camera 3 are respectively arranged on an optical platform, the camera 3 is located on the back focal plane of the imaging main lens group 1, the programmable LCD panel 2 is arranged between the imaging main lens group 1 and the camera 3, the position of the programmable LCD panel 2 relative to the camera 3 remains unchanged, the FZA amplitude coding pattern is displayed on the programmable LCD panel 2, the camera takes four FZA amplitude coding images with different phases for each sub-aperture, and the super-resolution module is used to reconstruct and process the four FZA amplitude coding images with different phases for each sub-aperture to obtain a super-resolution image.

2. The imaging system based on incoherent synthesis of amplitude coded modulation according to claim 1, characterized in that: The FZA amplitude coding pattern is specifically: Where β is the Fresnel parameter, φ F is the initial phase factor of FZA, r is the radial distance of the FZA plane, r k is the offset vector of different sub-apertures, k=1…N, N is the number of sub-apertures.

3. The imaging system based on incoherent synthesis of amplitude coded modulation according to claim 1, characterized in that: The specific method for the super-resolution module to reconstruct and process four FZA amplitude coded images of different phases of each sub-aperture to obtain a super-resolution image is as follows: Step 1: For each sub-aperture A k Four FZA amplitude coded images taken at different phases I B , B=1,2,3,4 to reconstruct and obtain the intensity value I of N sub-apertures k , k = 1…N; Step 2: Set the reconstructed intensity value I of the central sub-aperture o Perform Fourier transform to obtain frequency domain image, and use the frequency domain image as the initial high-resolution spectrum iter represents the number of iterations, iter=0 when initialized; Step 3: High-resolution spectrum Select the subaperture A under the support domain k , perform Fourier transform on it to the spatial domain to obtain a low-resolution image The amplitude of the image captured under the corresponding sub-aperture I k Mapping to low-resolution image Get the updated low-resolution image Step 4: Update the low-resolution image Transform back to frequency domain, replacing the high-resolution spectrum Corresponding subaperture A k The spectrum below; Step 5: Repeat steps 1 to 4 until all sub-apertures capture images I k All iterations are completed, which is regarded as one iteration process; Step 6: Set iter = iter + 1 and repeat steps 3 to 5 for the next round of iteration until the error converges.

4. The imaging system based on incoherent synthesis of amplitude coded modulation according to claim 3, characterized in that: For each subaperture A k Four FZA amplitude coded images taken at different phases I B , B=1,2,3,4 for reconstruction, specifically: Each subaperture A k Four FZA amplitude coded images taken at different phases I B ,B=1,2,3,4Specifically: Among them, r k is the offset vector (x k ,y k ), corresponding to the offset of the subaperture relative to the central aperture, β is the Fresnel parameter, φ B is the FZA phase factor; Each coded image I B Multiply Then add the results of the four phase shifts to get: The FZA amplitude modulation is converted into phase modulation by four-step phase shifting, and the following equation is obtained: exp[iβr 2 ]*exp[-iβr 2 ]=δ(r) Wherein, the "*" symbol represents convolution, r is the radial distance of the FZA plane, and δ(r) is the impulse response function; The intensity image I and exp[-iβr 2 ] to perform convolution and obtain the reconstructed image: Where f(x,y) is the reconstructed image, β is the Fresnel parameter, (x k ,y k ) is the two-dimensional coordinate of the intensity image I, and (x, y) is the two-dimensional coordinate of the reconstructed image f(x, y).

5. The imaging system based on incoherent synthesis of amplitude coded modulation according to claim 3, characterized in that: High-resolution spectrum The update formula is: W i =Diag(β i (|P i | 2 +c) -1 ) Among them, α iter is the step size, i is the sub-aperture number, is the error function ε i For high resolution spectrum Derivative, Ψ i is the subaperture spectrum, γ is a fixed constant, β i is the scaling factor, P i is the pupil function, A i is the subaperture support domain.

6. The imaging system based on incoherent synthesis of amplitude coded modulation according to claim 3, characterized in that: The convergence criterion is that all generated target low-resolution images Low-resolution image reconstructed with sub-aperture I k The sum of the mean square errors is less than a threshold T.

Images