Lensless double-camera system and motion correction biplane image reconstruction method

Abstract

The invention discloses a lensless double-camera system and a motion correction biplane image reconstruction method, and relates to a lensless double-camera system and an image reconstruction method. The objective of the invention is to solve the common problems of complex structure and troublesome operation in the traditional lens-free imaging technology. The lensless double-camera system comprises a laser optical fiber head, a collimating lens is arranged below the laser optical fiber head, a beam splitter prism is arranged below the collimating lens, a first camera is arranged below the beam splitter prism, a second camera is arranged on one side of the beam splitter prism, and a sample is arranged between the collimating lens and the beam splitter prism. The invention belongs to the technical field of image processing.

Description

Technical Field

[0001] This invention relates to a lensless dual-camera system and an image reconstruction method, belonging to the field of image processing technology. Background Technology

[0002] Lensless imaging is an innovative imaging method in computational optics. Compared to traditional imaging techniques, it does not rely on lenses to focus light rays for direct imaging. Instead, it captures the diffraction pattern of a sample using a sensor and then reconstructs the image using specific algorithms. This technology offers significant advantages, such as more compact and lightweight devices, making it ideal for portable devices or space-constrained scenarios. Furthermore, through specialized algorithms, it can recover the complex amplitude of the sample, thereby obtaining the "depth" information lost in traditional imaging methods.

[0003] From the perspective of the number of images acquired, lensless imaging technology can be divided into two main categories: multi-distance imaging technology and single-frame imaging technology. Multi-distance imaging acquires multiple sets of light field data (such as diffraction patterns or intensity distributions) at different distances, and uses the geometric or phase information between these data to reconstruct an image of the target scene. Its core idea is to increase information redundancy through multiple sets of measurements to compensate for the limitation of lensless systems lacking focal length adjustment. Multi-distance imaging systems often require a high-precision displacement stage to obtain the diffraction distances of different images, but the additional mechanical displacement also introduces new errors. Therefore, registration of the same set of images is often required before phase retrieval can be performed. To ensure the accuracy of phase retrieval, at least 6-8 images are usually needed to achieve a certain resolution, which also significantly increases the sampling time. In contrast, single-frame imaging technology only requires a single measurement (one image or a set of light intensity data) to reconstruct the target scene. It relies on specific hardware design (such as masks or spatial light modulators) and powerful computing algorithms. The core is to extract sufficient information from a single data frame by utilizing prior information of the scene (such as sparsity) or the modulation characteristics of the mask. Spatial light modulators are expensive and require stringent experimental conditions, making miniaturization and portability difficult. Furthermore, while single-frame imaging is faster than multi-range imaging due to the reduced number of images needed for phase retrieval, its resolution is inevitably lower. Therefore, some researchers have proposed an imaging method that combines the advantages of both—dual-frame imaging. Traditional dual-frame imaging uses the same setup as normal multi-range imaging, employing a precision motorized stage to move the camera, acquiring two images at different distances, and then using appropriate algorithms for phase retrieval. While this approach reduces image acquisition time, the mechanical displacement step still requires a precision stage during acquisition, making it unsuitable for live-body imaging.

[0004] To simultaneously retain high resolution and fast reconstruction speed, this invention proposes a motion-corrected dual-plane image reconstruction method based on a lensless dual-camera system. This method requires only two images. Its core idea is to acquire image data from two frames under different conditions (such as different illumination angles, different positions, or different time points), and utilize the differences between them to extract phase or depth information, thereby reconstructing a high-quality image. Compared to multi-distance imaging techniques, this invention requires significantly less data and eliminates the need for additional mechanical structures such as mechanical displacement stages. Furthermore, two cameras can be used simultaneously for image acquisition via a beam splitter, achieving a sampling speed consistent with single-frame imaging. Since there is no mechanical displacement step during acquisition, images can be captured continuously and reconstructed frame by frame, thus potentially enabling liveness detection. Compared to single-frame imaging techniques that do not require additional encoding devices, the use of two images for phase retrieval inevitably results in greater resolution.

[0005] In summary, traditional lensless imaging techniques generally suffer from disadvantages such as complex structure and cumbersome operation. The motion-corrected dual-plane image reconstruction method based on a lensless dual-camera system proposed in this invention eliminates the need for additional mechanical displacement or encoding devices, while retaining the high efficiency of single-frame imaging and the resolution of multi-distance imaging. Furthermore, since the method proposed in this invention can simultaneously capture images using two cameras, it has the potential to achieve live-body imaging. Summary of the Invention

[0006] To address the problems of complex structure and cumbersome operation in traditional lensless imaging technology, this invention proposes a lensless dual-camera system and a motion-corrected dual-plane image reconstruction method.

[0007] The technical solution adopted by the present invention to solve the above problems is as follows: The lensless dual-camera system of the present invention includes a laser fiber head, a collimating lens is provided below the laser fiber head, a beam splitter is provided below the collimating lens, a first camera is provided below the beam splitter, a second camera is provided on one side of the beam splitter, and the sample is placed between the collimating lens and the beam splitter.

[0008] The steps of the motion-corrected dual-plane image reconstruction method of the present invention include: Step 1: Set up a lensless dual-camera system, placing the first and second cameras close to the beam splitter to acquire two raw images. and ; Step 2: Use the focusing algorithm to obtain the diffraction distances corresponding to the two images. and ; Step 3: Generate a current guess about the reconstruction results. ; Step 4: Utilize the current guess Forward diffraction to and ; Step 5: Obtain two original images using a registration strategy. and rotation angle between and pixel translation ; Step 6, regarding step 4 The corresponding diffraction pattern is subjected to rotation compensation and translation compensation; Step 7: Iterate and update according to the optimization formula and perform noise reduction. Step 8: Repeat steps 1 to 7 until the reconstruction effect is achieved.

[0009] Furthermore, before performing phase retrieval, the diffraction distance of each diffraction pattern is recovered using a digital focusing algorithm, and this is treated as an optimization problem. (1), In formula (1), This indicates an estimate of the diffraction distance. This represents the sharpness evaluation function. This represents the gradient operator.

[0010] Furthermore, the solution process for the optimization problem is as follows: Step 1: For a captured image A range containing the focal distance is generated, and the image is inversely diffracted onto the mask plane to obtain a set of diffraction pattern sequences. Step 2: Select the sharpness quantification function to calculate the image sharpness for each diffraction pattern, thereby obtaining a focus curve; Step 3: Record the peak value of the focusing curve, which is the focusing distance. ; Step 4: Repeat steps 1 to 3, and record the focusing distance for each acquired image. .

[0011] Furthermore, in step 5, the rotation angle and pixel translation The acquisition process is as follows: Step 501: Assume the two images are respectively and The two images have the following relationship: (2), (3), In formulas (2) and (3), Indicates the rotation angle. Indicates the pixel shift amount; Step 502, Image Spectrum With images Spectrum The mathematical relationship between them is: (4), In formula (4), Representing spectral coordinates, their relationship with spatial coordinates is as follows: and , Indicates wavelength. Indicates the diffraction distance; Step 503: Define intermediate variables and Simplify formula (4) and take its modulus: (5), The image can be obtained from formula (5). and The rotation angle between them and their spectrum and The rotation angles are consistent; Step 504: Perform polar coordinate integration on the spectrum to convert the original two-dimensional matrix into a one-dimensional signal; (6), The corresponding curves are obtained by performing polar coordinate integration on the spectra of both. and ,in The angular range of the integration region is determined. and The bandwidth of the frequency band has been determined; Step 505 Perform lateral translation Again with The rotation angle can be obtained by calculating the correlation. The specific formula is as follows: (7), In formula (7), The range of values ​​is , indicating the range of angles traversed, for By performing a rotation, the effects of the rotation error can be eliminated through the inverse Fourier transform. Step 506: Assume the image after rotation correction is as follows: , ,at this time and The relationship is: (8), As can be seen from formula (8), the original pixel shift amount is converted into a spectrum shift amount; Step 507: Spectrum Apply translation amount before passing through with The initial pixel shift can be obtained by calculating the cross-correlation spectrum. The specific formula is as follows: (9), (10) In formulas (9) and (10), This represents the inverse Fourier transform operator; Step 508: Finally, the image after rotation registration and translation registration is obtained: (11).

[0012] Furthermore, the optimization formula in step 7 is as follows: (12) (13) (14) (15) (16) (17) (18) (19) In formulas (12) to (19), the operator This represents the diffraction operator, with the subscript corresponding to the diffraction distance.

[0013] The beneficial effects of this invention are: 1. The system designed in this invention has a compact structure, is simple to operate, and can change its placement orientation to perform phase imaging of live samples; 2. The imaging method proposed in this invention does not require a high-precision displacement device, the image acquisition process is simple, and the imaging quality is significantly better than that of single-frame imaging technology. 3. This invention proposes a registration algorithm for the corresponding system, which can effectively correct rotational registration and translational registration between two images. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the structure of a lensless dual-camera system according to the present invention; Figure 1In the image, 1-laser fiber optic head, 2-collimating lens, 3-sample, 4-beam splitter prism, 5-first camera, 6-second camera; Figure 2 This is a flowchart of a motion-corrected dual-plane image reconstruction method according to the present invention. Detailed Implementation

[0015] Example 1: As Figure 1 As shown, this embodiment includes a camera module and a guide rail module. The camera module uses Huagu Power's WP-UT1201, where 1 is the camera CMOS sensor plane and 2 is the power and signal interface. The power and signal interface is connected to the host computer. The guide rail module uses THORLABS' ELL17 / M, where 1 is the guide rail displacement platform and 2 is the power and signal interface. Power is supplied by connecting the power supply wires, and the signal interface is connected to the host computer.

[0016] The steps in this embodiment include: Step 1: Fix the laser fiber head 1 and adjust the collimating lens 2 to shape the spherical wave emitted by the laser fiber head 1 into a plane wave. Step 2: Set up a 4-beam splitter prism and adjust the angle to keep the overall optical axis of the optical path consistent; Step 3: Connect the first camera 5 and the second camera 6 to the entire system, as close as possible to the beam splitter 4, and adjust the angle to keep the overall optical axis of the optical path consistent; the beam splitter 4 is a 50:50 beam splitter. Step 4: Place sample 3 in front of beam splitter 4, and simultaneously sample using first camera 5 and second camera 6 to obtain two diffraction images.

[0017] Step 5: Phase recovery is performed using the motion-corrected dual-plane image reconstruction algorithm proposed by our team; The specific process of the multi-distance recovery algorithm used in step 5 is as follows: Step 1: Set up a dual-camera system, placing the two cameras as close as possible to the beam splitter prism and acquiring two raw images; Step 2: Use the focusing algorithm mentioned above to obtain the diffraction distances corresponding to the two images. and ; Step 3: Generate or update the current guess (complex matrix) for the reconstruction results. ; Step 4: Utilize the forward diffraction of the current guess to... and Place; Step 5: Obtain two original images using the registration strategy proposed above. and rotation angle between and pixel translation ; Step 6: Perform rotation compensation and translation compensation on the diffraction pattern corresponding to Step 4; Step 7: Iterate and update according to the optimization formula proposed in this method and perform noise reduction. Step 8: Repeat steps 1 through 7 until the reconstruction effect is achieved or the required number of iterations is reached.

[0018] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A lensless dual-camera system, characterized in that, The device includes a laser fiber head (1), a collimating lens (2) below the laser fiber head (1), a beam splitter (4) below the collimating lens (2), a first camera (5) below the beam splitter (4), a second camera (6) on one side of the beam splitter (4), and a sample (3) placed between the collimating lens (2) and the beam splitter (4).

2. A motion-corrected dual-plane image reconstruction method, characterized in that, The specific steps include: Step 1: Construct a lensless dual-camera system, placing the first camera (5) and the second camera (6) close to the beam splitter (4) to acquire two original images. and ; Step 2: Use the focusing algorithm to obtain the diffraction distances corresponding to the two images. and ; Step 3: Generate a current guess about the reconstruction results. ; Step 4: Utilize the current guess Forward diffraction to and ; Step 5: Obtain two original images using a registration strategy. and rotation angle between and pixel translation ; Step 6, regarding step 4 The corresponding diffraction pattern is subjected to rotation compensation and translation compensation; Step 7: Iterate and update according to the optimization formula and perform noise reduction. Step 8: Repeat steps 1 to 7 until the reconstruction effect is achieved.

3. The motion-corrected dual-plane image reconstruction method according to claim 2, characterized in that, Before performing phase retrieval, the diffraction distance of each diffraction pattern is recovered using a digital refocusing algorithm, and this is treated as an optimization problem. （1）， In formula (1), This indicates an estimate of the diffraction distance. This represents the sharpness evaluation function. This represents the gradient operator.

4. The motion-corrected dual-plane image reconstruction method according to claim 3, characterized in that, The solution process for the optimization problem is as follows: Step 1: For a captured image A range containing the focal distance is generated, and the image is inversely diffracted onto the mask plane to obtain a set of diffraction pattern sequences. Step 2: Select the sharpness quantification function to calculate the image sharpness for each diffraction pattern, thereby obtaining a focus curve; Step 3: Record the peak value of the focusing curve, which is the focusing distance. ; Step 4: Repeat steps 1 to 3, and record the focusing distance for each acquired image. .

5. The motion-corrected dual-plane image reconstruction method according to claim 1, characterized in that, Rotation angle in step 5 and pixel translation The acquisition process is as follows: Step 501: Assume the two images are respectively and The two images have the following relationship: （2）， （3）， In formulas (2) and (3), Indicates the rotation angle. Indicates the pixel shift amount; Step 502, Image Spectrum With images Spectrum The mathematical relationship between them is: （4）， In formula (4), Representing spectral coordinates, their relationship with spatial coordinates is as follows: and , Indicates wavelength. Indicates the diffraction distance; Step 503: Define intermediate variables and Simplify formula (4) and take its modulus: （5）， The image can be obtained from formula (5). and The rotation angle between them and their spectrum and The rotation angles are consistent; Step 504: Perform polar coordinate integration on the spectrum to convert the original two-dimensional matrix into a one-dimensional signal; （6）， The corresponding curves are obtained by performing polar coordinate integration on the spectra of both. and ,in This determines the angular range of the integration region. and The bandwidth of the frequency band has been determined; Step 505 Perform lateral translation Again with The rotation angle can be obtained by calculating the correlation. The specific formula is as follows: （7）， In formula (7), The range of values ​​is , indicating the range of angles traversed, for By performing a rotation, the effects of the rotation error can be eliminated through the inverse Fourier transform. Step 506: Assume the image after rotation correction is as follows: , ,at this time and The relationship is: （8）， As can be seen from formula (8), the original pixel shift amount is converted into a spectrum shift amount; Step 507: Spectrum Apply translation amount before passing through with The initial pixel shift can be obtained by calculating the cross-correlation spectrum. The specific formula is as follows: （9）， （10）， In formulas (9) and (10), Represents the inverse Fourier transform operator; Step 508: Finally, the image after rotation registration and translation registration is obtained: （11）。 6. The motion-corrected dual-plane image reconstruction method according to claim 1, characterized in that, The optimization formula in step 7 is: （12）， （13）， （14）， （15）， （16）， （17）， （18）， （19）， In formulas (12) to (19), the operator This represents the diffraction operator, with the subscript corresponding to the diffraction distance.

Images