Lens-coding-free single-exposure imaging device and method for double-working-distance view field

Abstract

The invention discloses a double-working-distance view field lens-free coding single exposure imaging device. The device comprises a coding plate, a beam splitter prism, a first image sensor, a second image sensor, a control system and a computer, the control system is respectively connected with the first image sensor, the second image sensor and the computer; the coding plate is placed in front of the beam splitter prism and is used for modulating optical information of a target sample; the beam splitter prism is used for dividing the modulated light of the target sample into two parts, one part is received by the first image sensor after being reflected, and the other part is received by the second image sensor after being transmitted. According to the method, information of a target sample can be reconstructed through an iterative algorithm on the basis that two coded images which aim at the same coding plate but have different working distance point spread functions are acquired through single exposure. The method is of great significance to high-quality lens-free coding dynamic imaging.

Description

Technical Field

[0001] The invention belongs to the field of optical imaging and microscopy technology, and in particular relates to a dual-working-distance field-of-view lens-free encoding single-exposure imaging device and method. Background Art

[0002] Existing optical imaging technologies use lenses, or optical elements with the same or similar wavefront modulation functions as lenses, such as metasurfaces, Fresnel zone plates, etc., to image the target sample. Although these methods can intuitively present the information of the target sample, they are still limited by the unavoidable imaging aberrations of lenses and the high processing costs of metasurfaces and Fresnel zone plates. Lensless imaging technology can avoid the introduction of additional wavefront modulation optical elements, and therefore has become an important technical means in the field of computational imaging. Among them, lensless coded imaging technology introduces a coding plate to modulate the signal of the target sample to collect a coded image, and then reconstructs the information of the target sample in combination with the coding plate information. It has a series of advantages such as avoiding aberrations and controllable costs.

[0003] In order to realize sample reconstruction of coded imaging, some devices and methods have been proposed. The most classic case is the deconvolution scheme. It first obtains the point spread function of the coding plate by calculation or calibration, then collects the coded image of the sample, and finally uses deconvolution to reconstruct the sample information based on the point spread function of the coding plate and the coded image. Although this scheme can achieve sample reconstruction through a single exposure, it is limited by the fact that deconvolution is greatly affected by noise, and its reconstruction effect is usually not ideal. In order to further improve the reconstruction effect, by changing different coding plate patterns and collecting corresponding coded images, the influence of noise can be effectively suppressed, thereby obtaining high-quality sample reconstruction results. However, these methods require changing the coding plate pattern and cannot achieve dynamic imaging. Although some deep learning-based methods can reconstruct samples from single-exposure coded images without calculating or calibrating the point spread function of the coding plate; however, these methods require a large number of data sets and have poor mobility, and are not ideal single-exposure lensless coded imaging solutions. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a dual-working-distance field-of-view lensless coded single-exposure imaging device and method, which can reconstruct the information of the target sample through an iterative algorithm based on the acquisition of two coded images of the same coding plate but with different working-distance point spread functions in a single exposure. This is of great significance for high-quality lensless coded dynamic imaging.

[0005] Technical solution: The present invention discloses a dual-working-distance field-of-view lensless coded single-exposure imaging device, which comprises: a coding plate, a beam splitter prism, a first image sensor, a second image sensor, a control system and a computer; the control system is respectively connected to the first image sensor, the second image sensor and the computer; the coding plate is placed in front of the beam splitter prism to modulate the optical information of the target sample; the beam splitter prism is used to divide the modulated light of the target sample into two parts: one part is received by the first image sensor after reflection, and the other part is received by the second image sensor after transmission.

[0006] Furthermore, the first image sensor and the second image sensor are 50 mm away from the exit surface of the beam splitter prism, ensuring that the encoded images collected by the two image sensors correspond to different point spread functions.

[0007] The present invention also discloses an imaging method using a dual-working-distance field-of-view lensless coded single-exposure imaging device, comprising the following steps:

[0008] S1. Device construction: Assemble the coding plate, beam splitter, image sensor and control system, ensure the optical axes of the optical components are consistent, and adjust the distances between the two image sensors and the coding plate to be different;

[0009] S2, device calibration: use a point light source to calibrate the point spread function of the two image sensors corresponding to the encoding plate;

[0010] S3, target sample imaging: set the target sample and collect the corresponding coded image;

[0011] S4. Target sample reconstruction: reconstruct the target sample information according to the calibrated point spread function and the collected coded image.

[0012] Furthermore, step S1 specifically comprises: constructing an optical system, wherein the optical components include a coding plate, a beam splitter prism and an image sensor; ensuring that the optical axes of the optical components are consistent and the coding plate is placed horizontally; adjusting the distance between the two image sensors and the coding plate, which is achieved by adjusting the image sensor and the exit surface of the beam splitter prism in the system; the distance between the first image sensor (4) and the coding plate (2) is 80-120 mm, and the distance between the second image sensor (5) and the coding plate (2) is 130-170 mm. The distance between the first image sensor (4) and the coding plate (2) and the distance between the second image sensor (5) and the coding plate (2) need to be accurately measured. Synchronizing the image sensors, controlling the two image sensors through the image sensor control system, so that the two image sensors simultaneously capture two coded images.

[0013] Furthermore, step S2 specifically includes the following steps:

[0014] S2.1. Set a point light source. Use a pinhole as the point light source. Place a pinhole with a diameter of 1-2μm behind the LED light source to simulate the point light source. The wavelength of the LED light source is 532nm (other narrow-band LED light sources are also acceptable). Place the pinhole 130-170mm in front of the encoding plate. The distance needs to be accurately measured.

[0015] S2.2, collecting coded images; by setting the exposure time of the image sensor, two coded images are collected at the same time, and the two collected coded images are considered to be the point spread functions PSF1 and PSF2 of the two image sensors corresponding to the coding plate.

[0016] Furthermore, step S3 specifically includes the following steps:

[0017] S3.1. Set the sample and place the target object 130-170mm in front of the coding plate. The distance must be accurately measured and consistent with the position of the point light source during calibration. The sample is illuminated by transmission or reflection of the light source.

[0018] S3.2, collecting coded images, by setting the exposure time of the image sensor, collecting two coded images at the same time; the collected images are described by formula (1) and formula (2) respectively; where i1 and i2 are the collected coded images, PSF1 and PSF2 are the point spread functions of the two image sensors corresponding to the coding plate in the device calibration, n1 and n2 are noises, o is the target sample information, and * is convolution;

[0019] i1=o*PSF1+n1 (1)

[0020] i2=o*PSF2+n2 (2)

[0021] S3.3, sample reconstruction, re-obtaining the target sample information o according to the point spread functions PSF1 and PSF2 of the two image sensors corresponding to the calibrated coding plate and the collected coded images i1 and i2 of the corresponding target samples.

[0022] Furthermore, step S4 specifically includes the following steps:

[0023] S4.1. Calculate the optical transfer function OTF. The optical transfer function is the normalized Fourier transform of the point spread function. Therefore, the optical transfer functions OTF1 and OTF2 corresponding to PSF1 and PSF2 are described according to formula (3) and formula (4), respectively.

[0024] Where FT() stands for Fourier transform, || is the absolute value, and max() stands for the maximum value;

[0025]

[0026] S4.2. Calculate the spectrum of the coded image. The spectrum of the coded image is the Fourier transform of the coded image. That is, the spectra I1 and I2 of the coded images i1 and i2 are the Fourier transforms of i1 and i2, respectively, and are described according to formula (5) and formula (6):

[0027] I1=FT(i1) (5)

[0028] I2=FT(i2) (6)

[0029] S4.3, update the target sample information spectrum O1,m according to the coded image i1; when updating for the first time, that is, m=1, set the guessed target sample information spectrum O2,0 to be a matrix of all 1s; if it is not the first update, that is, m>1, set the guessed target sample information spectrum O2,m-1 to the result of updating the target sample information spectrum O according to the coded image i2 in the last iterative calculation, and calculate the coded image spectrum I1,m' calculated according to the last iterative target sample information spectrum O2,m-1 and the calculated optical transfer function OTF1 according to formula (7);

[0030] I1,m'=O2,m-1×OTF1 (7)

[0031] Then the residual Δ1,m between the coded image spectrum I1 obtained according to the collected coded image i1 and the coded image spectrum I1,m' calculated according to the guessed target sample information spectrum O2,m-1 can be calculated by formula (8);

[0032] Δ1,m=I1-I1,m'=I1-O2,m-1×OTF1 (8)

[0033] Then the target sample information spectrum O1,m is updated according to formula (9), where ε is a small value to avoid the denominator being 0;

[0034]

[0035] S4.4, updating the target sample information spectrum O2,m according to the coded image i2, and calculating the coded image spectrum I2,m' obtained by the target sample information spectrum O1,m and the calculated optical transfer function OTF2 in the previous iteration according to formula (10);

[0036] I2,M'=O1,m×OTF1 (10)

[0037] Then the residual Δ2,m between the coded image spectrum I2 obtained according to the collected coded image i2 and the coded image spectrum I2,m' calculated according to the guessed target sample information spectrum O1,m can be calculated by formula (11);

[0038] Δ2,m=I2-I2,m'=I2-O1,m×OTF2 (11)

[0039] Then the target sample information spectrum O2,m is updated according to formula (12);

[0040]

[0041] S4.5, iterative reconstruction, repeating the above S4.3 and S4.4 for several times, so as to obtain the final target sample information spectrum O2,m, then the reconstructed target sample image o', is calculated according to the inverse Fourier transform of O2,m, as shown in formula (13), where IFT is the inverse Fourier transform;

[0042] o'=IFT(O2,m) (13)

[0043] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0044] The present invention aims at the technical shortcomings of the existing single-frame single-exposure lensless coding imaging method, which has poor reconstruction effect, and the existing multi-frame lensless coding imaging method, which requires a large number of coding patterns and cannot realize dynamic measurement, and proposes a dual-working-distance field-of-view lensless coding single-exposure imaging device and method. The present invention only uses a single coding plate, and combines a beam splitter prism and two image sensors that have been synchronously controlled to construct a dual-working-distance field-of-view lensless coding single-exposure imaging device. The two image sensors are at different distances from the coding plate, so that the point spread functions PSF1 and PSF2 of the two image sensors corresponding to the coding plate, and their corresponding optical transfer functions OTF1 and OTF2 can be realized. By using a point light source to calibrate PSF1 and PSF2 respectively, and collecting corresponding coded images i1 and i2 for the target sample, the target sample image can be reconstructed with high quality. Compared with the single-frame single-exposure lensless coded imaging method, the reconstruction quality of the present invention is significantly improved; compared with the multi-frame lensless coded imaging method, the present invention only needs to collect two coded images with a single exposure to reconstruct the target sample, thereby supporting dynamic imaging, thereby providing a new technical means for applications such as real-time lensless coded imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a schematic diagram of a dual-working-distance field-of-view lens-free coded single-exposure imaging device.

[0046] Figure 2 This is a schematic diagram of the system calibration of the present invention.

[0047] Figure 3 Schematic diagram of target sample reconstruction of the present invention.

[0048] Figure 4To apply the present invention, lens-free coded single-exposure imaging is performed on a resolution plate sample.

[0049] In the figure, 1-sample, 2-encoding plate, 3-beam splitter prism, 4-first image sensor, 5-second image sensor, 6-control system, 7-computer. DETAILED DESCRIPTION

[0050] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0051] The present invention discloses a dual-working-distance field-of-view lensless coding single-exposure imaging device, comprising 1 sample, 2 coding plate, 3 beam splitter prism, 4 first image sensor, 5 second image sensor, 6 image sensor control system, and 7 computer. The image sensor control system 6 is connected to the first image sensor 4 and the second image sensor 5 for controlling the synchronization of the two image sensors, and the computer 7 is connected to the image sensor control system 6 for image reconstruction and display.

[0052] First, follow the steps below to build a dual working distance field of view lensless coded single exposure imaging device, such as Figure 1 As shown:

[0053] Build an optical system, which includes 2 encoding plates, 3 beam splitter prisms, 4 first image sensors, and 5 second image sensors. Ensure that the optical axes of these optical components are consistent, and use laser reflection or a level to ensure that the encoding plate is completely horizontal. In addition, by adjusting the exit surface of the image sensor and the beam splitter prism, the distance between the two image sensors and the encoding plate is 100 mm and 150 mm respectively.

[0054] The synchronous image sensor mainly controls the first image sensor 4 and the second image sensor 5 through the control system of the image sensor 6, so that the two image sensors can simultaneously capture two coded images. The control system of the image sensor 6 is connected with a computer 7, so that the image reconstruction of the target sample can be realized.

[0055] Then, the point spread functions PSF1 and PSF2 of the dual working distance field lensless coding single exposure imaging device of the present invention are calibrated according to the following steps: Figure 2 As shown, follow these steps:

[0056] 1. Set up a point light source. Place a pinhole with a diameter of 1μm behind the LED light source to simulate a point light source. The wavelength of the LED light source is 532nm. Place the pinhole 150mm in front of the encoding plate.

[0057] 2. Collecting coded images. By setting the exposure time of the image sensor, two coded images can be collected simultaneously. The two collected coded images are the point spread functions PSF1 and PSF2 of the dual working distance field lensless coded single exposure imaging device of the present invention.

[0058] Then, set the target sample, collect the corresponding coded images i1 and i2, and follow the steps below:

[0059] 1. Set the sample. In this case, the resolution plate is used as the target sample, and the target sample is placed 150mm in front of the encoding plate, which is consistent with the position of the point light source when the system is calibrated. In addition, an LED light source with a wavelength of 532nm is used to illuminate the target sample. The light passing through the resolution plate carries the information of the target sample and can be received by the image sensor after passing through the encoding plate.

[0060] 2. Collecting coded images. By setting the exposure time of the image sensor, two coded images i1 and i2 can be collected simultaneously, which can be described by formula (1) and formula (2) respectively.

[0061] Where i1 and i2 are the collected coded images, PSF1 and PSF2 are the point spread functions of the two image sensors corresponding to the coding plate in the device calibration, n1 and n2 are noise, o is the target sample information, and * is convolution;

[0062] i1=o*PSF1+n1 (1)

[0063] i2=o*PSF2+n2 (2)

[0064] A dual working distance field of view lensless coded single exposure imaging method is provided, which reconstructs target sample information o according to the calibrated point spread functions PSF1 and PSF2 of the device and the collected coded images i1 and i2, and is carried out according to the following steps:

[0065] 1. Calculate the optical transfer function. Based on the point spread functions PSF1 and PSF2 of the device obtained by system calibration, use formula (3) and formula (4) to calculate the normalized Fourier transform, that is, the optical transfer functions OTF1 and OTF2 corresponding to PSF1 and PSF2.

[0066]

[0067] 2. Calculate the coded image spectrum. According to the collected coded images i1 and i2, use formula (5) and formula (6) to calculate the coded image spectrum I1 and I2.

[0068] I1=FT(i1) (5)

[0069] I2=FT(i2) (6)

[0070] 3. Update based on the coded image i1. Use OTF1 and I1, as well as the sample information spectrum updated in the previous iteration, and use formulas (7), (8) and (9) to update the target sample information spectrum.

[0071] I1,m'=O2,m-1×OTF1 (7)

[0072] Δ1,m=I1-I1,m'=I1-O2,m-1×OTF1 (8)

[0073]

[0074] 4. Update based on the coded image i2. Use OTF2 and I2, as well as the sample information spectrum updated in the previous iteration, and use formula (10), formula (11) and formula (12) to update the target sample information spectrum.

[0075] I2,m'=O1,m×OTF1 (10)

[0076] Δ2,m=I2-I2,m'=I2-O1,m×OTF2 (11)

[0077]

[0078] 5. Iterative reconstruction. Repeat (3) and (4) 50 times in total to obtain the final target sample information spectrum, and use formula (13) to calculate the target sample information through inverse Fourier transform.

[0079] o'=IFT(O2,m) (13)

[0080] The results are as follows Figure 4 As shown, the dual-working-distance field-of-view lensless coded single-exposure imaging device and method of the present invention can reconstruct the target sample information based on the two coded images collected by single exposure, and can support dynamic imaging. In addition, the system has a simple structure, controllable cost, fast reconstruction speed and high precision, which increases the practicality of the device.

[0081] The examples described in the present invention are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made to the technical solutions of the present invention by engineers and technicians in this field should all fall within the protection scope of the present invention.

Claims

1. A dual working distance field of view lensless coded single exposure imaging device, characterized in that: The device comprises: a coding plate (2), a beam splitter prism (3), a first image sensor (4), a second image sensor (5), a control system (6) and a computer (7); the control system (6) is connected to the first image sensor (4), the second image sensor (5) and the computer (7) respectively; the coding plate (2) is placed in front of the beam splitter prism (3) and is used to modulate the light information of the target sample (1); the beam splitter prism (3) is used to divide the modulated light of the target sample (1) into two parts: one part is received by the first image sensor (4) after reflection, and the other part is received by the second image sensor (5) after transmission.

2. The dual working distance field of view lensless encoding single exposure imaging device according to claim 1, characterized in that: The first image sensor (4) and the second image sensor (5) are located at different distances from the exit surface of the beam splitter prism (3), wherein the distance between the first image sensor (4) and the coding plate (2) is 80-120 mm, and the distance between the second image sensor (5) and the coding plate (2) is 130-170 mm, so that the coded images collected by the two image sensors correspond to different point spread functions.

3. An imaging method using the dual working distance field of view lensless encoding single exposure imaging device according to claim 1, characterized in that: The following steps are involved: S1. Device construction: Assemble the coding plate, beam splitter, image sensor and control system, make the optical axes of the optical components consistent, and adjust the distances between the two image sensors and the coding plate to be different; S2, device calibration: use a point light source to calibrate the point spread function of the two image sensors corresponding to the encoding plate; S3, Target sample imaging: First, carefully place the target object at a position of 130-170mm in front of the coding plate, and keep the position consistent with the point light source used during calibration. The sample obtains proper illumination through transmission or reflection of the light source. By setting the exposure time of the image sensor, two coded images are captured synchronously. These two images respectively incorporate the influence of the point spread functions of the two image sensors corresponding to the coding plate determined during the device calibration process, and are marked as PSF1 and PSF2 respectively. They also contain the information of the target sample and noise interference, which are represented by n1 and n2 respectively. S4, target sample reconstruction: After image acquisition, enter the sample reconstruction stage; using the known PSF1 and PSF2, combined with the collected coded images i1 and i2 of the corresponding target sample, through specific image processing technology, finally extract and restore the original information o of the target sample.

4. The imaging method of the dual working distance field of view lensless coded single exposure imaging device according to claim 3, characterized in that: Step S1 is specifically as follows: construct an optical system, wherein the optical components include a coding plate, a beam splitter prism and an image sensor; ensure that the optical axes of the optical components are consistent and the coding plate is placed horizontally; adjust the distance between the two image sensors and the coding plate, which is achieved by adjusting the image sensor and the exit surface of the beam splitter prism in the system; the distance between the first image sensor and the coding plate is 80-120 mm, and the distance between the second image sensor and the coding plate is 130-170 mm; synchronize the image sensors, and control the two image sensors through the control system of the image sensors so that the two image sensors simultaneously capture two coded images.

5. The imaging method of the dual working distance field of view lensless coded single exposure imaging device according to claim 3, characterized in that: Step S2 specifically includes the following steps: S2.

1. Set a point light source. Use a pinhole as the point light source. Place a pinhole with a diameter of 1-2 μm behind the LED light source to simulate the point light source. Place the pinhole 130-170 mm in front of the encoding plate. S2.2, collecting coded images; by setting the exposure time of the image sensor, two coded images are collected at the same time, and the two collected coded images are considered to be the point spread functions PSF1 and PSF2 of the two image sensors corresponding to the coding plate.

6. The imaging method of the dual working distance field of view lensless coded single exposure imaging device according to claim 3, characterized in that: Step S3 specifically includes the following steps: S3.

1. Set the sample and place the target object 130-170mm in front of the coding plate, which is consistent with the position of the calibration point light source; the sample is illuminated by transmission or reflection of the light source; S3.2, collecting coded images, by setting the exposure time of the image sensor, collecting two coded images at the same time; the collected images are described by formula (1) and formula (2) respectively; where i1 and i2 are the collected coded images, PSF1 and PSF2 are the point spread functions of the two image sensors corresponding to the coding plate in the device calibration, n1 and n2 are noises, o is the target sample information, and * is convolution; i1=o*PSF1+n1 (1) i2=o*PSF2+n2 (2) S3.3, sample reconstruction, re-obtaining the target sample information o according to the point spread functions PSF1 and PSF2 of the two image sensors corresponding to the calibrated coding plate and the collected coded images i1 and i2 of the corresponding target samples.

7. The imaging method of the dual working distance field of view lensless coded single exposure imaging device according to claim 3, characterized in that: Step S4 specifically includes the following steps: S4.

1. Calculate the optical transfer function OTF. The optical transfer function is the normalized Fourier transform of the point spread function. Therefore, the optical transfer functions OTF1 and OTF2 corresponding to PSF1 and PSF2 are described according to formula (3) and formula (4), respectively, where FT() represents Fourier transform, || represents the absolute value, and max() represents the maximum value; S4.

2. Calculate the spectrum of the coded image. The spectrum of the coded image is the Fourier transform of the coded image. That is, the spectra I1 and I2 of the coded images i1 and i2 are the Fourier transforms of i1 and i2, respectively, and are described according to formula (5) and formula (6): I1=FT(i1) (5) I2=FT(i2) (6) S4.3, update the target sample information spectrum O1,m according to the coded image i1; when updating for the first time, that is, m=1, set the guessed target sample information spectrum O2,0 to be a matrix of all 1s; if it is not the first update, that is, m>1, set the guessed target sample information spectrum O2,m-1 to the result of updating the target sample information spectrum O according to the coded image i2 in the last iterative calculation, and calculate the coded image spectrum I1,m' calculated according to the last iterative target sample information spectrum O2,m-1 and the calculated optical transfer function OTF1 according to formula (7); I1,m'=O2,m-1×OTF1 (7) Then the residual Δ1,m between the coded image spectrum I1 obtained according to the collected coded image i1 and the coded image spectrum I1,m' calculated according to the guessed target sample information spectrum O2,m-1 can be calculated by formula (8); Δ1,m=I1-I1,m'=I1-O2,m-1×OTF1 (8) Then the target sample information spectrum O1,m is updated according to formula (9), where ε is a small value to avoid the denominator being 0; S4.4, update the target sample information spectrum O2,m according to the coded image i2, and calculate the coded image spectrum I2,m' according to the target sample information spectrum O1,m of the previous iteration and the calculated optical transfer function OTF2 Calculate according to formula (10); I2,m'=O1,m×OTF1 (10) Then the residual Δ2,m between the coded image spectrum I2 obtained according to the collected coded image i2 and the coded image spectrum I2,m' calculated according to the guessed target sample information spectrum O1,m can be calculated by formula (11); Δ2,m=I2-I2,m'=I2-O1,m×OTF2 (11) Then the target sample information spectrum O2,m is updated according to formula (12); S4.5, iterative reconstruction, repeating the above S4.3 and S4.4 for several times, so as to obtain the final target sample information spectrum O2,m, then the reconstructed target sample image o', is calculated according to the inverse Fourier transform of O2,m, as shown in formula (13), where IFT is the inverse Fourier transform; o'=IFT(O2,m) (13).

Images