An edge enhancement method for a lensless imaging system

Abstract

This invention proposes an edge enhancement method for a lensless imaging system, applicable to edge detection fields such as fingerprints, gestures, license plates, and industrial parts. The edge enhancement method based on a helical zone plate lensless imaging system reconstructs high-contrast, high-signal-to-noise-ratio edge-enhanced images of the object. Compared to traditional lens systems, this invention features a smaller system size, simpler structure, lower cost, and lighter weight. The imaging effect more prominently highlights the detected subject and enhances edge contours, while reducing background and noise interference. This system can also perform lensless imaging, achieving multimodal imaging for both imaging and edge enhancement. The method is highly feasible and widely applicable, suitable for computational imaging such as holographic imaging and FINCH (Fresnel incoherent correlation holography).

Description

Technical Field

[0001] This invention relates to the field of edge detection, and more specifically to an edge enhancement method for a lensless imaging system. Background Technology

[0002] In recent years, with the development of the Internet of Things (IoT) and intelligent manufacturing, wearable devices, robots, IoT devices, virtual / augmented reality (AR / VR) devices, and human-computer interaction devices have tended to be miniaturized and intelligent. In order to meet the urgent need for miniaturization and intelligence of instruments and equipment for visual information acquisition as the IoT and intelligent manufacturing are booming, lensless imaging systems are gradually being used due to their low cost, small size, and high output.

[0003] In many scenarios, people pay more attention to the detailed features of an image, especially edge information. Edge information, as a key visual feature, is the prerequisite and foundation for image recognition or machine vision. Image edge enhancement methods can be broadly divided into two types: one is image processing performed in a computer, which belongs to post-processing; the other is to enhance the edge signal of an object through a specific optical imaging system, which belongs to pre-processing. For example, optical vortex filtering enhances the edge signal during the imaging stage, essentially a redistribution of energy, with more light intensity concentrated on the edge contour.

[0004] In edge enhancement, traditional lensed systems use spatial light modulators to control the light field within a 4f optical system, extending edge enhancement in a single direction using a one-dimensional Hilbert transform to isotropic edge enhancement in any direction. Edge enhancement is achieved through energy redistribution. However, the entire 4f system is complex and expensive, containing multiple lenses, spatial light modulators, lasers, and other hardware components. Furthermore, the inter-lens spacing requirements are relatively strict, resulting in a large system footprint. Additionally, when using spiral phase plates or spatial light modulators, the phase singularity of the vortex must be aligned with the zero-frequency position of the spectral plane to achieve isotropic edge enhancement; without calibration, the edge enhancement will be uneven. Moreover, due to low energy efficiency, experiments require high-power lasers. Existing lensless systems, on the other hand, cannot perform edge enhancement, compromising image quality.

[0005] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to overcome the technical problems in the prior art where traditional lens systems are complex in structure, large in size and weight, and cannot guarantee imaging quality without a lens system.

[0007] Therefore, the present invention provides an edge enhancement method for a lensless imaging system, wherein the lensless imaging system uses a helical zone plate as an optical mask, and includes the following steps:

[0008] S1: Construct a lensless imaging model;

[0009] S2: Determine the image edge enhancement and reconstruction model based on helical zone plates;

[0010] S3: Obtain the reconstruction model of the entire system based on the imaging model in S1;

[0011] S4: Apply the transfer function to add an amplitude modulation term to suppress low-frequency noise;

[0012] S5: Based on the reconstruction model of the entire system, the holographic images obtained by the sensors are used for reconstruction to obtain edge-enhanced images.

[0013] Furthermore, the imaging model in step S1 is specifically as follows:

[0014]

[0015] Where O(x,y) is the input function, and T G (x,y) is the mask transmittance function, I holo (x,y) represents the holographic-like image received by the sensor, indicating the convolution of the object and the magnified mask image, h0(x,y) is the lensless imaging transfer function, and e(x,y) represents the noise. Let U(x,y) be the conjugate of h0(x,y), and U(x,y) be the diffracted wavefront propagating at a virtual wavelength λ and a virtual distance d. * (x,y) is the conjugate of U(x,y).

[0016] Furthermore, the transmittance function of the spiral zone plate is:

[0017]

[0018] Where T SZ (r,θ) is the transmittance function of the helical zone plate, i is the imaginary unit, θ is the helical phase factor; r1 is the radius of the first ring of the zone plate, taken as l=1, and the virtual wavelength λ and virtual object distance d satisfy…

[0019] The transmittance function of a spiral zone plate includes a quadratic phase factor and a spiral phase factor, which can simultaneously achieve imaging and radial Hilbert transform functions. That is, the spiral zone plate itself can achieve energy redistribution in all directions, thus achieving edge enhancement.

[0020] According to the method described in claim 3, the image edge enhancement and reconstruction model based on helical zone plates in step S2 is as follows:

[0021]

[0022] Among them, I edge (x,y) represents the edge enhancement image received by the sensor, O edge Edge enhancement image of the original image, To achieve the transfer function for edge enhancement, twin images and background noise are treated as errors and removed during the denoising process. Only the first term of the formula is taken, and |O| is set. edge | 2 The intensity is the result of the final edge enhancement, and the formula is rearranged to obtain the expression:

[0023]

[0024] in, To implement the transfer function for edge enhancement, H = iexp[-iπr1] 2 (u 2 +v 2 ], where h(x,y) is the Fourier transform, u and v are frequency domain coordinates; A is the amplitude modulation term, and the identity matrix corresponding to the helical zone plate is . For Fourier transform, This is the inverse Fourier transform.

[0025] Holograms contain all the information of an object, making it possible to reconstruct the complete image information of the object, O(x,y). Due to the flexibility of the algorithm, only a spiral factor needs to be added to the reconstructed transfer function for compensation, corresponding to the spiral factor in the spiral phase plate in actual physical space.

[0026] Furthermore, step S3 specifically involves using T SZA The transfer function corresponding to (r,θ) is reconstructed to obtain a complete image:

[0027]

[0028] Where h SZA (x,y)=exp[i(θ-(π(x 2 +y 2 ) / r1 2 ))] is the transfer function for achieving imaging. The formula for a binary amplitude-type helical zone plate is given; (r, θ) represents the polar coordinate system; therefore, to achieve complete imaging, the reconstruction model of the entire system is as follows:

[0029]

[0030] in, It is h SZA Fourier transform of (x,y), where u and v are frequency domain coordinates; e is noise.

[0031] Furthermore, in step S4, low-frequency noise is suppressed in the edge-enhanced image in the frequency domain by using amplitude modulation terms, including using LG terms, Bessel terms, Airy terms, and Sinc terms to modulate the transfer function to suppress low-frequency noise in the frequency domain and obtain an image with higher contrast.

[0032] When applied to the edge enhancement of helical zone plates, A corresponds to the following: A Sin c = sin c(ar)sin(aπr), where w1, α, w0, and r0 are modulation parameters, w1 is the parameter controlling the position of maximum amplitude in the LG term, α is the radial amplitude modulation parameter, r0 is the radius of the main loop of the Airy function, and w0 is the beam waist radius. The transfer function, further incorporating the LG term and the equal-frequency domain amplitude modulation term, is: Here, J2() is a Bessel function of the second type, and Ai() is an Airy function.

[0033] Further, step S5 includes the following steps:

[0034] S5-1: Based on the reconstruction model of the entire system, the entire system device is then built, and sensors are used to acquire holographic-like images.

[0035] S5-2: The obtained holographic-like image is reconstructed through calculation to finally obtain the edge-enhanced image.

[0036] Furthermore, in step S5-1, the position of the mask is adjusted so that the center of the mask and the center of the sensor are on the same horizontal plane.

[0037] The present invention has the following beneficial effects:

[0038] This invention constructs a lensless imaging model, then determines an image edge enhancement reconstruction model based on a helical zone plate based on the lensless imaging model. After obtaining the edge enhancement reconstruction model, an amplitude modulation term is added to the transfer function to suppress low-frequency noise, thereby obtaining the reconstruction model of the entire system. The hologram-like image acquired by the sensor is used for reconstruction. Because the entire system adopts a lensless optical system, it is small in size, light in weight, and simple in structure. At the same time, considering the image edge enhancement requirements, the lensless imaging model can obtain edge-enhanced images with strong isotropy and high contrast, and can realize multimodal imaging of imaging and edge enhancement, thereby ensuring imaging quality. Attached Figure Description

[0039] Figure 1 This is a flowchart of the edge enhancement method of the lensless imaging system according to an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram illustrating the principle of edge enhancement achieved by the spiral phase factor in an embodiment of the present invention;

[0041] Figure 3 This refers to the amplitude term of the vortex filter used in the embodiments of the present invention;

[0042] Figure 4 This is a cross-sectional intensity distribution curve of the vortex filter used in the embodiments of the present invention;

[0043] Figure 5 It is a binarized helical zone plate according to an embodiment of the present invention;

[0044] Figure 6(a) is a schematic diagram of a lensless enhancement system based on a helical zone plate according to an embodiment of the present invention;

[0045] Figure 6(b) is a schematic diagram of the spiral zone plate in an embodiment of the present invention;

[0046] Figure 6(c) is a schematic diagram of the sensor according to an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of edge images reconstructed using different transfer functions in embodiments of the present invention;

[0048] Figure 8 This is the edge image reconstructed according to an embodiment of the present invention. Figure 7 The intensity distribution corresponding to the white intercepts in (a)(b)(c)(d)(e)(f);

[0049] Figure 9 This is a comparison of image edges reconstructed using a lens system and a lensless system. Detailed Implementation

[0050] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0051] The system solutions in the embodiments of the present invention will now be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0052] The overall system block diagram of the edge enhancement method of the lensless imaging system used in this invention is as follows: Figure 1As shown, the lensless edge enhancement system based on helical zone plates is applicable to most edge detection applications, and is not limited to the letter edge enhancement proposed in this example. It should be noted that the optical imaging system of this embodiment is equipped with a helical zone plate, and edge-enhanced image processing is achieved through helical zone plate and computational modulation.

[0053] like Figure 1 As shown, the lensless edge enhancement method based on helical zone plates includes the following steps:

[0054] 1. Construct a lensless imaging model;

[0055] 2. Determine the image edge enhancement and reconstruction model based on the helical zone plate according to the lensless imaging model. For example... Figure 2 The diagram shown illustrates the principle of edge enhancement achieved by the spiral phase factor. Figure 2 (a) Phase diagram θ, Figure 2 (b) Helical phase factor exp(iθ): The helical phase factor in the helical zone plate realizes the radial Hilbert transform, realizes the redistribution of energy, and thus realizes edge enhancement.

[0056] 3. After obtaining the edge-enhanced reconstruction model, an amplitude modulation term is added to the transfer function to suppress low-frequency noise, such as... Figure 3 The vortex filter amplitude term used in the embodiment of the present invention shown herein, wherein, Figure 3 (a)LG item A LG , Figure 3 (b) Bessel item A Bessel , Figure 3 (c) Airy item A Airy , Figure 3 (d)Sinc item A Sinc While spiral zone plates can enhance edges, they cannot eliminate noise, and pinhole imaging noise and twin noise still exist, resulting in low image contrast. Therefore, an amplitude modulation term of a vortex filter is introduced to suppress low-frequency noise. Figure 3 As shown, the common characteristics of the amplitude images of the four filters are a dark center and dark edges, indicating their ability to suppress low-frequency signals. Figure 4 The image shows the center profile intensity distribution of the amplitude terms for four types of vortex filters, which effectively suppress low-frequency noise. This results in edge-enhanced images with stronger isotropy and higher signal-to-noise ratios. (LG term A) LG The modulation parameter w1 = 0.6R, where R = 133.3 mm. -1 For the size of the aperture, Bessel term A Bessel The modulation parameter α = 0.1 mm -1 Airy item A Airy The modulation parameters w0 = 0.6R, r0 = 0.2R, and the Sinc term ASinc The modulation parameter a = 0.015mm -1 .

[0057] 4. Obtain the reconstructed model of the entire system and reconstruct it using the holographic-like image acquired by the sensors. Since continuously varying zone plates are difficult to fabricate, we use a binarized zone plate, such as... Figure 5 As shown, the helical zone plate serves as the optical mask in this lensless edge enhancement model. The object light passes through the helical zone plate and is projected onto the sensor to obtain an coded image. This image is then decoded using a corresponding imaging model to obtain the edge-enhanced image. For example... Figure 6a The image shows the lensless imaging experimental setup. The experimental hardware system includes a helical zone plate. Figure 6b Display screen (iPad 2019), CMOS (Complementary Metal Oxide Semiconductor) (QHY163M) Figure 6c The system includes a computer system, a spiral zone plate with a first ring radius of 0.32 mm and a theoretical resolution of 0.015 mm. The spiral zone plate is 2 mm thick and 1 mm from the sensor. In numerical reconstruction, the distance from the zone plate to the sensor is considered to be 3 mm. The display screen is an iPad 2019, an LED screen with a resolution of 2160×1620, which serves as both the target object and an incoherent light source in the experiment. The display screen is 300 mm from the zone plate. The CMOS sensor has a pixel pitch of 3.8 μm and a pixel count of 4656×3522. To facilitate subsequent processing, the collected images were cropped to 2400×2400.

[0058] 5. Reconstructed images of a lensless edge enhancement system based on helical zone plates, such as... Figure 7 As shown, Figure 7 As shown in (a), the transfer function is... Where H = iexp[-iπr1] 2 (u 2 +v 2 [], where h(x,y) is the Fourier transform, and S=exp(iθ) is the spiral phase factor used for edge enhancement. However, during deconvolution reconstruction, the spiral phase factor is not removed from the reconstructed transfer function, thus failing to achieve edge enhancement. Furthermore, noise and twin image interference remain. Figure 7 As shown in (b), although the edge enhancement image achieved by the helical zone plate enhances the edges, the contrast is poor due to noise and twin image interference. Meanwhile... Figure 7 In the reconstructed transfer functions corresponding to (c) to (f), a frequency domain amplitude modulation term is added, that is, an LG term A is added. LG Bessel item A Bessel Airy item A Airy Sinc item ASinc . Figure 8 for Figure 7 (a)(b)(c)(d)(e)(f) Edge intensity distribution of the white lines. It can be seen that the addition of the frequency domain amplitude modulation term eliminates most of the low-frequency signal, enhances the contrast and signal-to-noise ratio of the image edge enhancement, and achieves a contrast ratio of over 0.7. Compared with the edge enhancement image without amplitude modulation, the edge positioning accuracy is improved by 50% to 60%.

[0059] Figure 9 The images show a comparison of image edges reconstructed using a lens system and a lensless system. (a) is an image captured using a lens system; (b) is an image reconstructed using deconvolution without a lens system; (c) is an image reconstructed using compressed sensing algorithm without a lens system; (d) is an image with enhanced edges without a lens; (e) is a schematic diagram of the image edges corresponding to (a); (f) is a schematic diagram of the image edges corresponding to (b); (g) is a schematic diagram of the image edges corresponding to (c); and (h) is a schematic diagram of the image edges corresponding to (d). Figure 9 Images (a) and (e) are the image captured by the lens system and the edge-extracted image, respectively. It can be seen that the lens system produces a good image, but it is also significantly affected by background edges. In lensless imaging reconstruction algorithms, a reconstruction distance can be set to reconstruct the image at the axial position of the gesture. However, at this reconstruction distance, a clear image cannot be obtained due to background noise. Figure 9 As shown in (b) and (f), the low signal-to-noise ratio image obtained from deconvolution, followed by Canny edge extraction, has a computation time of 0.48s for both image reconstruction and edge extraction. Due to excessive noise, the edges are extracted as pseudo-edges, interfering with the accuracy of gesture recognition. Even after processing with a denoising algorithm (compressed sensing) before edge extraction, noise still interferes with the extraction of gesture edges, such as... Figure 9 As shown in (c) and (g), it will use more computing resources, with a total computation time of 3.1 seconds. Figure 9 As shown in (d) and (h), the obtained edge images exhibit very clear gesture features without noise interference. However, due to the use of the most basic image processing methods, the resulting edges are relatively coarse. Compared to the edge extraction results of images with lenses, the lensless edge enhancement image enhances the edge details of the hand. Because of the selectivity of the reconstruction distance calculated in lensless imaging, the main body of the detected object is emphasized more, reducing interference from background information. Compared to the edge extraction results of images without lenses, the lensless edge enhancement method obtains a high-contrast edge image because the vortex filtering algorithm significantly reduces noise interference. In summary, the lensless edge enhancement algorithm has a significant advantage in enhancing image edge information, which is more beneficial for edge-based gesture recognition algorithms.

[0060] In summary, the lensless edge enhancement system based on a helical zone plate according to the embodiments of the present invention can effectively improve the isotropy of edge enhancement, achieving edge-enhanced images with stronger isotropy and higher signal-to-noise ratio, while also realizing multimodal imaging for both imaging and edge enhancement. The entire system has advantages such as simple structure, low cost, light weight, small size, and utilization of incoherent light illumination.

[0061] In the description of this invention, the term "first" is used for descriptive purposes only, and the terms "one embodiment," "some embodiments," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example that are included in at least one embodiment of this invention.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An edge enhancement method for a lensless imaging system, characterized in that, The lensless imaging system uses a helical zone plate as an optical mask. The transmittance function of the helical zone plate includes a quadratic phase factor for imaging and a helical phase factor for edge enhancement. The system includes the following steps: S1: Construct a lensless imaging model; S2: Determine the image edge enhancement and reconstruction model based on helical zone plates; The image edge enhancement and reconstruction model based on helical zone plates is as follows: Among them, I edge (x,y) represents the edge enhancement image received by the sensor, O edge Edge enhancement image of the original image, The transfer function for edge enhancement; S3: Obtain the reconstruction model of the entire system based on the imaging model in S1; Using T SZA The transfer function corresponding to (r,θ) is reconstructed to obtain a complete image: Where h SZA (x,y)=exp[i(θ-(π(x 2 +y 2 ) / r1 2 ))] is the transfer function for achieving imaging. The formula for a binary amplitude-type helical zone plate is given; (r, θ) represents the polar coordinate system; therefore, to achieve complete imaging, the reconstruction model of the entire system is as follows: in, It is h SZA Fourier transform of (x,y), where u and v are frequency domain coordinates; e is noise; S4: Apply the transfer function to add an amplitude modulation term to suppress low-frequency noise; S5: Based on the reconstruction model of the entire system, the holographic images obtained by the sensors are used for reconstruction to obtain edge-enhanced images.

2. The method according to claim 1, characterized in that, The imaging model in step S1 is as follows: Where O(x,y) is the input function, and T G (x,y) is the mask transmittance function, I holo (x,y) represents the holographic-like image received by the sensor, indicating the convolution of the object and the magnified mask image, h0(x,y) is the lensless imaging transfer function, and e(x,y) represents the noise. Let U(x,y) be the conjugate of h0(x,y), and U(x,y) be the diffracted wavefront propagating at a virtual wavelength λ and a virtual distance d. * (x,y) is the conjugate of U(x,y).

3. The method according to claim 1, characterized in that, Step S2 further includes: removing the twin image and background noise as errors during the denoising process, taking only the first term of the formula, and setting |O edge | 2 The intensity is the result of the final edge enhancement, and the formula is rearranged to obtain the expression: in, To achieve the transfer function for edge enhancement, H = i exp[-iπr1] 2 (u 2 +v 2 ], where h(x,y) is the Fourier transform, u and v are frequency domain coordinates; A is the amplitude modulation term, and the identity matrix corresponding to the helical zone plate is . For Fourier transform, This is the inverse Fourier transform.

4. The method according to claim 1, characterized in that, In step S4, low-frequency noise is suppressed in the frequency domain of the edge-enhanced image through amplitude modulation terms, including using LG terms, Bessel terms, Airy terms, and Sinc terms to modulate the transfer function to suppress low-frequency noise in the frequency domain and obtain an image with higher contrast.

5. The method according to claim 1, characterized in that, Step S5 includes the following steps: S5-1: Based on the reconstruction model of the entire system, the entire system device is then built, and sensors are used to acquire holographic-like images. S5-2: The obtained holographic-like image is reconstructed through calculation to finally obtain the edge-enhanced image.

6. The method according to claim 5, characterized in that, In step S5-1, the position of the mask is adjusted so that the center of the mask and the center of the sensor are on the same horizontal plane.

Images