Differentiable end-to-end pixel-super-resolution lensless imaging

Abstract

A differentiable lensless imaging system includes a light source and a sensor positioned to receive diffracted light from the light source. A stage for the sample is located between the light source and the sensor so that the sensor receives and records diffracted light from the sample on the stage. The computer receives the output recorded by the sensor and reconstructs a complete image of the target at a higher resolution than the sensor by integrating illumination control, auto-focus, and complex field sample reconstruction within a single differentiatable optimizable frame. A sub-pixel offset is generated by translating the light source, the sensor, or the sample relative to each other. The processor performs reconstruction by executing a differentiatable forward model of light propagation, and automatic differentiation results in simultaneous reconstruction of (a) a complex field representing the sample, (b) a sub-pixel offset scan position, and (c) a distance of the sample to the sensor.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202511856138.6 (22) Application Date 2025.12.10 (30) Priority Data 63 / 734528 2024.12.16 US (71) Applicant: University of Hong Kong Address: Pok Fu Lam Road, Hong Kong, China (72) Inventors: Chen Ni, Lin Yanmin (74) Patent Agency: Beijing Panhua Weiye Intellectual Property Agency Co., Ltd. 11280 Patent Attorney: Wang Bo (51) Int.Cl. G01N 21 / 84 (2006.01) G01N 21 / 47 (2006.01) G01N 21 / 01 (2006.01) (54) Invention Title Differentiable End-to-End Pixel Super-Resolution Lensless Imaging (57) Abstract A differentiable lensless imaging system includes a light source and a sensor positioned to receive diffracted light from the light source. A platform for the sample is located between the light source and the sensor, so that the sensor receives and records the diffracted light from the sample on the platform. A computer receives the output recorded by the sensor and reconstructs a complete image of the target at a resolution higher than that of the sensor by integrating illumination control, autofocus, and complex field sample reconstruction within a single differentiable optimization framework. Subpixel offset is generated by translating the light source, sensor, or sample relative to each other. The processor performs the reconstruction by executing a differentiable forward model of light propagation, and the automatic differentiation results in the simultaneous reconstruction of (a) a complex field representing the sample, (b) the subpixel offset scan position, and (c) the distance from the sample to the sensor. Claims 2 pages, Description 9 pages, Drawings 7 pages, CN 122217955 A 2026.06.16 CN 1 22 21 79 55 A 1. A lensless imaging system comprising: a light source emitting a light beam; a sensor positioned to receive and record a diffraction pattern of the light beam; a platform for a sample or target located between the light source and the sensor, the sensor receiving a diffraction pattern of light passing through the sample from the light source, while the sample and the light are translated relative to each other with a subpixel offset; a processor receiving a series of images from the sensor and configured to reconstruct a complete holographic image of the target at a resolution higher than that of the sensor by automatic differentiation, resulting in the simultaneous reconstruction of: (a) a complex field representing the sample, (b) a subpixel offset scan position, and (c) the distance of the sample to the sensor. 2. The system according to claim 1, wherein all parameters are jointly optimized using gradient descent optimization, wherein, in each iteration, the gradient of the loss function is calculated and the parameters are updated accordingly, and in each iteration, both the distance z from the sample to the sensor and the scanning position r are also updated.3. The system of claim 1, wherein autofocus is achieved using an established Laplacian method. 4. The system of claim 1, wherein the light source is a laser mounted on a computer-controlled x-y translation stage for translating and scanning the sample, thereby presenting iterative images of the sample at a series of angles to a sensor, the sensor capturing the intensity of light passing through the sample at each scanning position, and wherein the translation stage and the sensor are synchronized by the computer. 5. The system of claim 1, wherein the light source is a two-dimensional LED array, which is illuminated in different ways under the control of the processor to present a series of images of the sample at a series of angles to the sensor. 6. The system of claim 1, wherein the sensor is a camera. 7. The system of claim 4, wherein the wavelength of the light source is 300-700 nm, and the pixel size of the sample and the sensor camera is approximately 1 µm. 8. The system of claim 5, wherein the distance between the sample and the sensor is set to approximately 500 nm. 9. The system of claim 6, wherein autofocus is performed when the back-propagation distance is in the range of approximately 490 nm to 510 nm, covering a position of approximately 500 nm from the sensor to the sample. 10. The system of claim 1, wherein the processor executes a differentiable forward model of light propagation between the sample and the sensor, wherein sub-pixel offset differences are generated by translating at least one of the light source, the sensor, or the sample, and wherein the processor jointly estimates the complex light field of the sample, the distance from the sample to the sensor, and translation parameters by automatic differentiation of the forward model to minimize a loss function including a data fidelity term and one or more differentiable regularization terms. 11. The system of claim 10, wherein the differentiable forward model includes an angular propagation kernel, a Fresnel propagation kernel, or a convolutional propagation kernel. 12. The system of claim 10, wherein the light source array is a two-dimensional array of programmable light-emitting elements, thereby providing spatial or angular diversity. 13. The system of claim 10, wherein the light source scanning system translates a single light source between multiple positions relative to the sample mechanically, optically, or electronically. 14. The system of claim 10, wherein the one or more regularization terms comprise any differentiable function, learned prior information, or model-based constraints, including but not limited to Laplace penalties, total variational penalties, sparsity penalties, or neural network-based penalties. 15. The system of claim 10, wherein the processor uses an automatic [processor / mechanical system] implemented on a graphics processing unit.The differential framework performs gradient-based optimization. 16. The system of claim 1, wherein the configuration operates in transmission mode, reflection mode, or scattering mode. 17. The system of claim 10, wherein the processor compensates for environmental disturbances or mechanical misalignment through differentiable optimization. 18. A method for differentiable lensless imaging, comprising: (a) capturing multiple diffraction patterns of a sample during a sub-pixel shift caused by moving a light source or sample; (b) simulating the propagation of light from the sample to a sensor in a differentiable computational model; (c) defining a loss function that combines intensity, consistency, and one or more differentiable regularization terms; (d) simultaneously updating the reconstructed complex field, translation parameters, and propagation distance by automatic differentiation to minimize the loss, thereby producing a high-resolution complex image without sequential phase diversity measurements. 19. The method of claim 18, wherein translating the light source comprises operating an array of light sources, scanning a light source, or the sensor. 20. The method of claim 18, wherein the loss function comprises an adaptive edge enhancement term or a learned focusing term. 21. The method of claim 18, wherein the reconstructed field is digitally refocused to a different axial plane without additional capture. 22. The method of claim 18, wherein optimization is performed on a parallel computing architecture. 23. The system of claim 1, which implements a differentiable pixel super-resolution dPSR lensless imaging system, wherein the distance between the light source and the platform is much greater than the distance between the platform and the sensor, the sub-pixel offset creates an iteration of light diffusion patterns from the sample at a series of angles, and the processor reconstructs a complete holographic image of the target by integrating differentiable PSR image hologram synthesis, autofocus, and complex field sample reconstruction and inverse problem solving algorithms into a single unified end-to-end framework, the framework representing a numerical forward model of the lensless system. Claims 2 / 2 Page 3 CN 122217955 A Differentiable End-to-End Pixel Super-Resolution Lensless Imaging

[0001] Cross-Reference to Related Patent Applications

[0002] This application claims priority to U.S. Application No. 63 / 734,528, filed December 16, 2024, pursuant to Section 119(e) of 35 USC, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to lensless computational imaging, and more particularly, to a differentiable framework for lensless imaging systems that jointly reconstructs complex optical fields and system misalignments with enhanced resolution without phase diversity measurements. Background Art

[0004] As shown in Figure 1, lensless imaging typically employs extremely small optical devices, which include a light source and a relatively distant...A sensor in which the sample is placed close to the sensor [1]. This configuration enables an excellent numerical aperture (NA) close to 1, thus achieving microscopic resolution with pixel-level features within a wide field of view (FOV). However, despite the excellent performance of modern sensors with up to megapixels, lensless imaging systems typically cannot reach their maximum diffraction-limited resolution, mainly due to the use of micrometer-sized pixels.

[0005] Techniques such as pixel super-resolution (PSR) have become indispensable means of overcoming the limitations caused by the use of micrometer-sized pixels [2,3]. By abandoning traditional lenses and employing computational imaging, lensless systems provide high-resolution imaging in a compact and cost-effective manner. The simplicity and adaptability of lensless optics have made them widely used in high-resolution imaging, thus making them increasingly popular in the scientific community. This versatility has led to their extensive applications in a variety of fields, including medical imaging of cells and tissues [4], protein crystallography [5], microbial monitoring [6], and environmental sensing [7]. PSR has become a powerful technique, in particular, for high-resolution, label-free cell imaging, with significant advantages in terms of cost and system simplicity. Despite the powerful capabilities of PSR technology in lensless imaging, several challenges limit its application. These challenges include the apparatus and computation required for PSR measurements, autofocus for determining the distance from the sample to the sensor, and the reconstruction of complex fields from PSR holograms. In particular, the conventional computational reconstruction methods used in PSR face challenges due to the need for multiplexing in subpixel and phase diversity measurements.

[0006] PSR measurements are typically obtained by capturing a series of low-resolution images with subpixel offsets, which requires scanning the light source, sample, or sensor. These images are then combined to synthesize a high-resolution measurement hologram for sample reconstruction. Techniques such as phase retrieval are typically used to reconstruct the sample from the PSR hologram [8,9]. However, these methods require phase diversity measurements, such as multi-wavelength

[10] , multi-angle illumination [11-14], multi-height

[15] measurements, or coded mask [11,16] and synthetic aperture

[10] techniques. These techniques increase the complexity of lensless imaging systems.

[0007] Furthermore, the reconstruction of PSR holograms requires accurate estimation of the distance from the sample to the sensor before applying any image reconstruction algorithm, a task that can be extremely challenging. Traditional autofocus techniques typically involve backpropagating measurements to a series of distances covering the sample location, and then using a focus evaluation function to assess the focus position of the sample [17,18]. Various methods have been employed to evaluate image focus in both the spatial and frequency domains. In the spatial domain, sharp edges are a characteristic of focused images, and operators such as the Sobel operator

[19] , the Laplacian operator

[20] , and gradient summation

[17] have been used as evaluation functions to identify focused images. In the frequency domain, focused images exhibit more focused characteristics compared to blurred images.The high-frequency components of CN 122217955 A on page 1 / 9 of the book make the analysis of high-frequency components (e.g., bandpass filter power spectrum analysis

[20] ) a valuable standard for evaluating focus. In addition, alternative model-based evaluation functions have emerged, including differential critical function (DIF)

[17] , self-entropy

[21] , spectral L1 norm

[22] , etc. However, the presence of twins leads to fusion, making it challenging to apply these algorithms directly to lensless imaging. In particular, twins superimpose on each other in the target image, making the target image difficult to identify. Furthermore, factors such as low absorption and large phase delay further exacerbate these problems, making accurate autofocus a difficult task. Despite some attempts, a practical and robust autofocus method is still urgently needed for complex field imaging [23,24].

[0008] Finally, solving the complex challenge of reconstructing complex fields from PSR holograms remains a thorny inverse problem. Traditional iterative algorithms [8,9] and optimization techniques [16,25,26] are often used to solve this problem, which requires precise autofocus to ensure the accuracy of the imaging model. Obviously, there is a mutual constraint between autofocus and complex field reconstruction. However, conventional methods usually take a sequential approach to estimate PSR holograms, autofocus, and complex field image inversion, which can easily accumulate errors at each step, thus significantly impairing the quality of the final image.

[0009] PSR hologram synthesis, autofocus, and complex field reconstruction are usually performed sequentially. Each sub-process may introduce errors that affect subsequent steps and overall imaging performance

[27] . U.S. Patent No. 8,866,063 discloses a hardware system similar to PSR that uses a conventional sequential input processing method. This introduces errors at each stage. The system disclosed in U.S. Application Publication 2017-0357083A1 relies on scanning light sources to introduce sub-pixel offsets for synthesizing pixel super-resolution holograms and relies on multiple sample-to-sensor distances for phase recovery during image reconstruction. This system also suffers from resolution problems caused by micrometer-scale pixel sizes.

[0010] The present invention provides a differentiable lensless imaging system and method that integrates hologram synthesis, focusing, and phase recovery into a single differentiable optimization. The system includes: (1) a light source assembly (array light source or scanning light source) configured to illuminate a sample from multiple lateral positions; (2) an image sensor configured to record corresponding diffraction patterns; and (3) a processor that executes a differentiable forward model of light propagation between the sample and the sensor. By translating the light source, sensor, or sample individually or in combination, the sub-pixel offset differences required for high-resolution reconstruction can be achieved.

[0011] Through automatic differentiation, the processor jointly estimates complex light fields and relative flatness by minimizing a differentiable loss function.The displacement parameter and the distance from the sample to the sensor are considered. This differentiable loss function combines intensity fidelity and one or more flexible regularization terms. This method eliminates explicit phase diversity, achieves sub-pixel registration and micrometer-level focusing accuracy, and enables resolution enhancement beyond the sensor's native pixel pitch while maintaining simple lensless hardware.

[0012] In an exemplary embodiment, to address the challenges of prior art systems, the present invention does not use sequential steps but instead employs a differentiable end-to-end PSR (dPSR) lensless imaging technique that integrates PSR image hologram synthesis, autofocus, and complex field sample reconstruction into a single unified process that represents a numerical forward model of the lensless system. Furthermore, a differentiable framework simulating physical uncertainties integrates an inverse problem-solving algorithm for lensless holographic imaging, i.e., reconstructing complex fields from PSR holograms. In particular, it compensates for environmental disturbances or mechanical misalignment.

[0013] By utilizing automatic differentiation and gradient descent optimization, the following terms are simultaneously reconstructed: (a) the complex field representing the sample, (b) the subpixel offset scan position, and (c) the sample-to-sensor distance. In fact, key system parameters including the scan position and the sample-to-sensor distance are modeled as variables in a differentiable function. This numerical model considers unknown scan positions and sample-to-sensor distances, providing a more accurate representation of the actual imaging system without requiring phase diversity measurements (e.g., multi-depth, multi-wavelength, or multi-angle scans). This joint optimization of multiple variables reduces error accumulation associated with sequential processing. These advances collectively provide a simple, cost-effective, and high-performance solution for lensless imaging, broadening the applicability of pixel super-resolution in biomedical and scientific imaging applications.

[0014] The integrated system of the present invention differs significantly from traditional sequential processing methods and offers the following key advantages:

[0015] 1. Improved accuracy: By eliminating the inherent error accumulation in sequential processing, the unified method of the present invention achieves more accurate reconstruction while maintaining system simplicity.

[0016] 2. Simplified data acquisition: End-to-end optimization enables more accurate numerical modeling, effectively avoiding complex phase diversity measurements (e.g., multi-depth, multi-wavelength, or multi-angle scanning).

[0017] 3. Improved resolution: Experimental results show that the resolution is significantly improved by two times compared to the pixel size of the sensor, verifying the effectiveness of the present invention.

[0018] These advancements collectively provide a simple, cost-effective, and high-performance solution for lensless imaging, broadening the applicability of pixel super-resolution in biomedical and scientific imaging applications. Furthermore, the objective function is designed using domain-specific prior information to handle noisy data while maintaining high-resolution PSR holograms. In particular, the L1 norm is used to ensure...Sparsity is addressed, and the total variational (TV) norm is used to handle noisy data. Furthermore, the Laplacian operator is applied to the hologram to preserve high-resolution details.

[0019] This invention achieves a resolution improvement of twice the pixel size of the sensor, demonstrating its effectiveness in high-resolution lensless imaging. Therefore, this invention provides a simple, economical, and high-performance solution. This invention addresses the physical variance in lensless imaging, including uncertainties in the scan position and the target sample position.

[0020] The foregoing and other objects and advantages of the invention will become more apparent from the following detailed description and accompanying drawings, in which similar reference numerals denote similar elements in the various views, and wherein:

[0021] FIG1A is a lensless device for pixel super-resolution (PSR) imaging in which a laser source moves in a series of iterations; FIG1B is a device for PSR imaging in which an LED array is used as the source;

[0022] FIG2A is an image of a material sheet used in a simulation of a numerical experiment for analyzing errors in autofocus of the present invention; FIG2B shows a graph of autofocus performance with a Laplace metric for a pure absorption sample; FIG2C shows a graph of autofocus performance with a Laplace metric for a pure phase sample; FIG2D shows a graph of autofocus performance with a Laplace metric for a sample with absorption and phase delay; FIG2E shows how close the position estimated by the cross-correlation method is to 25 randomly generated scan positions of the sensor;

[0023] Figure 3A shows the raw data of imaging the USAF resolution test card, and a magnified view of the central region (small square in the center) located in the lower left corner; Figure 3B shows the reconstruction of the central region; Figure 3C compares the scan position estimated by the cross-correlation method of the present invention and the differentiable pixel super-resolution (dPSR) method, with the dPSR method having higher accuracy; Figure 3D shows the accurate estimation of the sample-to-sensor distance by the dPSR method;

[0024] Figure 4A shows the raw data of the oral cell experiment; Figure 4B is a magnified view of the portion in the square in the lower left corner of Figure 4A; Figure 4C is a magnified view of the central portion of Figure 4B. Detailed Description

[0025] Lensless computational imaging, such as pixel super-resolution (PSR) imaging, is achieved using the optical scanning method schematically shown in Figure 1. A laser source (laser 10) is mounted on an x-y translation stage 16 controlled by a computer or processor 20. The computer or processor 20 may be a graphics processing unit and may have a parallel computing architecture. The distance between the light source and the sample object 12 is Zlo, and the distance between the sample object 12 and the camera sensor 14 is Zos, where Zlo ≫ Zos.The light source 10 is scanned by translation without specifying a position. The processor 20 uses Python code to synchronize the translation stage 16 and the camera sensor 14. The sensor camera captures the diffraction pattern of the light passing through the sample at each scan position. The system can work even if the diffraction pattern is caused by a reflection mode or a scattering mode. The scanning light source can also be replaced with a light source array, such as a two-dimensional LED array, for more robust scanning, as shown in Figure 1B.

[0026] In order to create a mathematical model for lensless PSR imaging, the forward process of lensless pixel super-resolution imaging is represented as a function that depends on three key factors: 1) the high-resolution complex field of the target sample, denoted as u; 2) the distance between the sample and the camera sensor, denoted as z; 3) the scan position of the nth measurement, denoted as rn, where rn = (xn, yn). The nth image captured by the camera sensor can be represented as:

[0027]

[0028] where P(·) characterizes the propagation of the wave, and Sdown is responsible for the downsampling of the camera sensor pixels [15,16]. The wave propagation function P(·) is determined by convolving u(r, z) with the free-space point spread function implemented using the angle spectrum method (ASM)

[28] . Taking the first measurement value y0 as a reference, the nth measurement value yn can be represented by shifting the first measurement value rn − r0. The shift operation can be expressed as follows:

[0029]

[0030] Here, fr represents the frequency coordinates along the x-axis and y-axis, and F and F−1 represent the Fourier transform and its inverse transform, respectively. This forward model effectively simulates the propagation of the sample after translation to the sensor plane and downsampling through the pixelated sensor.

[0031] To optimize differentiability, u is obtained from N measurements with unknown z and unknown scan positions r = {r1, ..., rN}, minimizing the error metric, which is defined as:

[0032]

[0033] where N is the number of measurements, and r = {r1, ..., rN} is the set of scan positions. The data fidelity term ensures that the reconstructed image matches the measured data. The second term enhances the high-resolution features of the hologram by introducing the second-order spatial derivative. The l1 norm Rl1(·) promotes sparsity, which is helpful for autofocus, as the focused sample typically has a sparse representation. The total variational (TV) norm RTV(·) helps suppress experimental noise. Solving the inverse problem is challenging due to the nonlinearity and nonconvexity of the model. However, since the forward model defined in Equation (1) is differentiable with respect to the parameters {u, z, r}, the optimization problem in Equation (3) can be solved using gradient descent as described in Algorithm 1. Specification 4 / 9 pages 7 CN 122217955 A

[0034]

[0035] In the field of imaging, "differentiability" refers to the mathematical differentiability of the image representation or processing, thereby allowing the calculation of gradients. Gradients are crucial for optimization techniques such as gradient descent, and can essentially be used to fine-tune images by making small adjustments to neural networks or computational imaging processes based on the impact of pixel value changes on the expected results. Differentiable holography solves the mismatch between numerical models and real-world scenarios by introducing system defects into the imaging model and overcoming the complex inversion of the forward model using differentiable optimization techniques, and has proven its effectiveness in various applications. In particular, differentiable holography can reconstruct complex fields from single-shot coaxial holograms, achieving high-performance lensless holographic imaging (e.g., PSR) without additional phase retrieval measures, and can resolve imaging of denser volumetric particles.

[0036] When executing Algorithm 1, all parameters of the model are jointly optimized using gradient descent. This is implemented by a processor, which may have a parallel computing architecture. In each iteration, the gradient of the loss function is calculated by automatic differentiation, and the parameters are updated accordingly. In each iteration, the forward model of the imaging system (affected by the distance z from the sample to the sensor and the scanning position r) is also updated.

[0037] This optimization process continues until convergence. Unlike traditional methods, PSR holograms are not explicitly acquired and do not have a separate autofocus step. Instead, high-resolution images are generated by optimizing the error metric defined in Equation (1) in an end-to-end manner. This approach eliminates the error accumulation problem that often affects sequential steps in traditional techniques. Algorithm 1 can be implemented using PyTorch, but the invention is not limited to this and can also be implemented using any automatic differentiation framework (e.g., TensorFlow, JAX, or a custom gradient calculation system).

[0038] Through automatic differentiation, the processor simultaneously estimates the complex light field, relative translation parameters, and sample-to-sensor distance by minimizing a differentiable loss function that combines intensity fidelity and one or more flexible regularization terms. These one or more regularization terms include any differentiable function, learned prior information, or model-based constraints, including but not limited to Laplacian penalties, total variational penalties, sparsity penalties, or neural network-based penalties. This method eliminates explicit phase diversity, achieves subpixel-level registration and micrometer-level focusing accuracy, and enables resolution enhancement beyond the sensor's native pixel pitch while maintaining simple lensless hardware.

[0039] Numerical analysis is performed to illustrate the challenges of autofocus for lensless imaging. To more closely approximate the optical properties of real objects, the object's transmission function is defined as to = exp(−αmax·I) × exp[jϕmax·I], where I is a grayscale image normalized to the range of 0 to 1, αmax represents the maximum absorptivity, and ϕmax represents the absorptivity caused by the sample.Maximum phase delay. The test image is shown in Figure 2A.

[0040] Within the scope of the invention, the wavelength of the light source can be between 300 and 700 nm, and the pixel size is about 1 micrometer. However, in the exemplary measurements described below, the parameters selected are: the wavelength of the light source is 405 nm, and the pixel size of both the sample and the sensor camera is 1.1 µm. The distance between the sample and the sensor is set to 500 nm. Autofocus is performed using the established Laplace method

[29] , where the backpropagation distance ranges from 490 nm to 510 nm, covering the position of the sample at 500 nm. As shown in Figure 2B, for a purely absorbing sample, the detected sample position remains accurate regardless of the absorptivity. However, Figure 2C shows that for a sample with a pure phase delay of ϕ max, the detected sample position becomes increasingly inaccurate, where the observed inaccuracy increases as the phase delay increases from 0.5π to 2.0π. Figure 2D illustrates the autofocus performance for a sample with absorption and phase delay, where absorption is set to 0.1 to simulate label-free cells with low absorption. The accuracy of the detected position is lower compared to the pure phase sample in Figure 2C.

[0041] For online holographic autofocus, the twin problem complicates distinguishing between sharp and defocused images, where the conjugate reconstruction of the target acts as noise in the reconstruction. While higher absorption leads to a more pronounced amplitude distribution and sharper edges, aiding autofocus, phase-dominant samples tend to have less sharp edges, making accurate focusing difficult, especially with a large phase range. For position estimation, 25 scan positions were randomly generated for the sensor, as shown by the points (true positions) in Figure 2E. The “x” offset indicates the position estimated using a cross-correlation method. The mean absolute position estimation error was 0.1172 pixels, which is quite large for the accuracy required for PSR holographic synthesis. This analysis highlights the challenges involved in scan position estimation and autofocus in lensless imaging and underscores the need for more robust and accurate methods.

[0042] In the experimental setup, a monochrome CMOS sensor (Jiangsu Qunyi Intelligent Technology Co., Ltd.) was used, with a pixel size of 0.9 µm and a resolution of 5664 × 4256 pixels, thus providing a field of view (FOV) of 5.1 mm × 3.8 mm. The light source was a Thorlabs LP405C1 laser diode with collimator output and a center wavelength of 405 nm. The sensor was placed very close to the sample, less than 1 mm away, while the light source was placed approximately 15 cm away from the sample.

[0043] Using a pure phase USAF resolution target from Benchmark Optics, images were captured at different light source positions.The resolution performance of the device was evaluated using a sequence of images. Figure 3A shows the original image of a stack of 25 images from the USAF resolution test card, along with a magnified view of the central region (small square in the center) located in the lower left corner. Figure 3B shows a reconstructed image of the central region of Figure 3A. The differentiable pixel super-resolution (dPSR) method of the present invention achieves a resolution of 435 nm (group 10, element 2), which is 2.0 times higher than the pixel size of the sensor. Figure 3C compares the scan position estimated by the cross-correlation method and the dPSR method, where dPSR shows higher accuracy. Similarly, Figure 3D shows the accurate estimation of the distance from the sample to the sensor by dPSR. These results validate the effectiveness of dPSR in position estimation and resolution enhancement for lensless imaging.

[0044] The submicron resolution capability of the present invention was demonstrated by imaging unlabeled COS7 cells. Figure 4A shows the unprocessed original image, while Figure 4B shows the reconstruction of the image from 25 scans. The magnified reconstructed image in Figure 4C clearly shows the cell nucleus, demonstrating the system's ability to capture key cellular structures without labeling. This capability enables non-invasive cell analysis and opens up new possibilities for live-cell imaging research.

[0045] Therefore, the present invention provides a differentiable end-to-end lensless imaging technique, such as PSR, which integrates hologram synthesis, autofocus, and inverse image reconstruction into a unified framework. This method achieves high-resolution imaging with minimal measurements and eliminates the need for phase diversity in phase retrieval. Joint optimization of scan position, sample-to-sensor distance, and sample reconstruction enables flexible scanning strategies. The framework can potentially accommodate full three-dimensional scan motion, which can further enhance measurement diversity and imaging performance.

[0046] The above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications or substitutions that are obvious to those skilled in the art should fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0047] References

[0048] The references cited in this application are incorporated herein by reference in their entirety, as follows:

[0049] [1] E. McLeod and A. Ozcan, “Microscopy without lenses,” Phys. Today 70, 50–56 (2017).

[0050] [2] W. Bishara, T.-W. Su, AF Coskun, and A. Ozcan, “Lensfree on-chip microscopy over a wide field-of-viewusing pixel super‑resolution,” Opt. Express 18, 11181 (2010) .

[0051] [3] A . Greenbaum and A . Ozcan , “Maskless imaging of dense samples using pixel super‑resolution based multi‑height lensfree on‑chip microscopy,” Opt. Express 20, 3129 (2012) .

[0052] [4] C. J. Potter, Y. Hu, Z. Xiong , et al., “Point‑of‑care SARS‑CoV‑2 sensing using lens‑free imaging and a deep learning‑assisted quantitative agglutination assay,” Lab on Chip 22, 3744‑3754 (2022) .

[0053] [5] M . U . Daloglu , A . Ray , M . J . Collazo , et al ., “Low‑cost and portable UV holographic microscope for high‑contrast protein crystal imaging,” APL Photonics 4 (2019) .

[0054] [6] S . Jiang , C . Guo , Z . Bian , et al ., “Ptychographic sensor for large‑ scale lensless microbial monitoring with high spatiotemporal resolution,” Biosensors and Bioelectronics. 196, 113699 (2022) .

[0055] [7] Y. Wu and A. Ozcan, “Lensless digital holographic microscopy and its applications in biomedicine and environmental monitoring ,” Methods136, 4–16 (2018) .

[0056] [8] R. W. Gerchberg , “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–246 (1972) .

[0057] [9] J . R . Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21, 2758 (1982) .

[0058]

[10] W . Luo , Y . Zhang , A . Feizi , et al ., “Pixel super‑resolution using wave‑length scanning,” Light. Sci. Appl. 5, e16060 (2015) .

[0059]

[11] Z . Zhang , Y . Zhou , S . Jiang , et al ., “Invited article: Mask‑ modulated lensless imaging with multi‑angle illuminations,” APL Photonics 3, 060803 (2018) .

[0060]

[12] T .‑W . Su , S . O . Isikman , W . Bishara , et al ., “Multi‑angle lensless digital holography for depth resolved imaging on a chip ,” Opt . Express 18, 9690 (2010) .

[0061]

[13] Y . Guo , R . Guo , P. Qi , et al ., “Robust multi‑angle structured illumination lensless microscopy via illumination angle calibration ,” Opt. Lett. 47, 1847 (2022) .

[0062]

[14] .Zhou, J. Wu, J. Suo, et al., “Singleshot lensless imaging via simultaneous multi‑angle LED illumination,” Opt. Express 26, 21418 (2018) .

[0063]

[15] J. Zhang, J. Sun, Q. Chen, and C. Zuo, “Resolution analysis in a lens‑ free on‑chip digital holographic microscope ,” IEEE Trans . on Comput. Imaging 6, 697–710 (2020) .

[0064]

[16] Y. Gao and L. Cao, “Generalized optimization framework for pixel super‑resolution imaging in digital holography ,” Opt . Express 29 , 28805 Description Page 7 of 9 10 CN 122217955 A (2021) .

[0065]

[17] M . Lyu , C . Yuan , D . Li , and G . Situ , “Fast autofocusing in digital holography using the magnitude differential ,” Appl . Opt. 56 , F152 (2017) .

[0066]

[18] Z . Ren , N . Chen , and E . Y . Lam , “Automatic focusing for multisectional objects in digital holography using the structure tensor ,” Opt. Lett. 42, 1720 (2017) .

[0067]

[19] F . Dubois , C . Schockaert , N . Callens , and C . Yourassowsky , “Focus plane detection criteria in digital holography microscopy by amplitude analysis,” Opt. Express14, 5895 (2006) .

[0068]

[20] P . Langehanenberg , B . Kemper , D . Dirksen , and G . von Bally , “Autofocusing in digital holographic phase contrast microscopy on pure phase objects for live cell imaging,” Appl. Opt. 47, D176 (2008) .

[0069]

[21] J . Gillespie and R . King , “The use of self‑entropy as a focus measure in digital holography,” Pattern Recognit. Lett. 9, 19–25 (1989) .

[0070]

[22] W . Li , N . C . Loomis , Q . Hu , and C . S. Davis , “Focus detection from digital in‑line holograms based on spectral l1 norms,” J. Opt. Soc. Am. A 24, 3054 (2007) .

[0071]

[23] M . Trusiak , J .‑A . Picazo‑Bueno , P . Zdankowski , and V . Micó , “DarkFocus: numerical autofocusing in digital in‑line holographic microscopy using variance of computational dark‑field gradient,” Opt. Lasers Eng . 134, 106195 (2020) .

[0072]

[24] Y . Wu , L . Lu , J . Zhang , et al ., “Autofocusing Algorithm for pixel‑super‑resolved lensfree on‑chip microscopy,” Front. Phys. 9 (2021)

[0073]

[25] Y . Gao , F . Yang, and L. Cao, “Pixel super‑resolution phase retrieval for lensless on‑chip microscopy via accelerated Wirtinger flow,” Cells 11, 1999 (2022)

[0074]

[26] Y. Gao and L. Cao, “High‑fidelity pixel‑super‑resolved complex field reconstruction via adaptive smoothing,” Opt. Lett. 45, 6807 (2020).

[0075]

[27] J. Zhang, J. Sun, Q. Chen, et al., “Adaptive pixel‑super‑resolved lens‑ free in‑line digital holography for wide‑field on‑chip microscopy,” Sci. Reports 7 (2017).

[0076]

[28] J. W. Goodman, Introduction to Fourier Optics (W .H. Freeman, Macmillan Learning, New York, 2017), fourth edition ed. Hier auch später,” erschienene, unveränderte Nachdrucke.

[0077]

[29] S. Jiang, C. Guo, P. Song, et al., “Resolution-enhanced parallel coded ptychography for high-throughput optical imaging,” ACS Photonics 8, 3261–3271 (2021).

[0078] Although the invention has been explained in conjunction with some embodiments, it should be understood that, after reading the specification, the various modifications of each specification 8 / 9 pages 11 CN 122217955 A will become apparent to those skilled in the art. Therefore, it should be understood that the invention disclosed herein is intended to cover such modifications falling within the scope of the appended claims. Specification 9 / 9 pages 12 CN122217955 A Figure 1A Figure 1B Instruction Manual Appendix 1 / 7 Page 13 CN 122217955 A Figure 2A Figure 2B Instruction Manual Appendix 2 / 7 Page 14 CN 122217955 A Figure 2C Figure 2D Instruction Manual Appendix 3 / 7 Page 15 CN 122217955 A Figure 2E Figure 3A Instruction Manual Appendix 4 / 7 Page 16 CN 122217955 A Figure 3B Figure 3C Instruction Manual Appendix 5 / 7 Page 17 CN 122217955 A Figure 3D Figure 4A Instruction Manual Appendix 6 / 7 Page 18 CN 122217955 A Figure 4B Figure 4C Instruction Manual Appendix 7 / 7 Page 19 CN 122217955 A Abstract A differentiable lensless imaging system includes a light source and a sensor positioned to receive diffracted light from the light source. A platform for a sample is located between the light source and the sensor, so that the sensor receives and records the diffracted light from the sample on the platform. A computer receives the output recorded by the sensor and reconstructs a complete image of the target at a resolution higher than that of the sensor by integrating illumination control, autofocus, and complex-field sample reconstruction within a single differentiable optimization framework. Subpixel shifts are generated by translating the light source, sensor, or samplerelative to each other. The processor performs the reconstruction by executing a differentiable forward model of light propagation, and the automatic differentiation results in the simultaneous reconstruction of: (a) a complex field representing the sample, (b) the subpixel shift scanning position, and (c) the distance from the sample to the sensor.

Claims

1. A lensless imaging system, comprising: A light source that emits a beam of light; A sensor, positioned to receive and record the diffraction pattern of the light beam; A platform for a sample or target is located between the light source and the sensor, which receives the diffraction pattern of light passing through the sample from the light source, while the sample and the light are translated relative to each other with subpixel offsets. A processor receives a series of images from the sensor and is configured to reconstruct a complete holographic image of the target at a resolution higher than that of the sensor by automatic differentiation, resulting in the simultaneous reconstruction of (a) a complex field representing the sample, (b) a subpixel offset scan position, and (c) the distance of the sample from the sensor.

2. The system according to claim 1, wherein, All parameters are jointly optimized using gradient descent optimization, where, in each iteration, the gradient of the loss function is calculated and the parameters are updated accordingly, and in each iteration, both the distance z from the sample to the sensor and the scanning position r are also updated.

3. The system according to claim 1, wherein, Autofocus is achieved using the established Laplacian method.

4. The system according to claim 1, wherein, The light source is a laser mounted on a computer-controlled xy translation stage to translate and scan the sample, thereby presenting the iterations to a sensor in the form of a series of images of the sample at a series of angles. The sensor captures the intensity of light passing through the sample at each scanning position, and the translation stage and the sensor are synchronized by the computer.

5. The system according to claim 1, wherein, The light source is a two-dimensional LED array. Under the control of the processor, the two-dimensional LED array is lit in different ways so as to present a series of images of the sample at a series of angles to the sensor.

6. The system according to claim 1, wherein, The sensor is a camera.

7. The system according to claim 4, wherein, The wavelength of the light source is 300-700 nm, and the pixel size of the sample and sensor camera is approximately 1 µm.

8. The system according to claim 5, wherein, The distance between the sample and the sensor is set to approximately 500 nm.

9. The system according to claim 6, wherein, Autofocus is performed with a back-propagation distance in the range of approximately 490 nm to 510 nm, covering a position where the sample is approximately 500 nm away from the sensor.

10. The system according to claim 1, wherein, The processor executes a differentiable forward model of light propagation between the sample and the sensor, wherein sub-pixel offset differences are generated by translating at least one of the light source, the sensor, or the sample, and wherein the processor jointly estimates the complex light field of the sample, the distance from the sample to the sensor, and translation parameters by automatic differentiation of the forward model to minimize a loss function including a data fidelity term and one or more differentiable regularization terms.

11. The system according to claim 10, wherein, The differentiable forward model includes angular spectrum propagation kernel, Fresnel propagation kernel, or convolution propagation kernel.

12. The system according to claim 10, wherein, A light source array is a two-dimensional array of programmable light-emitting elements, thereby providing spatial or angular diversity.

13. The system according to claim 10, wherein, A light source scanning system translates a single light source between multiple positions relative to the sample using mechanical, optical, or electronic means.

14. The system according to claim 10, wherein, The one or more regularization terms include any differentiable function, learned prior information, or model-based constraints, including but not limited to Laplace penalty, total variation penalty, sparsity penalty, or neural network-based penalty.

15. The system according to claim 10, wherein, The processor uses an automatic differentiation framework implemented on the graphics processing unit to perform gradient-based optimization.

16. The system according to claim 1, wherein, Run the configuration in transmission mode, reflection mode, or scattering mode.

17. The system according to claim 10, wherein, The processor compensates for environmental disturbances or mechanical misalignment through differentiable optimization.

18. A method for differentiable lensless imaging, comprising: (a) Capturing multiple diffraction patterns of a sample during a process of subpixel shift caused by moving a light source or sample; (b) Simulate the propagation of light from the sample to the sensor in a differentiable computational model; (c) Define a loss function for the combination strength, consistency, and one or more differentiable regularization terms; (d) By automatically differentiating and simultaneously updating the reconstructed complex field, translation parameters and propagation distance to minimize loss, high-resolution complex images are produced without sequential phase diversity measurements.

19. The method according to claim 18, wherein, Translating the light source includes operating a light source array, scanning a light source, or at least one of the sensors.

20. The method according to claim 18, wherein, The loss function includes an adaptive edge enhancement term or a learned focus term.

21. The method according to claim 18, wherein, The reconstructed field is digitally refocused onto a different axial plane without the need for additional capture.

22. The method according to claim 18, wherein, Perform optimizations on parallel computing architectures.

23. The system according to claim 1, which realizes a differentiable pixel super-resolution dPSR lensless imaging system, wherein, The distance between the light source and the platform is much greater than the distance between the platform and the sensor. The sub-pixel offset creates an iteration of light diffusion patterns from the sample at a series of angles. The processor reconstructs a complete holographic image of the target by integrating differentiable PSR image hologram synthesis, autofocus, and complex field sample reconstruction and inverse problem solving algorithms into a single unified end-to-end framework, which represents a numerical forward model of a lensless system.

Images