A synthetic aperture quantitative phase imaging method and system based on physically driven deep learning

Abstract

The application discloses a synthetic aperture quantitative phase imaging method and system based on physical driving deep learning. The method uses a multi-angle illumination module to illuminate a sample from multiple perspectives to collect multiple low-resolution intensity images containing bright field and dark field components; the images are input into an untrained unsupervised deep convolutional neural network to output an estimated phase reconstruction image; a sample complex transmission function is constructed based on the phase image, and a partial coherent physical forward model is used to generate a predicted intensity image in combination with optical parameters; finally, network parameters are iteratively updated by minimizing a normalized weighted intensity loss function between the measured and predicted intensity images, and a final high-resolution reconstructed phase image is output. The application can significantly reduce the amount of data collection without any hardware modification, realize high-precision and high-resolution phase reconstruction, and is suitable for label-free biological microscopic imaging with small amount of data and high resolution requirement.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computational optical imaging technology, and particularly relates to synthetic aperture super-resolution imaging and deep learning technology in the field of microscopic imaging, especially a synthetic aperture quantitative phase imaging method and system based on physical-driven deep learning. Background Technology

[0002] The spatial resolution of traditional optical imaging systems is limited by the numerical aperture of the objective lens ( The limit is usually determined by the Fraunhofer diffraction limit, while improving... This typically implies a smaller field of view or a complex optical structure. Traditional differential phase-contrast (DPC) quantitative phase imaging techniques, limited by the numerical aperture of the objective lens, can only achieve [specific parameters]. The imaging resolution is [not specified]. To further improve resolution, one approach is to use synthetic aperture imaging technology, which, by synthesizing spectral information acquired from multiple angles, can surpass the resolution of a single lens. The resolution improvement involves acquiring the target's light field through multiple sub-apertures or multiple viewing angles and processing the signals to achieve a larger equivalent aperture than a single aperture. Early synthetic aperture imaging was applied in astronomy and synthetic aperture radar (SAR), achieving super-resolution imaging by interferometric synthesis of signals received from multiple sub-apertures on the detection plane. For example, Fourier stacked microscopy (FPM) acquires images through illumination from multiple angled variable light sources and synthesizes a large aperture in computation, achieving high-resolution imaging of far-field transparent samples and obtaining a larger equivalent aperture than the original. Significantly higher resolution. However, traditional synthetic aperture methods require acquiring a large number of images and performing precise alignment, resulting in significant data redundancy, high processing complexity, and susceptibility to light source or system errors, making rapid and real-time imaging difficult. Existing patents disclose synthetic aperture telescope systems based on incoherent light sources, achieving high-resolution imaging through multi-channel filters, moving platforms, etc., but they still face the problems of large data volume and slow imaging speed.

[0003] In recent years, deep learning methods have been increasingly applied to computational imaging, providing a new technical path for super-resolution imaging. Among these, supervised learning-based deep neural networks typically require the construction of large amounts of high-quality pairwise training data to reconstruct images by learning the mapping relationship between low-resolution and high-resolution images. However, the reconstruction performance of such methods is often affected by the scale and distribution of the training data, as well as the method of sample acquisition. When actual imaging conditions differ from the training data, the adaptability of the network model is limited.

[0004] To address the aforementioned issues, deep neural network reconstruction methods based on unsupervised learning have gradually gained attention. These methods utilize image statistical features contained within the neural network structure as prior constraints, achieving image restoration by combining imaging physics and actual observation data without relying on large-scale pre-collected labeled data. Related research shows that combining untrained deep network structures with optical imaging models can leverage the constraint relationship between observational data and physical models to achieve tasks such as image denoising, deblurring, phase retrieval, and super-resolution reconstruction. For example, some studies embed untrained neural networks into diffraction propagation models, optimizing network parameters to ensure the model output aligns with actual optical measurement results, thus achieving phase retrieval and image reconstruction based on physical constraints. Furthermore, existing techniques have proposed image denoising methods based on unsupervised generative networks, combining prior network structure with noise modeling to restore image information under actual noise conditions.

[0005] Therefore, combining unsupervised deep network methods with imaging physics models provides a feasible technical approach to reduce the dependence of traditional super-resolution imaging methods on large amounts of training data. However, existing methods are mainly applied to general image restoration tasks. In the field of synthetic aperture super-resolution imaging, there is still a lack of imaging methods that combine multi-angle illumination information, partially coherent imaging models, and unsupervised network optimization mechanisms, making it difficult to achieve effective spectral expansion and high-resolution reconstruction while reducing data acquisition requirements.

[0006] In summary, how to fully utilize the distribution characteristics of axial diffraction information in the spatial frequency domain to overcome the spatial resolution limitations of traditional optical imaging systems and maintain reconstruction accuracy while significantly reducing data acquisition volume remains a key technical problem that urgently needs to be solved in the field of computational optical imaging.

[0007] There is still a lack of a synthetic aperture super-resolution imaging method that is data-efficient, requires no training data, and can significantly improve imaging resolution. Summary of the Invention

[0008] The purpose of this invention is to address the problems existing in the prior art by providing a physical-driven deep learning-based synthetic aperture quantitative phase imaging method for achieving high-fidelity super-resolution image reconstruction with only a small amount of multi-view acquisition data. This invention, based on a deep image prior framework, combines a partially coherent physical model with a neural network structure to achieve equivalent larger aperture imaging with minimal image acquisition, thus resolving the contradiction between resolution and acquisition efficiency in traditional methods.

[0009] The technical solution for achieving the objective of this invention is: a synthetic aperture quantitative phase imaging method based on physics-driven deep learning, the method comprising:

[0010] Step 1: Use a multi-angle illumination module to illuminate the sample from multiple perspectives and acquire multiple low-resolution intensity images from multiple perspectives, including bright field components and dark field components.

[0011] Step 2: Input the acquired multiple low-resolution intensity images from multiple viewpoints into an untrained unsupervised deep convolutional neural network, and the neural network outputs an estimated phase reconstruction image.

[0012] Step 3: Construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system.

[0013] Step 4: Solve the constructed intensity loss function using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images. Backpropagate the error of the intensity loss function to the neural network to update its network parameters. After the neural network is updated, continue to output a new estimated phase reconstruction image.

[0014] Step 5: Iterate through steps 3 and 4 until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

[0015] Further, step 1 specifically includes: using a programmable illumination module, sequentially subjecting the sample to two orthogonal semi-annular differential phase contrast illuminations matched with the objective lens numerical aperture, and a high-angle annular dark-field illumination with an illumination range extended to twice the objective lens numerical aperture, thereby simultaneously acquiring two frames of DPC intensity maps and one frame of annular dark-field intensity map as the multiple multi-view low-resolution intensity images; wherein, the orthogonal semi-annular differential phase contrast illumination is denoted as orthogonal semi-annular DPC illumination, including orthogonally distributed first semi-annular DPC illumination and second semi-annular DPC illumination.

[0016] Furthermore, the radii of the orthogonal semi-annular DPC illumination and the high-angle annular dark-field illumination are adaptively calculated by measuring the vertical distance from the illumination source to the sample stage, and their calculation formulas are expressed as follows:

[0017]

[0018]

[0019] In the formula, It is the numerical aperture of DPC illumination. It is the numerical aperture for dark field illumination. The numerical aperture of the objective lens. It is the distance between the lighting source and the sample stage. It is the distance from the illumination source to the center of the optical axis of the imaging system;

[0020] The illumination functions for the two orthogonal semi-circular DPC intensity maps and one circular dark field intensity map acquired are expressed as follows:

[0021]

[0022]

[0023]

[0024] in, It is the radial width of the lighting ring. and The parameter is used to control the ring width of the lighting, and 0 < <1< <2, , These represent the illumination functions used to obtain the intensity maps of two orthogonal semi-circular DPC frames, respectively. This represents the illumination function used to obtain a frame of annular dark field intensity map; The two-dimensional spatial frequency coordinates of the light source plane are represented by, where and These represent the spatial frequency components along the x-axis and y-axis, respectively.

[0025] Furthermore, in step 1, a programmable LED light source array is used as the illumination source. The programmable LED light source array is adaptively controlled by a preset control timing to sequentially switch on the corresponding orthogonal semi-circular illumination area and the annular dark field illumination area, and a trigger signal is sent to the camera simultaneously to stably acquire images of the sample under illumination at different angles.

[0026] Furthermore, in step 2, the unsupervised deep convolutional neural network adopts an encoder-decoder architecture with skip connections, wherein the encoder is used to extract multi-scale spatial feature information of the input image, the decoder is used to recover the spatial distribution layer by layer and output the estimated phase reconstruction image, and the skip connections are used to retain and transmit fine structural information at different feature scales.

[0027] Furthermore, step 2 also includes the following before image input:

[0028] Set network optimization hyperparameters, which include at least the learning rate, maximum number of iterations, loss function weight coefficients, regularization coefficients, and the number of network input channels;

[0029] The multiple intensity images acquired in step 1 are used as parallel inputs to different input channels of the unsupervised deep convolutional neural network.

[0030] Furthermore, the partially coherent physical forward model in step 3 is constructed based on the incoherent overlay framework of coherent images, and its formula for calculating the predicted intensity image generated for each illumination mode is expressed as follows:

[0031]

[0032] In the formula, It is the complex transmission function of the sample. For the absorption of the sample, For the phase of the sample, It is a light source. The cutoff frequency is The pupil function, Represents the coordinates on the camera plane. Represents the frequency domain coordinates on the light source plane. Represents the frequency domain coordinates at the pupil plane. Characterizing the absorption of the sample, These are spatial coordinates; This represents the intensity map obtained at the camera location;

[0033] Among them, the lighting sources corresponding to orthogonal semi-circular DPC lighting and circular dark field lighting They are represented as follows:

[0034]

[0035]

[0036]

[0037] In the formula, , These are the lighting sources corresponding to the first orthogonal semi-circular DPC lighting and the second orthogonal semi-circular DPC lighting, respectively. The lighting source corresponding to the annular dark field illumination; , These represent the illumination functions used to obtain the intensity maps of two orthogonal semi-circular DPC frames, respectively. This represents the illumination function used to obtain a frame of annular dark field intensity map.

[0038] Furthermore, the intensity loss function constructed in step 4 The normalized weighted combination of the orthogonal semi-ring DPC illumination component and the dark field illumination component is expressed as:

[0039]

[0040] In the formula, This represents the loss function between the acquired intensity image and the predicted intensity image under the first semi-circular DPC illumination condition. This represents the first semi-circular DPC illumination intensity loss function calculated during the first iteration. This represents the loss function between the acquired intensity image and the predicted intensity image under the second semi-circular DPC illumination condition. This represents the second semi-circular DPC illumination intensity loss function calculated during the first iteration. This represents the loss function between the acquired intensity image and the predicted intensity image under dark lighting conditions. This represents the dark field illumination intensity loss function calculated during the first iteration. , and The parameters are used to control the weights between the DPC lighting intensity loss function and the dark field lighting intensity loss function, and the weights satisfy the following conditions: ,and , , , This represents a regularization term used to suppress unstable high-frequency oscillations.

[0041] Furthermore, the aforementioned , , All use L1 reconstruction loss, specifically expressed as:

[0042]

[0043] In the formula, This represents the total number of pixels in the tensor. For L1 reconstruction loss function, This is the actual intensity map collected. For the predicted intensity map, i represents the i-th pixel in the tensor;

[0044] The regularization term Represented as:

[0045]

[0046] in, The regularization coefficient is . To output the pixel value in the i-th row and j+1-th column of the phase, To output the pixel value in the i-th row and j-th column of the phase, This is to output the pixel value in the (i+1)th row and jth column of the phase.

[0047] Furthermore, in step 5, in each iteration cycle, the frequency domain overlap integral calculation of the partially coherent physical forward model and the gradient backpropagation process of the neural network are accelerated in parallel using the GPU parallel computing architecture to improve the convergence rate of adaptive update of network parameters.

[0048] On the other hand, a synthetic aperture quantitative phase imaging system based on physics-driven deep learning is provided, implemented based on the physics-driven deep learning-based synthetic aperture quantitative phase imaging method, the system comprising the following sequentially executed steps:

[0049] The first module is used to: illuminate the sample from multiple angles using the multi-angle illumination module, and acquire multiple low-resolution intensity images from multiple angles containing bright field components and dark field components.

[0050] The second module is used to: input the acquired multiple low-resolution intensity images from multiple perspectives into an untrained unsupervised deep convolutional neural network, and output an estimated phase reconstruction image from the neural network;

[0051] The third module is used to: construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system.

[0052] The fourth module is used to: solve the constructed intensity loss function by using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images, backpropagate the error of the intensity loss function to the neural network to update its network parameters, and continue to output a new estimated phase reconstruction image after the neural network is updated;

[0053] The fifth module is used to: iteratively execute the third and fourth modules until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

[0054] Compared with the prior art, the significant advantages of this invention are:

[0055] (1) This method effectively integrates physical imaging models and untrained deep network priors, overcomes the data and speed bottlenecks of traditional synthetic aperture imaging, and improves imaging efficiency and image quality.

[0056] (2) This method utilizes the numerical aperture of the objective lens ( Matched orthogonal semi-circular illumination and annular high-angle dark-field illumination with twice the objective lens numerical aperture, by synthesizing information from multiple angles, expands the spatial spectral coverage, equivalent to having a larger numerical aperture (up to This method significantly breaks through the traditional partial coherence diffraction limit, achieving high-resolution quantitative phase reconstruction. In USAF resolution target simulations, experimental results show that only three images are needed to significantly improve the resolvable spatial frequency, surpassing the traditional differential phase contrast (DPC) method.

[0057] (3) Compared with the Fourier layered microscopy (FPM) method, which requires the acquisition of dozens or even a large number of images, this method only requires the acquisition of a small number of images (2 DPC intensity maps and 1 dark field intensity map), significantly reducing the amount of data acquisition. Therefore, this method is suitable for label-free biological microscopy imaging in situations with limited data, high resolution requirements, dynamic scenes, or weak signal scenes.

[0058] (4) This method adopts an unsupervised learning mechanism, using the image statistical features implicit in the deep network structure as prior constraints. The network parameters do not depend on large-scale pairwise training data, but are gradually and adaptively updated by fitting the difference between the actual measured values ​​and the predicted values ​​of the physical model. This solves the problem that traditional supervised deep learning requires massive training samples and has a wider range of applications.

[0059] (5) By closely integrating the unsupervised neural network optimization process with the physical model of partially coherent imaging, the phase output of the network is subject to the physical simulation and constraint of the forward model. This optimization process based on physical model constraints can ensure high robustness and stability, and can more accurately recover target details, reducing phase recovery deviations caused by noise, measurement errors and limited spectral coverage.

[0060] (6) Thanks to the good adaptability of deep network priors to natural images, the implicit regularization of the network can effectively suppress unstable high-frequency oscillations, local noise and artifacts during the iterative process of phase retrieval, so that the recovered high-resolution phase image retains richer details and complete phase information.

[0061] (7) Although this method includes an iterative optimization process, its forward model calculation and network parameter iterative update process can be accelerated by fully utilizing GPU parallel computing, thereby significantly improving the calculation speed while ensuring that the loss function error tends to stabilize and converges quickly within an acceptable time.

[0062] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0063] Figure 1 This is a flowchart of a synthetic aperture quantitative phase imaging method based on physics-driven deep learning in one embodiment.

[0064] Figure 2 This is a network framework diagram of a physical-driven deep learning-based synthetic aperture quantitative phase imaging method implemented in one embodiment.

[0065] Figure 3 This is a schematic diagram comparing the simulated resolution plate model in this invention with existing methods in tomographic imaging, as shown in one embodiment. Figure 3 (a) in the image shows the intensity maps that need to be collected using the three methods. Figure 3 (b) shows the final spectral results achieved by the three methods. Figure 3 (c) shows the phase reconstruction results of the three methods. Figure 3 (d) in the figure shows the results of the comparison of the high-frequency region resolution of phase reconstruction using the three methods. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0068] In one embodiment, combined Figure 1 and Figure 2 A method for quantitative phase imaging of synthetic aperture based on physics-driven deep learning is provided, the method comprising:

[0069] Step 1: Use a multi-angle illumination module to illuminate the sample from multiple perspectives and acquire multiple low-resolution intensity images from multiple perspectives, including bright field components and dark field components.

[0070] Step 2: Input the acquired multiple low-resolution intensity images from multiple viewpoints into an untrained unsupervised deep convolutional neural network, and the neural network outputs an estimated phase reconstruction image.

[0071] Step 3: Construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system.

[0072] Step 4: Solve the constructed intensity loss function using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images. Backpropagate the error of the intensity loss function to the neural network to update its network parameters. After the neural network is updated, continue to output a new estimated phase reconstruction image.

[0073] Step 5: Iterate through steps 3 and 4 until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

[0074] Further, in one embodiment, step 1 specifically includes: using a programmable illumination module, sequentially applying two orthogonal semi-annular differential phase contrast illuminations matching the numerical aperture of the objective lens and a high-angle annular dark-field illumination with an illumination range extended to twice the numerical aperture of the objective lens to the sample, thereby simultaneously acquiring two frames of DPC intensity maps and one frame of annular dark-field intensity map as the multiple multi-view low-resolution intensity images; wherein, the orthogonal semi-annular differential phase contrast illumination is denoted as orthogonal semi-annular DPC illumination, including orthogonally distributed first semi-annular DPC illumination and second semi-annular DPC illumination.

[0075] Furthermore, in one embodiment, the radii of the orthogonal semi-annular DPC illumination and the high-angle annular dark-field illumination are adaptively calculated by measuring the vertical distance from the illumination source to the sample stage, and the calculation formulas are expressed as follows:

[0076]

[0077]

[0078] In the formula, It is the numerical aperture of DPC illumination. It is the numerical aperture for dark field illumination. The numerical aperture of the objective lens. It is the distance between the lighting source and the sample stage. It is the distance from the illumination source to the center of the optical axis of the imaging system;

[0079] The illumination functions for the two orthogonal semi-circular DPC intensity maps (e.g., the upper and left semi-circular rings) and the one-frame annular dark field intensity map are expressed as follows:

[0080]

[0081]

[0082]

[0083] in, It is the radial width of the lighting ring. and The parameter is used to control the ring width of the lighting, and 0 < <1< <2, , These represent the illumination functions used to obtain the intensity maps of two orthogonal semi-circular DPC frames, respectively. This represents the illumination function used to obtain a frame of annular dark field intensity map; The two-dimensional spatial frequency coordinates of the light source plane are represented by, where and These represent the spatial frequency components along the x-axis and y-axis, respectively.

[0084] Furthermore, in one embodiment, in step 1, a programmable LED light source array is used as the illumination source. The programmable LED light source array is adaptively controlled by a preset control timing to sequentially switch on the corresponding orthogonal semi-circular illumination area and the annular dark field illumination area, and a trigger signal is sent to the camera simultaneously to stably acquire images of the sample under illumination at different angles.

[0085] Furthermore, in one embodiment, the unsupervised deep convolutional neural network in step 2 adopts an encoder-decoder architecture with skip connections, wherein the encoder is used to extract multi-scale spatial feature information of the input image, the decoder is used to recover the spatial distribution layer by layer and output the estimated phase reconstruction image, and the skip connections are used to retain and pass fine structural information at different feature scales.

[0086] Here, the network parameters do not depend on large-scale training data; instead, they are progressively updated by fitting the difference between actual measurements and current physical forward model predictions. This approach utilizes depth image priors to implicitly constrain the distribution of the phase to be recovered, enabling the network to reconstruct phase information even in the absence of pre-trained samples. By combining the neural network optimization process with a partially coherent imaging physical model, network parameters and reconstruction results can be updated synchronously during iteration, reducing phase recovery biases caused by noise, measurement errors, and limited spectral coverage. Simultaneously, the optimization process based on physical model constraints maintains consistency between the reconstruction results and the actual optical propagation process. This method avoids the dependence of traditional supervised learning methods on large labeled datasets, improving its applicability to different samples and imaging conditions.

[0087] Furthermore, in one embodiment, step 2 further includes the following before image input:

[0088] Set network optimization hyperparameters, which include at least the learning rate, maximum number of iterations, loss function weight coefficients, regularization coefficients, and the number of network input channels;

[0089] The multiple intensity images acquired in step 1 are used as parallel inputs to different input channels of the unsupervised deep convolutional neural network. Here, the network encoder extracts multi-scale feature information, and the decoder recovers the spatial distribution of the image layer by layer, finally outputting a two-dimensional phase distribution corresponding to the sample under test. The output phase map is updated with network parameters in subsequent iterations based on the error between the actual measurement data and the calculation results of the forward physics model.

[0090] Preferably, in some embodiments, in step 3, parameters corresponding to the forward model are set according to the actual system parameters, including wavelength, objective magnification, and numerical aperture. Three corresponding intensity maps are calculated using the forward model formula through different illumination functions; these are the DPC intensity maps. and and dark field intensity map .

[0091] Furthermore, in one embodiment, the partially coherent physical forward model in step 3 is constructed based on the incoherent overlay framework of coherent images, and its formula for calculating the predicted intensity image generated for each illumination mode is expressed as:

[0092]

[0093] In the formula, It is the complex transmission function of the sample. For the absorption of the sample, For the phase of the sample, It is a light source. The cutoff frequency is The pupil function, Represents the coordinates on the camera plane. Represents the frequency domain coordinates on the light source plane. Represents the frequency domain coordinates at the pupil plane. Characterizing the absorption of the sample, These are spatial coordinates; This represents the intensity map obtained at the camera location;

[0094] Among them, the lighting sources corresponding to orthogonal semi-circular DPC lighting (e.g., but not limited to the upper half-ring and the left half-ring) and annular dark field lighting. They are represented as follows:

[0095]

[0096]

[0097]

[0098] In the formula, , These are the lighting sources corresponding to the first orthogonal semi-circular DPC lighting and the second orthogonal semi-circular DPC lighting, respectively. The lighting source corresponding to the annular dark field illumination; , These represent the illumination functions used to obtain the intensity maps of two orthogonal semi-circular DPC frames, respectively. This represents the illumination function used to obtain a frame of annular dark field intensity map.

[0099] Furthermore, in one embodiment, the intensity loss function constructed in step 4... The normalized weighted combination of the orthogonal semi-ring DPC illumination component and the dark field illumination component is expressed as:

[0100]

[0101] In the formula, This represents the loss function between the acquired intensity image and the predicted intensity image under the first semi-circular DPC illumination condition. This represents the first semi-circular DPC illumination intensity loss function calculated during the first iteration. This represents the loss function between the acquired intensity image and the predicted intensity image under the second semi-circular DPC illumination condition. This represents the second semi-circular DPC illumination intensity loss function calculated during the first iteration. This represents the loss function between the acquired intensity image and the predicted intensity image under dark lighting conditions. This represents the dark field illumination intensity loss function calculated during the first iteration. , and The parameters are used to control the weights between the DPC lighting intensity loss function and the dark field lighting intensity loss function, and the weights satisfy the following conditions: ,and , , , This represents a regularization term used to suppress unstable high-frequency oscillations.

[0102] Preferably, in some embodiments, the , , All use L1 reconstruction loss, specifically expressed as:

[0103]

[0104] In the formula, This represents the total number of pixels in the tensor. For L1 reconstruction loss function, This is the actual intensity map collected. For the predicted intensity map, i represents the i-th pixel in the tensor;

[0105] The regularization term Represented as:

[0106]

[0107] in, The regularization coefficient is . To output the pixel value in the i-th row and j+1-th column of the phase, To output the pixel value in the i-th row and j-th column of the phase, This is the pixel value in the (i+1)th row and jth column of the output phase. The regularization weight is relatively small, and its main function is to suppress local noise and artifacts in the output image without significantly sacrificing L1 reconstruction accuracy.

[0108] Furthermore, in one embodiment, in step 5, in each iteration cycle, the frequency domain overlap integral calculation of the partially coherent physical forward model and the gradient backpropagation process of the neural network are accelerated in parallel using a GPU parallel computing architecture to improve the convergence rate of adaptive update of network parameters.

[0109] Preferably, in some embodiments, the network output phase in step 5 is constrained by the forward model to generate a predicted intensity map, minimize the loss function between the predicted intensity map and the true intensity map, and adaptively update the network parameters in each iteration until the loss function value tends to a stable value, at which point the network outputs a high-resolution phase.

[0110] In one embodiment, a synthetic aperture quantitative phase imaging system based on physics-driven deep learning is provided, implemented based on the physics-driven deep learning-based synthetic aperture quantitative phase imaging method, the system comprising the following sequentially executed steps:

[0111] The first module is used to: illuminate the sample from multiple angles using the multi-angle illumination module, and acquire multiple low-resolution intensity images from multiple angles containing bright field components and dark field components.

[0112] The second module is used to: input the acquired multiple low-resolution intensity images from multiple perspectives into an untrained unsupervised deep convolutional neural network, and output an estimated phase reconstruction image from the neural network;

[0113] The third module is used to: construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system.

[0114] The fourth module is used to: solve the constructed intensity loss function by using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images, backpropagate the error of the intensity loss function to the neural network to update its network parameters, and continue to output a new estimated phase reconstruction image after the neural network is updated;

[0115] The fifth module is used to: iteratively execute the third and fourth modules until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

[0116] Specific limitations regarding the physics-driven deep learning-based synthetic aperture quantitative phase imaging system can be found in the limitations of the physics-driven deep learning-based synthetic aperture quantitative phase imaging method described above, and will not be repeated here. Each module in the aforementioned physics-driven deep learning-based synthetic aperture quantitative phase imaging system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0117] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:

[0118] Step 1: Use a multi-angle illumination module to illuminate the sample from multiple perspectives and acquire multiple low-resolution intensity images from multiple perspectives, including bright field components and dark field components.

[0119] Step 2: Input the acquired multiple low-resolution intensity images from multiple viewpoints into an untrained unsupervised deep convolutional neural network, and the neural network outputs an estimated phase reconstruction image.

[0120] Step 3: Construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system.

[0121] Step 4: Solve the constructed intensity loss function using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images. Backpropagate the error of the intensity loss function to the neural network to update its network parameters. After the neural network is updated, continue to output a new estimated phase reconstruction image.

[0122] Step 5: Iterate through steps 3 and 4 until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

[0123] For specific limitations on each step, please refer to the limitations of the synthetic aperture quantitative phase imaging method based on physics-driven deep learning mentioned above, which will not be repeated here.

[0124] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:

[0125] Step 1: Use a multi-angle illumination module to illuminate the sample from multiple perspectives and acquire multiple low-resolution intensity images from multiple perspectives, including bright field components and dark field components.

[0126] Step 2: Input the acquired multiple low-resolution intensity images from multiple viewpoints into an untrained unsupervised deep convolutional neural network, and the neural network outputs an estimated phase reconstruction image.

[0127] Step 3: Construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system.

[0128] Step 4: Solve the constructed intensity loss function using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images. Backpropagate the error of the intensity loss function to the neural network to update its network parameters. After the neural network is updated, continue to output a new estimated phase reconstruction image.

[0129] Step 5: Iterate through steps 3 and 4 until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

[0130] For specific limitations on each step, please refer to the limitations of the synthetic aperture quantitative phase imaging method based on physics-driven deep learning mentioned above, which will not be repeated here.

[0131] As a specific example, in one embodiment, the effectiveness of the method proposed in this invention is verified through a series of simulation experiments. The experiments used a 10x objective lens with a numerical aperture of 0.3 to conduct imaging studies on a USAF resolution plate. Three sets of reconstruction results are shown below. Figure 3As shown in the diagram. Comparative analysis reveals that when reconstructing the resolution panel image using four intensity maps acquired via DPC, the resolution is limited, only clearly identifying 10⁻³ of the resolution panel information. While reconstructing using 2207 intensity maps acquired via FPM can broaden the image frequency domain through aperture synthesis, effectively improving imaging resolution, the large amount of data required reduces imaging efficiency. However, the method proposed in this invention only requires acquiring three sets of intensity maps for reconstruction, achieving accurate reconstruction of both low- and high-frequency information of the resolution panel, reaching a resolution level equivalent to FPM.

[0132] In summary, this method, by employing a method similar to the numerical aperture of the objective lens (…), Three intensity images are acquired using matched orthogonal ring illumination and high-angle dark-field illumination with a 2x objective lens numerical aperture. These three intensity images are then input into an untrained deep convolutional neural network (DCNN), with the network architecture itself serving as an implicit image prior. The network outputs an estimated reconstructed phase. Subsequently, a physical forward model of partially coherent imaging is embedded into the network, and a predicted intensity map is generated based on the estimated reconstructed phase through the forward model. Finally, the network parameters are optimized by minimizing the intensity loss function to reconstruct a high-resolution phase, achieving stable phase reconstruction without relying on any external training data. By synergistically fusing bright-field and dark-field information, this method breaks through the traditional limitations of partially coherent imaging. Due to frequency limitations, only 3 images are needed to achieve the equivalent effect. Synthetic aperture quantitative phase imaging effectively improves spatial resolution.

[0133] In summary, this invention significantly reduces the amount of data acquisition while ensuring imaging resolution and reconstruction accuracy, and improves the imaging efficiency of synthetic aperture super-resolution imaging methods, making it suitable for high-throughput super-resolution imaging of label-free biological samples.

[0134] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A synthetic aperture quantitative phase imaging method based on physics-driven deep learning, characterized in that, The method includes: Step 1: Use a multi-angle illumination module to illuminate the sample from multiple perspectives and acquire multiple low-resolution intensity images from multiple perspectives, including bright field components and dark field components. Step 2: Input the acquired multiple low-resolution intensity images from multiple viewpoints into an untrained unsupervised deep convolutional neural network, and the neural network outputs an estimated phase reconstruction image. Step 3: Construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system. Step 4: Solve the constructed intensity loss function using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images. Backpropagate the error of the intensity loss function to the neural network to update its network parameters. After the neural network is updated, continue to output a new estimated phase reconstruction image. Step 5: Iterate through steps 3 and 4 until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

2. The synthetic aperture quantitative phase imaging method based on physics-driven deep learning according to claim 1, characterized in that, Step 1 specifically includes: using a programmable illumination module, sequentially applying two orthogonal semi-annular differential phase contrast illuminations matching the numerical aperture of the objective lens, and a high-angle annular dark-field illumination with the illumination range extended to twice the numerical aperture of the objective lens, thereby simultaneously acquiring two frames of DPC intensity maps and one frame of annular dark-field intensity map as the multiple multi-view low-resolution intensity images; wherein, the orthogonal semi-annular differential phase contrast illumination is denoted as orthogonal semi-annular DPC illumination, including orthogonally distributed first semi-annular DPC illumination and second semi-annular DPC illumination.

3. The synthetic aperture quantitative phase imaging method based on physics-driven deep learning according to claim 2, characterized in that, The radii of the orthogonal semi-annular DPC illumination and the high-angle annular dark-field illumination are adaptively calculated by measuring the vertical distance from the illumination source to the sample stage, and their calculation formulas are expressed as follows: In the formula, It is the numerical aperture of DPC illumination. It is the numerical aperture for dark field illumination. The numerical aperture of the objective lens. It is the distance between the lighting source and the sample stage. It is the distance from the illumination source to the center of the optical axis of the imaging system; The illumination functions for the two orthogonal semi-circular DPC intensity maps and one circular dark field intensity map acquired are expressed as follows: in, It is the radial width of the lighting ring. and The parameter is used to control the ring width of the lighting, and 0 < <1< <2, , These represent the illumination functions used to obtain the intensity maps of two orthogonal semi-circular DPC frames, respectively. This represents the illumination function used to obtain a frame of annular dark field intensity map; The two-dimensional spatial frequency coordinates of the light source plane are represented by, where and These represent the spatial frequency components along the x-axis and y-axis, respectively.

4. The synthetic aperture quantitative phase imaging method based on physics-driven deep learning according to claim 3, characterized in that, In step 1, a programmable LED light source array is used as the illumination source. The programmable LED light source array is adaptively controlled by a preset control timing to sequentially switch on the corresponding orthogonal semi-circular illumination area and annular dark field illumination area, and a trigger signal is sent to the camera at the same time to stably acquire images of the sample under illumination at different angles.

5. The synthetic aperture quantitative phase imaging method based on physical-driven deep learning according to claim 1, characterized in that, In step 2, the unsupervised deep convolutional neural network adopts an encoder-decoder architecture with skip connections, wherein the encoder is used to extract multi-scale spatial feature information of the input image, the decoder is used to recover the spatial distribution layer by layer and output the estimated phase reconstruction image, and the skip connections are used to retain and transmit fine structural information at different feature scales.

6. The synthetic aperture quantitative phase imaging method based on physics-driven deep learning according to claim 1, characterized in that, Step 2, before image input, also includes: Set network optimization hyperparameters, which include at least the learning rate, maximum number of iterations, loss function weight coefficients, regularization coefficients, and the number of network input channels; The multiple intensity images acquired in step 1 are used as parallel inputs to different input channels of the unsupervised deep convolutional neural network.

7. The synthetic aperture quantitative phase imaging method based on physics-driven deep learning according to claim 2, characterized in that, The partially coherent physical forward model in step 3 is constructed based on the incoherent overlay framework of coherent images. The formula for calculating the predicted intensity image generated for each illumination mode is expressed as follows: In the formula, It is the complex transmission function of the sample. For the absorption of the sample, For the phase of the sample, It is a light source. The cutoff frequency is The pupil function, Represents the coordinates on the camera plane. Represents the frequency domain coordinates on the light source plane. Represents the frequency domain coordinates at the pupil plane. Characterizing the absorption of the sample, These are spatial coordinates; This represents the intensity map obtained at the camera location; Among them, the lighting sources corresponding to orthogonal semi-circular DPC lighting and circular dark field lighting They are represented as follows: In the formula, , These are the lighting sources corresponding to the first orthogonal semi-circular DPC lighting and the second orthogonal semi-circular DPC lighting, respectively. The lighting source corresponding to the annular dark field illumination; , These represent the illumination functions used to obtain the intensity maps of two orthogonal semi-circular DPC frames, respectively. This represents the illumination function used to obtain a frame of annular dark field intensity map.

8. The synthetic aperture quantitative phase imaging method based on physical-driven deep learning according to claim 2, characterized in that, The intensity loss function constructed in step 4 The normalized weighted combination of the orthogonal semi-ring DPC illumination component and the dark field illumination component is expressed as: In the formula, This represents the loss function between the acquired intensity image and the predicted intensity image under the first semi-circular DPC illumination condition. This represents the first semi-circular DPC illumination intensity loss function calculated during the first iteration. This represents the loss function between the acquired intensity image and the predicted intensity image under the second semi-circular DPC illumination condition. This represents the second semi-circular DPC illumination intensity loss function calculated during the first iteration. This represents the loss function between the acquired intensity image and the predicted intensity image under dark lighting conditions. This represents the dark field illumination intensity loss function calculated during the first iteration. , and The parameters are used to control the weights between the DPC lighting intensity loss function and the dark field lighting intensity loss function, and the weights satisfy the following conditions: ,and , , , This represents a regularization term used to suppress unstable high-frequency oscillations. The , , All use L1 reconstruction loss, specifically expressed as: In the formula, This represents the total number of pixels in the tensor. For L1 reconstruction loss function, This is the actual intensity map collected. For the predicted intensity map, i represents the i-th pixel in the tensor; The regularization term Represented as: in, The regularization coefficient is . To output the pixel value in the i-th row and j+1-th column of the phase, To output the pixel value in the i-th row and j-th column of the phase, This is to output the pixel value in the (i+1)th row and jth column of the phase.

9. The synthetic aperture quantitative phase imaging method based on physics-driven deep learning according to claim 1, characterized in that, In step 5, in each iteration cycle, the GPU parallel computing architecture is used to accelerate the frequency domain overlap integral calculation of the partially coherent physical forward model and the gradient backpropagation process of the neural network in parallel, so as to improve the convergence rate of adaptive update of network parameters.

10. A synthetic aperture quantitative phase imaging system based on physics-driven deep learning, implemented according to the method of any one of claims 1 to 9, characterized in that, The system includes sequential execution of: The first module is used to: illuminate the sample from multiple angles using the multi-angle illumination module, and acquire multiple low-resolution intensity images from multiple angles containing bright field components and dark field components. The second module is used to: input the acquired multiple low-resolution intensity images from multiple perspectives into an untrained unsupervised deep convolutional neural network, and output an estimated phase reconstruction image from the neural network; The third module is used to: construct the complex transmission function of the sample based on the estimated phase reconstruction image, and generate the corresponding predicted multi-view intensity image using a partially coherent physical forward model according to the actual optical parameters of the imaging system. The fourth module is used to: solve the constructed intensity loss function by using the acquired multiple multi-view low-resolution intensity images and the generated predicted multi-view intensity images, backpropagate the error of the intensity loss function to the neural network to update its network parameters, and continue to output a new estimated phase reconstruction image after the neural network is updated; The fifth module is used to: iteratively execute the third and fourth modules until the error of the intensity loss function converges to a preset stable state, then end the loop and output the final high-resolution reconstructed phase image.

Images