Reconstructing Phase Images Using Deep Learning

Abstract

Aspects relate to reconstructing a phase image from brightfield images at multiple focal planes using machine learning techniques. A machine learning model can be trained using a training data set that includes sets of matched images, each set of matched images including multiple brightfield images at different focal planes and optionally corresponding ground truth phase images. An initial training data set can include images selected based on views of the sample that are substantially free of unwanted visual artifacts such as dust. The brightfield images of the training data set can then be modified based on simulating at least one visual artifact, thereby generating an enhanced training data set for training the model. The output of the machine learning model can be compared to the ground truth phase images to train the model. The trained model can be used to generate a phase image from an input data set.

Description

[0001] Part of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner does not object to the reproduction of the patent document or the patent disclosure by anyone in the form in which it appears in the Patent and Trademark Office patent file or records, but reserves all copyright rights otherwise.

[0002] Cross - Reference to Related Applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 906,605, filed on September 26, 2019, entitled "Reconstructing Phase Images with Deep Learning", and U.S. Non - Provisional Patent Application No. 17 / 028,448, filed on September 22, 2020, entitled "Reconstructing Phase Images with Deep Learning", the contents of which are hereby incorporated by reference in their entirety. Technical Field

[0004] This disclosure generally relates to devices, methods, and systems for reconstructing phase images from sets of images, including sets of images from bright - field microscopy images, using machine - learning techniques. Background Art

[0005] Microscopes enable us to see certain parts of our world at levels of detail that are invisible to the human eye. Technological advancements have far exceeded the "mirror" methods of the past, and modern technology can provide increasingly clear images of the microscopic world. Imaging techniques seem to offer both high levels of zoom resolution to observe samples and the ability to isolate elements of interest. These techniques can be of particular interest in the study of biological cells.

[0006] Quantitative phase imaging is such a technique. Bright - field imaging techniques illuminate a sample and collect the light passing through the sample for the user to view. Due to the nature of the optical refraction through the different elements contained in the sample, a pair of bright - field images at different focal planes of the sample can be processed and combined to reconstruct a clear image of the sampled object (such as a cell contained in a sample on a plate for use in a microscope).

[0007] Simple brightfield imaging can be greatly enhanced by using phase imaging. For example, views of a sample (e.g., a microscope or other sample view to be imaged) can be imaged at two or more different focal planes to provide two or more brightfield images, and then contrast in refractive index can be used to generate a combined, enhanced phase image. For example, in a sample containing cells in water, the known refractive index of water (n = 1.33) can be used to isolate and identify the cells (e.g., n = 1.35 - 1.38) and generate a phase image focused on the cells in the sample. The brightfield images can be processed and combined based on a basic phase equation, which is a special form of the intensity transport equation for paraxial illumination (M.R. Teague, “Deterministic phase retrieval: a Green’s function solution,” Journal of the Optical Society of America, vol. 73, no. 11, pp. 1434–1441, 1983):

[0008]

[0009] where I is the light intensity as a function of the spatial coordinates x, y, and z, Φ is the optical wave phase as a function of the spatial coordinates, ▽ and Δ are the corresponding gradient operator and Laplacian operator in the xy plane, and is the intensity derivative along the z-axis. For more information on brightfield phase imaging, see the paper “Quantitative phase-amplitude microscopy I: optical microscopy” by E.D. Barone-Nugent et al., Journal of Microscopy, Vol. 206, Pt. 3 June 2002, pp. 193 - 203.

[0010] Brightfield phase imaging can be a powerful tool. While it works well in some use cases, when faced with common artifacts (e.g., dust particles, water droplets, hole edges / hole boundaries), this method may perform poorly and may take a significant amount of time (i.e., 3 - 5 seconds) to generate a phase image. Interferences in the optical path (e.g., visual artifacts, or simply “artifacts”) can present specific problems in the phase image because they do not refract light but absorb it. These interferences or artifacts can lead to unwanted results in the generated phase image because the phase equation does not capture absorption effects, and accounting for these effects is an ill-posed problem and computationally costly. Without proper consideration of absorption, the phase solution will be severely and non-locally distorted.

[0011] Another disadvantage of bright-field phase imaging is the boundary regions of the image. For example, since there is no useful data outside the image edge, the calculated phase image will include blurring at the boundary regions representing the image edge. In another example, the well boundaries of a sample well plate may cause similar problems because some light is completely absorbed within the imaging region. Other problems include the combination of illumination non-uniformity and incomplete paraxial illumination, which produces a significant background component. Sufficient background removal may also deteriorate the signal.

[0012] Aspects described herein can address these and other disadvantages using a novel machine learning model trained based on a training dataset and simulated visual artifacts. Through the new techniques described herein, improved phase images can be generated from multiple sets of bright-field images. This can enhance the ability of scientists and researchers to view the structure of cell samples, for example, through enhanced image clarity and the isolation of cells relative to background components. Summary of the Invention

[0013] Described herein are techniques that combine artificial intelligence and machine learning to utilize deep learning algorithms to automatically reconstruct high-quality phase images, which can reduce the impact of the artifacts and imaging quality issues identified herein. These techniques can also generate phase images in a shorter time than existing techniques (e.g., less than 1 second compared to existing techniques that require multiple seconds).

[0014] Aspects discussed herein relate to reconstructing phase images from bright-field images at multiple focal planes using machine learning techniques. A machine learning model can be trained to generate phase images based on multiple sets of bright-field images. In an example of imaging a sample that includes a view of a sample well plate that also includes one or more sample (micro) wells, the training dataset can be selected based on those views and / or images of the sample well plate that avoid visual artifacts (such as sample well plate boundaries, (micro) well boundaries, and / or other visual artifacts such as dust). By considering the entire sample and / or sub-regions within the samples contained therein, the machine learning model can be trained to reduce the impact of visual artifacts on the resulting phase image. The training dataset can be enhanced by creating copies of images in the training dataset and adding simulated visual artifacts to create simulated training dataset images. In a supervised training method, the machine learning model can be trained using the simulated training dataset images, and the machine learning model can learn to adjust the output based on the simulated visual artifacts.

[0015] The set of matching images in the dataset can be used as input to a machine learning model during training, where each set of matching images can include: two or more brightfield images corresponding to a sample view, each brightfield image at a different focal plane; and a ground truth phase image of the sample generated based on the corresponding two or more brightfield images. The output of the machine learning model generated based on the set of matching images can be compared with the corresponding ground truth phase image to train the model. Additionally and / or alternatively, an unsupervised training method that does not require a training dataset with corresponding ground truth images can be employed, where a phase equation is used to evaluate the generated output of the machine learning model. The trained model can be used to generate phase images from an input dataset of pairs of brightfield images.

[0016] Corresponding methods, devices, systems, and computer-readable media are also within the scope of the present disclosure.

[0017] These features, as well as many others, will be discussed in more detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present disclosure is illustrated by way of example and not limited to the drawings, in which like reference numerals indicate similar elements, and in which:

[0019] Figure 1 Illustrates an example of a computing device according to one or more illustrative aspects discussed herein, which can be used to implement one or more aspects of the present disclosure;

[0020] Figure 2 Shows an example generation of a brightfield phase image according to some aspects;

[0021] Figures 3A to 3C Shows an example visual artifact that may be present in the generated brightfield phase image;

[0022] Figure 4A Is a block diagram of an illustrative example of a method for developing a training dataset for training a machine learning model to generate phase images according to one or more aspects;

[0023] Figure 4B Is a block diagram of an illustrative example of a method for developing a training dataset for training a machine learning model to generate phase images according to one or more aspects;

[0024] Figure 5 Shows an exemplary set of matching images for training a machine learning model according to some aspects;

[0025] Figure 6 Shows an exemplary modified set of matching images for training a machine learning model according to some aspects;

[0026] Figure 7 Illustrate the optical principles associated with simulated visual artifacts according to some aspects; and

[0027] Figures 8A to 8B Illustrate the comparison results between classical methods and novel techniques according to one or more aspects described herein. Detailed Description

[0028] In the following description of the various embodiments, reference is made to the accompanying drawings which form a part hereof and in which are shown by way of illustration various embodiments in which aspects of the disclosure may be practiced. It is to be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope of the disclosure. Aspects of the disclosure are capable of having other embodiments and of being practiced or carried out in various ways. Further, it is to be understood that the terminology and phraseology used herein is for the purpose of description and should not be regarded as limiting. On the contrary, the phrases and terms used herein will be given their broadest interpretation and meaning. The use of "including" and "comprising" and their variants is meant to cover the items listed thereafter and their equivalents as well as additional items and their equivalents.

[0029] As an introduction, aspects discussed herein may relate to devices, systems, methods, and techniques for using deep learning to generate phase images that may reduce and / or eliminate the effects of visual artifacts (such as dust, water droplets, pore boundaries (e.g., in a porous plate), image boundaries, etc.), thereby addressing the drawbacks of classical methods associated with such artifacts. For the purposes of this disclosure, an “artifact-free” image should be understood as an image that is substantially free of visual artifacts and / or discontinuities that would produce significant image distortion without the application of the disclosed devices, systems, methods, and techniques. As also used herein, a “sample” refers to the content of the view or “image view” to be imaged, such as a microscope view, where such a sample may include one or more objects, structures, specimens, etc. In one embodiment, a training set of images of artifact-free views of a sample (e.g., a microscope view) obtained using at least two different focal planes may be provided to provide at least first and second brightfield images, and an artifact-free phase image of the view may be obtained therefrom. Thereafter, at least the first and second artifact-free brightfield images may be modified by simulating the presence of visual artifacts or other types of challenges known to exist in brightfield microscope images. The modified brightfield images, along with the artifact-free phase image (e.g., obtained from the unmodified brightfield image), are provided as input to a machine learning model as a training set. The machine learning model may generate an output phase image of a matching image based on the training set based on a phase equation and one or more model parameters. In such an embodiment, given the artifact-free phase image (based on the artifact-free brightfield image), the parameters of the machine learning model are adjusted to compensate for the simulated, artifact-induced brightfield image. After training, when presented with a sample brightfield image containing actual artifacts, the disclosed devices, methods, and systems are capable of using the trained machine learning model to compute a phase image of the corresponding view that compensates for the artifacts.

[0030] In some embodiments, a trainable machine learning model (e.g., a neural network) can be trained to generate a phase image from a pair of brightfield images of a sample acquired at different focal planes. The model can be trained to eliminate common problems that may render portions of a phase image generated in a classical manner unusable. A training dataset can be constructed by selecting a set of relatively clean matching images that can be chosen to provide example image views of the sample or a (sub)region thereof that avoid visual artifacts and other problems. For example, the model can be trained to operate on training images (e.g., from at least two different focal planes) of a sheared region within a full brightfield image, which can allow the model to learn to reduce and / or eliminate the effects of image boundaries. The training dataset can be extended by acquiring a set of matching images and modifying the images to simulate the presence of visual artifacts or other challenges faced by the practicality of brightfield microscopy. Since the visual artifacts are simulated, the output of the model can still be evaluated against known ground truth examples. This process can cause the trained model to learn to compensate for visual artifacts, thereby reducing their impact on the generated phase image. Once trained, the model can be used to generate an improved phase image from brightfield images / image views that may contain visual artifacts and / or discontinuities.

[0031] Model training can be a supervised learning, unsupervised learning, and / or hybrid learning method. In supervised learning, each set of matching images in the training dataset can include a corresponding ground truth phase image (e.g., a phase image obtained from an unmodified, artifact-free brightfield image). The output of the machine learning model during training can be compared with the ground truth phase image, and the model parameters can be adjusted appropriately. In unsupervised learning, the output of the machine learning model during training can be compared with an expected ground truth phase image calculated by applying a phase equation (and / or additional enhancement) to an unmodified, artifact-free brightfield image. In some embodiments, a hybrid method can also be used, employing aspects of both supervised and unsupervised learning to obtain improved model training.

[0032] According to some aspects, a supervised learning system and / or method may use a phase image reconstructed in a classical manner from an artifact-free image view of a sample or a portion thereof selected away from the image boundaries and without visual artifacts as the ground truth for training a machine learning algorithm. A deep artificial neural network model may be trained to reproduce the phase image reconstruction. To address the interferences and artifacts described herein, image enhancement tools may be employed to simulate those artifacts on the images of the artifact-free training dataset (and / or the corresponding ground truth phase images) to allow the artificial neural network to learn different ways of handling those (simulated) artifacts. For example, dark regions in the artifact-free training data and / or ground truth phase images may be created to simulate the effects of dust particles or droplets, where such artifact creation / simulation may account for the focal plane of the corresponding brightfield image and the effect (if any) of the artifact being simulated. The network may be trained to ignore the darkening contributions. Additionally and / or alternatively, some regions of the artifact-free training images may be "blacked out" to simulate the effects of image boundaries or hole boundaries. In such cases, the network may be trained to keep these regions as blacked-out regions and not affect the phase signal in nearby image regions. These enhancement tools / techniques may greatly improve the success rate of the supervised method. In some embodiments, the disclosed devices, methods, and systems may include projecting simulated visual artifacts onto different brightfield images based on the difference between a first focal plane associated with a first brightfield image and a second focal plane associated with a second focal plane.

[0033] According to certain aspects, an unsupervised learning method may not rely on a phase image reconstructed in a classical manner. Instead, the unsupervised method may primarily rely on the phase equation directly as the loss function when training an artificial neural network model. This method may not need to use image enhancement tools / techniques in the same way and is capable of addressing the previously identified artifacts and image quality issues. However, since the unsupervised method relies heavily on the phase equation (and related enhancements), it may reconstruct additional details in the phase image that may encounter similar problems in classical reconstructions. Thus, in some unsupervised embodiments, a brightfield image is provided as an input to the machine learning model, and random weights may be used to compute the phase image. The phase equation may be used to compute the loss and adjust the weights. This process may be repeated until a phase image is obtained that produces a predetermined or desired loss.

[0034] According to some aspects, a hybrid learning device, system, and / or method may use supervised and unsupervised methods to train a model to perform the reconstruction. Applying these learning methods to train a machine learning model may provide a model capable of automatically removing and / or reducing the effects of visual artifacts without the need for manual parameter tuning associated with classical adjustments.

[0035] The process of generating a phase image using a set of bright-field microscopy images measured at different focal planes is a known method. Although it works well in some use cases, when faced with common artifacts (e.g., dust particles, water droplets, hole boundaries), this method may perform poorly and may take time (i.e., 3 - 5 seconds) to generate a phase image.

[0036] However, before discussing these concepts in more detail, reference will first be made to Figure 1 discuss several examples of systems and / or methods that include computing devices that can be used to implement and / or otherwise provide various aspects of the present disclosure.

[0037] Figure 1 An example of a computing device 101 that can be used to implement one or more illustrative aspects discussed herein is shown. For example, in some embodiments, the computing device 101 can implement one or more aspects of the present disclosure by reading and / or executing instructions and performing one or more actions based on the instructions. In some embodiments, the computing device 101 can represent various devices, be incorporated in various devices, and / or include various devices, such as desktop computers, computer servers, mobile devices (e.g., laptop computers, tablet computers, smart phones, any other type of mobile computing device, etc.), and / or any other type of data processing device.

[0038] In some embodiments, the computing device 101 can operate in a stand-alone environment. In other cases, the computing device 101 can operate in a network environment. As Figure 1 shown, various network nodes 101, 105, 107, and 109 can be interconnected via a network 103 (such as the Internet). Other networks can also be used or alternatively, including private intranets, corporate networks, LANs, wireless networks, personal area networks (PANs), etc. The network 103 is for illustrative purposes and can be replaced with fewer or more computer networks. A local area network (LAN) can have one or more of any known LAN topologies and can use one or more of various different protocols, such as Ethernet. Devices 101, 105, 107, 109, and other devices (not shown) can be connected to one or more of the networks via twisted pair, coaxial cable, fiber optic, radio waves, or other communication media.

[0039] As Figure 1As shown, computing device 101 may include a processor 111, RAM 113, ROM 115, a network interface 117, an input / output interface 119 (e.g., keyboard, mouse, display, printer, etc.), and a memory 121. The processor 111 may include one or more central processing units (CPUs), a graphics processing unit (GPU), and / or other processing units, such as a processor suitable for performing computations associated with machine learning. The I / O 119 may include various interface units and drivers for reading, writing, displaying, and / or printing data or files. The I / O 119 may be coupled to a display such as display 120. The memory 121 may store software for configuring the computing device 101 as a dedicated computing device for performing one or more of the various functions discussed herein. The memory 121 may store an operating system software 123 for controlling the overall operation of the computing device 101, control logic 125 for instructing the computing device 101 to perform the aspects discussed herein, machine learning software 127, training set data 129, and other applications 129. The control logic 125 may be incorporated into the machine learning software 127 and / or may be part of the machine learning software 127. In other embodiments, the computing device 101 may include two or more of any and / or all of these components (e.g., two or more processors, two or more memories, etc.) and / or other components and / or subsystems not shown herein.

[0040] Devices 105, 107, 109 may have a similar or different architecture as described for computing device 101. Those skilled in the art will appreciate that the functionality of the computing device 101 (or devices 105, 107, 109) as described herein may be distributed across multiple data processing devices, e.g., to distribute the processing load across multiple computers, to separate transactions based on geographical location, user access level, quality of service (QoS), etc. For example, devices 101, 105, 107, 109 and other devices may operate to provide parallel computing features to support the operation of the control logic 125 and / or software 127.

[0041] One or more aspects discussed herein may be embodied as computer-usable or readable data and / or computer-executable instructions, such as embodied as one or more program modules, executed by one or more computers or other devices described herein. In general, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by a processor in a computer or other device. Modules may be written in a source code programming language and subsequently compiled for execution, or may be written in a scripting language such as (but not limited to) Python or R. The computer-executable instructions may be stored on a computer-readable medium such as a hard disk, optical disk, removable storage medium, solid-state memory, RAM, etc. As will be understood by those skilled in the art, the functionality of program modules may be combined or distributed as needed in various implementations. Additionally, such functionality may be embodied in whole or in part in firmware or hardware equivalents such as integrated circuits, field-programmable gate arrays (FPGAs), etc. Particular data structures may be used to more effectively implement one or more aspects discussed herein, and such data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein. The various aspects discussed herein may be embodied as a method, a computing device, a data processing system, or a computer program product.

[0042] Several examples of computing devices that may be used to implement some aspects for further discussion below have been discussed. Now the discussion will turn to the process for training a machine learning model to generate bright-field phase images.

[0043] Figure 2 An example of generating a bright-field phase image from two input bright-field images is shown. The first image 210 and the second image 220 may be images captured from a bright-field microscope using a first and a second focal plane, respectively. For example, a sample having a sample containing cells and water may be prepared in a well of a sample plate and appropriately positioned for imaging through the microscope. The first image 210 may be an image of a view of at least a portion of the sample captured at the first focal plane, while the second image 220 may be a corresponding image of the same view of the sample captured at the second focal plane. Applying the principles of bright-field phase imaging, the first image 210 and the second image 220 may be processed to generate a phase image 230 based on the phase equation 235. This process may utilize the different refractive indices of a known sample medium (e.g., water with n = 1.33) and the sample (e.g., cells with n = 1.35 - 1.38) to generate an image that isolates the sample from the background through visual contrast. Applying the phase equation to images at different focal planes is based on the recognition that cells work like lenses, gathering light passing through the cell as light travels from one layer to the next. The law of conservation of light governs how light moves from one place to another without losing photons. Applying the phase equation may transform the density information from the first and second focal planes to generate the phase image (Φ).

[0044] The strict application of the phase equation to the input bright-field image may generate a phase image with unusable portions due to the presence of visual artifacts such as dust or hole boundaries, etc. Since the present technique is not limited to these or other examples of artifacts and can be extended to other visual artifacts and / or discontinuities in the image. Classical methods have developed various adjustment enhancements to be applied to the phase image generated using the phase equation, but they typically require manual adjustment and application. The correction applied to the resulting phase image violates the phase equation but can reduce the impact of visual artifacts that are outliers in terms of refractive index.

[0045] Figures 3A to 3C Illustrates the complexity in phase imaging and what the corresponding appropriate results may look like. In practice, bright-field imaging is affected by defects in the illumination pattern, light loss, noise / interference, optical path interference, and other visual artifacts.

[0046] Figure 3A Illustrates the problems associated with background noise and image boundaries. In exemplary images 310 and 320, two views of a particular sample are provided, and visual artifacts and discontinuities in the form of background noise and image boundaries are used to blur or introduce unwanted contrast in regions near the boundaries of the image views of the sample. This background glow may be caused by the strict application of the phase equation. In the corresponding processed images 315 and 325, a more appropriate visualization of the contrast problems for images 310 and 320 is presented. Classical methods for compensating these problems include background cut-off / sliding cut-off methods. As described, these methods may violate the phase equation and may indeed have a higher error rate, but result in a generated phase image that is more suitable for viewing. Although the correction may deviate from the phase equation, they provide an improved separation of the useful signal from the useless signal. Another enhancement may include applying a high-pass filter to the generated phase image. However, the high-pass filter may skip over the image and may not satisfactorily address certain problems. A better approach may be to use a sliding parabola as a kind of "razor" to interpolate between the minima on the phase image to shave off the cells with background glow. However, the sliding parabola method may require careful adjustment and application by the operator.

[0047] Similarly, Figure 3BShows problems associated with the image view when dust particles are in the image view. Image 330 shows the resulting phase image of an example derived from a classical method, which assigns too much weight to dust particles surrounded by a blurred and high-contrast halo. Dust particles (such as dust in the air or fibers of a laboratory coat) contained in the sample image view can produce very poor results because they are optically absorbent and violate the conservation law phase equation used to generate the contrast phase image. Image 335 shows the processed image corresponding to Image 330, where the influence of the dust particles has been reduced. Image 340 shows a similar visual artifact where interference in the visual path has caused the phase image to turn black (white) in a portion of the image. Image 345 shows the appropriately adjusted phase image with the interference effects removed.

[0048] Figure 3C Shows the influence that the hole boundaries (e.g., on a porous plate) may have on the generated phase image. Samples within a sample (e.g., biological materials, non-biological materials, and / or materials in other types of research) can be measured within the small holes of a porous plate observable under a microscope, although such examples are provided for illustration only, and the disclosed methods and systems can be applied to other views of other objects / samples. The hole boundaries on the plate can act as a strong absorbing medium, optically drowning out useful signals. The unprocessed phase image 350 shows that the hole boundaries may absorb light passing through them and thus appear as high-contrast regions on the generated phase image, largely unusable for observation. Classical methods have significant problems with the hole boundaries in the image, and such processing methods may have a long-term destructive impact (e.g., affecting large visual areas) through the generated phase image. According to some aspects, the processed phase image 355 shows an enhanced image with the influence of the hole boundaries removed, revealing sample cells that may not have been observable in the original, unprocessed phase image. As further described herein, a significant advantage of using machine learning methods with simulated visual artifacts to generate phase images can be the robust handling of hole boundaries in the sample image by the trained model.

[0049] Figure 4AShows an embodiment of a novel method 4000 for training a machine learning model to generate a phase image. The method 4000 can train a machine learning model based on an initial training dataset that includes obtaining one or more sets of initial matched bright-field images, each bright-field image in a set being acquired at a different focal plane. Such initial dataset images can include views that are substantially free of visual artifacts and / or discontinuities. The training dataset can be enhanced by generating additional sets of matched images by modifying one or more of the initial matched image training set (bright-field images) to simulate visual artifacts and / or discontinuities, which classical methods may have classification difficulties when generating phase images from the initial matched image training set. The machine learning model can be trained based on the simulated visual artifacts and / or discontinuities to improve the generation of the phase image, even in the presence of such visual artifacts, which include but are not limited to dust, water droplets, boundaries (such as hole boundaries and plate boundaries), and other visual artifacts and discontinuities that may occur in bright-field imaging (e.g., as seen in Figures 3A to 3C discussed). The method 4000 can be implemented by any suitable computing device (such as the computing device 101 of Figure 1 ), embodied in a computer-readable medium, and / or embodied in a suitably configured system. The machine learning model can be an artificial neural network employing deep learning techniques, such as a convolutional neural network. The deep neural network can be a neural network that includes one or more hidden layers in addition to its input layer and output layer. Many types of artificial neural networks are suitable for generating a phase image based on the input bright-field image, and the aspects described herein can apply equally, regardless of the specific neural network structure employed by the machine learning model. As will be understood in the art, the candidate phase images generated by the machine learning module can be evaluated using supervised learning, unsupervised learning, and / or hybrid learning methods, and the parameters of the machine learning model can be adjusted based on the evaluation to improve the accuracy of the machine learning model.

[0050] At 4010, a computing system implementing method 4000 can optionally include one or more initial training data sets of matched image sets. Each matched image set in the initial training data set can include two or more images of a sample observed at different focal planes. For example, a matched image set can include a first bright-field image of a sample corresponding to a view at a first focal plane and a second bright-field image of the same sample corresponding to a view at a second focal plane. The initial training data set can be selected to generally include matched image sets that are substantially free of significant visual artifacts and / or discontinuities. For example, the training data set can include examples selected to substantially avoid dust particles or the well boundaries of a microplate. For supervised learning applications, the training data set can include corresponding phase images generated based on the matched image sets that are substantially artifact-free. Corresponding phase images can be generated using classical methods, such as by applying a phase equation to the first and second bright-field images. The phase images can be further enhanced by correction, such as by applying a sliding parabola correction to reduce background glow / noise.

[0051] At 4020, the computing system can enhance the training data set by generating additional matched image sets by modifying the initial matched image sets to simulate visual artifacts.

[0052] At 4030, the system can train a machine learning model based on the enhanced training data set (including the initial "artifact-free" images and the modified images). During training, at 4033, the machine learning model can generate candidate phase images from the matched bright-field image sets in the training data set. In an example implementation with two bright-field images, the first bright-field image at a first focal plane and the corresponding second bright-field image at a second focal plane of the matched image set can be input into the machine learning model. In some implementations, in cases where more than two focal planes are captured in the matched image set, the input can include each bright-field image at different focal planes. In the example implementations described herein, the machine learning model can output candidate phase images generated based on the first and second bright-field images of the matched set. At 4035, supervised learning, unsupervised learning, and / or hybrid learning methods as described below can be used to evaluate the generated candidate phase images. And at 4037, the parameters of the machine learning model can be adjusted based on the evaluation to improve the accuracy of the machine learning model.

[0053] In the supervised learning method, according to the foregoing example, the foregoing example can use Figure 5For the set of matching images shown, candidate phase images generated by the model and corresponding to the set of matching images from the training dataset can be compared with the corresponding ground truth phase images of the training dataset corresponding to the first and second brightfield images (e.g., phase images generated from a substantially artifact-free brightfield image dataset by classical methods). The difference between the ground truth phase images and the candidate phase images generated by the machine learning model can be used to further train the model. That is, the parameters of the model can be adjusted based on the difference to adjust and / or improve the output of the machine learning model.

[0054] In an unsupervised learning method, candidate phase images generated by a machine learning model can be evaluated using a phase equation and corrected if necessary. That is, the error rate of candidate phase images generated from a set of one or more brightfield images (by the model) can be evaluated against the optical laws that apply in multiple focal planes of the brightfield image. For example, when training a machine learning model, the phase equation can be used as a loss function, with the candidate phase image and the corresponding brightfield image as inputs, and the model can be trained to minimize the loss function when generating candidate phase images. Optimization can be performed by making small random changes to the neural network weights each time the model passes through a training cycle to achieve minimum loss, and the loss obtained can be observed at the end of each cycle. The loss obtained can be observed over many cycles to evaluate the performance of the best-preserved model.

[0055] In a hybrid learning method, aspects of supervised learning and unsupervised learning can be adopted. For example, candidate phase images generated by a machine learning model for a set of matching images can be evaluated against the corresponding ground truth phase images, while also considering the error rate with respect to the phase equation. Some model parameters can be better adjusted based on supervised learning, while other model parameters can be better learned based on unsupervised learning. The supervised and unsupervised methods can use the same or different datasets. For example, a first training dataset can support supervised learning with multiple sets of brightfield images and ground truth phase images, while a second training dataset can omit the ground truth phase images.

[0056] In step 4040, the trained model can be used to generate phase images from actual data. Data received from a brightfield image source, such as a set of live brightfield images, can be fed into the model to generate phase images that can be provided to researchers to assist in observation.

[0057] Figure 4B A method 400 for generating a training dataset is shown, which is used to train a machine learning model to generate the disclosed phase images by the devices, methods, and systems described herein. In Figure 4B the example, the view is a microscope view. Method 400 can implement method 4000 ( Figure 4A) aspects. The method 400 can train a machine learning model based on a training dataset that includes a matching set of at least two brightfield images of a microscopic view, where each brightfield image is acquired using a different focal plane. As Figure 4B shown, an initial training dataset can be constructed based on examples of matching images or image views that are substantially and / or to a significant extent free of visual artifacts or discontinuities. The initial training dataset can be enhanced by generating additional sets of matching images (modified brightfield images) from the initial training dataset images and simulating visual artifacts and / or discontinuities on the initial training dataset images, which classical methods may have difficulty classifying or otherwise processing. Modifications to such brightfield images can be based on the different focal planes associated with different brightfield images and the potential different visual effects of artifacts / discontinuities at different focal planes. In some embodiments of certain learning scenarios, the disclosed methods and systems can also include modifying the "ground truth" phase image (generated from a substantially artifact-free brightfield image) with corresponding visual artifacts and / or discontinuities as made to the brightfield images. A machine learning model can be trained based on simulated visual artifacts to improve the generation of phase images, even in the presence of such visual artifacts, which in the illustrated embodiments can include, but are not limited to, dust, water droplets, boundaries, and other visual artifacts that may occur in brightfield imaging (e.g., as discussed in Figures 3A to 3C ). The method 400 can be implemented by any suitable computing device (such as Figure 1 computing device 101), embodied in a computer-readable medium, and / or embodied in a suitably configured system.

[0058] At 401, a computing system implementing method 400 can generate an initial training data set by selecting a matching set of at least two bright-field images, where each image in the matching set includes an image view of a sample at different focal planes. For example, in the case of two bright-field images, the first bright-field image will be obtained from a specific microscopic view of the sample at a first focal plane, while the second bright-field image will be obtained from the same microscopic view of the same sample but corresponding to a second focal plane. The images selected at 401 can be substantially "artifact-free", preferably substantially without any significant and / or unwanted visual artifacts and / or discontinuities. In a supervised learning implementation, at 402, a training phase image can be generated by applying a phase equation to the corresponding matching set of bright-field images (artifact-free, initial training data) obtained at 401. Thus, a corresponding training phase image ("ground truth phase image") can be generated using classical methods, although other techniques can be used. The corresponding training phase image can be further enhanced by correction, for example, by applying a sliding parabola correction to reduce background glow / noise. The "artifact-free" bright-field images and their corresponding "artifact-free" phase images can be stored. In an unsupervised learning implementation, 402 can be omitted.

[0059] At 403, one or more visual artifacts can be selected, and any one or more visual artifacts can be included, which are typically present in bright-field images of the sample or sample type within the view, as previously described herein. As previously provided, such visual artifacts can include dust, holes or plate boundaries, blackened areas, etc., and the present disclosure is not limited to the types of visual artifacts that can be selected for simulation as provided herein, as such artifacts and / or discontinuities are based on the sample and its characteristics. At 404, known techniques can be used, such as by horizontally and vertically shifting pixels, rotating the pixels of the image, adding random background noise (e.g., Gaussian noise, salt-and-pepper noise, Poisson noise, speckle noise, etc.), blurring the pixels, generating blackened areas in specified regions to simulate micropores or microplate boundaries, etc. and / or generating random blackened areas throughout the (previous artifact-free bright-field) image, to apply such one or more selected visual artifacts to each of the "artifact-free" bright-field images in a simulated manner.

[0060] At 405, it can be determined whether the visual artifacts should also be applied to the "artifact-free" phase image generated from the "artifact-free" bright-field image and corresponding to the "artifact-free" bright-field image in a simulated manner. Such a decision can be based on the selected artifacts and the impact that such artifacts may have on the learning process. For example, microplate boundaries and hole boundaries and other fully absorbing visual artifacts can be selected to be replicated in the (artifact-free) phase image to better guide machine learning. Other artifacts may not be able to be simulated in the phase image.

[0061] If it is determined to modify the "artifact-free" phase image with visual artifacts, a corresponding modified phase image can be generated at 406. The modified brightfield image and the modified phase image include a matching image set and are stored as part of the training dataset at 407. Otherwise, in the case where visual artifacts are not applied to the phase image, the "artifact-free" phase image and the modified brightfield image include a matching image set and are stored as part of the training dataset at 408. After the modified matching image set is stored, it can be determined at 409 to create another matching image set using these same initial images but with different artifacts. In this case, the process described herein can be repeated at 410 to generate a new matching image set based on the same "artifact-free" brightfield image and corresponding "artifact-free" phase image but with different artifacts 410 to augment the training dataset. Additionally and / or alternatively, at step 411, a new set of "artifact-free" brightfield images can be obtained by returning to step 401, and the method 400 shown in FIG. 4 can be repeated to generate a modified matching image set based on other initial matching brightfield image sets and / or other previously generated matching training datasets. For example, in some implementations, rather than "resetting" as shown at 410, the system can select a matching set that has already been modified and further modify the matching set with another artifact. Once the training dataset includes all desired image sets, the training dataset can be provided at 412 to train a machine learning model to generate phase images for use by researchers and other users.

[0062] Figure 5 Illustrative matching image set 500 depicting an exemplary training dataset, where the number of brightfield images obtained (see Figure 4B , 401) is two. Matching image set 500 can include a first brightfield image 510 of a sample taken at a first focal plane and a second brightfield image 520 of the sample taken at a second focal plane. A phase image 530 can be generated by applying a phase equation to images 510 and 520 and performing appropriate corrections (such as sliding parabola correction).

[0063] Example images 510, 520, and 530 can be substantially free of unwanted artifacts, such as the image boundaries being removed so that the machine learning model is not trained based on anomalous results near the image boundaries. For example, image data within a certain pixel range of the image boundary can be removed from the training dataset images 510, 520, and 530. In some embodiments, the machine learning model can consider data from images 510 and 520 for more comprehensive training to obtain phase image 530 as the ground truth. Although the model is intended to generate a reduced region of phase image 530, in some implementations, it can consider additional data outside of that region from images 510 and 520 to better predict the content within the reduced region.

[0064] Figure 6 Illustrate a modified matching image set 600 of an exemplary training data set including two bright-field images. The first modified bright-field image at the first focal plane 610 can be modified to include simulated pore boundaries 611 and simulated dust particles 613. These simulated artifacts can be projected onto the corresponding second bright-field image and the corresponding ground truth phase image corresponding to the second focal plane. For example, the modified second bright-field image 620 is modified to include simulated pore boundaries 621, which may be substantially the same as the simulated pore boundaries 611 due to the nature of the pore boundaries when crossing the focal plane of the sample. However, due to the different focal planes between the first bright-field image and the second bright-field image, the dust particles 613 may be expected to have a greater impact on the second bright-field image 620 of the modified training data set. For example, using the optical properties associated with the dust and the distance between the first focal plane and the second focal plane, the system can calculate that the dust will appear less concentrated and more dispersed in the first focal plane 610 compared to the second focal plane 620 where the dust appears more prominent (e.g., darker and deeper). Figure 7 Provide a diagram 700 of the optical principles involved in the projection of simulated artifacts from one plane to another. Specifically, due to how dust particles actually interact with light, the upper plane 710 is more blurred than the lower plane 720. Additionally, the crescent effect may cause the projected dust particles to shift slightly in position when projected onto the second focal level. Thus, the disclosed methods and systems include simulating artifacts and / or discontinuities by modifying an artifact-free (e.g., "initial") training data set based on a selected artifact / discontinuity and the associated visual effects of such selected artifact / discontinuity on the corresponding focal plane.

[0065] As Figure 4B provided in, at 405 to 408, the ground truth phase image corresponding to the training data set can be modified, where appropriate, to project the desired effects of simulated visual artifacts. For example, the system can generate a modified training data set phase image 630 to include simulated pore boundaries 631 in substantially the same form as in images 610 and 620. However, as shown in region 633, the visual impact of the simulated dust particles 613 can be greatly reduced. Thus, the simulated visual artifacts can be configured to guide the machine learning model to the desired results, appropriately compensating for the visual artifacts in a way that maximizes useful signals and reduces useless signals.

[0066] Because the (initial or artifact-free) training data set includes examples selected to avoid visual artifacts and other complexities, such as Figure 5Those shown in, for example, can train a machine learning model to accurately generate phase images based on relatively clean input data. However, according to aspects described herein, a modified set of example matching images, such as Figure 6 those shown in, can be used to train a machine learning model to handle visual artifacts in a view of a sample.

[0067] In the disclosed devices, systems, and methods, by starting with clean (e.g., substantially free of unwanted artifacts) image / training data, a matching set of brightfield images can be modified while the system still knows the true phase image corresponding to the matching set. Thus, the performance of the machine learning model on the modified examples with simulated visual artifacts can be evaluated based on how accurately the machine learning model can obtain the simulated true phase image or the unmodified original true phase image.

[0068] As provided herein, for example, a first brightfield image at a first focal plane and a corresponding second brightfield image at a second focal plane of a modified training data set (of the same sample) can be input into a machine learning model. The machine learning model can output a candidate phase image generated based on the first and second brightfield images of the modified training data set. That is, the candidate phase image generated by the machine learning model can be based on simulated artifacts. Similar supervised learning, unsupervised learning, and / or hybrid learning methods as described above can be used to evaluate the generated candidate phase image. Specifically, the generated candidate phase image can be evaluated against the true phase image corresponding to the modified training data set in a supervised learning method (whether or not it also includes simulated visual artifacts, or is unmodified). In some embodiments, the generated candidate phase image can be evaluated using an unsupervised method, using a phase equation applied to the modified brightfield images of the examples, as a loss function to guide model training. In some embodiments, as described above, a hybrid method can be applied. As described herein, the parameters of the machine learning model can be adjusted based on the evaluation to improve the accuracy of the machine learning model.

[0069] Once trained, the machine learning model can be used to generate phase images from actual brightfield image data. Data received from a brightfield image source, such as a set of live brightfield images, can be fed into the model to generate a phase image that can be provided to a researcher to aid in observation. A model that has been trained on artifact simulation data during the training phase may be robust to visual artifacts in the actual data.

[0070] Figure 8A and Figure 8B shows an example of comparing the results of a machine learning model trained according to some aspects described herein with the results of a classical method. Figure 8AShows how the classical method will fail when presenting a pore boundary in image 810. Image 820 generated according to implementations of some aspects described herein provides a clear visualization of the pore boundary while still showing the cells near the pore boundary. As another example, Figure 8B Compare the processing of dust particles in a brightfield image with the classical method by implementations of the aspects described herein. The first brightfield image 831 and the second brightfield image 833 include dust particles that are highly optically absorbent compared to the cells and water of the sample. In the classical method, due to the unwanted signal caused by the dust particles, the resulting phase image 840 is practically unusable. In contrast, image 845 (generated by implementations of some aspects described herein) minimizes the unwanted signal introduced by the dust particles and retains the useful signal of the cells around the dust particles.

[0071] As described above, the aspects described herein relate to a process for training a machine learning model to generate a phase image from a brightfield image. Regardless of the structure of the machine learning model, the aspects described herein can be equally applicable. Exemplary implementations may use a convolutional deep neural network structure for the machine learning model. Specifically, an example implementation of the machine learning model that can be used is detailed below. In this implementation, the machine learning model can be implemented based on the TensorFlow library provided by Google. The model may have a "U" shape, gradually shrinking the input image to identify relevant features and then enlarging back to the result image. The U-shaped architecture can gradually shrink the image through a series of layers until reaching the single-pixel level, and then use deconvolution to restore to the full-size image.

[0072] The U-shaped network can iteratively reduce the resolution of the image, for example, 4 times per step. In each convolution, 1 pixel may correspond to 4 or 16 pixels at the previous level. Therefore, at each progressive layer, each pixel contains more information. This can facilitate the segmented training of the model and allow the neural network to identify relevant features for consideration at various levels of detail. In some implementations, the U-shaped network can consist of 3x3 or 4x4 convolutional deep neural network layers with 32 filters. An input image that may be 256x256 pixels in two planes (e.g., the upper plane and the lower plane) may be gradually filtered. At each layer, the image can be segmented into more constituent images. 256 overlapping image patches can be fed into the network and processed individually before being tiled together.

[0073] Example Python code uses the TensorFlow library to build a convolutional neural network that can be used to implement certain aspects described herein, as follows:

[0074] layer1 = tf.layers.conv2d(x, 64, 4, 2,'same', activation=tf.nn.leaky_relu)

[0075] layer2 = tf.layers.conv2d(layer1, 128, 4, 2,'same', activation=tf.nn.leaky_relu)

[0076] layer3 = tf.layers.batch_normalization(layer2)

[0077] layer4 = tf.layers.conv2d(layer3, 256, 4, 2,'same', activation=tf.nn.leaky_relu)

[0078] layer5 = tf.layers.batch_normalization(layer4)

[0079] layer6 = tf.layers.conv2d(layer5, 512, 4, 2,'same', activation=tf.nn.leaky_relu)

[0080] layer7 = tf.layers.batch_normalization(layer6)

[0081] layer8 = tf.layers.conv2d(layer7, 512, 4, 2,'same', activation=tf.nn.leaky_relu)

[0082] layer9 = tf.layers.batch_normalization(layer8)

[0083] layer10 = tf.layers.conv2d(layer9, 512, 4, 2,'same', activation=tf.nn.leaky_relu)

[0084] layer11 = tf.layers.batch_normalization(layer10)

[0085] layer12 = tf.layers.conv2d(layer11, 512, 4, 2,'same', activation=tf.nn.leaky_relu)

[0086] layer13 = tf.layers.batch_normalization(layer12)

[0087] layer14 = tf.layers.conv2d(layer13, 512, 4, 2,'same', activation=tf.nn.leaky_relu)

[0088] layer15 = tf.layers.batch_normalization(layer14)

[0089] layer16 = tf.layers.conv2d_transpose(layer15, 512, 4, 2,'same', activation=tf.nn.relu)

[0090] layer17 = tf.layers.batch_normalization(layer16)

[0091] layer18 = tf.layers.dropout(layer17, 0.5, training=is_training)

[0092] layer19 = tf.concat((layer12, layer18), axis=3)

[0093] layer20 = tf.layers.conv2d_transpose(layer19, 512, 4, 2,'same', activation=tf.nn.relu)

[0094] layer21 = tf.layers.batch_normalization(layer20)

[0095] layer22 = tf.layers.dropout(layer21, 0.5, training=is_training)

[0096] layer23 = tf.concat((layer10, layer22), axis=3)

[0097] layer24 = tf.layers.conv2d_transpose(layer23, 512, 4, 2,'same', activation=tf.nn.relu)

[0098] layer25 = tf.layers.batch_normalization(layer24)

[0099] layer26 = tf.layers.dropout(layer25, 0.5, training=is_training)

[0100] layer27 = tf.concat((layer8, layer26), axis=3)

[0101] layer28 = tf.layers.conv2d_transpose(layer27, 512, 4, 2,'same', activation=tf.nn.relu)

[0102] layer29 = tf.layers.batch_normalization(layer28)

[0103] layer30 = tf.concat((layer6, layer29), axis=3)

[0104] layer31 = tf.layers.conv2d_transpose(layer30, 256, 4, 2,'same', activation=tf.nn.relu)

[0105] layer32 = tf.layers.batch_normalization(layer31)

[0106] layer33 = tf.concat((layer4, layer32), axis=3)

[0107] layer34 = tf.layers.conv2d_transpose(layer33, 128, 4, 2,'same', activation=tf.nn.relu)

[0108] layer35 = tf.layers.batch_normalization(layer34)

[0109] layer36 = tf.concat((layer2, layer35), axis = 3)

[0110] layer37 = tf.layers.conv2d_transpose(layer36, 64, 4, 2,'same', activation = tf.nn.relu)

[0111] layer38 = tf.layers.batch_normalization(layer37)

[0112] layer39 = tf.concat((layer1, layer38), axis = 3)

[0113] layer40 = tf.layers.conv2d_transpose(layer39, 1, 4, 2,'same')

[0114] Aspects described herein focus on the training process of a neural network model. The model can be configured to apply corrections to a candidate phase image using aspects of classical techniques, such as offset correction and background glow reduction. The model can be designed to use a phase equation to test quality (e.g., as a loss function). However, the corrections typically violate the phase equation, so the model can be appropriately configured to evaluate the accuracy of the generated phase image before applying the corrections in some implementations. As discussed above regarding the sample neural network structure, the input image can be tiled for ease of processing and then stitched together to form the final phase image. Aspects of the neural network structure can be adjustable. For example, meta-parameters (and / or hyper-parameters) can be adjusted to further improve the performance of the machine learning model. As a specific example, the number of filters employed in each layer can be adjusted depending on the implementation.

[0115] Additional neural networks and corrections can be employed to further improve the resulting phase image. For example, in some implementations, there may be higher errors at short resolution scales. High-frequency issues can cause noise in the generated phase image. Additional neural networks can be employed to denoise the generated phase image. Similarly, blur detection techniques can be employed to further improve the generated phase image.

[0116] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims need not be limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A computer-implemented method for generating a phase image, the method comprising: Receiving a training data set comprising one or more sets of matching images, wherein each set of matching images comprises: A plurality of bright-field images corresponding to an image view of a sample, each of the plurality of bright-field images corresponding to a different focal plane; and A ground-truth phase image generated based on the plurality of bright-field images; Selecting at least one visual artifact for simulation; Generating one or more additional sets of matching images by modifying at least one of the one or more sets of matching images of the received training data set to simulate the at least one selected visual artifact in the image view of the sample, wherein modifying a given set of matching images comprises modifying at least one of the corresponding plurality of bright-field images based on the at least one selected visual artifact; Adding the one or more additional sets of matching images to the training data set; Training a machine learning model based on the training data set, wherein training the machine learning model is based on comparing the generated output of the machine learning model for a given set of matching images with the corresponding ground-truth phase image; Receiving a first input bright-field image and a second input bright-field image; and Using the trained machine learning model to generate a phase image corresponding to the first input bright-field image and the second input bright-field image.

2. The method of claim 1, wherein modifying a given set of matching images to simulate the at least one selected visual artifact comprises: Modifying a first bright-field image of the given set of matching images based on the at least one selected visual artifact; And Based on the difference between the at least one selected visual artifact and the focal planes of the plurality of bright-field images in the given set of matching images, modifying additional bright-field images in the given set of matching images based on the at least one selected visual artifact.

3. The method of claim 2, wherein modifying the bright-field image comprises at least one of the following: Shifting pixels horizontally, Shifting pixels vertically, Rotating pixels, Adding random background noise, Blurring pixels, Generating a blackened region in a specified region of the bright-field image, Generating a random blackened region in the bright-field image, or Darkening at least one region of the bright-field image.

4. The method of claim 3, wherein adding random background noise comprises adding background noise in the form of at least one of: Gaussian noise, salt-and-pepper noise, Poisson noise, or speckle noise.

5. The method of claim 1, wherein selecting the one or more sets of matching images to avoid plate boundaries within the image view of the sample.

6. The method of claim 1, wherein selecting the one or more sets of matching images to avoid imaging visual artifacts within the imaging view of the sample.

7. The method of claim 1, wherein each ground-truth phase image is generated by applying a phase equation to the corresponding plurality of bright-field images.

8. The method of claim 7, wherein the generated ground-truth phase image is further processed by applying a sliding parabola correction.

9. The method according to claim 1, wherein modifying at least one of the matched image sets to simulate the at least one selected visual artifact in the image view of the sample further comprises: modifying the ground truth phase image based on the at least one selected visual artifact.

10. The method according to claim 1, wherein the machine learning model comprises a neural network.

11. A computer-implemented method for generating a phase image, the method comprising: receiving a training data set comprising one or more matched image sets, wherein each matched image set comprises a plurality of bright field images corresponding to an image view of a sample, each bright field image being taken at a different focal plane; generating one or more additional matched image sets by modifying at least one of the one or more matched image sets of the received training data set to simulate at least one visual artifact, wherein modifying a given matched image set comprises modifying at least one of the corresponding plurality of bright field images; adding the one or more additional matched image sets to the training data set; training a machine learning model based on the training data set, wherein training the machine learning model is based on evaluating the accuracy of the generated output of the machine learning model for a given matched image set by applying a phase equation to the corresponding plurality of bright field images; receiving an input data set comprising at least a first input bright field image and a second input bright field image; and generating a resulting phase image using the trained machine learning model based on the first input bright field image and the second input bright field image.

12. The method according to claim 11, wherein modifying a given matched image set to simulate at least one visual artifact comprises: selecting a simulated visual artifact to apply to a first bright field image within the corresponding plurality of bright field images; visually modifying the first bright field image based on the focal plane associated with the first bright field image to simulate the at least one selected visual artifact; projecting the at least one selected visual artifact onto at least one other bright field image of the corresponding plurality of bright field images based on a difference between the focal plane associated with the first bright field image and the focal plane associated with at least one other bright field image of the corresponding plurality of bright field images; and visually modifying the at least one other bright field image of the corresponding plurality of bright field images based on the projection to simulate the at least one selected visual artifact.

13. The method according to claim 11, wherein each matched image set further comprises a corresponding training phase image, and wherein modifying at least one of the one or more matched image sets to simulate at least one visual artifact in the image view of the sample further comprises: modifying the training phase image based on the at least one visual artifact.

14. The method according to claim 11, wherein the one or more matched image sets are selected to avoid associated plate boundaries within the imaging view of the sample.

15. The method according to claim 11, wherein each set of matching images further includes a corresponding training phase image, and wherein evaluating the accuracy of the generated output of the machine learning model for a given set of matching images further includes applying a sliding parabola correction to the corresponding training phase image.

16. The method according to claim 11, wherein the machine learning model includes a neural network.

17. A device configured to generate a phase image, the device comprising: a data storage device that stores a training data set including one or more sets of matching images, each set of matching images including: a plurality of bright-field images, each bright-field image corresponding to a different focal plane of an imaging view of a sample; and a ground-truth phase image based on the plurality of bright-field images; one or more processors; and a memory that stores instructions which, when executed by the one or more processors, cause the device to perform the following operations: modifying one or more of the one or more sets of matching images to simulate at least one visual artifact in the image view, wherein modifying a set of matching images includes modifying at least one of the corresponding plurality of bright-field images based on the at least one visual artifact and the focal plane associated with the bright-field image; adding the modified set of matching images to the training data set; training a machine learning model based on the training data set, wherein the training is based on comparing the generated output of the machine learning model for a given set of matching images with the corresponding ground-truth phase image; receiving an input data set including a first input bright-field image and a second input bright-field image; and using the trained machine learning model to generate a resulting phase image based on the first input bright-field image and the second input bright-field image.

18. The device according to claim 17, wherein the ground-truth phase image associated with the set of matching images is generated based on applying a phase equation to the corresponding plurality of bright-field images.

19. The device according to claim 17, wherein the instructions cause the device to modify at least one of the plurality of bright-field images of a given set of matching images by: selecting at least one simulated visual artifact; modifying a first bright-field image of the corresponding plurality of bright-field images to simulate the at least one selected visual artifact at a first focal plane associated with the first bright-field image; projecting the at least one selected visual artifact onto at least a second bright-field image of the corresponding plurality of bright-field images based on a difference between the first focal plane and a second focal plane associated with the second bright-field image; and and modifying the second bright-field image based on the projection to simulate the at least one visual artifact at the second focal plane.

20. The device according to claim 17, wherein the one or more sets of matching images are selected to avoid visual artifacts within the imaging view of the sample.

Images