Biological image translation using machine learning model

Abstract

To provide biological image translation using suitable machine learning models.SOLUTION: A system and method trains a machine learning model and generates images of biological samples, and the system and method generates enhanced images of the biological samples. The method for training the machine learning model and generating the images of the biological sample may include a step for obtaining a plurality of training images comprising a first type of training images and a second type of training images. The method may also include steps for generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images, generating a second type of synthetic image based on the plurality of wavelet coefficients, comparing the second type of synthetic image to the second type of training images, and updating the machine learning model based on the comparison.SELECTED DRAWING: Figure 3B

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Application No. 63 / 075,751, filed September 8, 2020, and U.S. Provisional Application No. 63 / 143,707, filed January 29, 2021, the contents of which are incorporated by reference and are hereby incorporated in their entireties.

[0002] The present disclosure relates generally to machine learning techniques, and more particularly to low-cost machine learning-based generation of image data at large scale. The generated image data (e.g., image data of biological samples) can provide sufficient richness and depth for optimized downstream processing (e.g., phenotyping). Some embodiments of the system include a programmable spatial light modulator ("SLM") that rapidly produces optimized data for downstream processing (e.g., phenotyping) without mechanical modification to the system. Some embodiments of the system include a machine learning model with an attention layer that includes multiple weights corresponding to multiple illumination settings (e.g., different illumination emitters of an illumination source) to identify optimal illumination patterns for capturing image data. Some embodiments of the system include techniques for evaluating candidate treatments for a disease of interest. [Background technology]

[0003] Brightfield images of biological samples can be acquired on a large scale and at low cost due to inexpensive equipment, ease of clinical deployment, and low processing and storage resource requirements for the captured images. Acquiring brightfield images is generally non-invasive and involves low phototoxicity. However, low-contrast images lack rich visual details, thus making them unsuitable for many downstream analyses (e.g., phenotyping of microscopic samples). In comparison, other image modalities (e.g., fluorescent images) can provide rich visual information of the captured sample. However, acquiring fluorescent images requires additional equipment and materials, and can be time-consuming and computing resource intensive. Thus, fluorescent images can be difficult to acquire in a large scale and low-cost manner.

[0004] Transforming bright-field images of biological samples into enhanced high-quality images can be challenging for several reasons. First, bright-field images suffer from an inherent class imbalance problem (i.e., abundant low-frequency signals, but fewer high-frequency signals). Furthermore, the overall geometry of the bright-field image needs to be extracted and maintained throughout the transformation. Furthermore, many factors, such as the illumination pattern under which the bright-field image is taken, can affect the effectiveness of the transformation. Furthermore, the robustness of the transformation in supporting downstream analysis needs to be quantified and verified. Summary of the Invention [Means for solving the problem]

[0005] Described are systems and methods for training a machine learning model and generating images of a biological sample. Also described are systems and methods for generating enhanced images of a biological sample. The systems and methods can be used, for example, to acquire a first type of image, for example, a bright field image type. The acquired image of the first image type can then be used by the systems and methods to generate a composite image of a second type, for example, a fluorescent image.

[0006] In some embodiments, a method for training a machine learning model and generating images of a biological sample includes acquiring a plurality of training images, the plurality of images comprising a first type of training images and a second type of training images. The method also includes generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images, generating a second type of synthetic image based on the plurality of wavelet coefficients, comparing the second type of synthetic image to the second type of training images, and updating the machine learning model based on the comparison.

[0007] In some embodiments, a method for generating an enhanced image of a biological sample includes acquiring an image of the biological sample using a microscope, and generating the enhanced image of the biological sample based on the image using a machine learning model. The machine learning model may be trained by acquiring a plurality of training images comprising a first type of training image and a second type of training image, generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images, generating a second type of synthetic image based on the plurality of wavelet coefficients, comparing the second type of synthetic image to the second type of training images, and updating the machine learning model based on the comparison.

[0008] In some embodiments, a system for training a machine learning model and generating images of a biological sample comprises a computing system comprising one or more processors and one or more memories that store the machine learning model, the computing system configured to receive a plurality of training images of a first type and one training image of a second type. The computing system may be configured to generate a plurality of wavelet coefficients using the machine learning model based on the training images of the first type, generate a synthetic image of a second type based on the plurality of wavelet coefficients, compare the synthetic image of the second type to the training images of the second type, and update the machine learning model based on the comparison.

[0009] In some embodiments, a system for generating an enhanced image of a biological sample comprises a computing system comprising one or more processors and one or more memories storing a machine learning model. The computing system may be configured to receive an image of the biological sample acquired from a microscope and generate an enhanced image of the biological sample using the machine learning model based on the image. The machine learning model may be trained by acquiring a plurality of training images comprising a first type of training image and a second type of training image, generating a plurality of wavelet coefficients using the machine learning model based on the first type of training image, generating a second type of synthetic image based on the plurality of wavelet coefficients, comparing the second type of synthetic image to the second type of training image, and updating the machine learning model based on the comparison.

[0010] An exemplary method for training a machine learning model and generating images of a biological sample includes obtaining a plurality of training images comprising a first type of training images and a second type of training images; generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating a second type of synthetic image based on the plurality of wavelet coefficients; comparing the second type of synthetic image to the second type of training images; and updating the machine learning model based on the comparison.

[0011] In some embodiments, the first type of training image is a bright field image of the biological sample.

[0012] In some embodiments, the second type of training images are fluorescent images of the biological sample.

[0013] In some embodiments, the machine learning model comprises a generator and a discriminator.

[0014] In some embodiments, the machine learning model comprises a conditional GAN ​​model.

[0015] In some embodiments, the generator comprises multiple neural networks corresponding to multiple frequency groups.

[0016] In some embodiments, each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

[0017] In some embodiments, the multiple neural networks comprise multiple U-Net neural networks.

[0018] In some embodiments, the discriminator is a PatchGAN neural network.

[0019] In some embodiments, the method further comprises generating a third type of image based on the training images of the first type.

[0020] In some embodiments, the third type of image is a phase shift image.

[0021] In some embodiments, the method further comprises generating a fourth type of image based on the training images of the first type.

[0022] In some embodiments, the fourth type of image comprises segmentation data.

[0023] In some embodiments, the first type of training images are captured using a microscope and according to a first illumination scheme.

[0024] In some embodiments, the first lighting scheme comprises one or more lighting patterns.

[0025] In some embodiments, the first type of training image is a portion of a bright field image array.

[0026] In some embodiments, the plurality of training images is a first plurality of training images, and the method further includes identifying a second lighting scheme based on the comparison; acquiring a second plurality of training images comprising one or more images of a first type, where the one or more images of the first type are acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images.

[0027] In some embodiments, the method further includes acquiring a plurality of images of a first type using a microscope, and generating a plurality of synthetic images of a second type based on the acquired plurality of images using a machine learning model.

[0028] In some embodiments, the method further includes training a classifier based on the plurality of synthetic images of a second type.

[0029] In some embodiments, the microscope is a first microscope and the classifier is a first classifier, and the method further includes acquiring a plurality of images of a second type using a second microscope, training the second classifier based on the plurality of images of the second type, and comparing performance of the first classifier and the second classifier.

[0030] In some embodiments, the second microscope is a fluorescence microscope.

[0031] An exemplary method for generating an enhanced image of a biological sample includes acquiring an image of the biological sample using a microscope; and generating an enhanced image of the biological sample using a machine learning model based on the image, the machine learning model being trained by acquiring a plurality of training images comprising a first type of training image and a second type of training image; generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating a second type of synthetic image based on the plurality of wavelet coefficients; comparing the second type of synthetic image to the second type of training images; and updating the machine learning model based on the comparison.

[0032] In some embodiments, the first type of training image is a bright field image of the biological sample.

[0033] In some embodiments, the second type of training images are fluorescent images of the biological sample.

[0034] In some embodiments, the machine learning model comprises a generator and a discriminator.

[0035] In some embodiments, the machine learning model comprises a conditional GAN ​​model.

[0036] In some embodiments, the generator comprises multiple neural networks corresponding to multiple frequency groups.

[0037] In some embodiments, each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

[0038] In some embodiments, the multiple neural networks comprise multiple U-Net neural networks.

[0039] In some embodiments, the discriminator is a PatchGAN neural network.

[0040] In some embodiments, the method further comprises generating a third type of image based on the training images of the first type.

[0041] In some embodiments, the third type of image is a phase shift image.

[0042] In some embodiments, the method further comprises generating a fourth type of image based on the training images of the first type.

[0043] In some embodiments, the fourth type of image comprises segmentation data.

[0044] In some embodiments, the first type of training images are captured using a microscope and according to a first illumination scheme.

[0045] In some embodiments, the first lighting scheme comprises one or more lighting patterns.

[0046] In some embodiments, the first type of training image is a portion of a bright field image array.

[0047] In some embodiments, the plurality of training images is a first plurality of training images, and the method further includes identifying a second lighting scheme based on the comparison; acquiring a second plurality of training images comprising one or more images of a first type, where the one or more images of the first type are acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images.

[0048] In some embodiments, the method further includes acquiring a plurality of images of a first type using a microscope, and generating a plurality of synthetic images of a second type based on the acquired plurality of images using a machine learning model.

[0049] In some embodiments, the method further includes training a classifier based on the plurality of synthetic images of a second type.

[0050] In some embodiments, the microscope is a first microscope and the classifier is a first classifier, and the method further includes acquiring a plurality of images of a second type using a second microscope, training the second classifier based on the plurality of images of the second type, and comparing performance of the first classifier and the second classifier.

[0051] In some embodiments, the second microscope is a fluorescence microscope.

[0052] An exemplary system for training a machine learning model and generating images of a biological sample comprises a computing system comprising one or more processors and one or more memories that store the machine learning model, the computing system configured to receive a plurality of training images of a first type and one training image of a second type, generate a plurality of wavelet coefficients using the machine learning model based on the training images of the first type, generate a composite image of a second type based on the plurality of wavelet coefficients, compare the composite image of the second type to the training images of the second type, and update the machine learning model based on the comparison.

[0053] In some embodiments, the first type of training image is a bright field image of the biological sample.

[0054] In some embodiments, the second type of training images are fluorescent images of the biological sample.

[0055] In some embodiments, the machine learning model comprises a generator and a discriminator.

[0056] In some embodiments, the machine learning model comprises a conditional GAN ​​model.

[0057] In some embodiments, the generator comprises multiple neural networks corresponding to multiple frequency groups.

[0058] In some embodiments, each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

[0059] In some embodiments, the multiple neural networks comprise multiple U-Net neural networks.

[0060] In some embodiments, the discriminator is a PatchGAN neural network.

[0061] In some embodiments, the computing system is further configured to generate a third type of image based on the training images of the first type.

[0062] In some embodiments, the third type of image is a phase shift image.

[0063] In some embodiments, the computing system is further configured to generate a fourth type of image based on the training images of the first type.

[0064] In some embodiments, the fourth type of image comprises segmentation data.

[0065] In some embodiments, the first type of training images are captured using a microscope and according to a first illumination scheme.

[0066] In some embodiments, the first lighting scheme comprises one or more lighting patterns.

[0067] In some embodiments, the first type of training image is a portion of a bright field image array.

[0068] In some embodiments, the plurality of training images is a first plurality of training images, and the computing system is further configured to identify a second lighting scheme based on the comparison and obtain a second plurality of training images comprising one or more images of the first type, the one or more images of the first type obtained based on the second lighting scheme, and train the machine learning model based on the second plurality of training images.

[0069] In some embodiments, the computing system is further configured to acquire a plurality of images of a first type using the microscope and generate a plurality of synthetic images of a second type based on the plurality of acquired images using the machine learning model.

[0070] In some embodiments, the computing system is further configured to train the classifier based on the plurality of synthetic images of the second type.

[0071] In some embodiments, the microscope is a first microscope, the classifier is a first classifier, and the computing system is further configured to acquire a plurality of images of a second type using a second microscope, train a second classifier based on the plurality of images of the second type, and compare performance of the first classifier and the second classifier.

[0072] In some embodiments, the second microscope is a fluorescence microscope.

[0073] An exemplary system for generating an enhanced image of a biological sample comprises a computing system comprising one or more processors and one or more memories storing a machine learning model, the computing system being configured to receive images of the biological sample acquired from a microscope and generate an enhanced image of the biological sample based on the images using the machine learning model, the machine learning model being trained by acquiring a plurality of training images comprising a first type of training images and a second type of training images, generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images, generating a second type of synthetic image based on the plurality of wavelet coefficients, comparing the second type of synthetic image to the second type of training images, and updating the machine learning model based on the comparison.

[0074] In some embodiments, the first type of training image is a bright field image of the biological sample.

[0075] In some embodiments, the second type of training images are fluorescent images of the biological sample.

[0076] In some embodiments, the machine learning model comprises a generator and a discriminator.

[0077] In some embodiments, the machine learning model comprises a conditional GAN ​​model.

[0078] In some embodiments, the generator comprises multiple neural networks corresponding to multiple frequency groups.

[0079] In some embodiments, each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

[0080] In some embodiments, the multiple neural networks comprise multiple U-Net neural networks.

[0081] In some embodiments, the discriminator is a PatchGAN neural network.

[0082] In some embodiments, the machine learning model is further trained by generating images of a third type based on training images of the first type.

[0083] In some embodiments, the third type of image is a phase shift image.

[0084] In some embodiments, the machine learning model is trained by generating images of a fourth type based on training images of the first type.

[0085] In some embodiments, the fourth type of image comprises segmentation data.

[0086] In some embodiments, the first type of training images are captured using a microscope and according to a first illumination scheme.

[0087] In some embodiments, the first lighting scheme comprises one or more lighting patterns.

[0088] In some embodiments, the first type of training image is a portion of a bright field image array.

[0089] In some embodiments, the plurality of training images is a first plurality of training images, and the machine learning model has been trained by identifying the second lighting scheme based on the comparison; acquiring a second plurality of training images comprising one or more images of a first type, where the one or more images of the first type are acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images.

[0090] In some embodiments, the machine learning model is trained by acquiring a plurality of images of a first type using a microscope, and generating a plurality of synthetic images of a second type using the machine learning model based on the plurality of acquired images.

[0091] In some embodiments, the machine learning model is trained by training a classifier based on a plurality of synthetic images of a second type.

[0092] In some embodiments, the microscope is a first microscope, the classifier is a first classifier, and the machine learning model is trained by acquiring a plurality of images of a second type using a second microscope, training a second classifier based on the plurality of images of the second type, and comparing the performance of the first classifier and the second classifier.

[0093] In some embodiments, the second microscope is a fluorescence microscope.

[0094] An exemplary method of processing images of a biological sample and obtaining one or more output images includes obtaining multiple images of the biological sample using multiple configurations of an SLM of an optical system, where the SLM is located in an optical path between the biological sample and an image recording device, and inputting the multiple images of the biological sample into a trained machine learning model and obtaining one or more output images.

[0095] In some embodiments, at least one configuration of the plurality of configurations of the SLM is for generating one or more optical aberrations.

[0096] In some embodiments, generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof.

[0097] In some embodiments, at least one of the configurations of the SLM is intended to accentuate one or more features.

[0098] In some embodiments, the one or more features include cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof.

[0099] In some embodiments, at least one of the configurations of the SLM is for reducing optical aberrations.

[0100] In some embodiments, multiple SLM configurations are for acquiring images of a biological sample at different depths.

[0101] In some embodiments, the machine learning model is configured to generate an image of a second type based on the image of the first type.

[0102] In some embodiments, the first type of image is a bright field image.

[0103] In some embodiments, the second type of image is a fluorescent image.

[0104] In some embodiments, the second type of image is an enhanced version of the first type of image.

[0105] In some embodiments, the machine learning model is a GAN model or a self-supervised model.

[0106] In some embodiments, multiple images are acquired using multiple configurations of the light source of the optical system.

[0107] In some embodiments, the light source is an LED array of the optical system.

[0108] In some embodiments, at least one configuration of the plurality of SLM configurations is obtained by training a machine learning model, evaluating the trained machine learning model, and identifying at least one configuration based on the evaluation.

[0109] In some embodiments, the trained machine learning model is configured to receive an input image and output an enhanced version of the input image.

[0110] In some embodiments, the enhanced version of the input image comprises one or more enhanced cellular phenotypes.

[0111] An exemplary electronic device for processing images of a biological sample and obtaining one or more output images comprises one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for obtaining multiple images of the biological sample using multiple configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device, and inputting the multiple images of the biological sample into a trained machine learning model and obtaining one or more output images.

[0112] An exemplary non-transitory computer-readable storage medium stores one or more programs for processing images of a biological sample and obtaining one or more output images, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to obtain multiple images of the biological sample using multiple configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device, inputting the multiple images of the biological sample into a trained machine learning model, and obtaining one or more output images.

[0113] An exemplary method for classifying images of a biological sample includes acquiring multiple images of the biological sample using multiple configurations of an SLM of an optical system, where the SLM is located in an optical path between the biological sample and an image recording device, and inputting the multiple images of the biological sample into a trained machine learning model and obtaining one or more classification outputs.

[0114] In some embodiments, at least one configuration of the plurality of configurations of the SLM is for generating one or more optical aberrations.

[0115] In some embodiments, generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof.

[0116] In some embodiments, at least one of the configurations of the SLM is intended to accentuate one or more features.

[0117] In some embodiments, the one or more features include cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof.

[0118] In some embodiments, at least one of the configurations of the SLM is for reducing optical aberrations.

[0119] In some embodiments, multiple SLM configurations are for acquiring images of a biological sample at different depths.

[0120] In some embodiments, multiple images are acquired using multiple configurations of the light source of the optical system.

[0121] In some embodiments, the light source is an LED array of the optical system.

[0122] In some embodiments, at least one configuration of the plurality of SLM configurations is obtained by training a machine learning model, evaluating the trained machine learning model, and identifying at least one configuration based on the evaluation.

[0123] In some embodiments, the trained machine learning model is configured to receive an input image and detect one or more pre-defined objects in the input image.

[0124] In some embodiments, the predefined object comprises diseased tissue.

[0125] An exemplary electronic device for classifying images of a biological sample comprises one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for acquiring multiple images of the biological sample using multiple configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device, and inputting the multiple images of the biological sample into a trained machine learning model and obtaining one or more classification outputs.

[0126] An exemplary non-transitory computer-readable storage medium stores one or more programs for classifying images of a biological sample, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to acquire multiple images of the biological sample using multiple configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device, input the multiple images of the biological sample into a trained machine learning model, and obtain one or more classification outputs.

[0127] An exemplary method for training a machine learning model includes acquiring multiple images of a biological sample using multiple configurations of an SLM of an optical system, where the SLM is located in an optical path between the biological sample and an image recording device, and training the machine learning model using the multiple images.

[0128] In some embodiments, at least one configuration of the plurality of configurations of the SLM is for generating one or more optical aberrations.

[0129] In some embodiments, generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof.

[0130] In some embodiments, at least one of the configurations of the SLM is intended to accentuate one or more features.

[0131] In some embodiments, the one or more features include cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof.

[0132] In some embodiments, at least one of the configurations of the SLM is for reducing optical aberrations.

[0133] In some embodiments, at least one configuration of the multiple configurations of the SLM is for acquiring images of the biological sample at different depths.

[0134] In some embodiments, the machine learning model is configured to generate an image of a second type based on the image of the first type.

[0135] In some embodiments, the first type of image is a bright field image.

[0136] In some embodiments, the second type of image is a fluorescent image.

[0137] In some embodiments, the machine learning model is a GAN model or a self-supervised model.

[0138] In some embodiments, the machine learning model is a classification model.

[0139] In some embodiments, multiple images are acquired using multiple configurations of the light source of the optical system.

[0140] In some embodiments, the light source is an LED array of the optical system.

[0141] In some embodiments, training the machine learning model includes (a) training the machine learning model using a first image, where the first image is acquired using a first configuration of the SLM of the optical system; (b) evaluating the trained machine learning model; (c) identifying a second configuration of the SLM based on the evaluation; and (d) training the machine learning model using a second image, where the second image is acquired using the second configuration of the SLM of the optical system.

[0142] In some embodiments, the evaluation is based on a loss function of the machine learning model.

[0143] In some embodiments, the method further comprises repeating steps (a)-(d) until a threshold is met.

[0144] In some embodiments, the threshold indicates convergence of training.

[0145] In some embodiments, the trained machine learning model is configured to receive an input image and output an enhanced version of the input image.

[0146] In some embodiments, the enhanced version of the input image comprises one or more enhanced cellular phenotypes.

[0147] In some embodiments, the trained machine learning model is configured to receive an input image and detect one or more pre-defined objects in the input image.

[0148] In some embodiments, the predefined object comprises diseased tissue.

[0149] An exemplary electronic device for training a machine learning model comprises one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for acquiring multiple images of a biological sample using multiple configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device, and training a machine learning model using the multiple images.

[0150] An exemplary non-transitory computer-readable storage medium stores one or more programs for training a machine learning model, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to acquire multiple images of a biological sample using multiple configurations of an SLM of an optical system, the SLM being positioned in an optical path between the biological sample and an image recording device, and use the multiple images to train the machine learning model.

[0151] An exemplary method for generating an enhanced image of a biological sample includes the steps of: acquiring, using a microscope, an image of the biological sample illuminated using an illumination pattern of an illumination source, where the illumination pattern is determined by: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of illumination settings; and identifying the illumination pattern based on the plurality of weights of the trained machine learning model; and generating an enhanced image of the biological sample by inputting the acquired image of the biological sample into the trained machine learning model.

[0152] In some embodiments, the acquired image is a bright field image.

[0153] In some embodiments, the enhanced image is a fluorescence image, a phase image, or a combination thereof.

[0154] In some embodiments, the illumination source comprises an array of illumination emitters.

[0155] In some embodiments, the illumination source is an LED array.

[0156] In some embodiments, the lighting pattern indicates whether each lighting emitter is turned on or off and the intensity of each lighting emitter.

[0157] In some embodiments, each light setting of the plurality of light settings corresponds to a respective light emitter of the illumination source, and each weighting corresponds to an intensity of the respective light emitter.

[0158] In some embodiments, the classification model is configured to receive an input phase image or an input fluorescence image and output a classification result indicative of one of a plurality of predefined classes.

[0159] In some embodiments, the plurality of predefined classes includes a healthy class and a diseased class.

[0160] In some embodiments, the machine learning model is a GAN model that includes an attention layer, a discriminator, and a generator with multiple weights.

[0161] In some embodiments, the machine learning model is a conditional GAN ​​model.

[0162] In some embodiments, the generator comprises multiple neural networks corresponding to multiple frequency groups.

[0163] In some embodiments, each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

[0164] In some embodiments, the multiple neural networks comprise multiple U-Net neural networks.

[0165] In some embodiments, the discriminator is a PatchGAN neural network.

[0166] In some embodiments, training a machine learning model using the trained classification model includes applying a plurality of weights to a plurality of bright-field training images; aggregating the plurality of weighted bright-field training images into an aggregated bright-field image; inputting the aggregated bright-field training images into the machine learning model to obtain enhanced training images and a generator loss; inputting the enhanced training images into a trained classifier to obtain a classifier loss; augmenting the generator loss based on the classifier loss; and updating the plurality of weights based on the augmented generator loss.

[0167] In some embodiments, the method further comprises classifying the enhanced image using the trained classifier.

[0168] In some embodiments, the method further comprises displaying the enhanced image.

[0169] An exemplary system for generating an enhanced image of a biological sample comprises one or more processors, a memory, and one or more programs, the one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: acquiring, using a microscope, an image of the biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern being determined by: training a classification model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of illumination settings; and identifying the illumination pattern based on the plurality of weights of the trained machine learning model; and generating an enhanced image of the biological sample by inputting the acquired image of the biological sample into the trained machine learning model.

[0170] An exemplary non-transitory computer-readable storage medium stores one or more programs for generating an enhanced image of a biological sample, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to acquire, using a microscope, an image of the biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern being determined by: training a classification model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of illumination settings; and identifying the illumination pattern based on the plurality of weights of the trained machine learning model, and generate an enhanced image of the biological sample by inputting the acquired image of the biological sample into the trained machine learning model.

[0171] An exemplary method for evaluating a treatment for a disease of interest includes receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest, receiving a second plurality of images depicting a second set of untreated biological samples affected by the disease of interest, receiving a third plurality of images depicting a third set of treated biological samples affected by the disease of interest and treated with a treatment, inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images, inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images, inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images, and comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment.

[0172] In some embodiments, the first plurality of images, the second plurality of images, and the third plurality of images are bright field images.

[0173] In some embodiments, the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images are fluorescent images.

[0174] In some embodiments, the first plurality of weighted images, the second plurality of weighted images, and the third plurality of weighted images are phase images.

[0175] In some embodiments, comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment includes identifying a signal in each image that is associated with a biomarker.

[0176] In some embodiments, comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment further includes determining a first distribution based on the signals of the biomarkers in the first plurality of enhanced images, determining a second distribution based on the signals of the biomarkers in the second plurality of enhanced images, and determining a third distribution based on the signals of the biomarkers in the third plurality of enhanced images.

[0177] In some embodiments, comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment further includes comparing the first distribution, the second distribution, and the third distribution and evaluating the treatment.

[0178] In some embodiments, comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment includes determining a score for each image, indicative of a status of the disease of interest.

[0179] In some embodiments, comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment further includes determining a first distribution based on the scores of the first plurality of enhanced images, determining a second distribution based on the scores of the second plurality of enhanced images, and determining a third distribution based on the scores of the third plurality of enhanced images.

[0180] In some embodiments, comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment further includes comparing the first distribution, the second distribution, and the third distribution and evaluating the treatment.

[0181] In some embodiments, the treatment is a first treatment, and the method further includes receiving a fourth plurality of images depicting a fourth set of treated biological samples afflicted by a disease of interest and treated with a second treatment; inputting the fourth plurality of images into the trained machine learning model to obtain a fourth plurality of enhanced images; and comparing the first plurality of enhanced images, the second plurality of enhanced images, the third plurality of enhanced images, and the fourth plurality of enhanced images to compare the first treatment and the second treatment.

[0182] In some embodiments, the method further comprises selecting a treatment from the first treatment and the second treatment based on the comparison.

[0183] In some embodiments, the method further comprises administering a selected treatment.

[0184] In some embodiments, the method further comprises providing a medical recommendation based on the selected treatment.

[0185] In some embodiments, the trained machine learning model is a GAN model, which comprises a discriminator and a generator.

[0186] In some embodiments, the machine learning model is a conditional GAN ​​model.

[0187] In some embodiments, the generator comprises multiple neural networks corresponding to multiple frequency groups.

[0188] In some embodiments, each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

[0189] In some embodiments, the discriminator is a PatchGAN neural network.

[0190] An exemplary system for evaluating a treatment for a disease of interest includes one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, and includes steps of receiving a first plurality of images depicting a first set of healthy biological samples that are not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples that are affected by the disease of interest; and receiving a second plurality of images depicting a second set of untreated biological samples that are affected by the disease of interest and treated with the treatment. and one or more programs including instructions for receiving a third plurality of images depicting a third set of treated biological samples, inputting the first plurality of images into the trained machine learning model and obtaining a first plurality of enhanced images, inputting the second plurality of images into the trained machine learning model and obtaining a second plurality of enhanced images, inputting the third plurality of images into the trained machine learning model and obtaining a third plurality of enhanced images, and comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment.

[0191] An exemplary non-transitory computer readable storage medium stores one or more programs for evaluating a treatment for a disease of interest, the one or more programs comprising instructions, when executed by one or more processors of an electronic device, to cause the electronic device to receive a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest, receive a second plurality of images depicting a second set of untreated biological samples affected by the disease of interest, receive a third plurality of images depicting a third set of treated biological samples affected by the disease of interest and treated with a treatment, input the first plurality of images into a trained machine learning model, obtain a first plurality of enhanced images, input the second plurality of images into the trained machine learning model, obtain the second plurality of enhanced images, input the third plurality of images into the trained machine learning model, obtain the third plurality of enhanced images, compare the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images, and evaluate the treatment. The present invention provides, for example, the following: (Item 1) 1. A method for training a machine learning model and generating images of a biological sample, comprising: acquiring a plurality of training images, the plurality of training images comprising: A first type of training images; and The second type of training images and and generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; and A method comprising: (Item 2) 2. The method of claim 1, wherein the first type of training image is a bright-field image of a biological sample. (Item 3) 3. The method of claim 2, wherein the second type of training images are fluorescent images of the biological sample. (Item 4) 4. The method of any of claims 1-3, wherein the machine learning model comprises a generator and a discriminator. (Item 5) 5. The method of claim 4, wherein the machine learning model comprises a conditional GAN ​​model. (Item 6) 6. The method of claim 4, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. (Item 7) 7. The method of claim 6, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. (Item 8) 8. The method of claim 6, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. (Item 9) 9. The method of any of items 5-8, wherein the discriminator is a PatchGAN neural network. (Item 10) 10. The method of any of items 1-9, further comprising generating a third type of image based on the first type of training images. (Item 11) Item 11. The method according to item 10, wherein the third type of image is a phase shift image. (Item 12) 12. The method of any of items 1-11, further comprising generating a fourth type of image based on the first type of training images. (Item 13) Item 13. The method of item 12, wherein the fourth type of image comprises segmentation data. (Item 14) 14. The method according to any of items 1-13, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. (Item 15) Item 15. The method of item 14, wherein the first lighting scheme comprises one or more lighting patterns. (Item 16) 16. The method of any of items 14-15, wherein the first type of training images is part of a bright field image array. (Item 17) The plurality of training images is a first plurality of training images, and the method further comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 17. The method according to any one of items 14 to 16, comprising: (Item 18) acquiring a plurality of images of the first type using a microscope; and generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 17. The method of any of items 1-16, further comprising: (Item 19) 20. The method of claim 18, further comprising training a classifier based on a plurality of synthetic images of the second type. (Item 20) the microscope is a first microscope and the classifier is a first classifier; acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. 20. The method of claim 19, further comprising: (Item 21) 21. The method of claim 20, wherein the second microscope is a fluorescent microscope. (Item 22) 1. A method for generating an enhanced image of a biological sample, comprising: obtaining an image of the biological sample using a microscope; generating an enhanced image of the biological sample based on the image using a machine learning model, the machine learning model comprising: acquiring a plurality of training images, the plurality of training images comprising: A first type of training images; and The second type of training images and and generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; and Trained by A method comprising: (Item 23) 23. The method of claim 22, wherein the first type of training images are bright-field images of a biological sample. (Item 24) 23. The method of claim 22, wherein the second type of training images are fluorescent images of the biological sample. (Item 25) 25. The method of any of claims 22-24, wherein the machine learning model comprises a generator and a discriminator. (Item 26) 26. The method of claim 25, wherein the machine learning model comprises a conditional GAN ​​model. (Item 27) 27. The method of any of claims 25-26, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. (Item 28) 28. The method of claim 27, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. (Item 29) 29. The method of any of claims 27-28, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. (Item 30) 30. The method of any of items 26-29, wherein the discriminator is a PatchGAN neural network. (Item 31) 31. The method according to any of items 23-30, further comprising generating an image of a third type based on training images of the first type. (Item 32) 32. The method of claim 31, wherein the third type of image is a phase shift image. (Item 33) 33. The method according to any of items 23-32, further comprising generating an image of a fourth type based on training images of the first type. (Item 34) Item 34. The method of item 33, wherein the fourth type of image comprises segmentation data. (Item 35) 35. The method according to any of items 23-34, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. (Item 36) Item 36. The method of item 35, wherein the first lighting scheme comprises one or more lighting patterns. (Item 37) 37. The method of any of items 35-36, wherein the first type of training images is part of a bright field image array. (Item 38) The plurality of training images is a first plurality of training images, and the method further comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 38. The method according to any one of items 35 to 37, comprising: (Item 39) acquiring a plurality of images of the first type using a microscope; and generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 39. The method according to any one of items 35-38, further comprising: (Item 40) 40. The method of claim 39, further comprising training a classifier based on a plurality of synthetic images of the second type. (Item 41) the microscope is a first microscope and the classifier is a first classifier; acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. 41. The method of claim 40, further comprising: (Item 42) Item 42. The method of item 41, wherein the second microscope is a fluorescence microscope. (Item 43) 1. A system for training a machine learning model and generating images of a biological sample, comprising: 1. A computing system comprising one or more processors and one or more memories storing a machine learning model, the computing system configured to receive a plurality of training images of a first type and one training image of a second type, the computing system comprising: generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; and A computing system configured to: A system comprising: (Item 44) Item 44. The system of item 43, wherein the first type of training image is a bright-field image of a biological sample. (Item 45) 45. The system of any of items 43-44, wherein the second type of training images are fluorescent images of the biological sample. (Item 46) 46. ​​The system of any of claims 43-45, wherein the machine learning model comprises a generator and a discriminator. (Item 47) Item 47. The system of item 46, wherein the machine learning model comprises a conditional GAN ​​model. (Item 48) 48. The system of any of claims 46-47, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. (Item 49) Item 49. The system of item 48, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. (Item 50) 50. The system of any of claims 48-49, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. (Item 51) 51. The system of any of items 46-50, wherein the discriminator is a PatchGAN neural network. (Item 52) 52. The system of any of items 43-51, wherein the computing system is further configured to generate a third type of image based on the first type of training images. (Item 53) Item 53. The system of item 52, wherein the third type of image is a phase shift image. (Item 54) 54. The system of any of items 43-53, wherein the computing system is further configured to generate a fourth type of image based on the first type of training images. (Item 55) Item 55. The system of item 54, wherein the fourth type of image comprises segmentation data. (Item 56) 56. The system of any of items 43-55, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. (Item 57) Item 57. The system of item 56, wherein the first lighting scheme comprises one or more lighting patterns. (Item 58) 58. The system of any of items 56-57, wherein the first type of training image is a portion of a bright field image array. (Item 59) The plurality of training images is a first plurality of training images, and the computing system further comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 59. The system of any of items 56-58, configured to: (Item 60) The computing system further comprises: acquiring a plurality of images of the first type using a microscope; and generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 59. The system of any of items 43-59, configured to: (Item 61) 60. The system of claim 59, wherein the computing system is further configured to train a classifier based on the plurality of synthetic images of the second type. (Item 62) The microscope is a first microscope, the classifier is a first classifier, and the computing system further comprises: acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. Item 62. The system of item 61, configured to perform the following: (Item 63) Item 63. The system of item 62, wherein the second microscope is a fluorescence microscope. (Item 64) 1. A system for generating an enhanced image of a biological sample, comprising: 1. A computing system comprising one or more processors and one or more memories storing a machine learning model, the computing system configured to receive an image of a biological sample obtained from a microscope and generate an enhanced image of the biological sample based on the image using a machine learning model, the machine learning model comprising: acquiring a plurality of training images, the plurality of training images comprising: A first type of training images; and The second type of training images and and generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; and Trained by,Computing Systems A system comprising: (Item 65) Item 65. The system of item 64, wherein the first type of training images are bright-field images of a biological sample. (Item 66) Item 65. The system of item 64, wherein the second type of training images are fluorescent images of the biological sample. (Item 67) 67. The system of any of claims 64-66, wherein the machine learning model comprises a generator and a discriminator. (Item 68) Item 68. The system of item 67, wherein the machine learning model comprises a conditional GAN ​​model. (Item 69) 69. The system of any of claims 67-68, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. (Item 70) 70. The system of claim 69, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. (Item 71) 71. The system of any of claims 69-70, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. (Item 72) 72. The system of any of claims 67-71, wherein the discriminator is a PatchGAN neural network. (Item 73) 73. The system of any of claims 64-72, wherein the machine learning model is further trained by generating a third type of image based on the first type of training images. (Item 74) Item 74. The system of item 73, wherein the third type of image is a phase shift image. (Item 75) 75. The system of any of claims 64-74, wherein the machine learning model is trained by generating a fourth type of image based on the first type of training images. (Item 76) Item 76. The system of item 75, wherein the fourth type of image comprises segmentation data. (Item 77) 77. The system of any of items 64-76, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. (Item 78) Item 78. The system of item 77, wherein the first lighting scheme comprises one or more lighting patterns. (Item 79) 8. The system of claim 7, wherein the first type of training image is a portion of a bright field image array. (Item 80) The plurality of training images is a first plurality of training images, and the machine learning model comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 80. The system of any of items 77-79, wherein the system is trained by (Item 81) The machine learning model is acquiring a plurality of images of the first type using a microscope; and generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 81. The system of any of items 77-80, wherein the system is trained by (Item 82) Item 82. The system of item 81, wherein the machine learning model is trained by training a classifier based on a plurality of synthetic images of the second type. (Item 83) The microscope is a first microscope, the classifier is a first classifier, and the machine learning model is acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. Item 83. The system of item 82, which is trained by (Item 84) Item 84. The system of item 83, wherein the second microscope is a fluorescence microscope. (Item 85) 1. A method for processing an image of a biological sample to obtain one or more output images, comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model to obtain the one or more output images; A method comprising: (Item 86) Item 86. The method of item 85, wherein at least one of the plurality of configurations of the SLM is for generating one or more optical aberrations. (Item 87) 87. The method of claim 86, wherein generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof. (Item 88) 87. The method according to any of items 85-86, wherein at least one of the configurations of the SLM is for enhancing one or more features. (Item 89) 89. The method of claim 88, wherein the one or more features comprise cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. (Item 90) 90. The method according to any of items 85-89, wherein at least one configuration of the plurality of configurations of the SLM is for reducing optical aberrations. (Item 91) 91. The method according to any of items 85-90, wherein the multiple SLM configurations are for acquiring images of the biological sample at different depths. (Item 92) 92. The method of any of items 85-91, wherein the machine learning model is configured to generate an image of a second type based on an image of a first type. (Item 93) Item 93. The method of item 92, wherein the first type of image is a bright field image. (Item 94) 93. The method of claim 92, wherein the second type of image is a fluorescent image. (Item 95) Item 93. The method of item 92, wherein the second type of image is an enhanced version of the first type of image. (Item 96) The method according to any of items 92-95, wherein the machine learning model is a GAN model or a self-supervised model. (Item 97) 97. The method of any of items 85-96, wherein the multiple images are acquired using multiple configurations of a light source of the optical system. (Item 98) Item 98. The method of item 97, wherein the light source is an LED array of the optical system. (Item 99) At least one of the plurality of SLM configurations comprises: training the machine learning model; evaluating the trained machine learning model; and identifying the at least one configuration based on the evaluation; and 99. The method according to any of items 85-98, obtained by (Item 100) 99. The method of claim 85, wherein the trained machine learning model is configured to receive an input image and output an enhanced version of the input image. (Item 101) Item 101. The method of item 100, wherein the enhanced version of the input image comprises one or more enhanced cell phenotypes. (Item 102) 1. An electronic device for processing images of a biological sample and obtaining one or more output images, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model to obtain the one or more output images; and one or more programs containing instructions for carrying out the An electronic device comprising: (Item 103) 1. A non-transitory computer readable storage medium storing one or more programs for processing images of a biological sample and obtaining one or more output images, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model to obtain the one or more output images; A non-transitory computer-readable storage medium for causing (Item 104) 1. A method for classifying an image of a biological sample, comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model and obtaining one or more classification outputs; A method comprising: (Item 105) Item 105. The method of item 104, wherein at least one of the plurality of configurations of the SLM is for generating one or more optical aberrations. (Item 106) Item 106. The method of item 105, wherein generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof. (Item 107) 107. The method according to any of claims 104-106, wherein at least one of the plurality of configurations of the SLM is for enhancing one or more features. (Item 108) 108. The method of claim 107, wherein the one or more features comprise cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. (Item 109) Item 109. The method according to any of items 104-108, wherein at least one configuration of the plurality of configurations of the SLM is for reducing optical aberrations. (Item 110) 109. The method according to any of claims 104-109, wherein the multiple SLM configurations are for acquiring images of the biological sample at different depths. (Item 111) 111. The method of any of items 104-110, wherein the multiple images are acquired using multiple configurations of a light source of the optical system. (Item 112) Item 112. The method of item 111, wherein the light source is an LED array of the optical system. (Item 113) At least one of the plurality of SLM configurations comprises: training the machine learning model; evaluating the trained machine learning model; and identifying the at least one configuration based on the evaluation; and The method according to any of items 104-112, obtained by (Item 114) Item 114. The method of any of Items 104-113, wherein the trained machine learning model is configured to receive an input image and detect one or more predefined objects within the input image. (Item 115) 15. The method of claim 114, wherein the predefined object comprises diseased tissue. (Item 116) 1. An electronic device for classifying images of a biological sample, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model and obtaining one or more classification outputs; one or more programs including instructions for performing An electronic device comprising: (Item 117) 1. A non-transitory computer readable storage medium storing one or more programs for classifying images of biological samples, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model and obtaining one or more classification outputs; A non-transitory computer-readable storage medium for causing (Item 118) 1. A method for training a machine learning model, comprising: - acquiring a plurality of images of a biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; training the machine learning model using the plurality of images; and A method comprising: (Item 119) Item 119. The method of item 118, wherein at least one of the plurality of configurations of the SLM is for generating one or more optical aberrations. (Item 120) Item 120. The method of item 119, wherein generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof. (Item 121) 121. The method according to any of items 118-120, wherein at least one of the plurality of configurations of the SLM is for enhancing one or more features. (Item 122) 122. The method of claim 121, wherein the one or more features comprise cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. (Item 123) Item 123. The method according to any of items 118-122, wherein at least one configuration of the plurality of configurations of the SLM is for reducing optical aberrations. (Item 124) 124. The method according to any of items 118-123, wherein at least one configuration of the plurality of configurations of the SLM is for acquiring images of the biological sample at different depths. (Item 125) 125. The method of any of claims 118-124, wherein the machine learning model is configured to generate an image of a second type based on an image of a first type. (Item 126) Item 126. The method of item 125, wherein the first type of image is a bright field image. (Item 127) Item 126. The method of item 125, wherein the second type of image is a fluorescent image. (Item 128) The method according to any of items 118-127, wherein the machine learning model is a GAN model or a self-supervised model. (Item 129) 129. The method of any of claims 118-128, wherein the machine learning model is a classification model. (Item 130) 130. The method of any of items 118-129, wherein the multiple images are acquired using multiple configurations of a light source of the optical system. (Item 131) Item 131. The method of item 130, wherein the light source is an LED array of the optical system. (Item 132) Training the machine learning model includes: (a) training the machine learning model using a first image, the first image being acquired using a first configuration of an SLM of the optical system; (b) evaluating the trained machine learning model; and (c) identifying a second configuration of the SLM based on the evaluation; and (d) training the machine learning model using a second image, the second image being acquired using a second configuration of the SLM of the optical system; 132. The method according to any one of items 118-131, comprising: (Item 133) Item 113. The method of item 112, wherein the evaluation is based on a loss function of the machine learning model. (Item 134) 114. The method of any of items 112-113, further comprising repeating steps (a)-(d) until a threshold is met. (Item 135) Item 115. The method of item 114, wherein the threshold indicates convergence of the training. (Item 136) Item 136. The method of any of Items 118-135, wherein the trained machine learning model is configured to receive an input image and output an enhanced version of the input image. (Item 137) Item 137. The method of item 136, wherein the enhanced version of the input image comprises one or more enhanced cell phenotypes. (Item 138) Item 136. The method of any of Items 118-135, wherein the trained machine learning model is configured to receive an input image and detect one or more predefined objects within the input image. (Item 139) 19. The method of claim 138, wherein the predefined object comprises diseased tissue. (Item 140) 1. An electronic device for training a machine learning model, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: - acquiring a plurality of images of a biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; training the machine learning model using the plurality of images; and and one or more programs containing instructions for carrying out the An electronic device comprising: (Item 141) 1. A non-transitory computer-readable storage medium storing one or more programs for training a machine learning model, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: - acquiring a plurality of images of a biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; training the machine learning model using the plurality of images; and A non-transitory computer-readable storage medium for causing (Item 142) 1. A method for generating an enhanced image of a biological sample, comprising: 2. Using a microscope, acquiring an image of a biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern comprising: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of lighting settings; identifying the lighting pattern based on a plurality of weights of the trained machine learning model; and generating an enhanced image of the biological sample by inputting an acquired image of the biological sample into the trained machine learning model; A method comprising: (Item 143) Item 143. The method of item 142, wherein the acquired image is a bright field image. (Item 144) 144. The method according to any of items 142-143, wherein the enhanced image is a fluorescence image, a phase image, or a combination thereof. (Item 145) 5. The method of claim 142, wherein the illumination source comprises an array of illumination emitters. (Item 146) Item 146. The method of item 145, wherein the illumination source is an LED array. (Item 147) Item 147. The method of any of items 142-146, wherein the lighting pattern indicates whether each lighting emitter is turned on or off and the intensity of each lighting emitter. (Item 148) Item 148. The method of any of items 145-147, wherein each lighting setting of the plurality of lighting settings corresponds to a respective lighting emitter of the lighting source, and each weighting corresponds to an intensity of the respective lighting emitter. (Item 149) Item 149. The method of any of items 142-148, wherein the classification model is configured to receive an input phase image or an input fluorescence image and output a classification result indicative of one of a plurality of predefined classes. (Item 150) Item 149. The method of item 149, wherein the plurality of predefined classes comprises a healthy class and a diseased class. (Item 151) Item 151. The method of item 150, wherein the machine learning model is a GAN model comprising an attention layer comprising the plurality of weights, a discriminator, and a generator. (Item 152) Item 152. The method of item 151, wherein the machine learning model is a conditional GAN ​​model. (Item 153) Item 153. The method of any of items 151-152, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. (Item 154) Item 154. The method of item 153, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. (Item 155) 5. The method of claim 153, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. (Item 156) Item 156. The method of any of items 151-155, wherein the discriminator is a PatchGAN neural network. (Item 157) training the machine learning model using the trained classification model, applying the plurality of weights to a plurality of bright-field training images; aggregating the plurality of weighted brightfield training images into an aggregated brightfield image; inputting the aggregated bright-field training images into the machine learning model to obtain enhanced training images and a generator loss; inputting the enhanced training images into the trained classifier and obtaining a classifier loss; augmenting the generator loss based on the classifier loss; updating the weights based on the extended generator loss; and 157. The method according to any one of items 151-156, comprising: (Item 158) 158. The method of any of claims 142-157, further comprising classifying the enhanced image using the trained classifier. (Item 159) 159. The method of any of items 142-158, further comprising displaying the enhanced image. (Item 160) 1. A system for generating an enhanced image of a biological sample, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: 2. Using a microscope, acquiring an image of a biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern comprising: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of lighting settings; identifying the lighting pattern based on a plurality of weights of the trained machine learning model; and generating an enhanced image of the biological sample by inputting an acquired image of the biological sample into the trained machine learning model; and one or more programs containing instructions for carrying out the A system comprising: (Item 161) 1. A non-transitory computer readable storage medium storing one or more programs for generating an enhanced image of a biological sample, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: 2. Using a microscope, acquiring an image of a biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern comprising: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of lighting settings; identifying the lighting pattern based on a plurality of weights of the trained machine learning model; and generating an enhanced image of the biological sample by inputting an acquired image of the biological sample into the trained machine learning model; A non-transitory computer-readable storage medium for causing (Item 162) 1. A method for evaluating a treatment for a disease of interest, comprising: receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples afflicted by the disease of interest; receiving a third plurality of images depicting a third set of treated biological samples afflicted with the disease of interest and being treated with the treatment; inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images; inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images; inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment; A method comprising: (Item 163) Item 163. The method of item 162, wherein the first plurality of images, the second plurality of images, and the third plurality of images are bright field images. (Item 164) Item 164. The method of any of items 162-163, wherein the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images are fluorescence images. (Item 165) Item 164. The method of any of items 162-163, wherein the first plurality of weighted images, the second plurality of weighted images, and the third plurality of weighted images are phase images. (Item 166) 166. The method of any of claims 162-165, wherein comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment includes identifying a signal in each image that is associated with a biomarker. (Item 167) Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment further includes: determining a first distribution based on signals of the biomarkers in the first plurality of enhanced images; determining a second distribution based on signals of the biomarkers in the second plurality of enhanced images; determining a third distribution based on signals of the biomarkers in the third plurality of enhanced images; Item 167. The method according to item 166, comprising: (Item 168) Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment further includes: 168. The method of claim 167, comprising comparing the first distribution, the second distribution, and the third distribution to evaluate the treatment. (Item 169) 166. The method of any of items 162-165, wherein comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment includes determining a score for each image indicative of a state of the disease of interest. (Item 170) Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment further includes: determining a first distribution based on the scores of the first plurality of enhanced images; determining a second distribution based on the scores of the second plurality of enhanced images; determining a third distribution based on the scores of the third plurality of enhanced images; Item 169. The method according to item 169, comprising: (Item 171) Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment further includes: 171. The method of claim 170, comprising comparing the first distribution, the second distribution, and the third distribution to evaluate the treatment. (Item 172) The treatment is a first treatment and the method further comprises: receiving a fourth plurality of images depicting a fourth set of treated biological samples afflicted with the disease of interest and being treated with a second treatment; inputting the fourth plurality of images into the trained machine learning model to obtain a fourth plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, the third plurality of enhanced images, and the fourth plurality of enhanced images and comparing the first treatment and the second treatment; The method according to any one of items 162-171, comprising: (Item 173) 173. The method of claim 172, further comprising selecting a treatment from the first treatment and the second treatment based on the comparison. (Item 174) 174. The method of claim 173, further comprising administering the selected treatment. (Item 175) Item 174. The method of item 173, further comprising providing a medical recommendation based on the selected treatment. (Item 176) Item 162-175. The method of any of items 162-175, wherein the trained machine learning model is a GAN model comprising a discriminator and a generator. (Item 177) Item 177. The method of item 176, wherein the machine learning model is a conditional GAN ​​model. (Item 178) Item 178. The method of any of items 176-177, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. (Item 179) Item 179. The method of item 178, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. (Item 180) 180. The method of any of claims 176-179, wherein the discriminator is a PatchGAN neural network. (Item 181) 1. A system for evaluating a treatment for a disease of interest, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples afflicted by the disease of interest; receiving a third plurality of images depicting a third set of treated biological samples afflicted with the disease of interest and being treated with the treatment; inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images; inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images; inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment; and one or more programs containing instructions for carrying out the A system comprising: (Item 182) 1. A non-transitory computer readable storage medium storing one or more programs for evaluating a treatment for a disease of interest, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples afflicted by the disease of interest; receiving a third plurality of images depicting a third set of treated biological samples afflicted with the disease of interest and being treated with the treatment; inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images; inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images; inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment; A non-transitory computer-readable storage medium for causing [Brief description of the drawings]

[0192] [Figure 1] FIG. 1 illustrates an example process for training a machine learning model configured to generate synthetic data (e.g., image data) in a large-scale and low-cost manner, according to some embodiments.

[0193] [Figure 2A] FIG. 2A illustrates an example process for training and applying a machine learning model, according to some embodiments.

[0194] [Figure 2B] FIG. 2B illustrates an example process for training and applying a machine learning model, according to some embodiments.

[0195] [Figure 3A] FIG. 3A illustrates a process for training a machine learning model, according to some embodiments.

[0196] [Figure 3B] FIG. 3B illustrates the operation of an example generator, according to some embodiments.

[0197] [Figure 3C] FIG. 3C illustrates a process for training a machine learning model, according to some embodiments.

[0198] [Figure 3D] FIG. 3D illustrates the process of training a machine learning model, according to some embodiments.

[0199] [Figure 4] FIG. 4 illustrates an example image formation model, according to some embodiments.

[0200] [Diagram 5] FIG. 5 illustrates the output of an example trained generator using a static microscope setting, according to some embodiments.

[0201] [Figure 6A] FIG. 6A illustrates an example process for determining robustness of a generated image, according to some embodiments.

[0202] [Figure 6B] FIG. 6B illustrates an example comparison between two classifiers, according to some embodiments.

[0203] [Figure 7A] FIG. 7A illustrates an exemplary method for training a machine learning model and generating images of a biological sample.

[0204] [Figure 7B] FIG. 7B illustrates an exemplary method for generating an enhanced image of a biological sample.

[0205] [Figure 8] FIG. 8 illustrates an exemplary electronic device, according to some embodiments.

[0206] [Figure 9] FIG. 9 illustrates an example optical system, according to some embodiments.

[0207] [Figure 10] FIG. 10 illustrates an exemplary method for analyzing an image of a biological sample using a programmable SLM of an optical system, according to some embodiments.

[0208] [Figure 11] FIG. 11 illustrates an example method for training a machine learning model, according to some embodiments.

[0209] [Figure 12-1] 12A and 12B illustrate side-by-side comparisons of classification results of two classification models based on the same input images, according to some embodiments. Figures 12C and 12D illustrate side-by-side comparisons of the same classification results in Figures 9A and 9B, respectively, using different color schemes to show the resistance of the two models to batch effects, according to some embodiments.

[0210] [Figure 12-2] FIG. 12E illustrates Euclidean distance metrics corresponding to the actual image set, the generated image set, and the true batch-invariant image set in accordance with some embodiments.

[0211] [Figure 13A] FIG. 13A illustrates an example generated image, according to some embodiments.

[0212] [Figure 13B] FIG. 13B illustrates an example generated image, according to some embodiments.

[0213] [Figure 13C] FIG. 13C illustrates an example generated image, according to some embodiments.

[0214] [Figure 14A] 14A and 14B illustrate an example process for training a machine learning model (e.g., a GAN model) configured to generate synthetic data (e.g., image data) and identifying optimal lighting patterns for capturing input data for the machine learning model, according to some embodiments. [Figure 14B]14A and 14B illustrate an example process for training a machine learning model (e.g., a GAN model) configured to generate synthetic data (e.g., image data) and identifying optimal lighting patterns for capturing input data for the machine learning model, according to some embodiments.

[0215] [Figure 15] FIG. 15 illustrates a schematic for training a machine learning model, according to some embodiments.

[0216] [Figure 16-1] 16A illustrates a synthetic image generated by an exemplary GAN model to analyze NASH disease, according to some embodiments. FIG. 16B illustrates downstream analysis of the generated image, according to some embodiments.

[0217] [Figure 16-2] FIG. 16C illustrates an exemplary process for evaluating candidate treatments, according to some embodiments.

[0218] [Figure 16-3] FIG. 16D illustrates an exemplary process for evaluating candidate treatments, according to some embodiments.

[0219] [Figure 17A] FIG. 17A illustrates a synthetic image generated by an exemplary GAN model for considering tuberous sclerosis complex ("TSC"), according to some embodiments.

[0220] [Figure 17B] FIG. 17B illustrates an exemplary process for evaluating candidate treatments, according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0221] Detailed Description of the Invention The present disclosure includes methods, systems, electronic devices, non-transitory storage media, and apparatuses for implementing ML-based generation of image data on a large scale. The generated image data (e.g., image data of biological samples) can provide sufficient richness and depth for downstream processing (e.g., phenotyping). Furthermore, embodiments of the present disclosure comprise a set of computer and hardware optimization methods that extend the current dimensions of classical microscopy techniques.

[0222] The embodiments of the present disclosure can process multiple bright-field images of a biological sample to produce an enhanced image of the biological sample. Enhanced images include, but are not limited to, fluorescent images, phase shift images, semantic maps, polarization maps, refraction maps (2D and 3D), absorbance maps, and other image modalities. Bright-field images of biological samples can be acquired at large scale and low cost due to inexpensive equipment (e.g., relative to fluorescent microscopes), ease of clinical deployment, and low processing and storage resource requirements. Acquiring bright-field images is generally non-invasive and involves low phototoxicity. Thus, bright-field images can be acquired efficiently and at large scale. Enhanced images provide sufficient richness and depth for downstream processing (e.g., phenotype exploration).

[0223] An embodiment of the present disclosure includes a machine learning model that is trained to receive a first type of image and convert the input image to another imaging modality. An exemplary machine learning model can receive a first type of image and convert the input image to a second type of image (e.g., an enhanced image). In some embodiments, the different image types refer to different imaging modalities. In some embodiments, the first type of image is a bright field image. For example, the bright field image can be captured from illuminating an in vitro (or biopsy) cell sample with an inexpensive LED array. The second type of image includes a fluorescent image. The resulting fluorescent image exhibits high contrast features that are not directly visible in the bright field image and can be used for downstream processing (e.g., phenotyping).

[0224] The embodiments of the present disclosure reduce or eliminate the need to capture actual fluorescent images (or other specialized image modalities) of biological samples for downstream analysis, allowing brightfield images to be widely used for a variety of purposes. This is particularly beneficial for live cell imaging. For example, the disclosed methods may be used for monitoring and optimization of cell differentiation experimental protocols. In the context of chemical or genetic perturbations, time-consuming activities coupled with cell staining and fixation may be avoided. In some embodiments, the dosing time, which is the incubation time of the drug with the cells under observation, may also be optimized by the software. Researchers will no longer need to arbitrarily determine the best incubation time, as the software will be able to inform the researcher of the optimized incubation time. Furthermore, the machine learning techniques used to convert brightfield images to other modalities require lower processing and storage resource utilization. Thus, the embodiments of the present disclosure present a technical improvement to the field of medical imaging while highlighting the operability and functionality of computing systems.

[0225] Embodiments of the present disclosure further include a machine learning model that is trained to receive an image of the first type and convert the input image into an image of a third type, in some embodiments, the image of the third type includes image data indicative of various optical properties (e.g., phase shifts) of the captured biological sample.

[0226] Embodiments of the present disclosure further include a machine learning model that is trained to receive an image of the first type and convert the input image into an image of a fourth type, in some embodiments, the image of the fourth type includes image data indicative of compartmentalization data (e.g., cell boundaries).

[0227] Those skilled in the art will appreciate that embodiments of the present disclosure may further convert the input image into a number of other types of images that capture various imaging characteristics, such as semantic maps, polarization maps, refraction maps (2D or 3D), absorbance maps, etc.

[0228] In some embodiments, a single machine learning model is trained to perform multiple transformation tasks simultaneously. For example, the same machine learning model can receive a first type of image and generate multiple types of images (e.g., a second type, a third type, a fourth type of image). The machine learning model can be a generative adversarial network ("GAN") model. For example, the GAN network can be a conditional GAN ​​("cGAN") model.

[0229] In some embodiments, the machine learning model converts the input image into its corresponding wavelet coefficients and generates one or more output images in the wavelet coefficients. The compact and multi-scale representation of the image, combined with the inherent nonlinear options of the neural network, achieves several goals at once. First, the wavelet-based representation solves the inherent class imbalance problem faced by most of the generative models. Specifically, most input image data comprises abundant low-frequency signals, but fewer high-frequency signals. Second, the wavelet-based representation extracts and maintains the overall geometry of the input image data. The discriminator of the model ensures that the actual and generated images (e.g., actual vs. generated fluorescent images) are indistinguishable. The generated fluorescent images or other enhanced image modalities correspond to virtual staining of the sample. Due to the low phototoxicity of the bright-field imaging modality and its availability in clinical settings, the virtual staining according to the embodiments of the present disclosure can be performed on live cells as well as biopsy samples.

[0230] The embodiments of the present disclosure further include a hardware optimization method. For example, the embodiments of the present disclosure further can dynamically optimize the illumination scheme of a microscope (e.g., a microscope that acquires a first type of image). In some embodiments, the microscope used to capture the first type of image can be adjusted or programmed to provide a different illumination scheme during the training process. During the training of the machine learning model, an optimal illumination scheme for capturing the first type of image can be identified. The optimal illumination scheme can be used to capture the first type of image (e.g., a bright field image) to extract the best representation of the biological sample for wavelet-based image transformation (e.g., for downstream phenotype exploration).

[0231] The embodiment of the present disclosure further includes a step of evaluating the robustness of the generated image of the machine learning model. In some embodiments, the first downstream classifier is trained using real images (e.g., real fluorescent images), and the second downstream classifier is trained using generated images (e.g., generated fluorescent images). The performance of the two classifiers can be compared to evaluate the robustness of the generated image as training data in downstream tasks.

[0232] Thus, the present disclosure provides an integrated platform that simultaneously solves many problems, namely image enhancement, phase recovery, low phototoxicity, realistic virtual staining of brightfield images, and robustness in downstream tasks. The present disclosure can evaluate the robustness of generated images through downstream classification tasks. These tasks are integrated into the platform, closing the loop of data generation from non-invasive brightfield images to fluorescent images. For example, the present system can optimize the parameters of the brightfield microscope acquisition system during use. The illumination pattern of the LED array and other parameters of the brightfield microscope acquisition system, including, for example, the focal position of the microscope objective lens and the activation timing of the spatial light modulator (SLM), can be optimized by backpropagation during downstream classification tasks.

[0233] Furthermore, in some embodiments, the platform learns to subject cells to a series of perturbations, optimize lighting schemes, and extract the best representation of the cells for phenotype exploration.

[0234] Some embodiments of the present disclosure can identify one or more optimal lighting patterns for capturing image data. In some embodiments, the lighting patterns can indicate whether each lighting emitter of the lighting source (e.g., each LED on an LED array) should be turned on or off and the intensity of each lighting emitter. The system can determine the optimal lighting patterns by training a machine learning model having an attention layer with multiple weights (e.g., multiple weights corresponding to the intensities of the multiple lighting emitters) corresponding to the intensities of the multiple lighting emitters. During training of the machine learning model, the multiple weights are applied to multiple training images (e.g., bright field images) illuminated by different lighting emitters. The aggregated images are input into the machine learning model, and a loss can be determined, and the model, including the weights in the attention layer, can be updated accordingly based on the loss. After training, the lighting patterns can be determined based on the weights in the attention layer of the trained machine learning model, as each weight can correspond to a desired intensity level of a corresponding lighting emitter. Thus, the process only involves capturing images using a limited number of lighting settings (e.g., turning on a single lighting emitter at a time and capturing images) and does not require physically adjusting the intensity of the lighting emitters to identify the optimal lighting pattern.

[0235] Some embodiments of the present disclosure can train a machine learning model such that synthetic image data generated by the machine learning model can provide the same performance as real images in downstream analysis. In some embodiments, training the machine learning model includes first training a classifier corresponding to the downstream task (e.g., classifying healthy vs. diseased tissue based on the image) and then using the output of the classifier to guide the training of the machine learning model.

[0236] Some embodiments of the present disclosure can evaluate a candidate treatment for a disease of interest. In some embodiments, the system receives a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest, receives a second plurality of images depicting a second set of untreated biological samples affected by the disease of interest, and receives a third plurality of images depicting a third set of treated biological samples affected by the disease of interest and treated with the candidate treatment. The images are input into a machine learning model to obtain an enhanced image, and the enhanced images are compared to evaluate the treatment (e.g., by analyzing the distribution of the images).

[0237] The following description sets forth example methods, parameters, and equivalents, however, it should be recognized that such description is not intended as a limitation on the scope of the present disclosure, but is instead provided as a description of example embodiments.

[0238] In some embodiments, an exemplary optical system comprises a programmable spatial light modulator ("SLM"). The SLM of the optical system can improve the performance of the machine learning model through a training phase (e.g., by providing an abundant training data set) and / or through an inference phase (e.g., by providing input data under various optical settings or optimal settings). The SLM is programmed without requiring any mechanical movement or modification of the optical system.

[0239] The SLM provides additional degrees of freedom and a source of contrast and controls the microscope in a programmable manner. For example, the SLM can be programmed to generate optical aberrations that highlight important phenotypes. As another example, the SLM can be programmed to provide different modulations and thus produce a variety of images that allow rapid exploration of deep samples. The SLM also allows the identification of optimal imaging settings to infer cell phenotypes and reconstruct alternative image modalities in a supervised manner. Multifocal acquisition is possible without any mechanical movement, thus accelerating and improving downstream tasks. Three-dimensional phase tomography and reconstruction are therefore accelerated and improved.

[0240] The following description uses the terms "first", "second", etc. to describe various elements, but these elements should not be limited by the terms. These terms are used only to distinguish one element from another. For example, a first graphical representation may be referred to as a second graphical representation, and similarly, a second graphical representation may be referred to as a first graphical representation, without departing from the scope of various described embodiments. Although the first graphical representation and the second graphical representation are both graphical representations, they are not the same graphical representation.

[0241] The terminology used in the description of the various illustrated embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various illustrated embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Also, the term "and / or," as used herein, is to be understood to refer to and encompass all possible combinations of one or more of the associated listed items. Furthermore, it is to be understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0242] The term "if" is interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context, as appropriate. Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" are interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context, as appropriate.

[0243] FIG. 1 illustrates an example process for training a machine learning model configured to generate an enhanced image of a biological sample, according to some embodiments.

[0244] 1 , the training data 120 comprises a first type of image 122 and a second type of image 124. In some embodiments, the images are images of biological samples, where the biological samples may include one or more sets of stained, unstained, perturbed, and / or unperturbed biological samples.

[0245] In some embodiments, the first type of image data 122 comprises a set of brightfield images and the second type of image data 124 comprises images in a different modality (e.g., fluorescent images). The first type of images 122 can be acquired using a brightfield microscope, while the second type of images 124 can be acquired using a fluorescent microscope.

[0246] As shown in Fig. 1, a first type of image 122 is acquired based on one or more illumination patterns 110. An illumination pattern indicates a setting with which an object is illuminated. In some embodiments, an illumination pattern can be defined by one or more parameters indicating a spatial relationship (e.g., distance, angle) between the object and the illumination source, one or more parameters indicating a setting of the illumination source, or a combination thereof. For example, an illumination pattern can indicate a set of LED light sources to be activated, a specific out-of-focus / polarized setting, etc.

[0247] In some embodiments, the microscope that captures the first type of image can be a microscope that supports multiple illumination patterns. For example, the microscope can provide a programmable illumination source (e.g., LED array, laser), an adaptive optics system (SLM, micromirror), or a combination thereof. By updating the illumination pattern (e.g., controlling the pupil function of the illumination source and / or the optics system), multiple representations of the biological sample corresponding to the multiple illumination patterns can be obtained.

[0248] In some embodiments, the training data 120 can be organized as three-dimensional image data (e.g., an image array). For example, the training data 120 can be of dimensions (B, C, H, W), where B indicates the batch size, C indicates the number of channels (i.e., the illumination pattern), H indicates the height, and W indicates the width. C is equal to 1 if only a single brightfield image is present, and C is greater than 1 if a stack of brightfield images is present.

[0249] In some embodiments, one or more images in the training data 120 may be normalized before they are used to train the machine learning model 100. For example, fluorescent images may be normalized based on illumination or intensity parameters.

[0250] Referring to FIG. 1, training data 120 is used to train a machine learning model 100. In some embodiments, the model 100 is a generative adversarial network ("GAN") model. For example, the GAN network can be a conditional GAN ​​("cGAN") model. As described in detail below, the GAN network comprises a generator and a discriminator. The generator receives a first type of image (e.g., a bright field image) and is trained to convert the input image into a second type of image (e.g., a fluorescent image). The training step includes comparing the first type of actual image 122 and the second type of actual image 124 to determine a ground truth baseline for the wavelet-based transformation that the first type of image 122 should seek to perform. During training, the generator output (i.e., the generated fluorescent image) can be directly connected to the discriminator input. The discriminator is trained to distinguish the second type of generated image (e.g., the generated fluorescent image) from the second type of actual image (e.g., the actual fluorescent image). Through backpropagation, the output of the discriminator can be used by the generator to update the weights of the generator so that the generator learns to generate images that the discriminator would classify as real images.

[0251] In some embodiments, the lighting parameters can be updated during the training of model 100. Thus, during the training of model 100, the lighting scheme can be continually updated, and training data can be obtained according to the updated lighting scheme to further train model 100, as described in detail below.

[0252] 2A and 2B illustrate example processes 200 and 250 for training and applying a machine learning model (e.g., model 100) according to some embodiments. Fig. 2A illustrates the process when the parameters of the microscope cannot be changed during acquisition of the training data. Fig. 2B illustrates the process when the parameters of the microscope can be changed during acquisition of the training data, and thus an optimal illumination pattern can be identified.

[0253] Each process can be implemented, at least in part, using one or more electronic devices. In some embodiments, the blocks of each process step depicted in FIGS. 2A and 2B can be divided among multiple electronic devices. In each process, some blocks are optionally combined, the order of some blocks is optionally changed, and some blocks are optionally omitted. In some examples, additional steps may be implemented in combination with a process. Thus, the operations as illustrated (and described in more detail below) are exemplary in nature and should not be considered as limiting.

[0254] 2A, the parameters of the microscope cannot be changed during acquisition of the brightfield images, so all of the brightfield images in the training data (e.g., training data 120) are acquired based on the same illumination pattern.

[0255] At block 204, training data (e.g., training data 120 in FIG. 1) is acquired, at least in part, according to a default illumination pattern. As described above with reference to FIG. 1, the training data comprises a first type of image (e.g., bright field images) acquired according to the default illumination pattern by a bright field microscope and a second type of image (e.g., fluorescent images) acquired by a fluorescent microscope.

[0256] In block 206, a machine learning model (e.g., model 100 in FIG. 1) is trained based on the training data. The model can be a GAN model. For example, the GAN network can be a conditional GAN ​​("cGAN") model.

[0257] 3A, 3C, and 3D illustrate a training process of a machine learning model, according to some embodiments. The GAN model comprises a generator 302 and a discriminator 304. The generator 302 is trained to receive a first type of image 310 (e.g., a brightfield image) and generate a second type of image 312 (e.g., a fluorescent image). In some embodiments, the first type of image 310 comprises a brightfield image array, as described above.

[0258] During training, the generator output (i.e., the generated fluorescent images) can be directly connected to the discriminator input. The discriminator 304 is trained to distinguish the second type of generated images 312 (e.g., the generated fluorescent images) from the second type of actual images 314 (e.g., the actual fluorescent images). In some embodiments, the second type of actual images 314 comprises a fluorescent image array, as described above.

[0259] Through backpropagation, the output of the discriminator can be used by the generator to update the weights of the generator so that the generator learns to generate images that the discriminator would classify as real images, as described in more detail below. In some embodiments, the generator 302 and the discriminator 304 are neural networks.

[0260] 3B illustrates the operation of an example generator, according to some embodiments. The input image 352 can be a single image from a brightfield image array or the entire brightfield image array. For example, the input image 352 can be of dimensions (B, C, H, W), where B indicates the batch size, C indicates the number of channels, H indicates the height, and W indicates the width. C is equal to 1 if only a single brightfield image is present, and C is greater than 1 if a stack of brightfield images is present.

[0261] 3B, the generator comprises a series of convolution layers to downsample an input image 352 to obtain a downsampled input image 354. In some embodiments, the input image 353 can be downsampled to half its original size. For example, if the input is 256×256 in spatial size, it will be reduced to 128×128.

[0262] The downsampled image 354 is then passed to multiple neural networks. In the depicted embodiment, the multiple neural networks comprise four U-Net neural networks. The U-Net network is a convolutional network for image-to-image transformation. Details of the design and implementation of the U-Net network can be found, for example, in Ronneberger et al. "U-Net: Convolutional Networks for Biomedical Image Segmentation," which is incorporated herein by reference in its entirety.

[0263] The multiple neural networks may correspond to different frequency groups in the wavelet domain. In the depicted example, the four U-Net networks are responsible for low frequencies, high frequencies (horizontal), high frequencies (vertical), and high frequencies (diagonal), respectively. In signal processing, the low frequency signals correspond to features that are very large relative to the size of the image (e.g., about the size of the cytoplasm or nucleus when imaging cells). The high frequency information is very fine small image features (e.g., about the size of mitochondria, microtubules). The low frequency signals correspond to the first magnitude of the wavelet coefficients. High frequencies are encoded in the higher magnitude wavelet coefficients. In some embodiments, the multiple neural networks operate independently and do not share weights.

[0264] As shown, three of the four U-Net branches correspond to high-frequency blocks. Low-frequency information is relatively easy to recover. Therefore, having more computing power dedicated to high frequencies ensures the reconstruction of fine details in the image. Thus, the loss function, operating in the wavelet domain, benefits from the original organization of the signal (three times more high-frequency information than low-frequency information).

[0265] Each neural network is configured to output (or predict) wavelet coefficients for a distinct set of frequencies. A loss function is applied to the predicted wavelet coefficients 356 and the true wavelet coefficients of the actual fluorescence image. The loss function is further described below with reference to Figures 3C and 3D.

[0266] A resulting fluorescence image 358 in the image domain can be obtained by applying an inverse wavelet transform to the predicted coefficients 356 .

[0267] 3C illustrates the backpropagation process of the discriminator 304, according to some embodiments. The discriminator 304 is a model (e.g., a neural network) that is trained to provide an output based on a given image. The training data for the discriminator 304 comprises actual images 314 of a second type (e.g., actual fluorescent images) and synthetic images 312 of a second type (e.g., synthetic fluorescent images) generated by the generator 302.

[0268] In some embodiments, the discriminator 304 is a PatchGAN network. Details of the design and implementation of PatchGAN networks can be found, for example, in Isola et al., “Image-to-Image Translation with Conditional Adversarial Networks,” incorporated by reference in its entirety.

[0269] During training of the discriminator 304, a discriminator loss 322 can be calculated based on the generator output (i.e., the predicted wavelet coefficients). In some embodiments, the discriminator loss function is the Wasserstein discriminator loss, calculated as follows:

[0270] [ka]

[0271] where f(x) is the output of the discriminator based on the wavelet coefficients of the actual fluorescence image, w is the model weight of the discriminator, m is the mini-batch size, f is the discriminator model, x is the actual image, z is the input (bright field), G is the generator model, and f(G(z)) is the output of the discriminator based on the predicted wavelet coefficients corresponding to the synthetic fluorescence image.

[0272] The discriminator 304 is configured to maximize this function. In other words, it attempts to maximize the difference between its output based on the actual image and its output based on the synthetic image. As depicted in Figure 3C, the discriminator updates its weights through backpropagation based on the discriminator loss 332 through the discriminator network.

[0273] 3D illustrates the backpropagation process of the generator 302, according to some embodiments. The generator 302 is a neural network configured to receive a first type image 310 and generate a second type image 312, as described with reference to FIGS. 3A and 3B. The wavelet coefficients predicted by the generator are input into a discriminator 304. Depending on the output of the discriminator, a generator loss 324 can be calculated. In some embodiments, the generator loss function is the Wasserstein generator loss, calculated as follows:

[0274] [ka]

[0275] where f(x) is the output of the discriminator based on the wavelet coefficients of the actual fluorescence image, m is the size of the mini-batch, f is the discriminator model, z is the input (bright field), G is the generator model, and f(G(z)) is the output of the discriminator based on the predicted wavelet coefficients.

[0276] Reconstruction losses operating in the wavelet domain naturally have the property of balancing low and high frequency contributions. As shown in FIG. 3B, the wavelet coefficients can be split into two categories, namely low (one block) and high frequency (three blocks). Three of the four UNet branches can be dedicated to the high frequency block. Low frequency information can be more easily retrieved. Thus, dedicating more computing power to high frequencies helps in the reconstruction of fine details in the image. The loss function operating in the wavelet domain benefits from the original organization of the signal (three times more high frequency information than low frequency information). In some embodiments, the wavelet coefficients can be split into more than two categories, for example, three categories (high, medium, low frequency), four categories, or more than four categories.

[0277] The generator 302 is configured to maximize this function, in other words, it attempts to maximize the output of the discriminator based on its synthetic image. In some embodiments, the generator loss is back-propagated through both the discriminator 304 and the generator 302 to obtain gradients, which are then used to adjust the generator weights.

[0278] In some embodiments, the generator 302 and the discriminator 304 are trained in alternating cycles. In each cycle, the discriminator trains for one or more epochs and the generator trains for one or more epochs. During discriminator training, the generator may remain constant. Similarly, during generator training, the discriminator may remain constant.

[0279] In some embodiments, the generator can convert the input image into a third type of image (e.g., a phase-shifted image). For example, in addition to the generated fluorescence image, the generator may also output a phase-shifted image, in which each pixel indicates the local value of the phase in the image (e.g., -5 to 5 phase information), which can be transformed. A physics-based image formation model can be used to generate a real phase-shifted image (i.e., a ground truth phase-shifted image). The image formation model generates an image given absolute knowledge of the microscope (e.g., aberrations of the optical system) as well as the optical properties of the captured sample (e.g., refractive index, phase). The optical properties of the samples can be compared between samples, so the risk of batch effects in downstream tasks is nearly null. The physics-based model allows the incorporation of robust a priori knowledge in the generation process.

[0280] Referring to Figure 4, during training, the illumination source S is optimized to read out X with high fidelity. The microscope PSF (or pupil function) can also be optimized using a spatial light modulator or a set of micromirrors. In addition, the polarization of S can also be modulated to inject more contrast into the collected images.

[0281] [ka]

[0282] In the above equation, S(f) refers to the partially coherent illumination source (LED array). X(r) refers to the complex electron field of the sample. P(r) refers to the point spread function (PSF) of the microscope. RI(r) refers to the refractive index. I refers to the image. The forward model is applied outside the training loop to obtain the ground truth phase.

[0283] In some embodiments, the generator can convert the input image into a fourth type of image. In some embodiments, the fourth type of image includes image data that indicates segmentation data (e.g., cell boundaries). In some embodiments, the loss function depends on the supported image modality. For semantic segmentation, the L1 norm can be used to infer discrete labels in the image. For another image modality, another branch can be added to the generator to output the new modality.

[0284] 2A, in block 208, metadata can be associated with the trained machine learning model. The metadata can include default illumination patterns (e.g., parameters of a brightfield microscope).

[0285] At block 210, one or more images of a first type (e.g., bright field images) are acquired. In some embodiments, the images are acquired using the same illumination pattern (e.g., as indicated in the metadata at block 208).

[0286] In block 212, one or more images are input into a generator of a trained machine learning model (e.g., generator 302). The generator is trained to convert an image of a first type into an image of a second type (e.g., a fluorescent image). In block 214, one or more images of the second type are acquired. As described below, the generated fluorescent images can then be used as training data for other machine learning models, thus eliminating the need to acquire actual fluorescent images as training data.

[0287] 2B illustrates a process by which the parameters of the microscope can be changed during acquisition of training data so that an optimal illumination pattern can be identified. For example, the parameters of the microscope can be programmable such that multiple illumination patterns can be provided by the microscope.

[0288] In block 252, an illumination pattern is loaded onto the microscope. In block 254, brightfield training images are captured according to the illumination pattern (e.g., brightfield images), and fluorescent training images are also captured by the fluorescent microscope. Blocks 252 and 254 can be performed iteratively, as indicated by arrow 253. In other words, a sequence of illumination patterns can be loaded onto the microscope, and training data corresponding to the sequence of illumination patterns can be acquired. The sequence of illumination patterns is also referred to herein as an illumination scheme.

[0289] At block 256, a machine learning model is trained based on the training data. The model can be a GAN model that operates as described with reference to Figures 3A-D. For example, the GAN network can be a conditional GAN ​​("cGAN") model.

[0290] During training, the model iteratively updates the illumination pattern by backpropagating the gradient of the loss (e.g., generator loss) to the parameters of the microscope. The model training procedure minimizes the overall imaging time and minimizes a loss function associated with the transformation or classification task.

[0291] During training, the system determines the illumination pattern that leads to the smallest loss (e.g., generator loss). In some embodiments, the model produces a first generator loss when transforming a first image, a second generator loss when transforming a second image, and so on. The losses can be compared and the smallest loss can be identified. In block 260, the illumination pattern that produced the smallest loss can be identified and a new illumination scheme can be identified, if appropriate. For example, the new illumination scheme can include the best illumination pattern and / or one or more new illumination patterns similar to the best illumination pattern. The new illumination scheme can also exclude the previously included illumination pattern that is resulting in the largest loss. As indicated by arrow 258, the identified illumination scheme can be loaded into the microscope and additional training data can be acquired. This process can be repeated until no further improvement (e.g., in terms of generator loss) is observed. The optimal illumination scheme can be stored in block 261.

[0292] At block 262, one or more images of a first type (e.g., bright field images) are acquired. In some embodiments, the images are acquired using an optimal illumination scheme, which is stored at block 261.

[0293] In block 264, one or more images are input into a generator of trained machine learning models (e.g., generator 302). The generator is trained to convert images of a first type into images of a second type (e.g., fluorescent images). In block 266, one or more images of the second type are acquired. As described below, the generated fluorescent images can then be used as training data for other machine learning models, thus eliminating the need to acquire actual fluorescent images as training data.

[0294] FIG. 5 illustrates the output of an exemplary trained generator using a static microscope setup. The trained generator receives an input brightfield image and generates phase and fluorescence images 504. For comparison, FIG. 5 also shows the corresponding actual phase and fluorescence images 506. In FIG. 5, each input image is a composite of four different illuminations. In some embodiments, the input images are false color images representing different illumination patterns.

[0295] 6A illustrates an exemplary process for determining the robustness of a generated image, according to some embodiments. Two classifiers 612 and 614 are trained to receive a fluorescent image and provide an output (e.g., a phenotype classification result). Classifier 614 is trained based on an actual fluorescent image 604 (e.g., an actual fluorescent image), while classifier 612 is trained based on a generated fluorescent image 610. The generated fluorescent image is generated by a trained generator 608 (e.g., generator 102, 302) based on a brightfield image 506.

[0296] In some embodiments, classifier 612 is validated using synthetic images, while classifier 614 is validated using real images.

[0297] The performance of classifiers 612 and 614 can be compared to determine the robustness of the generated images. Figure 6B illustrates an exemplary comparison between two such classifiers. In this particular setting, a dataset is created from fluorescent and 3D brightfield images of primary cultured hepatocytes from 12 healthy donors and 12 NASH donors. A GAN model is trained to generate fluorescent images from a stack of out-of-focus brightfield images.

[0298] A comparison is made to determine whether a classifier trained on generated images and validated on real images performs equally well as when trained on real images and validated on generated images. As shown in Figure 6B, there is no degradation in classification accuracy, and more importantly, the geometry of the embedding space of the real images is preserved on the generated images.

[0299] Some embodiments include a backpropagation module 616. The backpropagation module 616 can be used to refine image acquisition parameters, for example, used to acquire the brightfield image to be enhanced. For example, Xi can include a set of parameters for an illumination pattern applied to an LED illumination array of a brightfield microscope. Xi can also include imaging parameters related to the focal position of the microscope objective lens, and / or activation parameters of a spatial light modulator (SLM). All elements of Xi can be variables in the optimization procedure. Thus, in 616, the gradient of the loss function estimated on the downstream task (e.g., classification, image-to-image transformation, regression) may be backpropagated to optimize the parameters Xi. For each update to the value of Xi, a new set of images may be acquired, generating a new dataset of images.

[0300] In some embodiments, the data used to train the generator 608, the data used to train the first classifier 614 or the second classifier 612, and the data used to evaluate the performance of the classifiers can include overlapping images. Images used in any of the processes described above can be annotated based on the captured biological sample (e.g., type of cell, whether diseased or healthy) and perturbations. These annotations can be used for downstream classification tasks (e.g., as training data for training the classifiers 612 and 614 in FIG. 6A) and validation of predictive models (e.g., to evaluate the performance of the trained classifiers 612 and 614).

[0301] 7A illustrates an example of a method 700 for training a machine learning model and generating images of a biological sample, according to one embodiment. The method includes obtaining 702 a plurality of training images comprising a first type of training image and a second type of training image. The method further includes generating 704 a plurality of wavelet coefficients using the machine learning model based on the first type of training image, generating 706 a second type of synthetic image based on the plurality of wavelet coefficients, comparing 708 the second type of synthetic image to the second type of training image, and updating 710 the machine learning model based on the comparison.

[0302] 7B illustrates an example of a method 750 for generating an enhanced image of a biological sample, according to one embodiment. The method includes acquiring an image of the biological sample using a microscope, at 752. The method also includes generating an enhanced image of the biological sample, at 754, based on the image and using the machine learning model of FIG. 7A.

[0303] FIG. 8 illustrates an example of a computing device, according to one embodiment. The device 800 can be a host computer connected to a network. The device 800 can be a client computer or a server. As shown in FIG. 8. The device 800 can be any suitable type of microprocessor-based device, such as a personal computer, a workstation, a server, or a handheld computing device (portable electronic device), such as a phone or tablet. The device can include, for example, one or more of a processor 810, an input device 820, an output device 830, a storage device 840, and a communication device 860. The input device 820 and the output device 830 can generally correspond to those described above and can be either connectable or integrated with the computer.

[0304] The input device 820 can be any suitable device that provides input, such as a touch screen, a keyboard or keypad, a mouse, or a voice recognition device. The output device 830 can be any suitable device that provides output, such as a touch screen, a tactile device, or a speaker.

[0305] The storage device 840 can be any suitable device that provides storage, for example, electrical, magnetic, or optical memory, including RAM, cache, a hard drive, or a removable storage disk. The communication device 860 can include any suitable device capable of transmitting and receiving signals over a network, such as a network interface chip or device. The components of a computer can be connected in any suitable manner, for example, via a physical bus or wirelessly.

[0306] Software 850, which may be stored in memory 840 and executed by processor 810, may include, for example, programming that embodies functionality of the present disclosure (e.g., as embodied in the present device, as described above).

[0307] The software 850 can also be stored and / or transported in any non-transitory computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch instructions associated with the software from and execute the instructions. In the context of this disclosure, a computer-readable storage medium can be any medium, such as storage device 840, that can contain or store programming for use by or in connection with an instruction execution system, apparatus, or device.

[0308] The software 850 can also be propagated in any carrier medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch instructions associated with the software from and execute the instructions. In the context of this disclosure, a carrier medium can be any medium that can communicate, propagate, or carry programming for use by or in connection with an instruction execution system, apparatus, or device. Carrier-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation media.

[0309] The device 800 may be connected to a network, which can be any suitable type of interconnected communication system. The network can implement any suitable communication protocol and can be secured by any suitable security protocol. The network can include any suitable arrangement of network links that can implement the transmission and reception of network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0310] Device 800 can implement any operating system suitable for operating on a network. Software 850 can be written in any suitable programming language, for example, C, C++, Java, or Python. In various embodiments, application software embodying the functionality of the present disclosure can be deployed in different configurations, such as in a client / server arrangement or as, for example, through a web browser as a web-based application or web service.

[0311] Optimization using spatial light modulators

[0312] In some embodiments, the exemplary optical system comprises a programmable SLM. The exemplary SLM comprises a high-resolution liquid crystal panel with micron-sized individually addressable pixels, which can be used to shape the wavefront of an optical beam. The gray level values ​​on the panel are converted to phase shifts. The SLM can be used in some embodiments as a programmable Fourier filter to generate contrast enhancement, or as a programmable diffractive optical element for quantitative phase microscopy.

[0313] The SLM can be programmed to generate various images during the training phase of the machine learning model and / or during the inference phase to improve the performance of the machine learning model, as described herein.

[0314] During the inference stage of the trained machine learning model, the SLM can be programmed to generate different input images for the trained machine learning model. For example, the SLM can be programmed to generate input images for the trained image translation model to obtain an enhanced version of the input image. The enhanced image can then be used for downstream operations. As another example, the SLM can be programmed to generate input images for the trained classification model to obtain a more accurate classification result.

[0315] Additionally, the SLM can be programmed to generate different images as training data during the training phase of the machine learning model. Additionally, the SLM can be iteratively programmed to identify optimal settings for capturing images that lead to the best performance of a given machine learning model during either the training or inference phase.

[0316] Thus, the SLM of the optical system can improve the performance of the machine learning model through a training phase (e.g., by providing a rich training data set) and / or through an inference phase (e.g., by providing input data under various settings or optimal settings). The SLM is programmed without requiring any mechanical movement or modification to the optical system (e.g., microscope). The SLM provides an additional degree of freedom for controlling the microscope. Multifocal acquisition is possible without any mechanical movement, thus accelerating and improving downstream tasks in an efficient manner.

[0317] 9 illustrates an exemplary optical system, according to some embodiments. The exemplary optical system includes a light source 902 (e.g., an LED array), an objective lens 904, an SLM 906, and a camera 908. The optical system has a reflection mode. In the reflection mode, the light source 902 provides illumination to a biological sample 910, which generates reflected light. The reflected light travels through the objective lens 904 and is captured by the camera 908.

[0318] 9, dashed line 912 indicates an intermediate image plane. As shown, an SLM is placed in the imaging path between the biological sample 910 and the camera 908. The SLM is configured to impose a spatially varying modulation on the reflected light. For example, the SLM allows for shaping an alternate Fourier plane before the reflected light is focused onto the camera 908.

[0319] In some embodiments, both the light source 902 and the SLM 906 can be programmable, thus allowing additional degrees of freedom for controlling and optimizing the optical system without mechanically moving components of the optical system.

[0320] The configuration of the optical system in Figure 9 is merely exemplary: those skilled in the art will appreciate that other configurations of the optical system can also be used to place the SLM in the imaging path of the optical system and apply the optimization techniques described herein.

[0321] FIG. 10 illustrates an exemplary method for analyzing an image of a biological sample using a programmable SLM of a system according to some embodiments. The process 1000 is implemented, for example, using one or more electronic devices implementing a software platform. In some examples, the process 1000 is implemented using a client-server system, and the blocks of the process 1000 are divided in any manner between a server and one or more client devices. In other examples, the process 1000 is implemented using only one or more client devices. In the process 1000, some blocks are optionally combined, the order of some blocks is optionally changed, and some blocks are optionally omitted. In some examples, additional steps may be implemented in combination with the process 1000. Thus, the operations as illustrated (and described in more detail below) are exemplary in nature and should not be considered as limiting.

[0322] In block 1002, an example system (e.g., one or more electronic devices) acquires multiple images of a biological sample (e.g., sample 910 in FIG. 9). The multiple images are generated using multiple configurations of an SLM (e.g., SLM 906 in FIG. 9) of an optical system. The SLM is located in an optical path between the biological sample and an image recording device (e.g., camera 908).

[0323] In some embodiments, at least one configuration of the SLM is for generating one or more optical aberrations in the resulting image. The optical aberrations can include spherical aberration, astigmatism, exfocality, tilt, or any combination thereof. In some embodiments, more information can be captured in and / or derived from an image with optical aberrations. As an example, astigmatism allows for the collection of multi-focal plane information in one image. As another example, exfocality allows the system to scan the sample without any mechanical movement. Exemplary methods for generating optical aberrations are provided in "Quantitative Phase Imaging and Complex Field Reconstruction by Pupil Modulation Differential Phase Contrast" by Lu et al., the contents of which are incorporated herein by reference in their entirety.

[0324] In some embodiments, at least one configuration of the SLM is for enhancing one or more features. The one or more features may comprise cell boundaries, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. In some embodiments, the system provides enhancement of neurites with specific convolution kernels encoded in the transfer function of the microscope. In some embodiments, the system provides enhancement of the response function of the microscope for certain liquid-phase separated biological objects (proteins, RNA, lipids). This enhancement would provide the ability to detect and characterize the composition of these objects. For example, using a spiral phase pattern as a phase filter in the Fourier plane gives rise to a doughnut-shaped point spread function. With extended amplitude or phase objects, convolution leads to enhancement of strong isotropic edges in the image. In homogeneous samples (homogeneous regions thereof), destructive interference occurs due to a π phase shift across the doughnut (for any angle along the ring). Structures within the sample may give rise to less complete cancellation and therefore local brightening in the image. As a result, light is redistributed into edges and boundaries within the sample. Exemplary methods for enhancing features are described in "Spiral phase contrast imaging in microscopy" by Furhapter et al., "Shadow effects in spiral phase contrast microscopy" by Furhapter et al., "Quantitative imaging of complex samples in spiral phase contrast microscopy" by Bernet et al., and "Upgrading a microscope with a spiral phase plate, the contents of which are incorporated by reference herein in their entirety.

[0325] In some embodiments, at least one configuration of the SLM is for reducing optical aberrations. For example, for live cell imaging and continuous monitoring of cells, it is important to reduce sample variability arising from debris in the plate or well. The SLM can allow these aberrations to be corrected. In some embodiments, the SLM can be used for iterative correction of phase aberrations, for example based on the Gerchberg-Saxton algorithm. For example, the dark center of a single optical vortex can be used as a critical sensor for residual phase aberrations. An exemplary method for highlighting features is the “Wavefront correction” method by Jesacher et al. of spatial light modulators using an optical vortex image” and “Phase contrast microscopy with full numerical aperture illumination” by Maurer et al., the contents of which are incorporated by reference herein in their entirety.

[0326] In some embodiments, the multiple SLM configurations are for acquiring images of biological samples at different depths. For example, the SLM allows flexible image multiplexing, for example, to combine images from different depths of the sample, or different settings of imaging parameters within one recorded image. Image multiplexing can facilitate quantitative phase microscopy. Imaging live cell samples needs to be as fast as possible to minimize stress on the cells. Mechanical movement for 3D scanning of the sample is prohibitive. The system allows Fourier ptychography to be performed and reconstruction of large 3D volumes from low-resolution images of the entire well. Exemplary methods of multiplexing are provided in "Depth-of-field-multiplexing in microscopy" by Maurer et al., "Differential interference contrast imaging using a spatial light modulator" by McIntyre et al., and "Quantitative SLM-based differential interference contrast Imaging" by McIntyre et al., the contents of which are incorporated by reference and are incorporated herein in their entirety.

[0327] In some embodiments, the system allows the reconstruction of high-resolution images through optimized Fourier ptychography. Fourier ptychography is a computational imaging technique based on optical microscopy that involves the synthesis of a wider numerical aperture from a set of full-field images acquired using different optical settings, resulting in increased resolution compared to conventional microscopes. Images in an image set can be acquired using different configurations of LEDs and / or SLMs, and the acquired image sets can then be combined using an iterative phase retrieval algorithm into a final high-resolution image that can contain up to a billion pixels (gigapixels) with diffraction-limited resolution and result in a high spatial-bandwidth product.

[0328] The light source of the optical system can also be programmed. In some embodiments, multiple images are acquired in block 1002 using multiple configurations of the light source of the optical system (e.g., light source 902). Exemplary methods for programming the light source are described herein, for example, with reference to Figures 1 and 2B.

[0329] Returning to FIG. 10, at block 1004, the system inputs a plurality of images of a biological sample into a trained machine learning model and obtains one or more outputs.

[0330] In some embodiments, the trained model is an image transformation model configured to generate an output image of a second type (e.g., an enhanced version of the input image) based on an input image of a first type. In some embodiments, the enhanced version of the input image comprises an enhanced cell phenotype. In some embodiments, the trained model is a GAN model or a self-supervised model. For example, the trained model can be model 100, configured to receive a bright field image and generate a fluorescent image.

[0331] In some embodiments, the trained model is a classification model configured to provide a classification output, for example, the model receives an input image and detects one or more predefined objects, such as diseased tissue, in the input image.

[0332] FIG. 11 illustrates an exemplary method for training a machine learning model, according to some embodiments. Process 1100 is implemented using, for example, one or more electronic devices implementing a software platform. In some examples, process 1100 is implemented using a client-server system, and blocks of process 1100 are divided in any manner between a server and one or more client devices. In other examples, process 1100 is implemented using only one or more client devices. In process 1100, some blocks are optionally combined, the order of some blocks is optionally changed, and some blocks are optionally omitted. In some examples, additional steps may be implemented in combination with process 1100. Thus, the operations as illustrated (and described in more detail below) are exemplary in nature and should not be considered as limiting.

[0333] In block 1102, an exemplary system (e.g., one or more electronic devices) acquires multiple images of a biological sample. The multiple images are generated using multiple configurations of an SLM (e.g., SLM 906 in FIG. 9) of an optical system. The SLM is located in an optical path between the biological sample and an image recording device (e.g., camera 908).

[0334] In some embodiments, the SLM configurations are configured to create desired effects in the resulting image: As discussed above, some of the SLM configurations are for generating one or more optical aberrations (e.g., spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof), for enhancing one or more features (e.g., cell boundaries, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof), for reducing optical aberrations, for acquiring images at different depths of a biological sample, etc.

[0335] At block 1104, the system uses the plurality of images to train a machine learning model. In some embodiments, the trained model is an image translation model configured to generate an output image of a second type (e.g., an enhanced version of the input image) based on an input image of a first type. In some embodiments, the enhanced version of the input image comprises an enhanced cell phenotype. In some embodiments, the trained model is a GAN model or a self-supervised model. For example, the trained model can be model 100, configured to receive a bright field image and generate a fluorescent image. Exemplary methods of training a model are described herein with reference to FIGS. 2A-B and 3A-3D.

[0336] In some embodiments, the trained model is a classification model configured to provide a classification output. For example, the model can receive an input image and detect one or more predefined objects in the input image, such as diseased tissue. Training of the classification model can be performed using multiple images and associated labels.

[0337] In some embodiments, at block 1104, the SLM can be iteratively programmed to identify an optimal SLM configuration for capturing an image that leads to improved performance of a given machine learning model. Specifically, at block 1106, the system trains a machine learning model using a first image, the first image being acquired using a first configuration of the SLM of the optical system. At block 1108, the system evaluates the trained machine learning model. At block 1110, the system identifies a second configuration of the SLM based on the evaluation. At block 1112, the system trains a machine learning model using a second image, the second image being acquired using a second configuration of the SLM of the optical system.

[0338] For example, in block 1106, the system trains the model using images corresponding to a first set of SLM configurations. Each image results in a corresponding loss based on a loss function of the model. In block 1108, the system determines an SLM configuration from the first set of SLM configurations that results in a minimum loss (e.g., generator loss). In block 1110, the SLM configuration that produced the minimum loss can be identified, and a new set of second SLM configurations can be identified, if appropriate. For example, the new set of SLM configurations can include the best SLM configuration from the first set (i.e., the configuration that produced the minimum loss) and / or one or more new SLM configurations similar to the best SLM configuration. The new set of SLM configurations can also exclude the SLM configuration from the first set that is producing the maximum loss. The new set of SLM configurations can be loaded onto the optical system, and additional training data can be acquired. This step can be repeated until a threshold is met, e.g., when no further improvement (e.g., with respect to generator loss) is observed. The optimal SLM configuration can be stored and used to acquire an input image.

[0339] Although steps 1106-1112 are described as part of a training process, they can be implemented at other stages of the pipeline (e.g., the inference stage) to identify the optimal SLM configuration for generating the input image. In some embodiments, the light source and the SLM of the optical system can be iteratively programmed together to identify the best combination of illumination pattern and SLM configuration for generating the input image.

[0340] 9-11 illustrate the optimization technique using an SLM component of the optical system, it should be understood that the SLM may be replaced with another hardware component, such as a micromirror, that may modify the optical function (e.g., pupil function) of the system without departing from the spirit of the invention.

[0341] 12A and 12B illustrate a side-by-side comparison of classification results of two classification models, according to some embodiments. The classification model in FIG. 12A is trained using actual images captured by a microscope (e.g., actual fluorescent images), while the classification model in FIG. 12B is trained using synthetic images (e.g., fluorescent images generated from bright field images) generated using techniques described herein. The classification models determine the chemical compounds to which the tissue depicted in the input images responded. Specifically, each model is configured to receive an input image and output a classification result indicating one of 150 predefined chemical compounds. In the depicted example, FIGS. 12A and 12B each show a uniform manifold approximation and projection (UMAP), in which each input image is represented as a point in the UMAP. The color of the point represents the chemical compound for which the image is classified by the classification model. In some embodiments, the images input into the model in FIG. 12A are actual images, while the images input into the model in FIG. 12B are generated images.

[0342] Figures 12C and 12D respectively illustrate side-by-side comparisons of the same classification results in Figures 12A and 12B using different color schemes to demonstrate the resistance of the two models to batch effects. Batch effects refer to situations where subsets of data (i.e., batches) differ significantly in distribution due to non-instrument-related factors. Batch effects are undesirable because they can introduce systematic errors, produce spurious results in downstream statistical analysis, and / or obscure signals of interest. In each of Figures 12C and 12D, the input images belong to three different batches (e.g., from different plates or experiments), as indicated by different gray levels. As shown, Figure 12D shows a larger overlap between the points corresponding to the three batches, indicating that the generated images are more resistant to batch effects. FIG. 12E illustrates the Euclidean distance metric corresponding to the actual image set (e.g., input images as shown in FIGS. 12A and 12C), the generated image set (e.g., input images as shown in FIGS. 12B and 12D), and the true batch invariant image set. The Euclidean distance metric for each image set measures the Euclidean distance between image embeddings from two separate batches (e.g., actual image batch 1 and actual image batch 3 in FIG. 12C, generated image batch 1 and generated image batch 3 in FIG. 12D). For a true batch invariant image set, the average score should be 0 (i.e., no batch effect). As shown, the average score for the generated image set is lower than the average score for the actual images, thus demonstrating good batch invariance.

[0343] 13A and 13B illustrate an exemplary generated phase image and an exemplary generated fluorescence image according to some embodiments. FIG. 13A is a phase image generated by the GAN model described herein, for example, from a brightfield image. FIG. 13B is a fluorescence / bodipy image generated by the GAN from the same brightfield image. The bodipy image includes a virtual green hue to highlight the presence of biomarkers. FIG. 13C shows FIG. 13B overlaid on the phase image in FIG. 13A. Thus, the GAN model can be used for disease modeling. For example, multiple samples can be acquired and imaged from a subject. The resulting series of brightfield images can be analyzed by the GAN model to generate synthetic phase and fluorescence images to consider disease perturbations.

[0344] 14A-B illustrate an exemplary process for training a machine learning model (e.g., a GAN model) configured to generate synthetic data (e.g., image data) and identify an optimal lighting scheme for obtaining input data for the machine learning model, according to some embodiments. The process 1400 is performed using one or more electronic devices implementing a software platform. In some examples, the process 1400 is performed using a client-server system, and the blocks of the process 1400 are divided in any manner between a server and one or more client devices. In other examples, the process 1400 is performed using only one or more client devices. In the process 1400, some blocks are optionally combined, the order of some blocks is optionally changed, and some blocks are optionally omitted. In some examples, additional steps may be performed in combination with the process 1400. Thus, the operations as illustrated (and described in more detail below) are exemplary in nature and should not be considered as limiting, therefore.

[0345] In block 1402, an exemplary system (e.g., one or more electronic devices) receives a plurality of training images. The plurality of training images are actual images of a biological sample, also referred to as ground truth data. The plurality of training images comprises a type of image data that the GAN model is configured to receive and a type of image data that the GAN model is configured to output. For example, if the GAN model is configured to receive brightfield images and output fluorescence and phase images, the plurality of images received would include a plurality of brightfield training images 1402a (i.e., a GAN input data type), a plurality of fluorescence training images 1402b (i.e., a GAN output data type), and a plurality of phase training images (i.e., a GAN output data type).

[0346] The multiple brightfield training images 1402a can be captured by illuminating an in vitro (or biopsy) cell sample with an inexpensive LED array using different illumination settings. The multiple fluorescent training images 1402b can be captured after a dye is applied to the biological sample (e.g., to enhance the visibility of biomarkers). The phase training images can be acquired using a physics or optics-based model. It will be understood by those skilled in the art that the multiple training images used in the process 1400 can vary depending on the type of image data the GAN is configured to receive and output. In some embodiments, the multiple training images comprise paired image data. For example, brightfield, fluorescent, and phase images of the same biological sample can be included in the sets 1402a, 1402b, and 1402c, respectively.

[0347] In some embodiments, a plurality of training images are obtained to enable training of the GAN model such that a synthetic image (e.g., a synthetic fluorescence image, a synthetic phase image) generated by the GAN model will provide the same performance as a real image (e.g., a real fluorescence image, a real phase image) in downstream analysis. In some embodiments, the downstream analysis includes a classification task of classifying an image to correspond to one class of M classes. For example, the classification task can involve classifying an image to correspond to a particular cell state from a plurality of cell state classes (e.g., a healthy state, a diseased state). As another example, the classification task can involve classifying an image to correspond to a particular perturbation from a plurality of perturbation classes. To train the GAN model and generate synthetic images that can be accurately classified in the same way as real images, the training images include images corresponding to M classes (or conditions). For example, if the M classes include a healthy cell state class and a diseased cell state class, the plurality of brightfield training images 1402a can include a brightfield image depicting a healthy cell and a brightfield image depicting a diseased cell, the plurality of fluorescent training images 1402b can include a fluorescent image depicting a healthy cell and a fluorescent image depicting a diseased cell, and the plurality of phase training images 1402c can include a phase image depicting a healthy cell and a phase image depicting a diseased cell. For example, if the M classes include M perturbations, the plurality of brightfield training images 1402a can include a brightfield image depicting the M perturbations, etc. Each training image can be labeled with a corresponding condition. For example, a phase image depicting a diseased cell state can be associated with a disease label.

[0348] In an exemplary implementation, the multiple training images include X fields of view per condition, i.e., M×X fields of view in total. Specifically, in the multiple brightfield images 1402a, each field of view includes N brightfield images captured using N illumination settings, thus resulting in a total of M×X×N brightfield images. In the multiple fluorescence images 1402b, each field of view includes one fluorescence image, thus resulting in a total of M×X fluorescence images. In the multiple phase images 1402c, each field of view includes one phase image, thus resulting in a total of M×X phase images. In some embodiments, the brightfield images are magnified by a magnification factor=m in On the other hand, the fluorescence image is magnified by m out ≧m in The phase image is m out ≧m in It is.

[0349] At block 1404, the system trains a classifier configured to receive an input image and output a classification result indicative of one of the M conditions. For example, if the M conditions include healthy conditions and diseased conditions, the classifier is configured to receive an input image and output a classification result indicative of either a healthy condition or a diseased condition. After the classifier is trained, it is used during training of the GAN model to ensure that the GAN model can generate synthetic image data that can be classified to the same or a similar level of accuracy as the actual image data, as described below.

[0350] In some embodiments, the classifier is trained using the same type of image data that the GAN model is configured to output. In the example depicted in FIGS. 14A and 14B, the GAN model is configured to output a fluorescence image and a phase image. Thus, the classifier trained in block 1404 is trained using a plurality of fluorescence training images 1402b and a plurality of phase training images 1402c. During training, each fluorescence or phase image is input into the classifier to obtain a predicted classification result (e.g., healthy or diseased). The predicted classification result is then compared against the actual class associated with the training image (e.g., whether the training image actually depicts a healthy or diseased cell state), and based on the comparison, the classifier can be updated accordingly. The classifier can be implemented using any classification algorithm, such as a logistic regression model, a naive Bayes model, a decision tree model, a random forest model, a support vector machine model, etc.

[0351] In block 1406, the system trains the GAN model based on the training images. Block 1406 can include steps 1408a-1408e, which can be repeated until training is complete (e.g., when convergence is reached). Steps 1408a-e are described below with reference to FIG. 15, which is a schematic diagram illustrating steps according to some embodiments.

[0352] As shown in Figure 15, the GAN model is a matrix of weights, specifically, weights w1-w n The GAN model includes a multi-head attention layer 1502 with K sets of K, a generator 1504, a discriminator 1508, and a classifier 1506 trained from block 1404. During training of the GAN model, the classifier remains fixed while the generator, discriminator, and attention layer are updated as described below.

[0353] In block 1408a, the system applies each of the K sets of weights in the attention layer of the GAN model to a set of bright-field training images. The set of bright-field training images is obtained from a plurality of bright-field images 1402a. In some embodiments, the set of bright-field training images corresponds to the same field of view and depicts the same biological sample, but are captured using different illumination settings. For example, if the LED array comprises N illumination emitters (e.g., LEDs), each illumination emitter can be turned on one by one, and a bright-field image of the biological sample illuminated by each illumination emitter can be captured, thus resulting in a set of N bright-field training images.

[0354] 15, the system receives a set of N brightfield training images corresponding to illumination settings 1-N. For example, a first brightfield image depicts a biological sample illuminated using illumination setting 1 (e.g., only the first LED in the array is turned on), a second brightfield image depicts a biological sample illuminated using illumination setting 2 (e.g., only the second LED in the array is turned on), ... an Nth brightfield image depicts a biological sample illuminated using illumination setting N (e.g., only the Nth LED in the array is turned on).

[0355] The attention layer generates a set of K weights, each set with N weights. n Each set of weights is applied to the N images to generate an aggregated image. For each set of weights, the attention layer 1502 assigns a successive weight (e.g., a normalized scalar weight) in the set to each of the set of bright-field training images. These weights correspond to the intensity values ​​of the corresponding lighting settings (e.g., the corresponding LEDs). As shown, w1 is applied (e.g., multiplied with) to the first bright-field image, w2 is applied to the second bright-field image, and w nis applied to the Nth brightfield image. After the weights are applied, the weighted images can be aggregated (e.g., summed) to obtain one aggregated brightfield image. Since there are K sets of weights, K aggregated images 1512 can be generated. In some embodiments, the attention layer is an adapted multi-head attention layer. The attention mechanism allows for the natural generation of K linear combinations (i.e., aggregated images) of the brightfield images. These aggregated images are fed to the rest of the network as described herein.

[0356] At block 1408b, the system inputs each of the aggregated brightfield images into a GAN model. With reference to Figure 15, each of the aggregated brightfield images 1512 is input into a generator 1504, which outputs a composite fluorescence image 1514a and a composite phase image 1514b. The generators can be implemented in a similar manner to the generators described above with reference to Figures 3A-D.

[0357] During training, the generator output (i.e., the generated fluorescence image 1514a and the generated phase image 1514b) can be directly connected to the discriminator input. The discriminator 1508 is trained to distinguish between the generated image and the real image. During training, the generated image can be input into the discriminator 1508 as described above to obtain a discriminator loss and a generator loss. In addition, the real image can also be input into the discriminator 1508 to generate a discriminator loss and a generator loss. The real image can be an actual fluorescence image 1516a (from the multiple fluorescence training images 1402b in FIG. 14A ) corresponding to the same field of view as the input brightfield image, or an actual phase image 1516b (from the multiple phase training images 1402c in FIG. 14A ) corresponding to the same field of view as the input brightfield image.

[0358] In some embodiments, the discriminator loss function is the Wasserstein discriminator loss, calculated as follows:

[0359] [ka]

[0360] where f(x) is the output of the discriminator based on the wavelet coefficients of the real fluorescence or phase image, w is the model weight of the discriminator, m is the size of the mini-batch, f is the discriminator model, x is the actual image, z is the input (brightfield image 1512), G is the generator model, and f(G(z)) is the output of the discriminator based on the predicted wavelet coefficients corresponding to the synthetic fluorescence or phase image.

[0361] In some embodiments, the generator loss function is the Wasserstein generator loss, calculated as follows:

[0362] [ka]

[0363] where f(x) is the output of the discriminator based on the wavelet coefficients of the real fluorescence or phase image, m is the size of the mini-batch, f is the discriminator model, z is the input (bright field image 1512), G is the generator model, and f(G(z)) is the output of the discriminator based on the predicted wavelet coefficients.

[0364] In block 1408c, the system inputs each generated image and the corresponding actual image into a trained classifier to obtain a classifier loss. For example, the generated fluorescence image 1514a can be input into the classifier 1506 to obtain a first classification result, the actual fluorescence image 1516a can be input into the classifier 1506 to obtain a second classification result, and the classifier loss can be calculated based on the difference between the first classification result and the second classification result. As another example, the generated phase image 1514b can be input into the classifier 1506 to obtain a third classification result, and the actual phase image 1516b can be input into the classifier 1506 to obtain a fourth classification result, and the classifier loss can be calculated based on the difference between the third classification result and the fourth classification result.

[0365] At block 1408d, the system augments the generator loss based on the classifier loss. For example, the generator loss can be augmented with the L2 norm of the classification scores from the actual images. In some embodiments, if a classifier is not available or classification is not desired, the classifier loss is replaced by a constant (e.g., 0).

[0366] In block 1408e, the system updates the GAN model based on the augmented generator loss. Backpropagation follows the same procedure as described above. The discriminator updates its weights through backpropagation based on the discriminator loss through the discriminator network. Furthermore, the augmented generator loss is backpropagated to update the weights in the attention layer (e.g., the weights with which the aggregated image is calculated) and the generator. For example, the generated loss calculated based on the aggregated image corresponding to the Kth set of weights can be used to update the Kth set of weights in the attention layer.

[0367] In block 1410, the system obtains one or more optimal illumination patterns based on weights in the attention layer of the trained GAN model. As explained above, the weights in the attention layer (e.g., w1-w n) may indicate the intensity values ​​of the corresponding lighting settings (e.g., the LEDs in the corresponding LED array). As discussed above, the attention layer may be a multi-head attention layer that provides a set of K weights (i.e., K linear combinations), thus resulting in K lighting patterns.

[0368] In some embodiments, before block 1406, the generator and discriminator of the GAN model are pre-trained using a bright field image in which all LEDs in the LED array are turned on. After pre-training, block 1406 is performed to update the attention weights while the generator and discriminator remain fixed. An optimal combination of illumination can be obtained based on the updated weights.

[0369] FIG. 16A illustrates a synthetic image generated by an exemplary GAN model to study NASH disease, according to some embodiments. In the depicted example, Hepg2 cells with NASH genetic background are illuminated using an optimal illumination pattern identified using the techniques described herein, and a brightfield image is captured. The brightfield image is input into the GAN model, which outputs the resulting phase image and the resulting fluorescence / body pie image in FIG. 16A. The GAN model used in FIG. 16A can be configured to include a trained classifier as described with reference to FIGS. 14A-B and 15. The classifier has two conditions: a healthy condition (e.g., no perturbation) and a diseased condition (e.g., inflammatory cocktail + fatty acids). As described, the GAN model can be trained such that the generated image can be classified by the classifier to the same or similar accuracy as the actual image.

[0370] The synthetic image generated by the GAN model can be used to create a disease model. Figure 16B illustrates the downstream analysis of the generated image according to some embodiments. As shown, the generated image can be used to perform nuclear compartmentalization and speckle detection (e.g., using image processing algorithms). Thus, the generated image can be used to create a disease model for NASH disease and to consider chemical perturbations in NASH.

[0371] The synthetic images generated by the GAN model can also be used to evaluate the effectiveness of treatment. The system processes three groups of generated images: a first group of images depicting healthy tissue that does not have disease, a second group of images depicting untreated diseased tissue, and a third group of images depicting diseased tissue that has been treated (e.g., using a specific drug). In the example depicted in FIG. 16C, the first group of images (labeled as "untreated") comprises images of healthy tissue that does not have non-alcoholic steatohepatitis (NASH) disease, the second group of images (labeled as "NASH 2X") comprises images of untreated tissue with NASH disease, and the third group of images (labeled as "NASH 2X+ACC inhibitor") comprises images of tissue with NASH that has been treated with a drug (e.g., 50 μM ACC inhibitor or filsocostat). In some embodiments, the three groups of images capture tissue of the same subject at different times. In some embodiments, the group of three images capture different tissues of interest. The images can be phase images or fluorescence images generated using the techniques described herein (e.g., from a brightfield image). Each phase or fluorescence image is generated based on multiple brightfield images and can therefore comprise richer information, such as phase shift information. Thus, the generated phase or fluorescence images can enable greater accuracy in downstream analysis, as described below.

[0372] Specifically, to evaluate the effectiveness of a drug, the system generates a distribution for each group of images and determines whether the distribution reflects the effect of the drug on a disease state. In FIG. 16C, three probability distributions are generated using a classifier trained on the generated images. The X-axis shows the probabilities output by the classifier in response to receiving an input image (e.g., the generated phase image). As shown by the three distributions, the generated image of healthy tissue (i.e., distribution 1620) is generally classified as having a lower probability of having disease than the generated image of diseased tissue (i.e., distribution 1624). Furthermore, the generated image of treated tissue (i.e., distribution 1622) is generally classified as having a lower probability of having disease than the generated image of diseased tissue. Comparison of these distributions may indicate that the drug is effective in regressing or reducing the disease state because the treatment has caused the diseased tissue to contain features that are more similar to the healthy state and less similar to the diseased state. Thus, the composite images can enable a biophysical understanding of the distribution of lipids within cells and provide insight for treatment.

[0373] In some embodiments, rather than using a distribution, the system can identify image clusters in an embedding space (e.g., a UMAP), as shown in Figure 16D. In a UMAP, each point represents an image embedding of an image (e.g., a generated phase image). As shown, the generated images of treated tissue form a cluster that moves away from the cluster of generated diseased images and towards the cluster of generated healthy images, which may indicate that the drug is effective in regressing or reducing the disease state.

[0374] The analysis in Figures 16C and 16D can be applied to evaluate multiple drug candidates for a disease of interest. For example, tissues treated with each drug candidate can be imaged, for example, by a microscope to obtain a bright field image. The bright field image can be converted to a phase image using the techniques described herein. A distribution or cluster can be generated for each treatment phase image. Thus, the resulting plot can comprise a distribution or cluster representing a disease state, a distribution or cluster representing a healthy state, and multiple distributions or clusters each representing a candidate drug. The system can then identify the distribution or cluster closest to the healthy state and identify the most effective candidate drug candidate.

[0375] FIG. 17A illustrates a synthetic image generated by an exemplary GAN model to study tuberous sclerosis ("TSC"), according to some embodiments. In the depicted example, NGN2 neurons are illuminated using an optimal illumination pattern identified using the techniques described herein, and a brightfield image is captured. The brightfield image is input into the GAN model, which outputs the resulting phase image and the resulting fluorescence / body-py image in FIG. 17A. The GAN model used in FIG. 17A can be configured to include a trained classifier, as described with reference to FIGS. 14A-B and 15. The classifier has two conditions: a healthy condition (e.g., wild type) and a diseased condition (e.g., TSC KO). As described, the GAN model can be trained such that the generated image can be classified by the classifier to the same or similar accuracy as the actual image.

[0376] The synthetic images generated by the GAN model can also be used to evaluate the effectiveness of treatment. The system processes three groups of generated images: a first group of images depicting healthy tissue that does not have the disease, a second group of images depicting untreated diseased tissue, and a third group of images depicting diseased tissue that has been treated (e.g., with a specific drug). In the example depicted in FIG. 17B, the first group of images (labeled as "wild type") comprises images of healthy tissue that does not have TSC disease, the second group of images (labeled as "TSC") comprises images of untreated tissue that has TSC disease, and the third group of images (labeled as "TSC+rapamycin") comprises images of tissue with TSC that has been treated with a drug. In some embodiments, the three groups of images capture tissue of the same subject at different times. In some embodiments, the three groups of images capture tissue of different subjects. The images are phase images, generated using techniques described herein (e.g., from bright field images).

[0377] Specifically, to evaluate the effectiveness of a drug, the system generates a distribution for each group of images and determines whether the distribution reflects the effect of the drug on the disease. In FIG. 17B, three biomarker distributions are generated. As shown by these distributions, the drug is likely to be effective in regressing or reducing the disease state because the treatment causes the diseased tissue to contain characteristics that are more similar to the healthy state and less similar to the diseased state. The analysis in FIG. 17B can be applied to evaluate multiple drug candidates for a disease of interest, as described above.

[0378] Exemplary methods, non-transitory computer-readable storage media, systems, and electronic devices are described in the following sections. 1. 1. A method for training a machine learning model and generating images of a biological sample, comprising: acquiring a plurality of training images, the plurality of training images comprising: A first type of training images; and The second type of training images and and generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; A method comprising: 2. 2. The method of claim 1, wherein the first type of training image is a bright-field image of a biological sample. 3. 3. The method according to claim 2, wherein the second type of training images are fluorescent images of the biological sample. 4. 4. The method of any one of claims 1 to 3, wherein the machine learning model comprises a generator and a discriminator. 5. 5. The method of claim 4, wherein the machine learning model comprises a conditional GAN ​​model. 6. 6. The method of claim 4, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. 7. 7. The method of claim 6, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. 8. 8. The method of any of items 6-7, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. 9. 9. The method according to any of items 5-8, wherein the discriminator is a PatchGAN neural network. 10. 10. The method of any of items 1-9, further comprising generating a third type of image based on the first type of training images. 11. 11. The method according to claim 10, wherein the third type of image is a phase shift image. 12. 12. The method according to any of items 1-11, further comprising generating a fourth type of image based on the first type of training images. 13. Item 13. The method according to item 12, wherein the fourth type of image comprises segmentation data. 14. 14. The method according to any of items 1-13, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. 15. Item 15. The method of item 14, wherein the first lighting scheme comprises one or more lighting patterns. 16. 16. The method of any of items 14-15, wherein the first type of training images is a portion of a bright field image array. 17. The plurality of training images is a first plurality of training images, and the method further comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 17. The method according to any one of items 14 to 16, comprising: 18. acquiring a plurality of images of said first type using a microscope; generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 17. The method of any of items 1-16, further comprising: 19. 20. The method of claim 18, further comprising training a classifier based on a plurality of synthetic images of the second type. 20. the microscope is a first microscope and the classifier is a first classifier; acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. 20. The method of claim 19, further comprising: twenty one. 21. The method according to claim 20, wherein the second microscope is a fluorescent microscope. twenty two. 1. A method for generating an enhanced image of a biological sample, comprising: obtaining an image of the biological sample using a microscope; generating an enhanced image of the biological sample based on the image using a machine learning model, the machine learning model comprising: acquiring a plurality of training images, the plurality of training images comprising: A first type of training images; and The second type of training images and and generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; Being trained by A method comprising: twenty three. 23. The method according to claim 22, wherein the first type of training images are bright-field images of a biological sample. twenty four. 23. The method according to claim 22, wherein the second type of training images are fluorescent images of the biological sample. twenty five. 25. The method of any of claims 22-24, wherein the machine learning model comprises a generator and a discriminator. 26. 26. The method of claim 25, wherein the machine learning model comprises a conditional GAN ​​model. 27. 27. The method of any of claims 25-26, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. 28. 28. The method of claim 27, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. 29. 29. The method of any of claims 27-28, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. 30. 30. The method of any of items 26-29, wherein the discriminator is a PatchGAN neural network. 31. 31. The method according to any of items 23-30, further comprising generating an image of a third type based on the training images of the first type. 32. 32. The method according to claim 31, wherein the third type of image is a phase shift image. 33. 33. The method according to any of items 23-32, further comprising generating a fourth type of image based on the first type of training images. 34. 34. The method of claim 33, wherein the fourth type of image comprises segmentation data. 35. 35. The method according to any of items 23-34, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. 36. Item 36. The method of item 35, wherein the first lighting scheme comprises one or more lighting patterns. 37. 37. The method according to any of items 35-36, wherein the first type of training images is part of a bright field image array. 38. The plurality of training images is a first plurality of training images, and the method further comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 38. The method according to any one of items 35 to 37, comprising: 39. acquiring a plurality of images of said first type using a microscope; generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 39. The method according to any one of items 35-38, further comprising: 40. 40. The method of claim 39, further comprising training a classifier based on a plurality of synthetic images of the second type. 41. the microscope is a first microscope and the classifier is a first classifier; acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. 41. The method of claim 40, further comprising: 42. Item 42. The method according to item 41, wherein the second microscope is a fluorescence microscope. 43. 1. A system for training a machine learning model and generating images of a biological sample, comprising: 1. A computing system comprising one or more processors and one or more memories storing a machine learning model, the computing system configured to receive a plurality of training images of a first type and one training image of a second type, the computing system comprising: generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; A computing system configured to: A system comprising: 44. Item 44. The system of item 43, wherein the first type of training image is a bright-field image of a biological sample. 45. 45. The system of any of items 43-44, wherein the second type of training images are fluorescent images of the biological sample. 46. 46. ​​The system of any of items 43-45, wherein the machine learning model comprises a generator and a discriminator. 47. Item 47. The system of item 46, wherein the machine learning model comprises a conditional GAN ​​model. 48. 48. The system of any of claims 46-47, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. 49. Item 49. The system of item 48, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. 50. 50. The system of any of claims 48-49, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. 51. 51. The system of any of items 46-50, wherein the discriminator is a PatchGAN neural network. 52. 52. The system of any of items 43-51, wherein the computing system is further configured to generate a third type of image based on the first type of training images. 53. 53. The system of claim 52, wherein the third type of image is a phase shift image. 54. 54. The system of any of items 43-53, wherein the computing system is further configured to generate a fourth type of image based on the first type of training images. 55. Item 55. The system of item 54, wherein the fourth type of image comprises segmentation data. 56. 56. A system according to any of items 43-55, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. 57. Item 57. The system of item 56, wherein the first lighting scheme comprises one or more lighting patterns. 58. 58. The system of any of items 56-57, wherein the first type of training image is part of a bright field image array. 59. The plurality of training images is a first plurality of training images, and the computing system further comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 59. The system of any of items 56-58, configured to: 60. The computing system further comprises: acquiring a plurality of images of said first type using a microscope; generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 59. The system of any of items 43-59, configured to: 61. 60. The system of claim 59, wherein the computing system is further configured to train a classifier based on a plurality of synthetic images of the second type. 62. The microscope is a first microscope, the classifier is a first classifier, and the computing system further comprises: acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. Item 62. The system of item 61, configured to perform the following: 63. Item 63. The system of item 62, wherein the second microscope is a fluorescence microscope. 64. 1. A system for generating an enhanced image of a biological sample, comprising: 1. A computing system comprising one or more processors and one or more memories storing a machine learning model, the computing system configured to receive an image of a biological sample obtained from a microscope and generate an enhanced image of the biological sample based on the image using a machine learning model, the machine learning model comprising: acquiring a plurality of training images, the plurality of training images comprising: A first type of training images; and The second type of training images and and generating a plurality of wavelet coefficients using the machine learning model based on the first type of training images; generating the second type synthetic image based on the plurality of wavelet coefficients; comparing said second type synthetic image with said second type training image; updating the machine learning model based on the comparison; Trained by,Computing Systems A system comprising: 65. Item 65. The system of item 64, wherein the first type of training image is a bright-field image of a biological sample. 66. Item 65. The system of item 64, wherein the second type of training images are fluorescent images of the biological sample. 67. 67. The system of any of claims 64-66, wherein the machine learning model comprises a generator and a discriminator. 68. Item 68. The system of item 67, wherein the machine learning model comprises a conditional GAN ​​model. 69. 69. The system of any of claims 67-68, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. 70. 70. The system of claim 69, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. 71. 71. The system of any of items 69-70, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. 72. 72. The system of any of items 67-71, wherein the discriminator is a PatchGAN neural network. 73. 73. The system of any of items 64-72, wherein the machine learning model is further trained by generating a third type of image based on the first type of training images. 74. Item 74. The system of item 73, wherein the third type of image is a phase shift image. 75. 75. The system of any of claims 64-74, wherein the machine learning model is trained by generating a fourth type of image based on the first type of training images. 76. Item 76. The system of item 75, wherein the fourth type of image comprises segmentation data. 77. 77. The system of any of items 64-76, wherein the first type of training images are captured using a microscope and according to a first illumination scheme. 78. Item 78. The system of item 77, wherein the first lighting scheme comprises one or more lighting patterns. 79. 8. The system of any of items 77-78, wherein the first type of training image is a portion of a bright field image array. 80. The plurality of training images is a first plurality of training images, and the machine learning model comprises: identifying a second lighting scheme based on the comparison; and acquiring a second plurality of training images comprising one or more images of the first type, the one or more images of the first type being acquired based on the second lighting scheme; and training the machine learning model based on the second plurality of training images; 80. The system of any of items 77-79, wherein the system is trained by 81. The above machine learning model is acquiring a plurality of images of said first type using a microscope; generating a plurality of synthetic images of the second type based on the plurality of captured images using the machine learning model; and 81. The system of any of items 77-80, wherein the system is trained by 82. Item 82. The system of item 81, wherein the machine learning model is trained by training a classifier based on a plurality of synthetic images of the second type. 83. The microscope is a first microscope, the classifier is a first classifier, and the machine learning model is acquiring a plurality of images of the second type using a second microscope; and training a second classifier based on the plurality of images of the second type; Comparing the performance of the first classifier and the second classifier. Item 83. The system of item 82, which is trained by 84. Item 84. The system of item 83, wherein the second microscope is a fluorescence microscope. 85. 1. A method for processing an image of a biological sample to obtain one or more output images, comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model to obtain the one or more output images; A method comprising: 86. Item 86. The method of item 85, wherein at least one of the configurations of the SLM is for generating one or more optical aberrations. 87. 87. The method of claim 86, wherein generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof. 88. 87. The method according to any of items 85-86, wherein at least one of the configurations of the SLM is for enhancing one or more features. 89. 89. The method of claim 88, wherein the one or more characteristics include cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. 90. 80. The method according to any of items 85-89, wherein at least one of the configurations of the SLM is for reducing optical aberrations. 91. 91. The method according to any of items 85-90, wherein the multiple SLM configurations are for acquiring images of the biological sample at different depths. 92. 92. The method of any of items 85-91, wherein the machine learning model is configured to generate an image of a second type based on an image of a first type. 93. Item 93. The method according to item 92, wherein the first type of image is a bright field image. 94. 93. The method according to claim 92, wherein the second type of image is a fluorescent image. 95. Item 93. The method of item 92, wherein the second type of image is an enhanced version of the first type of image. 96. The method according to any one of items 92-95, wherein the machine learning model is a GAN model or a self-supervised model. 97. 97. The method of any of items 85-96, wherein the multiple images are acquired using multiple configurations of a light source of the optical system. 98. Item 98. The method of item 97, wherein the light source is an LED array of the optical system. 99. At least one of the plurality of SLM configurations comprises: training said machine learning model; Evaluating the trained machine learning model; and identifying the at least one configuration based on the evaluation; and 99. The method according to any of items 85-98, obtained by 100. 99. The method of any of claims 85-99, wherein the trained machine learning model is configured to receive an input image and output an enhanced version of the input image. 101. Item 101. The method of item 100, wherein the enhanced version of the input image comprises one or more enhanced cell phenotypes. 102. 1. An electronic device for processing images of a biological sample and obtaining one or more output images, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model to obtain the one or more output images; and one or more programs containing instructions for carrying out the An electronic device comprising: 103. 1. A non-transitory computer readable storage medium storing one or more programs for processing images of a biological sample and obtaining one or more output images, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of the biological sample into a trained machine learning model to obtain the one or more output images; A non-transitory computer-readable storage medium for causing 104. 1. A method for classifying an image of a biological sample, comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of said biological sample into a trained machine learning model to obtain one or more classification outputs; A method comprising: 105. Item 105. The method of item 104, wherein at least one of the plurality of configurations of the SLM is for generating one or more optical aberrations. 106. Item 106. The method of item 105, wherein generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof. 107. 107. The method according to any of claims 104-106, wherein at least one of the plurality of configurations of the SLM is for enhancing one or more features. 108. 108. The method of claim 107, wherein the one or more characteristics include cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. 109. Item 109. The method according to any of items 104-108, wherein at least one of the configurations of the SLM is for reducing optical aberrations. 110. 109. The method of claim 104, wherein the multiple SLM configurations are for acquiring images of the biological sample at different depths. 111. 111. The method of any of items 104-110, wherein the multiple images are acquired using multiple configurations of a light source of the optical system. 112. Item 112. The method of item 111, wherein the light source is an LED array of the optical system. 113. At least one of the plurality of SLM configurations comprises: training said machine learning model; Evaluating the trained machine learning model; and identifying the at least one configuration based on the evaluation; and The method according to any of items 104-112, obtained by 114. Item 114. The method of any of Items 104-113, wherein the trained machine learning model is configured to receive an input image and detect one or more predefined objects within the input image. 115. 15. The method of claim 114, wherein the predefined object comprises diseased tissue. 116. 1. An electronic device for classifying images of a biological sample, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of said biological sample into a trained machine learning model to obtain one or more classification outputs; one or more programs including instructions for performing An electronic device comprising: 117. 1. A non-transitory computer readable storage medium storing one or more programs for classifying images of biological samples, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: acquiring a plurality of images of the biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; inputting a plurality of images of said biological sample into a trained machine learning model to obtain one or more classification outputs; A non-transitory computer-readable storage medium for causing 118. 1. A method for training a machine learning model, comprising: - acquiring a plurality of images of a biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; training said machine learning model using said plurality of images; A method comprising: 119. Item 119. The method of item 118, wherein at least one of the plurality of configurations of the SLM is for generating one or more optical aberrations. 120. Item 120. The method of item 119, wherein generating one or more optical aberrations includes spherical aberration, astigmatism, exfocality, distortion, tilt, or any combination thereof. 121. 121. The method according to any of items 118-120, wherein at least one of the plurality of configurations of the SLM is for enhancing one or more features. 122. 122. The method of claim 121, wherein the one or more characteristics include cell borders, actin filaments, nuclear shape, cytoplasmic compartmentalization, or any combination thereof. one two three. Item 123. The method according to any of items 118-122, wherein at least one of the configurations of the SLM is for reducing optical aberrations. 124. 124. The method according to any of items 118-123, wherein at least one configuration of the plurality of configurations of the SLM is for acquiring images of the biological sample at different depths. 125. 125. The method of any of items 118-124, wherein the machine learning model is configured to generate an image of a second type based on an image of a first type. 126. Item 126. The method according to item 125, wherein the first type of image is a bright field image. 127. Item 126. The method according to item 125, wherein the second type of image is a fluorescent image. 128. The method according to any one of items 118-127, wherein the machine learning model is a GAN model or a self-supervised model. 129. 129. The method according to any of items 118-128, wherein the machine learning model is a classification model. 130. 130. The method of any of items 118-129, wherein the multiple images are acquired using multiple configurations of a light source of the optical system. 131. Item 131. The method of item 130, wherein the light source is an LED array of the optical system. 132. Training the above machine learning model involves: (a) training the machine learning model using a first image, the first image being acquired using a first configuration of an SLM of the optical system; (b) evaluating the trained machine learning model; and (c) identifying a second configuration of the SLM based on the evaluation; and (d) training the machine learning model using a second image, the second image being acquired using a second configuration of the SLM of the optical system; 132. The method according to any one of items 118-131, comprising: 133. Item 113. The method of item 112, wherein the evaluation is based on a loss function of the machine learning model. 134. 114. The method of any of items 112-113, further comprising repeating steps (a)-(d) until a threshold is met. 135. Item 115. The method of item 114, wherein the threshold indicates convergence of the training. 136. 136. The method of any of claims 118-135, wherein the trained machine learning model is configured to receive an input image and output an enhanced version of the input image. 137. Item 137. The method of item 136, wherein the enhanced version of the input image comprises one or more enhanced cellular phenotypes. 138. 136. The method of any of claims 118-135, wherein the trained machine learning model is configured to receive an input image and detect one or more predefined objects within the input image. 139. 19. The method of claim 138, wherein the predefined object comprises diseased tissue. 140. 1. An electronic device for training a machine learning model, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: - acquiring a plurality of images of a biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; training said machine learning model using said plurality of images; and one or more programs containing instructions for carrying out the An electronic device comprising: 141. 1. A non-transitory computer-readable storage medium storing one or more programs for training a machine learning model, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: - acquiring a plurality of images of a biological sample using a plurality of configurations of an SLM of an optical system, the SLM being located in an optical path between the biological sample and an image recording device; training said machine learning model using said plurality of images; A non-transitory computer-readable storage medium for causing 142. 1. A method for generating an enhanced image of a biological sample, comprising: Using a microscope, acquiring an image of a biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern comprising: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of lighting settings; identifying the lighting pattern based on a plurality of weights of the trained machine learning model; and and generating an enhanced image of the biological sample by inputting an acquired image of the biological sample into the trained machine learning model; A method comprising: 143. Item 143. The method of item 142, wherein the acquired image is a bright field image. 144. 144. The method according to any of items 142-143, wherein the enhanced image is a fluorescence image, a phase image, or a combination thereof. 145. 5. The method of claim 142, wherein the illumination source comprises an array of illumination emitters. 146. Item 146. The method of item 145, wherein the illumination source is an LED array. 147. Item 147. The method of any of items 142-146, wherein the lighting pattern indicates whether each lighting emitter is turned on or off and the intensity of each lighting emitter. 148. Item 148. The method of any of items 145-147, wherein each lighting setting of the plurality of lighting settings corresponds to a respective lighting emitter of the lighting source, and each weighting corresponds to an intensity of the respective lighting emitter. 149. Item 149. The method of any of items 142-148, wherein the classification model is configured to receive an input phase image or an input fluorescence image and output a classification result indicative of one of a plurality of predefined classes. 150. Item 149. The method of item 149, wherein the plurality of predefined classes comprises a healthy class and a diseased class. 151. Item 151. The method of item 150, wherein the machine learning model is a GAN model comprising an attention layer comprising the plurality of weights, a discriminator, and a generator. 152. Item 152. The method of item 151, wherein the machine learning model is a conditional GAN ​​model. 153. 153. The method of any of claims 151-152, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. 154. Item 154. The method of item 153, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. 155. 5. The method of any of claims 153-154, wherein the plurality of neural networks comprises a plurality of U-Net neural networks. 156. Item 156. The method of any of items 151-155, wherein the discriminator is a PatchGAN neural network. 157. Training the machine learning model using the trained classification model, applying the weights to a plurality of bright-field training images; and aggregating the plurality of weighted brightfield training images into an aggregated brightfield image; inputting the aggregated bright-field training images into the machine learning model to obtain enhanced training images and a generator loss; inputting the enhanced training images into the trained classifier and obtaining a classifier loss; augmenting the generator loss based on the classifier loss; updating the weights based on the extended generator loss; and 157. The method according to any one of items 151-156, comprising: 158. 158. The method of any of claims 142-157, further comprising classifying the enhanced image using the trained classifier. 159. 159. The method of any of items 142-158, further comprising displaying the enhanced image. 160. 1. A system for generating an enhanced image of a biological sample, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: Using a microscope, acquiring an image of a biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern comprising: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of lighting settings; identifying the lighting pattern based on a plurality of weights of the trained machine learning model; and and generating an enhanced image of the biological sample by inputting an acquired image of the biological sample into the trained machine learning model; and one or more programs containing instructions for carrying out the A system comprising: 161. 1. A non-transitory computer readable storage medium storing one or more programs for generating an enhanced image of a biological sample, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: Using a microscope, acquiring an image of a biological sample illuminated using an illumination pattern of an illumination source, the illumination pattern comprising: training a classification model configured to receive an input image and output a classification result; training a machine learning model using the trained classification model, the machine learning model having a plurality of weights corresponding to a plurality of lighting settings; identifying the lighting pattern based on a plurality of weights of the trained machine learning model; and and generating an enhanced image of the biological sample by inputting an acquired image of the biological sample into the trained machine learning model; A non-transitory computer-readable storage medium for causing 162. 1. A method for evaluating a treatment for a disease of interest, comprising: receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples afflicted by the disease of interest; receiving a third plurality of images depicting a third set of treated biological samples afflicted by the disease of interest and being treated with the treatment; inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images; inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images; inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment; A method comprising: 163. Item 163. The method of item 162, wherein the first plurality of images, the second plurality of images, and the third plurality of images are bright field images. 164. Item 164. The method of any of items 162-163, wherein the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images are fluorescence images. 165. Item 164. The method of any of Items 162-163, wherein the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images are phase images. 166. 166. The method of any of claims 162-165, wherein comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment includes identifying a signal in each image that is associated with a biomarker. 167. Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment may further include: determining a first distribution based on signals of the biomarkers in the first plurality of enhanced images; determining a second distribution based on signals of the biomarkers in the second plurality of enhanced images; determining a third distribution based on signals of the biomarkers in the third plurality of enhanced images; Item 167. The method according to item 166, comprising: 168. Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment may further include: Item 168. The method of item 167, comprising comparing the first distribution, the second distribution, and the third distribution and evaluating the treatment. 169. 166. The method of any of claims 162-165, wherein comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images and evaluating the treatment includes determining a score for each image indicative of a status of the disease of interest. 170. Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment may further include: determining a first distribution based on the scores of the first plurality of enhanced images; determining a second distribution based on the scores of the second plurality of enhanced images; determining a third distribution based on the scores of the third plurality of enhanced images; Item 169. The method according to item 169, comprising: 171. Comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment may further include: Item 171. The method of item 170, comprising comparing the first distribution, the second distribution, and the third distribution and evaluating the treatment. 172. The treatment is a first treatment, and the method further comprises: receiving a fourth plurality of images depicting a fourth set of treated biological samples afflicted with the disease of interest and being treated with a second treatment; inputting the fourth plurality of images into the trained machine learning model to obtain a fourth plurality of enhanced images; and comparing the first plurality of enhanced images, the second plurality of enhanced images, the third plurality of enhanced images, and the fourth plurality of enhanced images and comparing the first treatment and the second treatment; The method according to any one of items 162-171, comprising: 173. 173. The method of claim 172, further comprising selecting a treatment from the first treatment and the second treatment based on the comparison. 174. 174. The method of claim 173, further comprising administering said selected treatment. 175. Item 174. The method of item 173, further comprising providing a medical recommendation based on the selected treatment. 176. Item 162-175. The method of any of Items 162-175, wherein the trained machine learning model is a GAN model comprising a discriminator and a generator. 177. Item 177. The method of item 176, wherein the machine learning model is a conditional GAN ​​model. 178. 178. The method of any of claims 176-177, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups. 179. Item 179. The method of item 178, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective set of frequencies. 180. Item 179. The method of any of items 176-179, wherein the discriminator is a PatchGAN neural network. 181. 1. A system for evaluating a treatment for a disease of interest, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples afflicted by the disease of interest; receiving a third plurality of images depicting a third set of treated biological samples afflicted by the disease of interest and being treated with the treatment; inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images; inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images; inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment; and one or more programs containing instructions for carrying out the A system comprising: 182. 1. A non-transitory computer readable storage medium storing one or more programs for evaluating a treatment for a disease of interest, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: receiving a first plurality of images depicting a first set of healthy biological samples not affected by the disease of interest; receiving a second plurality of images depicting a second set of untreated biological samples afflicted by the disease of interest; receiving a third plurality of images depicting a third set of treated biological samples afflicted by the disease of interest and being treated with the treatment; inputting the first plurality of images into a trained machine learning model to obtain a first plurality of enhanced images; inputting the second plurality of images into the trained machine learning model to obtain a second plurality of enhanced images; inputting the third plurality of images into the trained machine learning model to obtain a third plurality of enhanced images; comparing the first plurality of enhanced images, the second plurality of enhanced images, and the third plurality of enhanced images to evaluate the treatment; A non-transitory computer-readable storage medium for causing

[0379] Although the present disclosure and embodiments have been described in full with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art, and such changes and modifications are to be understood as being included within the scope and embodiments of the present disclosure as defined by the claims.

[0380] The foregoing description has been described with reference to specific embodiments for illustrative purposes. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been chosen and described in order to best explain the principles of the technique and its practical application. Those skilled in the art will thereby be enabled to best utilize the technique and various embodiments with various modifications as may be suitable for the particular use contemplated.

Claims

1. A system for evaluating a treatment for a disease of interest, comprising: one or more processors; Memory, one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising: receiving a first one or more images depicting one or more untreated biological samples afflicted by the disease of interest; receiving a second one or more images depicting one or more treated biological samples afflicted by the disease of interest and being treated with the treatment; inputting the first one or more images into a trained machine learning model to obtain a first one or more transformed images; inputting the second one or more images into the trained machine learning model to obtain a second one or more transformed images; comparing the first one or more translated images and the second one or more translated images to evaluate the treatment; one or more programs containing instructions for carrying out the steps of: A system comprising:

2. The one or more programs, receiving a third one or more images depicting one or more healthy biological samples not affected by the disease of interest; inputting the third one or more images into the trained machine learning model to obtain a third one or more transformed images; comparing the first one or more transformed images, the second one or more transformed images, and the third one or more transformed images to evaluate the treatment; The system of claim 1 , further comprising instructions for:

3. The system described in claim 2, wherein the first one or more images, the second one or more images, and the third one or more images are bright field images.

4. The system described in claim 2, wherein the first one or more converted images, the second one or more converted images, and the third one or more converted images are fluorescent images.

5. The system described in claim 2, wherein the first one or more transformed images, the second one or more transformed images, and the third one or more transformed images are phase images.

6. The system of claim 2, wherein comparing the first one or more transformed images, the second one or more transformed images, and the third one or more transformed images and evaluating the treatment includes identifying a signal in each image that is associated with a biomarker.

7. Comparing the first one or more converted images, the second one or more converted images, and the third one or more converted images to evaluate the treatment, determining a first distribution based on signals of the biomarkers in the first one or more transformed images; determining a second distribution based on signals of the biomarkers in the second one or more transformed images; determining a third distribution based on signals of the biomarkers in the third one or more transformed images; The system of claim 6 further comprising:

8. The system of claim 7, wherein comparing the first one or more transformed images, the second one or more transformed images, and the third one or more transformed images and evaluating the treatment further includes comparing the first distribution, the second distribution, and the third distribution and evaluating the treatment.

9. The system of claim 2, wherein comparing the first one or more converted images, the second one or more converted images, and the third one or more converted images and evaluating the treatment includes determining a score for each image indicative of a status of the disease of interest. Comparing the first one or more converted images, the second one or more converted images, and the third one or more converted images to evaluate the treatment includes: determining a first distribution based on the scores of the first one or more transformed images; determining a second distribution based on the scores of the second one or more transformed images; and determining a third distribution based on the scores of the third one or more transformed images; and The system of claim 9 further comprising:

11. The system of claim 10, wherein comparing the first one or more transformed images, the second one or more transformed images, and the third one or more transformed images and evaluating the treatment further includes comparing the first distribution, the second distribution, and the third distribution and evaluating the treatment.

12. The treatment is a first treatment, and the one or more programs include: receiving a fourth one or more images depicting a fourth one or more treated biological samples afflicted by the disease of interest and being treated with a second treatment; inputting the fourth one or more images into the trained machine learning model to obtain a fourth one or more transformed images; comparing the first one or more transformed images, the second one or more transformed images, the third one or more transformed images, and the fourth one or more transformed images; and comparing the first treatment and the second treatment. The system of claim 2 , further comprising instructions for:

13. The system of claim 12, wherein the one or more programs include instructions for selecting a treatment from the first treatment and the second treatment based on the comparison.

14. The system of claim 13, wherein the one or more programs include instructions for administering the selected treatment.

15. The system of claim 13, wherein the one or more programs include instructions for providing medical recommendations based on the selected treatment.

16. The system of claim 1, wherein the trained machine learning model is a GAN model comprising a discriminator and a generator.

17. The system described in claim 16, wherein the trained machine learning model is a conditional GAN ​​model.

18. The system of claim 16, wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups.

19. The system of claim 18, wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a distinct set of frequencies.

20. The system of claim 16, wherein the discriminator is a PatchGAN neural network.

21. A method for evaluating a treatment for a disease of interest, the method comprising: receiving a first one or more images depicting one or more untreated biological samples afflicted by the disease of interest; receiving a second one or more images depicting one or more treated biological samples afflicted by the disease of interest and being treated with the treatment; inputting the first one or more images into a trained machine learning model to obtain a first one or more transformed images; inputting the second one or more images into the trained machine learning model to obtain a second one or more transformed images; comparing the first one or more translated images and the second one or more translated images to evaluate the treatment; A method comprising:

22. A non-transitory computer readable storage medium storing one or more programs for evaluating a treatment for a disease of interest, the one or more programs including instructions that, when executed by one or more processors of an electronic device, perform the following: receiving a first one or more images depicting one or more untreated biological samples afflicted by the disease of interest; receiving a second one or more images depicting one or more treated biological samples afflicted by the disease of interest and being treated with the treatment; inputting the first one or more images into a trained machine learning model to obtain a first one or more transformed images; inputting the second one or more images into the trained machine learning model to obtain a second one or more transformed images; comparing the first one or more translated images and the second one or more translated images to evaluate the treatment; A non-transitory computer-readable storage medium that causes the electronic device to perform the steps of:

23. The system of claim 1, wherein the first one or more images and the second one or more images are bright field images.

24. The system of claim 1, wherein the first one or more converted images and the second one or more converted images are fluorescent images.

25. The system of claim 1, wherein the first one or more transformed images and the second one or more transformed images are phase images.

26. The system of claim 1, wherein comparing the first one or more converted images and the second one or more converted images and evaluating the treatment includes identifying a signal in each image that is associated with a biomarker.

27. Comparing the first one or more converted images and the second one or more converted images to evaluate the treatment, comprising: determining a first distribution based on signals of the biomarkers in the first one or more transformed images; determining a second distribution based on signals of the biomarkers in the second one or more transformed images; 27. The system of claim 26, further comprising:

28. The system of claim 27, wherein comparing the first one or more transformed images and the second one or more transformed images and evaluating the treatment further comprises comparing the first distribution and the second distribution and evaluating the treatment.

29. The system of claim 1, wherein comparing the first one or more converted images and the second one or more converted images and evaluating the treatment includes determining a score for each image indicative of a status of the disease of interest.

30. The system of claim 1, wherein the first one or more images include a plurality of images.

Images