Phenotypic responses of drug-induced organelle morphology

Abstract

The disclosure provides a method of training a deep learning model on microscopy images of cells that are under the influence of different drugs or affected by different disease conditions and a database atlas to map how different drugs or disease conditions affect cellular morphology such that the drug or disease condition can be predicted from the microscopy image once the atlas has been trained.

Description

Attorney Docket No.00015-429WO1 PHENOTYPIC RESPONSES OF DRUG-INDUCED ORGANELLE MORPHOLOGY CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority of U.S. Provisional Application No. 63 / 522,681, filed June 22, 2023, the disclosures of which are incorporated herein by reference in their entirety. FIELD OF THE INVENTION

[0002] The disclosure provides a method of training a deep learning model on microscopy images of cells that are under the influence of different drugs or affected by different disease conditions and a database atlas to map how different drugs or disease conditions affect cellular morphology such that the drug or disease condition can be predicted from the microscopy image once the atlas has been trained. BACKGROUND

[0003] The goal of high-content phenotypic screening is to use the link between the morphology of a cell and its state to identify drug candidates that move the cell’s state in a desired manner. The morphology of a cell can thus guide the identification of a disease and its underlying aberrant states and the identification of drugs to correct these states.

[0004] In traditional high-content phenotypic profiling, a set of features is manually defined and quantified from each microscopic image. Examples include size, shape, and texture of an organelle. Such feature sets can then be processed to find a subset of features that constitute a ‘fingerprint’ of each individual sample condition, for example healthy versus diseased. Prominent successful examples of this profiling technique include the NCI-60 tumor cell line panel where 60 different tumor cell lines were used to extract patterns of anticancer drug mechanisms and the relation between small molecule drugs, genes, and diseases. Further examples include the determination of mechanism of action of pharmacological compounds, target identification, gene relationship identification and characterization of cellular heterogeneity. Classic modeling techniques in this method, e.g. hierarchical clustering, have in recent years been replaced with sophisticated deep learning methodsAttorney Docket No.00015-429WO1 in phenotypic drug development. Specifically, convolutional neural networks (CNNs), have demonstrated superior classification performance through enhanced extraction of image-intrinsic features. SUMMARY

[0005] The disclosure provides (1) a method of training a deep learning model on microscopy images of cells that are under the influence of different drugs and (2) the product of this training which is an atlas of how different drugs affect cellular morphology. In effect the disclosure provides a new method to extract meaning from microscopy images of a cell.

[0006] The method can be used for the following applications: (a) For drug discovery: In high content screening, the atlas can be used to predict the effect of novel drug compounds on the cell from taking images of the cell; (b) For disease diagnosis: In a clinical setting, images of cells extracted from biopsies can be used for diagnosis; (c) For biomarker generation: In clinical trials, a certain healthy cellular state might be the endpoint and the tool can be used to define and measure these endpoints; (d) For basic discovery and research: In a research setting, the atlas can be used to better understand cellular states and investigate their causes and distributions better. Since the atlas is the first of its kind, it enables new avenues for scientific discovery.

[0007] The disclosure provides a predictive atlas that links a microscopy image of a cell to the state that the cell is currently in and how that state is influenced by drugs or disease. The resulting atlas obtained on the model system of mitochondria organelles in the context of cervical and head and neck cancer is exemplified herein. But the invention is general and can be applied to any model system and any cell or tissue type. The approach is unique in that it combines: 1) self-supervised deep learning with 2) fluorescence microscopy data of cells that 3) have been subjected to a large panel of drugs to 4) create a latent space or atlas that 5) fully captures how cellular morphology is related morphology changing drugs.

[0008] The disclosure is based on creating low-dimensional embedding spaces from microscopy images using deep learning.Attorney Docket No.00015-429WO1 Specifically, microscopy images of mitochondria centered at a single cell nucleus are fed into the deep learning model. Each cell is treated with one from a fixed set of drugs and induces a particular visual phenotype (visual characteristic) in the mitochondria. The model converts these images into a compact and expressive atlas that places images of cells which show similar phenotypes together, thus delineating the variety of phenotypes visible in all the images. In this way, a map of phenotype is created from microscopy images.

[0009] The disclosure provides a method comprising creating low-dimensional embedding spaces from microscopy images using deep learning by: determining one or a plurality of subcellular structure or organelles of interest; capturing a plurality of images of the structures or organelle of interest in a cell under the influence of an agent suspected of modifying the phenotype of the structures or organelles of interest; training a self- supervised deep learning model on the plurality of images such that no labels are provided to the model as to which agent was contacted with the cell; repeating (i) – (iii) for a plurality of cells under different culture conditions; and converting the images obtained to a compact and expressive atlas that places images of cells showing similar visual phenotypes together, thus generating a map that links phenotypes from microscope images to the agent’s function. In one embodiment, the plurality of cells under different culture conditions comprise cells exposed to different agents suspected of modifying their phenotypes. In another embodiment, the structure or organelle of interest is a mitochondria. In still another embodiment, the images are obtained from cells stained with a fluorescent or luminescent agent. In another embodiment, the training of the self-supervised deep learning model comprises analysis of a plurality of visual characteristics obtained using one or more channel images that highlight different functional and morphological aspects of the structure or organelle of interest. In still another embodiment, the method further comprises obtaining a large number of multi-channel images of single cells treated with different agents. In a further embodiment, the method further comprises normalizing each channel of the large number of multi-Attorney Docket No.00015-429WO1 channel images with the 95 percentile of the maximum intensity value of each channel. In another or further embodiment, the method further comprise performing gamma correction and denoising to highlight the structure or organelle of interest in each image of the large number of multi-channel images. In still another or further embodiment, the method further comprises stochastically performing data augmentation on the large number of multi-channel images. In a further embodiment, data augmentation includes random rotation, flipping, additive noise, multiplicative brightness, additive brightness, random crops and combinations thereof. In another embodiment, images are applied to a deep learning algorithm selected from the group consisting of SimCLR, transformer-based (DINO), autoencoder based (VQVAE) and bootstrapping-based (VYOL). In a further embodiment, the deep learning algorithm maps images into a low dimensional (128-2048) latent space. In still a further embodiment, the method further comprises mapping cell images to a single vector in the low dimensional latent space. In another embodiment, the method further comprises mapping the biological effect of a test agent on a cell contacted with the test agent comprising generating a low dimensional latent space image of the cell contacted with the test agent and applying an atlas of vectors to the image to map a phenotypic changes of the cell contacted with the test agent. In another embodiment, the method further comprises creating a mapping that is robust to day-to-day variations that occur between microscopy images captured on different days. In yet another embodiment, the method further comprises a robust prediction technique to classify the agent acting on the cell from the microscopy image. In another embodiment, the method further comprises the ability to not only work on images that have been captured to assess an agent’s effect on the cell for a single time window but also work on temporal series of images that have been captured to assess an agent’s effect on the cell for a longer time period, and thereby consisting of images that capture a subcellular structure or organelle that evolves over time.

[0010] The disclosure also comprises a method for automated cell analysis, the method comprising: receiving, by a computer apparatus, a first cellular image associated with a first cellAttorney Docket No.00015-429WO1 treated with a first culture condition or having a first phenotype; determining, by the computer apparatus, one or more cellular structures or organelles in the first cell; creating low- dimensional embedding spaces from the first cellular image using deep learning by: determining a subcellular structure or organelle of interest; capturing a plurality of images of the structure or organelle of interest; training a self-supervised deep learning model on the plurality of images such that no labels are provided to the model as to which culture condition or phenotype was used on the first cell; repeating (i) – (iii) for a plurality of cells under different culture conditions or different disease types; and converting the images obtained to a compact and expressive atlas that places images of cells showing similar visual phenotypes together, thus generating a map of phenotypes from cellular images; providing a second cellular image of a second cell to the computer apparatus; determining, by the computer apparatus, one or more structures or cellular organelles in the second cell; measuring a plurality of visual characteristics from the structures or organelles in the second cell; identifying, by the computer apparatus, cellular structures or organelles in the compact and expressive atlas that have the same or similar visual characteristics to the one or more structure or organelles of the second cell; identifying a culture condition or phenotypes associated with the identified cellular structures or organelles in the compact and expressive atlas to a user as being associated with the second cell. In another embodiment, the plurality of cells under different culture conditions comprise cells exposed to different agents suspected of modifying their phenotypes. In still another embodiment, the structure or organelle is a mitochondria. In yet another embodiment, the images are obtained from cells stained with a fluorescent or luminescent agent. In still another embodiment, the plurality of visual characteristics are obtained using one or more channel images that highlight different functional and morphological aspects of the organelle. In another embodiment, the method further comprises obtaining a large number of multi-channel images of single cells treated with different agents. In another embodiment, the method further comprisesAttorney Docket No.00015-429WO1 normalizing each channel of the large number of multi-channel images with the 95 percentile of the maximum intensity value of each channel. In still another embodiment, the method further comprises gamma correction and denoising to highlight the organelle in each image of the large number of multi-channel images. In still another embodiment, the method further comprises stochastically performing data augmentation on the large number of multi-channel images. In still a further embodiment, data augmentation includes random rotation, flipping, additive noise, multiplicative brightness, additive brightness, random crops and combinations thereof. In still another embodiment, image data is applied to a deep learning algorithm selected from the group consisting of SimCLR, transformer-based (DINO), autoencoder based (VQVAE) and bootstrapping-based (VYOL). In a further embodiment, the deep learning algorithm maps each image of the large number of images into a low dimensional (128-2048) latent space. In still a further embodiment, the method further comprises mapping each cell image to a single vector in the low dimensional latent space. In an further embodiment, the method further comprises mapping the biological effect of a test agent on a cell contacted with the test agent comprising generating a low dimensional latent space image of the cell contacted with the test agent and applying an atlas of vectors to the image to map a phenotypic changes of the cell contacted with the test agent. In yet another embodiment, the method further comprises mapping the biological effect of a test agent on a cell contacted with the test agent comprising generating a low dimensional latent space embedding that relates how the subcellular structure’s or organelle’s morphology once contacted with the test agent is related to the subcellular structure’s or organelle’s morphology contacted with other agents and thereby creating an atlas to map phenotypic changes of the cell contacted with the test agent. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A-E provides self-supervised deep learning of drug-induced mitochondria phenotype changes with MitoSpace. (A) Mitochondrial network morphologies (white) are known to show a large cell to cell variability in control cells (HeLa) in controlAttorney Docket No.00015-429WO1 conditions (wt). (B) Mitochondrial networks (white) are reported to be distributed on an axis that ranges from highly fragmented (left) to highly fused (right). (C) Additional mitochondrial morphologies are hard to place on the fragmented-fused axis and have historically been hard to contextualize. (D) Depicted are the 9 major drug classes affecting mitochondria: i,ii) fission / fusion (yellow, beige), iii) interaction with other organelles such as the ER and the cytoskeleton (red), iv) mitochondrial DNA (brown), v,vi) mitochondrial membrane potential through uncouplers and ionophores (dark and light blue), vii) mitochondrial electron transport chain complexes (orange), and viii,ix) reduction (purple) or elevation (pink) of reactive oxygen species (ROS). 25 drugs belonging to these 9 drug classes were applied to cells to test the hypothesis that there are additional dimensions beyond the canonical fragmented to fused dimension. (E) High content fluorescence microscopy data of cells (HeLa, Cal27) was collected to capture mitochondrial morphology (Mitotracker, green), mitochondrial membrane potential (TMRM, red), and the location of the nucleus (DAPI, white) and were used to train a self-supervised deep learning model. After a data augmentation step, the model consists of two encoders that learn representations that aim to maximize agreement, thereby extracting signals that are common to entire classes of images while ignoring spurious factors such as background. The result is a reduced parameter representation of the cell images - an atlas that maps mitochondrial morphology to drug action.

[0012] Figure 2A-B shows MitoSpace’s ability create an atlas of drug-induced mitochondrial phenotypes. (A) Depicted is the image- extracted semantically meaningful latent space of drug-induced mitochondrial phenotypes. Every pixel represents a cell image taken under control or drug treatment conditions. A larger distance between images indicates greater differences in mitochondrial phenotype. Note how drugs of the same class (e.g. uncouplers, ionophores, agents that affect the respiratory chain complexes, etc.) cluster in the same regions of the space, indicating that mitochondria react to similar treatment similarly and that MitoSpace can semantically extract these similarities. (B)Attorney Docket No.00015-429WO1 Representative images of mitochondrial network phenotypes for each cluster are shown in this panel (green, mitochondrial mass, MitoTracker; red, mitochondrial membrane potential, TMRM). Note how similar morphologies (e.g. punctate for uncouplers and ionophores in the bottom left) and similar membrane potentials (e.g. high membrane potentials in hyperpolarized, oligomycin-treated cells on the right side of the space) cluster together in the space.

[0013] Figure 3A-G shows how MitoSpace introduces semantic directions that integrate both mitochondrial form and function. (A) Principal component (PC) analysis of MitoSpace shows that the first five PCs (blue bars) explain 58% of the variance in the dataset (light blue line, cumulative variance). (B) To interpret MitoSpace’s PCs, the mitochondrial network was quantified with respect to i) fluorescence intensity of MitoTracker and TMRM, and morphometric features of ii) mitochondrial network fragments and iii) the mitochondrial networks. (C) The different principal components (PCs) correlate differently with different features that together show the emergence of a pattern. Traditional semantic directions (axes) such as TMRM intensity axes or punctate-to- networked axes (aspect ratio) are highlighted with a star. (D) Traditional axes of mitochondrial characterization such as i) mitochondrial membrane potential (TMRM), ii) mitochondrial area, and iii) mitochondrial aspect ratio are preserved in MitoSpace. Clusters of morphological responses to individual drugs appear as superpositions of multiple axes. Note the clear patterning in form of a gradient that is present for each depicted mitochondrial feature. MitoSpace integrates all of them into a unified semantically meaningful space. (E) Mitochondria membrane potential axis: Note how the TMRM signal along this axis changes from very high intensity to medium and eventually to low intensity, indicating that the mitochondrial membrane potential is vanishing along this axis. Also note how the mitochondrial morphology changes from highly tubular networks to small punctate structures where the mitochondrial membrane potential is lowest. (F) Mitochondria area axis: Note how the mitochondrial area in the MitoTracker channel changes from large volumes of the cytoplasm covered by mitochondrial density to medium and then low volume ofAttorney Docket No.00015-429WO1 mitochondrial density. Also note that this change is not accompanied by a drastic change in TMRM intensity indicating that the mitochondrial membrane potential is less important in this axis. (G) Mitochondria aspect ratio axis: Note how the mitochondrial morphology changes from a tubular reticulon, to a mixture of tubules and fragments, to a very fragmented network along this axis.

[0014] Figure 4A-F shows self-supervised models generate higher quality latent spaces than supervised models. (A) Self-supervised models correctly place cells A1 and A2, with similar phenotype, together, in spite of them being treated with different drugs. (B) Supervised models overfit to predefined drug labels and incorrectly place A1 and A2 away from each other even though they have the same phenotype. (C) Self-supervised models correctly separate cells B1 and B2, even though they belong to the same drug, since they have different phenotypes, and place them close to other cells with similar respective phenotypes. (D) Supervised models place B1 and B2 close together, i.e. they pack all cells treated with the same drug together even if they respond differently. (E) Variation of the Mitochondria Area over the self-supervised latent space. Note how there is a clear semantic pattern (color gradient). (F) Variation of the Mitochondria Area over the supervised latent space. Note how there is no clear semantic pattern but that rather each cluster groups drastically different phenotypes together.

[0015] Figure 5A-H shows MitoSpace generalizes to different cell types and drugs without explicit training. (A) Cal27 HNSCC cells form tightly packed colonies from compact polygonal cells. (B) HeLa cells form loose assemblies from individual spread-out polygonal cells. (C) Predicted Cal27 mitochondria latent space using a model trained on the same Cal27 cell images using 100% of the training data. (D) Predicted Cal27 mitochondria latent space using a model that has been trained on HeLa cell images using 100% of the training data. Note that the predicted latent spaces trained on Cal27 and HeLa are remarkably similar. (E) Predicted Cal27 mitochondria latent space trained on only 25% of Cal27 training data. I.e. predict 24 mitochondria drug responses in Cal27 cells using a space trained on 6 mitochondria drug responses in Cal27Attorney Docket No.00015-429WO1 cells. Note how similar the predicted locations are between C and E even though the model has only been trained on a quarter of the drug conditions. (F) Predicted Cal27 mitochondria latent space trained on only 25% of HeLa training data. I.e. predict 24 mitochondria drug responses in Cal27 cells using a space trained on 6 mitochondria drug responses in HeLa cells. Note the similarity between E and F and D and F, even though different cells were used for training and only a quarter of that data was used for training. (G) Prediction accuracy if the ground truth drug morphology is contained in the top three predictions given by the model (top-3) when the model is trained on Cal27 data. Cal27 predictions are consistently around ∼80% even if the number of drugs used for training is reduced. HeLa predictions are consistently around 57% even if the number of drugs used for training is reduced. (H) Top-3 Prediction accuracy of Cal27 and HeLa drugs based on the mitochondrial morphology in HeLa cells. A similar pattern is observed as in G. Note that the prediction accuracy falls only by ∼5% when training and prediction are not done on the same cell line.

[0016] Figure 6 shows a flow chart of the disclosure.

[0017] Figure 7 shows MitoSpace use over multiple days and results. The figure represents MitoSpace constructed from data collected over multiple days. The model was trained on data from only one of the five days and then applied to all five days. In the figure, each day is represented by a different color / gray-scale. It is easy to observe that all the scales overlap, forming consistent clusters across all the days. This visual overlap supports the claim that the model is robust to day-to-day variations and effectively captures relevant patterns in the data.

[0018] Figure 8A-C shows 2.5D MitoSpace. The figure illustrates the temporal effects of different drugs in MitoSpace, represented with scale-coding where the transition from red to blue denotes time progression, with red indicating the initial time step and blue representing the final time step. (A) illustrates the temporal effect for Antimycin A, a drug known for its relatively slow action compared to (B) CCCP. The length of the box along the x-axis in MitoSpace clearly reflects this temporal trend. (C) depicts theAttorney Docket No.00015-429WO1 Control group. In MitoSpace, all colors appear intermixed with no discernible clustering based on color. This observation aligns with the control condition, indicating the absence of any drug or temporal effect. This figure highlights how MitoSpace effectively captures and visualizes the temporal dynamics of drug-induced changes in mitochondrial properties over time.

[0019] Figure 9A-B shows Robust Drug Prediction Algorithm. The figure illustrates the robust drug prediction algorithm. (A) displays the reference confusion matrix constructed solely from the training data, while (B) presents the confusion matrix constructed from the testing set against the training data. The lines between the two plots indicate the process of distribution matching performed for all the drugs from the evaluation confusion matrix to the reference confusion matrix. This figure visually demonstrates the methodology of the algorithm, highlighting how distribution matching enhances the robustness and accuracy of drug classification in MitoSpace despite inherent biological and imaging variations. DETAILED DESCRIPTION

[0020] As used herein and in the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a compound” includes a plurality of such compounds and reference to “the subject” includes reference to one or more subjects and so forth.

[0021] Also, the use of “or” means “and / or” unless stated otherwise. Similarly, “comprise,” “comprises,” “comprising” “include,” “includes,” and “including” are interchangeable and not intended to be limiting.

[0022] It is to be further understood that where descriptions of various embodiments use the term “comprising,” those skilled in the art would understand that in some specific instances, an embodiment can be alternatively described using language “consisting essentially of” or “consisting of.”

[0023] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood to one of ordinary skill in the art to which this disclosure belongs.Attorney Docket No.00015-429WO1 Although methods and materials similar or equivalent to those described herein can be used in the practice of the disclosed methods and compositions, the exemplary methods, devices and materials are described herein.

[0024] The publications discussed above and throughout the text are provided solely for their disclosure prior to the filing date of the present application. Nothing herein is to be construed as an admission that the inventors are not entitled to antedate such disclosure by virtue of prior disclosure.

[0025] The term “about” as used herein can allow for a degree of variability in a value or range, for example, within 10%, within 5%, or within 1% of a stated value or of a stated limit of a range. When a range or a list of sequential values is given, unless otherwise specified any value within the range or any value between the given sequential values is also disclosed.

[0026] In representing a range of positions on a structure, the notation “from X to Y” or “X-Y” may be used, wherein X and Y represent numbers. Then unless otherwise specified, this notation is intended to include not only the numbers represented by X and Y themselves, but all the numbered positions that are bounded by X and y. For example, “from 1 to 4” or “1-4” would, unless otherwise specified, be equivalent to 1, 2, 3, and 4.

[0027] The term “subject” refers to an animal, including, but not limited to, a primate (e.g., human, monkey, chimpanzee, gorilla, and the like), rodents (e.g., rats, mice, gerbils, hamsters, ferrets, and the like), lagomorphs, Swine (e.g., pig, miniature pig), equine, canine, feline, and the like. The terms “subject' and “patient” are used interchangeably herein in reference, for example, to a mammalian subject, such as a human patient.

[0028] The term “substantially” as used herein refers to a majority of, or mostly, as in at least about 51%, 60%, 70%, 80%, 90%, 95%, 96%, 97%, 98%, 99%, 99.5%, 99.9%, 99.99%, or at least about 99.999% or more.

[0029] Classic modeling techniques, e.g. hierarchical clustering, have in recent years been replaced with sophisticated deep learning methods in phenotypic drug development. Specifically,Attorney Docket No.00015-429WO1 convolutional neural networks (CNNs), have demonstrated superior classification performance through enhanced extraction of image- intrinsic features. However, at the core of all of these approaches remains human supervision. Human annotation of training data and creation and curation of feature lists are used to train supervised machine learning algorithms.

[0030] Advances in self-supervised deep learning have recently allowed the modeling of complex data distributions without explicit human input or the a-priori selection of features, for example for natural language processing29,30 and images31,32,33. Such methods have been applied before in the medical domain34, for example for protein structure prediction35, medical imaging36, and microscopy37

[0031] Imaging-based high content screening aims to identify substances that modulate cellular phenotypes. Traditional approaches screen compounds for their ability to shift disease phenotypes toward healthy phenotypes, but these end point-based screens lack an atlas-like mapping between phenotype and cell state that covers the full spectrum of possible phenotypic responses. To address this, the disclosure provides a method referred to herein as “MitoSpace”: a mitochondrial phenotypic atlas that leverages self-supervised deep learning to create a semantically meaningful latent space from images without relying on any data labels for training. The approach employs a dataset of (e.g., ~120,000) microscopy images of Cal27 cells treated with 25 drugs affecting mitochondria, but can be generalized to any cell-type, cell organelle, or drug library with no changes to the methodology. MitoSpace enhances the understanding of the range of mitochondrial phenotypes induced by pharmacological interventions. It is shown that i) self-supervised learning can automatically uncover the semantic landscape of drug induced latent mitochondrial phenotypes and can map individual cells to the correct functional area of the drug they are treated with, ii) the traditional classification of mitochondrial morphology along a fragmented to fused axis is more complex than previously thought, with additional axes being identified, and iii) latent spaces trained in a self-supervised manner are superior to those trained with supervised models, and generalize to other cell types and drug conditions without explicitAttorney Docket No.00015-429WO1 training on those cell types or drug conditions. Some applications of MitoSpace include creating mitochondrial biomarkers for drug discovery and determining the effects of unknown drugs and diseases for diagnostic purposes.

[0032] In a recent biological study, self-supervised deep learning was successfully applied to profile subcellular protein localization in microscopy data using an autoencoders and a proxy classification task involving protein identifier labels (Kobayashi et al. 2022).

[0033] The disclosure presents an approach to phenotypic screening that is based on self-supervised deep learning. Specifically, the approach is not based on traditional cell profiling techniques but is rather based on ‘cell perceiving’ techniques that do not require the pre-definition of features nor the labeling of the data, i.e., there is no human or other input telling the model which image is associated with which drug treatment. The model learns the inherent similarity and dissimilarity between images using relative associations between data as opposed to absolute given labels. Also note that in cell perceiving, the self-supervised neural networks do not require feature selection nor feature extraction steps. The result is that human bias in feature conception, feature selection, and data annotation is eliminated. In this approach, the models detect these features themselves internally. As such one can map the true latent factors of variation in the data without human intervention.

[0034] The mitochondrial network stands out among the different organelles of a cell as one of the most diverse and responsive to cellular state and cellular health. Mitochondria are small organelles that provide up to 90% of a cell’s energy and are implicated in a vast array of diseases such as metabolic disorders, developmental disabilities, epilepsy, neurodegenerative disease, cancer, and aging. For that reason, mitochondrial morphology has been the target of high content phenotypic screening in cancer, neurodevelopment, metabolic diseases, and neurodegeneration. These high-content screening studies are all of the ‘profiling’ type and follow the pattern that 1) hit parameters in the screen are defined, e.g., increase mitochondrial length by > 2.5 z-scoreAttorney Docket No.00015-429WO1 (Kobayashi et al. 2022), 2) control cells and disease cells are exposed to a drug compound library, 3) imaged using fluorescence microscopy, and 4) the resulting images analyzed using software to quantify the hit desired parameters.

[0035] Given that the mitochondrial morphology is exceedingly complex, it was hypothesized that manual definition of features and hit parameters was masking a multitude of factors that influence mitochondrial phenotype in response to drug candidates. Thus, important features would have remained so far unreported and unquantified.

[0036] As described herein, a contrastive learning approach and bootstrapping-based self-supervised deep learning approach was used to create image-based semantically meaningful latent spaces which can act as a map for understanding mitochondrial phenotypes and for modeling the effect of pharmacological interventions. The method requires no human labeling or feature definition. By not having human-defined classification labels, the approach is robust against overfitting and can correctly delineate generalizable factors of variation present in the data.

[0037] By focusing on drug-induced mitochondrial morphology and function changes the method allows one to create an interpretable atlas of the accessible function / morphology space of the cell that links to cellular state and thereby drug function. The results show how the traditional dimension of mitochondrial phenotypic change, i.e. the change from fragmented to tubular, is augmented by additional dimensions that have so far remained hidden in previous manual or supervised approaches. The disclosure shows how MitoSpace (i) captures meaningful functional, and morphological information with no human supervision, and (ii) generalizes across cell-types and to drugs unseen during training, and (iii) is semantically superior to supervised methods. MitoSpace provides a powerful method to characterize and contextualize the effects of disease, novel drugs and other interventions on mitochondria.

[0038] The disclosure provides a novel approach using self- supervised deep learning to create a one-to-one mapping of mitochondrial form and function to drug effect, referred to herein as MitoSpace. In addition, the disclosure provides a newAttorney Docket No.00015-429WO1 perspective, called ‘cell perceiving’, as opposed to traditional ‘cell profiling’. Cell perceiving is characterized by automated extraction and contextualization of phenotypic information from microscopic images without requiring human-defined labels or feature lists. The disclosure thus allows for the creation of a phenotypic latent space without requiring any additional information beyond the images themselves. This latent space acts as a phenotypic atlas that reliably clusters cells treated with functionally similar drugs into the same areas and captures all mitochondrial phenotypic features present in the data without any explicit human-defined feature selection steps. This includes mitochondrial features previously defined in literature but may also include other, more complex features inherent in the data that cannot be easily characterized manually.

[0039] Limiting the set of features to those defined by human experts may not capture all dimensions of phenotypic variation present in the data. The disclosure demonstrates this by showing that a model based on human-defined features captured using Mitochondria Analyzer does not capture the semantic meaning of the combined mitochondria form and function space. In fact, biasing a model using manual drug labels may hide true phenotypes expressed by individual cells and may lead to overfitting.

[0040] The disclosure also shows that phenotypic latent spaces obtained using self-supervised models are biologically meaningful. Specifically, this was validated by (i) the latent spaces reliably capture and delineate previously defined features related to mitochondrial phenotype in a semantically meaningful way, with cells having similar feature values being mapped close together and vice versa (i.e., the latent space is locally meaningful), and (ii) cells treated with functionally similar drugs are mapped to the same area (i.e. the space is globally meaningful).

[0041] The overall location of drug-induced morphology clusters in MitoSpace makes sense in that regard where, for example, membrane uncouplers and ionophores are both mapped in a similar region of MitoSpace. Locations that are at first glance unexpected turn out to be supported by the literature. For example, MitoQ canAttorney Docket No.00015-429WO1 cause both mitochondrial swelling and depolarization in some cell types.

[0042] Such biological validity opens up avenues for defining computational biomarkers that can characterize all phenotypes expressed by a cell or organelle without manually defining them. For example, one can choose the principal components of the resulting latent space as the major directions of variation in the data, and represent each cell by the values of the first N principal components at that point.

[0043] The method of the disclosure is independent of cell type, cell organelle, and drug library used and can be extended without any human annotation or effort except for sample preparation and image acquisition. This was validated by showing how the semantic structure and classification performance of the obtained latent space remains consistent when trained with a different cell type or when withholding drugs during the training phase in order to simulate unseen drugs during the test phase.

[0044] Given its generalizability, MitoSpace and similar latent spaces may be useful to determine the likely function of unknown drugs based on their effect on mitochondrial phenotype, as well as to identify the functional effect of diseases on mitochondria. They could be used for in vitro pharmacology and as endpoints in clinical trials.

[0045] Beyond drug discovery and disease profiling, MitoSpace, and more broadly ‘cell perceiving’, offers exciting opportunities for basic research. The distinct mitochondrial morphologies that appear in response to specific drug effects likely result from unique signaling pathways that optimize cellular health under stress conditions. MitoSpace provides a window into these pathways and opens up avenues for further research.

[0046] The MitoSpace model is exemplified herein using 25 drugs as the basis for its applications. However, expanding the scope of drugs will significantly enhance the potential of the tool. By testing a broader range of drugs, one can uncover a wider array of effects on organelle morphology and function, leading to novel insights into drug mechanisms and potential therapeutic targets.Attorney Docket No.00015-429WO1

[0047] In addition, the work disclosed herein is exemplified with a limited selection of cell lines. To improve the robustness and generalizability of the findings, a larger and more diverse set of cell lines can be incorporated into the model. Different cell types can react differently to the same drugs, making it crucial to understand these variable responses across cell lines.

[0048] While the application of MitoSpace to mitochondria provides a strong proof of concept, other organelles can benefit from this technology (e.g., cell membranes, nuclei, golgi etc.). Certain organelles like the endoplasmic reticulum and the cytoskeleton present unique complexities in their morphology that pose similar challenges for classical profiling methods as mitochondria. By extending MitoSpace to these organelles, one could provide more in-depth and accurate phenotypical quantification, helping researchers better understand their functions and interactions within the cell.

[0049] In the examples, provided herein MitoSpace was used to analyze two-dimensional images, but organelles are inherently three-dimensional structures that change over the fourth dimension of time. Therefore, integrating three-dimensional imaging and time- lapse movies into MitoSpace can significantly increase its predictive power. This expansion would enable one to capture the dynamic nature of organelles and better understand how they behave and interact under perturbations on cellular states.

[0050] The disclosure can be carried out using a computer machine or apparatus upon which one or more embodiments (e.g., the foregoing discussed methodologies) can be implemented (e.g., run). Examples of machines can include logic, one or more components, circuits (e.g., modules), or mechanisms. Circuits are tangible entities configured to perform certain operations. In an example, circuits can be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner. In an example, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware processors (processors) can be configured by software (e.g., instructions, an application portion, or an application) as a circuit that operates to perform certain operations as describedAttorney Docket No.00015-429WO1 herein. In an example, the software can reside (1) on a non- transitory machine readable medium or (2) in a transmission signal. In an example, the software, when executed by the underlying hardware of the circuit, causes the circuit to perform the certain operations.

[0051] In an example, a circuit can be implemented mechanically or electronically. For example, a circuit can comprise dedicated circuitry or logic that is specifically configured to perform one or more techniques such as discussed above, such as including a special-purpose processor, a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). In an example, a circuit can comprise programmable logic (e.g., circuitry, as encompassed within a general-purpose processor or other programmable processor) that can be temporarily configured (e.g., by software) to perform the certain operations. It will be appreciated that the decision to implement a circuit mechanically (e.g., in dedicated and permanently configured circuitry), or in temporarily configured circuitry (e.g., configured by software) can be driven by cost and time considerations.

[0052] The various operations of method examples described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented circuits that operate to perform one or more operations or functions. In an example, the circuits referred to herein can comprise processor-implemented circuits.

[0053] Similarly, the methods described herein can be at least partially processor-implemented. For example, at least some of the operations of a method can be performed by one or processors or processor-implemented circuits. The performance of certain of the operations can be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In an example, the processor or processors can be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other examples the processors can be distributed across a number of locations.Attorney Docket No.00015-429WO1

[0054] The one or more processors can also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., Application Program Interfaces (APIs)).

[0055] Example embodiments (e.g., apparatus, systems, or methods) can be implemented in digital electronic circuitry, in computer hardware, in firmware, in software, or in any combination thereof. Example embodiments can be implemented using a computer program product (e.g., a computer program, tangibly embodied in an information carrier or in a machine readable medium, for execution by, or to control the operation of, data processing apparatus such as a programmable processor, a computer, or multiple computers).

[0056] A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a software module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

[0057] In an example, operations can be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Examples of method operations can also be performed by, and example apparatus can be implemented as, special purpose logic circuitry (e.g., a field programmable gate array (FPGA) or an application- specific integrated circuit (ASIC)).

[0058] The computing system can include clients and servers. A client and server are generally remote from each other and generally interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client- server relationship to each other. In embodiments deploying aAttorney Docket No.00015-429WO1 programmable computing system, it will be appreciated that both hardware and software architectures require consideration. Specifically, it will be appreciated that the choice of whether to implement certain functionality in permanently configured hardware (e.g., an ASIC), in temporarily configured hardware (e.g., a combination of software and a programmable processor), or a combination of permanently and temporarily configured hardware can be a design choice. Below are set out hardware and software architectures that can be deployed in example embodiments.

[0059] In an example, the computer machine / apparatus can operate as a standalone device or the machine / apparatus can be connected (e.g., networked) to other machines.

[0060] In a networked deployment, the machine / apparatus can operate in the capacity of either a server or a client machine in server-client network environments. In an example, machine / apparatus can act as a peer machine in peer-to- peer (or other distributed) network environments. The machine / apparatus can be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) specifying actions to be taken (e.g., performed) by the machine / apparatus.

[0061] A typical computer machine / apparatus can include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU) or both), a main memory and a static memory , some or all of which can communicate with each other via a bus. The computer machine / apparatus can further include a display unit, an alphanumeric input device (e.g., a keyboard), and a user interface (UI) navigation device (e.g., a mouse).

[0062] The storage device can include a machine readable medium on which is stored one or more sets of data structures or instructions (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. The instructions can also reside, completely or at least partially, within the main memory, within static memory, or withinAttorney Docket No.00015-429WO1 the processor during execution thereof by the computer machine / apparatus.

[0063] The term “machine readable medium” can include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that configured to store the one or more instructions or data. The term “machine readable medium” can also be taken to include any tangible medium that is capable of storing, encoding, or carrying instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure or that is capable of storing, encoding or carrying data structures utilized by or associated with such instructions. The term “machine readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media. Specific examples of machine readable media can include non-volatile memory, including, by way of example, semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0064] The instructions can further be transmitted or received over a communications network using a transmission medium via the network interface device utilizing any one of a number of transfer protocols (e.g., frame relay, IP, TCP, UDP, HTTP, etc.). Example communication networks can include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., IEEE 802.16 standards family known as Wi-Fi®, IEEE 802.16 standards family known as WiMax®), peer-to-peer (P2P) networks, among others. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.Attorney Docket No.00015-429WO1

[0065] The term “computer-readable medium” (or “machine- readable medium”) as used herein is an extensible term that refers to any medium or any memory, that participates in providing instructions to a processor for execution, or any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). Such a medium may store computer- executable instructions to be executed by a processing element and / or control logic, and data which is manipulated by a processing element and / or control logic, and may take many forms, including but not limited to, non-volatile medium, volatile medium, and transmission medium. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise a bus. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infrared data communications, or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch-cards, paper- tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

[0066] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to processor for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer machine / apparatus can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on a bus. The bus carries the data to main memory, from which processor retrieves and executes the instructions. The instructions received by main memory may optionally be stored on storage device either before or after execution by processor.Attorney Docket No.00015-429WO1

[0067] The methods and computer implemented methods of the disclosure can be implemented on the computers, systems and architecture described herein. The methods may be implemented by a computer program or programs stored on a computer readable medium such that the program causes the computer to carry out the methods and steps set forth above.

[0068] It is to be understood that while the disclosure has been described in conjunction with specific embodiments thereof, that the foregoing description as well as the examples which follow are intended to illustrate and not limit the scope of the disclosure. Other aspects, advantages and modifications within the scope of the disclosure will be apparent to those skilled in the art to which the disclosure. EXAMPLES

[0069] Cell culture. Cal27 and HeLa cells were cultured in 89% DMEM, high glucose, GlutaMAX (Gibco, #10566016), 10% FBS (GenClone, #25-550), 1% Penicillin-Streptomycin (Gibco, #15140122). Cells were washed with DPBS (Gibco, #14190144) and detached with 0.05% Trypsin-EDTA (Gibco, #25300054) before passaging onto imaging plates.

[0070] Live cell imaging. Cells were passaged onto 96-well #1.5 glass bottom plates (Cellvis, #P96-1.5H-N) and grown for one day to reach a final density of 60-80%. After drug treatment the cells were stained with 50 nM TMRM (Invitrogen, #T668), 100 nM MitoTracker Green FM (Invitrogen, #M7514), and 1 drop / mL NucBlue Live (Invitrogen, #R37605). After 20 min staining, the medium was aspirated and washed with DPBS (Gibco, #14190144). The cells were then incubated in phenol red-free growth media.

[0071] During imaging, cells were kept under 5% CO and 37 °C. Spinning disk confocal microscope was used. The microscope was equipped with a spinning disk scanner (X-Light V3, CrestOptics), and an ORCA-Fusion digital CMOS camera (Hamamtsu). A water- immersion Plan Apo VC 60x WI NA 1.2 objective was used. 3 laser lines (405 nm, 477 nm, 546 nm) were generated with a Lumencor Celesta light engine. To automate the data acquisition for multiple conditions, the JOBS high-content screening module was used on NIS Element software. For each well, the optimal plane of focus wasAttorney Docket No.00015-429WO1 first determined by a two-pass autofocus with a range of 80 microns. Inside each well, the software generated a random path consisting of 25 field of views (FOVs). For each FOV visited, another single-pass autofocus was performed with a range of 20 microns to account for focus drift due to cell and plate variations in z-direction. Three channels were then captured simultaneously using triggered acquisition. After all 25 FOVs were imaged, the next well was automatically visited and the protocol was repeated until all 25 drugs + 1 control wells were imaged.

[0072] Drug Treatments. Each drug was incubated for 30 minutes prior to imaging. Drugs were present during staining and imaging. All 25 drugs were treated and imaged during one experiment, so the total incubation time ranged from 1-3 hours.

[0073] Data Preprocessing. We start with 25 FOVs for each cell type with three-channels (NucBlue Live, MitoTracker Green, TMRM). Each FOV contains a plurality of cells. The patch sizes vary for different cell types depending on the cell size. For Cal27256 x 256 patches were chosen while for HeLa cells a patch sizes of 384 x 384 was chosen to reflect the larger cell size. Note that the models are generalizable across cell size, e.g., they can beAttorney Docket No.00015-429WO1 trained on different image sizes and perform inference on different cell sizes.

[0074] We first compute channel wise 98th percentile values over the dataset and threshold each image channel-wise by these values to get rid of outlier high intensity values. Next, we normalized the images patch-wise, and apply patch-wise midtone- balancing or gamma correction on each image to enhance contrast and suppress noise. To get patches centered at a single cell nucleus, we apply thresholding on the nucleus channel followed by morphological opening. We then apply connected component search to identify each cell nucleus. The centroid of each nucleus was used to take a fixed size 3-channel patch around that to get the final patches centered at a single nucleus. The position was randomly jittered for each patch by a few pixels to introduce natural variability into the position of the cell in the image. An average of 80 cell patches were extracted from each FOV, amounting to a dataset of anywhere between 30,000 to 80,000 cell patches for different cell types.

[0075] Self-Supervised Deep Learning. A self-supervised framework was used for extracting the embeddings, where each embedding is the representation of an image in a lower dimensional space. Specifically, SimCLR and BYOL were used, two methods which work on the principle of comparing embeddings of augmented views of the same sample.

[0076] SimCLR is a contrastive learning method that works on the principle that augmented patches of the same images should be close together in the embedding space. The deep learning model is a ResNet18 backbone encoder, followed by a multi-linear perceptron (MLP) head, which together reduce each multi-channel image down to a vector of 128 dimensions. It was determined that ResNet18 backbones are enough to model the data, but other backbones can also be used, for example, ResNet50 for larger datasets, as well as larger embedding dimensions. For self-supervised training, we feed in stochastically augmented patches of single cells to the model (the details of the augmentations are described in the next section). The model is trained to match the embeddings of augmented versions of the same patches, while pushing semantically differentAttorney Docket No.00015-429WO1 patches far apart. A contrastive infoNCE loss63 is used for this purpose, with a temperature value fixed at 0.01. We train the model with a learning rate of 0.0003, a decay rate of 0.0001, and a batch size of 256. By learning to recognize similarities and dissimilarities between various augmented images, the model delineates the inherent factors of variation in the data without requiring any labels.

[0077] BYOL is similar to SimCLR in that it also compares augmented views of the same samples. However, unlike SimCLR, it uses two networks, the online and target network, and tries to directly align the latent space embeddings of the two views projected using the two networks via a simple mean-squared error loss. We use the same ResNet18 backbone for the online and target networks, followed by an MLP projection head that reduces dimensionality down to 128 dimensions. Unlike SimCLR, we use a mean-squared-error (MSE) loss to directly align the embeddings of the online and target. The optimizer hyperparameters like learning rate and batch size are kept the same as the SimCLR models.

[0078] For the autoencoder model, we use a hierarchical vector quantized variational autoencoder64 with a classifier head as in Kobayashi et al.38. The classifier head predicts the drug label that each cell is treated with. As opposed to proxy labels, here we train the model with the actual label itself. The model has two encoders and decoders each with 4 residual blocks, with a top embedding dimension of 32 and a bottom embedding dimension of 64. We use a dictionary size of 512 embeddings. We use MSE as the reconstruction loss and categorical cross-entropy as the classification loss. The learning rate, weight decay, and batch size are kept the same as the other methods.

[0079] The supervised learning model is a ResNet18 model trained with a categorical cross-entropy loss function. Once again, the optimizer parameters are kept the same.

[0080] All models use an Adam optimizer and are trained on graphical processing units (GPU). The end result of the deep learning models is a latent space that maps each cell image to a lower dimensional vector. Each vector captures the relevant phenotypic properties of that cell image. The latent space acts asAttorney Docket No.00015-429WO1 a search space or a map, and can be reduced to a lower dimensionality for easier visualization.

[0081] The data augmentations are defined in the next section, and are critical to implicitly teach the model to ignore spurious factors of variation. For example, to teach the deep learning model that cell orientation is not a relevant phenotypic property, we feed in pairs of rotated cell images; the model is trained to reduce the distance between them in embedding space..

[0082] Data Augmentation. We design the augmentations to make the model specifically robust to experimental variation such as noise or cell shape. All augmentations are stochastic and are sequentially and randomly added to each individual image. - To make the model robust to imaging noise, we add Gaussian noise of the same order as seen in the original images. Noise is added randomly to 50% of the patches with varying mean and a standard deviation which is fixed to be a particular fraction of the maximum value in each image. The mean of the additive Gaussian noise is drawn from a normal distribution centered at the mean background value of patches over the entire training set, while the variance is set at 2% to 10% of the maximum value of each patch, channel wise. - To make the model robust to slight variations in intensity caused by factors like imaging variation and dye uptake in cells, we use additive brightness augmentation, where we randomly add or subtract a value drawn from a Gaussian distribution with a fixed mean and variance to each image. - To make the model robust to cell shape, orientation and other spurious structural factors, we use random horizontal and vertical flips, random crops, random rotation, and patch- reordering. (In patch reordering, we split the image into 4 patches and randomly jumble the patches). Once again, these augmentations are only applied to a portion of images chosen randomly.

[0083] Visualizing and Understanding the Phenotypic Latent Space. The feature representations hence created from the self- supervised method described above are 128 dimensional vectors. For visualization purposes, we reduce the dimensionality of thisAttorney Docket No.00015-429WO1 feature representation to 3 dimensions by using UMAP (McInnes, Healy, and Melville 2020). UMAP (Uniform Manifold Approximation and Projection) is a dimensionality reduction algorithm which uses a graph based approach to represent data and transform high dimensional data into lower dimensions.

[0084] The data points in UMAP are denoted by nodes in the graph and the distances or similarities between these nodes is depicted by the edges. It tries to reduce the dimensionality of the space while preserving the global structure of original data.

[0085] The UMAP algorithm is computationally efficient and can handle large datasets with millions of data points. It has been widely used in data visualization, clustering, and machine learning applications.

[0086] For gaining a semantic understanding of the phenotypic latent space, we perform Principal Component Analysis to delineate the major axes of variation. We find that ~10 principal components are enough to explain ~75% of the variance in the embedding space. We also compute the Pearson correlation coefficient of each of these axes with cell-level mitochondrial features extracted using Mitochondria Analyzer.

[0087] Generating an Image Dataset that Captures the Full Variability of Drug-Induced Phenotypic Responses of Mitochondria. Mitochondria form a three dimensional network in the cell that changes over time through fission, fusion, and motility. Consequently, it is established that the mitochondrial network naturally displays a wide range of morphologies in control conditions (Figure 1A). Beyond this cell to cell variability, it is established that mitochondrial morphology responds to stress or a metabolic switch to glycolysis with a punctate / fragmented morphology (Figure 1B, left) and to stress relief and a metabolic switch to oxidative phosphorylation with an elongated or hyperfused morphology (Figure 1B, right). However, there is a wide variety of additional reported mitochondrial morphology phenotypes that cannot easily be placed on the punctate-to-networked axis (see examples in Figure 1C). Morphologies like these have been widely observed but have traditionally been classified as outliers. It was hypothesized that these mitochondrial morphologies belong to additional,Attorney Docket No.00015-429WO1 currently uncharacterized dimensions of mitochondrial morphology responses that have so far remained hidden because of insufficient sampling of these states and manual analysis methodologies that predefined the goal observables to be along the punctate-to- networked axis.

[0088] To test the hypothesis, we set out to create an imaging dataset that contains the full variability of drug-induced mitochondria phenotype responses in order to contextualize them using self-supervised deep learning.

[0089] Two cell lines were used, Cal27 head and neck squamous carcinoma cells58 and HeLa cervical cancer cells, as model systems. Eight classes of mitochondria drugs were employed to induce the spectrum of mitochondrial morphology and function responses (Figure 1D): 1) fission inhibitors (Mdivi-1, P110); 2) fusion inhibitors (MYLS22, MFI8); 3) agents affecting mitochondrial motility (Nocodazole for microtubules, Cytochalastin D for actin filaments, and Ciliobrevin D for dynein) and other cellular organelles that interact with mitochondria (Thapsigargin for ER); 4) agents that affect mitochondrial DNA (Mitomycin C and Cisplatin); 5) agents that dissipate the mitochondrial membrane potential (CCCP and DNP); 6) agents that act as ionophores (Valinomycin and Nigericin); 7) agents that affect the mitochondrial electron transport chain complexes (Rotenone for Complex I, Lonidamine for Complex II, Antimycin A for Complex III, Azide for Complex IV and Oligomycin for Complex V); and agents that affect the production of reactive oxygen species (ROS) (Paraquat, TBHP, and Cyclosporine A to 8) induce ROS, MitoQ, Tiron and Resveratrol to 9) buffer ROS).

[0090] As described above, an automated confocal microscope was used to fluorescently image the mitochondrial morphology using Mitotracker Green and the mitochondrial function using the mitochondrial membrane potential marker TMRM. To facilitate automated cell segmentation cell nuclei were labeled using DAPI (see Figure 1E). For each cell line, ∼2,000 individual cells were imaged for each drug condition, resulting in a total of 2 cell lines x 25 drugs x 2,000 cells = 100,000 individual cell images. This dataset was used to generate the self-supervised deep learning-based mitochondria phenotype space.Attorney Docket No.00015-429WO1

[0091] Self-supervised Deep Learning to Generate the Latent Space of Drug-Induced Mitochondrial Phenotypes. Self-supervised contrastive learning was used to generate the latent space of drug- induced mitochondrial phenotypes, without requiring any human labeling. The ‘latent space’ in this context refers to a lower dimensional vector space (for example, lower than the 256×256 = 65,536 dimensional space of the images) that captures only the important factors of variation in the data.

[0092] Two self-supervised deep learning architectures were compared, SimCLR31, a contrastive learning approach, and BYOL32, a related method leveraging bootstrapping, to learn semantically meaningful latent representations from the image dataset while ignoring spurious factors. Briefly, 256×256 images of single cells of both mitochondrial morphology (MitoTracker) and mitochondrial membrane potential (TMRM) were passed through a data augmentation step. The data augmentations was designed to explicitly make the model robust to potential spurious imaging and experimental factors such as noise and cell shape. The data augmentation helps the network learn invariance to any spurious features which are not related to morphology / function, for example rotation of mitochondria, cell shape, translation, imaging noise, or brightness variations in the images. In the second critical step, the augmented images enter two separate encoder pipelines that are compared to each other so that contrastive loss between the images is minimized. The two heads share weights and are encouraged to learn consistent representations of the same cell image (and hence the same phenotype), irrespective of changes to the spurious factors mentioned above. The resulting embedding is the low dimensional phenotypic feature space that encodes and contextualizes mitochondrial morphology and function. As a result of this training, the model, MitoSpace, naturally learns the inherent phenotypic similarities and differences between different cells, grouping similar phenotypes together (Figure 1E). The resulting lower dimensional latent space is an atlas of drug- induced mitochondrial phenotypes which can act as a map for understanding and relating mitochondrial morphology to drug function.Attorney Docket No.00015-429WO1

[0093] MitoSpace Delineates Phenotypic Activity from Images. The phenotypic atlas learnt by MitoSpace for Cal27 cells is shown in Figure 2. The method is learning phenotypic similarity, i.e., locating similar phenotypes close together in space. Being at one point in the map and moving gradually into one direction one would expect the mitochondrial morphology to gradually change and morph into a new phenotype. UMAP was used to reduce the 128 dimensional latent space learnt by the model down to 3 dimensions for visualization purposes. Note that the results are drawn from the 128 dimensional space and that UMAP is used for easier visualization. Each point in this latent space represents an image of a single control or drug-treated cell. The position of the point and its relation to all the other points (images) reflects the model’s understanding of the morphological (MitoTracker) and functional (TMRM) phenotypes expressed in that cell image in relation to the other points. Each point is colored by the drug that the cell was treated with. Colors are chosen for each drug based on the drug’s reported mechanism of action (Figure 2A). The disclosure shows that MitoSpace reliably maps the semantic landscape of drug-induced latent mitochondrial phenotypes, grouping drugs with similar effects on mitochondria together. This can be observed based on the color coding in the latent space, where drugs that have similar mechanisms of action are colored by a similar color.

[0094] The latent phenotypes were then analyzed more closely and found that the atlas self-organizes to capture a wide variety of phenotypes inherent in the data. The different drug conditions are found to be placed radiating into different directions from a common center in the space where the control condition is found. This indicates that these drugs have resulted in quantifiably different mitochondrial phenotypes that are distinct from each other. Figure 2B depicts MitoSpace in the form of a density plot to aid visualization surrounded by representative cell images taken from the center of each drug cluster. The center or mode of each cluster is where the most characteristic images from each drug condition are located. The data show that each image cluster shows clear differences in mitochondrial morphology. The controlAttorney Docket No.00015-429WO1 condition images are found clustered at the center of the latent space and show the baseline mitochondrial morphology and membrane potential. In drug conditions that affect the mitochondrial membrane potential through uncoupling (CCCP, DNP) and through acting as ionophores (Valinomycin, Nigericin), mitochondria show punctate morphology and low membrane potential (Fig. 2B left). Fission (Mdivi-1, P110) and fusion inhibitors (MYLS22, MFI8) cluster close to the control condition in the center of the space and show more and less punctate morphology, respectively, and close to control membrane potential (Fig. 2B bottom). Images taken under mitochondrial motility affecting drugs (Nocodazole, Cytochalastin D, Ciliobrevin D) and other organelle affecting drugs (Thapsigargin) cluster to the right side of the space, showing a more filamentous mitochondrial structure and high mitochondrial membrane potential (Fig. 2B right). Images taken under the effect of agents that affect the electron transport chain complexes (Rotenone, Lonidamine, Antimycin A, Azide, Oligomycin) cluster to the top right of the space (Fig. 2B top right), and images taken under mitochondrial DNA affecting drugs cluster to the top left of the space (Fig. 2B top left). Agents that affect the production of reactive oxygen species (Paraquat, TBHP, and Cyclosporine A, MitoQ, Tiron, Resveratrol) were found to be clustered in both ends of the space. Overall, the model clustered similar morphologies and similar mitochondrial function conditions in similar regions of the space. This means that cells are mapped in the direction of the respective phenotypes in varying degrees, correlating to how strongly they are affected by a drug intervention.

[0095] Not being constrained by human labels, MitoSpace focuses on actual phenotypic responses of cells. This means that MitoSpace does not overfit to try and conform to predefined labels. This is important because even cells treated with the same drug may show a range of phenotypes, with some cells responding mildly and other cells responding strongly to the same drug. Not enforcing the same label on such widely responding cells allowed one to capture the true response of each condition, resulting in a multidimensional continuum of phenotypes where each cell image is mapped to its correct phenotypic position irrespective of what drug it is treatedAttorney Docket No.00015-429WO1 with. The results also show supervised models trained on the same data, constrained by predefined labels, suffer from overfitting and are hence unable to create semantically meaningful latent spaces. Such automatic feature selection, extraction, and mapping are the keystones of the ‘cell perceiving’ approach as compared to ‘cell profiling’.

[0096] MitoSpace Introduces Principal Semantic Directions that Combine Form and Function of Mitochondria. MitoSpace captures various phenotypic features inherent in the data, without having to manually define them. It was hypothesized that there would be mitochondrial axes beyond the established punctate-to-networked axis, and self-supervision would allow this to be captured. It is important to recognize that the deep learning model may be capturing higher order features that may not be easily defined by simple features previously defined in the literature. This analysis is thus meant to be representative, not exhaustive.

[0097] A principal component analysis (PCA) was conducted in the latent space generated by the MitoSpace model in order to discover phenotypic axes and to gain deeper insights into the connections and clusters that MitoSpace found in mitochondrial phenotype space (Figure 3). The data show that > 70% of the variance in the latent space was accounted for by the first 10 principal components (PCs) and > 55% by the first 5 PCs (Figure 3A). The correlation of these PCs was analyzed with independently computed mitochondrial features extracted from each cell image. Specifically, measured correlation with (i) the fluorescence intensity of MitoTracker and TMRM, and (ii) ten morphology and network features extracted using Mitochondria Analyzer (Figure 3B). Specifically, total branch length per mitochondria was quantified, mean form factor, number of branch junctions, number of branch end- points per mitochondrion, total area of the mitochondrial network per cell, total mitochondrial branch length, number of mitochondrial network branches, total perimeter of the mitochondrial network, number of mitochondria per cell, and mean aspect ratio of mitochondria. Figure 3C shows the Pearson correlation coefficient (PCC) of these independently extracted features with the PCs extracted from the learned latent space.Attorney Docket No.00015-429WO1 MitoSpace captures the variation in these features, in spite of there being no explicit feature selection step anywhere in the pipeline. This is reflected in the high and statistically significant correlation coefficients in Figure 3C. For the first PC, three features can be highlighted (Figure 3C, star) that are closest to the reported axes along which mitochondrial phenotype and function are arranged: a) TMRM fluorescence intensity which has been reported to correlate with mitochondrial membrane potential, b) mitochondrial aspect ratio, a feature that closely tracks the punctate-to-networked axes of mitochondrial morphology, and c) mitochondrial area. For the first PC, TMRM is not correlated, aspect ratio is somewhat negatively correlated, and mitochondrial area is strongly negatively correlated. This pattern however is different for each PC and for some, e.g., PC-3, it is reversed with TMRM being strongly positively correlated and aspect ratio exhibiting a correlation close to zero. It was hypothesized that while the individual PCs might show one or the other pattern or correlation, the full MitoSpace model which integrates the entire variance of phenotypes available in the dataset would exhibit recognizable patterns. The intensity of each feature was plotted for each image as a color gradient on top of the MitoSpace- generated latent space and found that such patterns are indeed recognizable (Figure 3D). TMRM shows a clear trend from low TMRM intensity at the bottom left end of the latent space to high intensity at the right end of the latent space (Figure 3D, left). A similarly clear patterning exists for mitochondrial area from low mitochondrial area (top) to high mitochondrial area (bottom) of the latent space (Figure 3D, middle), and for mitochondrial aspect ratio (low aspect ratio in the bottom left to high aspect ratio in the top right, Fig. 3D, right). These continuous feature variations in the latent space can be thought of as novel axes that encode simultaneous descriptions of mitochondrial form and function that augment traditional ‘punctate-to-networked’ axes into a much richer description of mitochondrial phenotypes. To probe these axes data was sampled along their gradients to show representative images (Figure 3E). Indeed, the TMRM axis embedded in the latent space shows not only the feature itself (continuously decreasing TMRMAttorney Docket No.00015-429WO1 intensity, red signal, Fig. 3E, bottom) but simultaneously shows a mitochondria morphology continuum from more punctate to networked morphologies. In contrast, sampling along the mitochondrial area axis shows relatively constant TMRM intensity and network morphology while it is mainly the mitochondrial area that changes (Fig. 3F). In addition, sampling along the mitochondrial aspect ratio feature axis in the latent space shows the expected reticular to punctate morphology gradient while TMRM signal remains constant (Fig. 3G).

[0098] Together, these results show how MitoSpace directly captures and quantifies human-interpretable and previously validated features related to mitochondrial morphology and function, which lends validity to its semantic structure. Within this semantic structure, each drug class has its own location where the combination of mitochondrial morphology features and function (TMRM) act as a unique identifier that would allow the characterization of an acting drug from images alone.

[0099] MitoSpace Accurately Captures the Semantic Meaning of Drug-Induced Mitochondria Phenotypes. Next the accuracy of MitoSpace in predicting drugs from images alone was analyzed. Since each cell may react differently even to the same pharmacological intervention, drug labels and drug function labels may not always be truly representative of the phenotype expressed in each cell image. Each cell may express multiple phenotypes in various proportions in spite of the drug it is treated with, and hence the true latent space is more akin to a mixture model over latent phenotypes rather than points with hard drug label assignments. Self-supervised models allow one to capture this latent phenotype of each cell, independent of which drug it is treated with. This is a factor that allows the construction of biologically and semantically meaningful latent spaces. However, it also makes the models hard to evaluate. Hard classification accuracy based on drug labels as a metric does not fully capture the semantic validity of the generated latent space. Cognizant of this, a top-3 accuracy was used as a proxy metric for evaluating performance of the latent space taking into account its mixed nature. Above 84% top-3 drug-ID accuracy and close to 86% drug-class accuracy with the chosen modelAttorney Docket No.00015-429WO1 BYOL was obtained (see Table 1, row 1). This indicates that the model correctly places each cell image in the latent space close to the mode of the pharmacological intervention it is treated with. Experiments tested if other self-supervised model architectures would yield higher accuracies for MitoSpace but that was not the case. Slightly lower accuracies were obtained with SimCLR, with a more recent transformer-based approach, DINO, and with an autoencoder-based approach (a VQVAE with a classifier head) (Table 1, rows 2-4).

[0100] Table 1: MitoSpace is highly accurate at predicting drugs from images while simultaneously creating a semantically meaningful latent space. Displayed are top 1 and top 3 prediction accuracies for individual drugs, top 1 and top 3 accuracies for drug function, and the semantic quality metric for MitoSpace and other model architectures. The supervised models achieve slightly higher accuracies in raw prediction tasks but fail at the task of transfer learning, i.e. creating a semantically / biologically meaningful latent space that is interpretable (high semantic quality scores). Also note that the supervised KNN model on mitochondria analyzer features does not have a latent space and thus a ‘semantic quality metric’ cannot be computed.Attorney Docket No.00015-429WO1

[0101] Next, the self-supervised approach was compared to traditional, supervised approaches. As a comparison, a baseline supervised ResNet18 was trained on the dataset and also included a naive K-nearest neighbors (KNN) model trained on a set of 17 features previously defined in literature, extracted from each image using Mitochondria Analyzer. While the KNN performs expectedly poorly (18% and 24% drug-ID and drug-class performance respectively), the ResNet18 model performs very well and better than the self-supervised approaches (92% and 93% drug-ID and drug- class performance respectively) (Table 1, rows 5-6).

[0102] This is not surprising. Self-supervised models typically show slightly lower classification performance than supervised models on standard image benchmarks like ImageNet-1K. However, they are known to exhibit better transfer learning performance, that is, they generalize better to concepts and data that are out-of- distribution which the model has not seen at training time. This is especially important for a phenotypic atlas like MitoSpace, where the effects of unseen drugs and the resulting phenotypes must be accurately captured.Attorney Docket No.00015-429WO1

[0103] To show the superior transfer learning performance of MitoSpace compared to ResNet18 and its superior performance as a phenotypic atlas, the test-set latent spaces generated by MitoSpace and ResNet18 were plotted (Figure 4). MitoSpace’s latent space shows a continuous map in which individual images of drug treated cells blend into each other (Fig 4A). In contrast, the ResNet18 latent space shows each drug cluster separated from each other with large distances in between them (Fig. 4B). This structure reflects the supervised model’s superior performance in raw accuracy, but it comes at the expense of lost semantic / biological meaning.

[0104] First, consider two cells, A1 and A2, treated with different drugs, but having the same phenotypic values (A1: TMRM intensity=0.12, Mean Mitochondria Area=0.70, Mean Mitochondria Aspect Ratio=2.17, A2: TMRM intensity=0.13, Mean Mitochondria Area=0.74, Mean Mitochondria Aspect Ratio=2.26). Since the self- supervised model placed cells close to any other cells showing similar phenotype irrespective of what drug they are treated with, it correctly mapped A1 and A2 close together (Figure 4A). The supervised model, on the other hand, overfitted on the predefined drug label and incorrectly put A1 and A2 far away from each other (Figure 4B). Second, consider two cells, B1 and B2, treated with the same drug but showing completely different phenotypes (B1: TMRM intensity=0.13, Mean Mitochondria Area=0.53, Mean Mitochondria Aspect Ratio=1.76, B2: TMRM intensity=0.02, Mean Mitochondria Area=0.90, Mean Mitochondria Aspect Ratio=1.82). The self- supervised model, again correctly placed B1 and B2 far away from each other, closer to other cells showing phenotypes similar to each of them (Figure 4C). The supervised model again overfitted on the predefined drug label and incorrectly placed these images close together (Figure 4C). Such behavior is prevalent in all the supervised and self-supervised models.

[0105] While it is now apparent that the supervised latent space is not meaningful in terms of pharmacological function (i.e., it does not map drugs with similar function close together), it does not correctly capture individual features either.

[0106] Figure 4E shows the variation of a particular feature, the mean mitochondria area, over MitoSpace’s self-supervised latentAttorney Docket No.00015-429WO1 space. The consistent variation of the feature is correctly captured by the model as discussed previously. In contrast, the same feature is plotted on the supervised latent space in Figure 4F. Note, there is no consistent gradient over the latent space that captures the variation of the feature. Instead, cells with different mitochondrial areas are mixed together without any particular pattern. The model gives priority to the drug label provided during supervision as opposed to true underlying phenotype. Essentially, supervised classification models overfit to predefined drug labels leading to a latent space with inferior semantic / biological meaning. Thus, they may not generalize to out- of-distribution drugs or cell types, and no qualitative or quantitative biological conclusions can be drawn from latent spaces generated in such a manner.

[0107] To quantitatively capture this semantic quality of the latent spaces of self-supervised models, a semantic quality metric for each model was computed. The semantic quality metric quantifies how well the latent space captures previously known features related to mitochondria. This was defined as the aggregate Pearson Correlation Coefficient of the first 10 principal components of the latent space with each of the 17 features extracted from Mitochondria Analyzer. Intuitively, the higher the semantic quality metric, the better the latent space captures these predefined features, i.e. its directions correlate better with semantic biological concepts. The metric is provided for the different models in Table 1. All self-supervised models showed higher semantic quality as compared to the supervised model.

[0108] MitoSpace Generalizes to Different Cell Types and Drug Interventions without Explicit Training. Next, the generalizability of MitoSpace was analyzed in two ways: (a) generalization to a different but similar cell type, and (b) generalization to drugs unseen during model training. Mitochondria can be found in all eukaryotic cells and, depending on cell type, assume different or similar roles between different cell types. On that basis, it was hypothesized that the model would generalize across different cell types to the extent of the similarity between the cell types. To investigate generalizability of trained models two cell types wereAttorney Docket No.00015-429WO1 used: Cal27 and HeLa cells. Similar to Cal27, HeLa is an immortalized cancer cell line that heavily relies on oxidative phosphorylation in mitochondria for their large energy demand. Different from the head and neck cancer cell line Cal27 however, HeLa has been derived from cervical cancer.

[0109] To investigate generalization across cell types, the method trained models on one cell type and predicted all images of the alternate cell type using that model, i.e., models were trained using HeLa cells and predicted Cal27 cells and vice-versa. The top- 3 accuracy of the base model (the model trained on the same cell type) to the alternate model (the model trained on the other cell type) were compared. In addition, the resultant latent spaces were compared visually.

[0110] Figure 5 shows the results of the generalization experiments based on the drug-induced imaging dataset for Cal27 cells (Figure 5A) and HeLa cells (Figure 5B). Figure 5C and Figure 5D show the predicted Cal27 latent space for all drugs in the form of density plots, for models trained on Cal27 cell images (i.e. the base model) and HeLa cell images (i.e. the alternate model) respectively. While there was a minor differences, overall structure of the space is maintained, both globally and locally, irrespective of which cell type the model is trained on. Note that the deep learning latent spaces are quite robust and reproducible (see the consistent latent spaces in Figs. 5C-F). The UMAP visualization can be reproduced by fixing the random seed. Note that the UMAP embedding is for visualization purposes only. Any downstream analytical conclusions are drawn directly from the 128 dimensional latent space.

[0111] To investigate generalization to drugs unseen during training, drugs were withheld during training, and predicted the entire set of drugs during testing. The resulting latent space was then compared to the oracle model that has access to all drugs during training. In this example, the model was trained with as little as a quarter of the total drug library, ensuring that the training drug set was chosen so that there was still adequate representation from each functional category.Attorney Docket No.00015-429WO1

[0112] Figure 5E and Figure 5F show the same Cal27 latent spaces trained on the two different cell types, but only using a quarter of the drugs at training time (but predicting all the drugs at test time). The reduced training set of drugs was chosen such that the set still maximally represented the extent of the latent space. Once again, the structure of the latent space was maintained. Figure 5G shows the top-3 accuracy of predicting each image into the correct drug category using a KNN predictor on top of the embeddings, for models trained on different cell types and number of drugs. Only minor drops were observed in accuracy when the underlying model is trained on a smaller number of drugs. As an example, prediction accuracy for train Cal27 test Cal27 drops from 81% to 78% when the number of training drugs is reduced from 24 to only 6 (Figure 5G left). Similarly, prediction accuracy for train Cal27 test HeLa drops from 56% to 51% when the number of training drugs is reduced (Figure 5G right). It is worth noting that the prediction success for Cal27 was high even when trained on HeLa data (Fig. 5G,H, left) while the prediction success for HeLa remained low even when trained on HeLa (Fig. 5G,H right).

[0113] The results indicate two advantages of the approach: (i) latent spaces trained using the method generalize across cell types, i.e., a model can be trained on a different but related cell type and it still shows meaningful results on the original cell type, and (ii) global and local structure of the latent space and quantitative measures like accuracy are maintained even when only a subset of drugs is used for training, implying that models can generalize to drugs unseen during training.

[0114] Such out-of-distribution generalization capability is possible because of the completely unsupervised nature of the cell perceiving method, and opens up possibilities for understanding behavior of a wide range of cell types with minimal data collection and deep learning model complexity. It would also allow the delineation of similarities and differences in pharmacological effects and therapeutic interventions across cell types.

[0115] MitoSpace is Robust to Natural Imaging Variations. To evaluate the robustness of the model against natural imaging variations that occur over multiple imaging sessions, a series ofAttorney Docket No.00015-429WO1 experiments were conducted imaging and collecting data over five different days. During each session, the same drug conditions and imaging parameters were maintained to the best ability. The model was initially trained on data from a single day’s imaging session to construct the MitoSpace.

[0116] Next, the model's performance was evaluated on a small evaluation set from the same day, as well as the complete datasets from the other four days. The results indicated that MitoSpace is robust to natural variations, consistently learning relevant patterns in the data across different imaging sessions. These findings are summarized in Table 2, which presents the model's performance metrics for each evaluation set.

[0117] Furthermore, to better understand the consistency of the model across different imaging days, the overlap of the samples in the 3D projected space was visualized (Figure 7). This visualization demonstrates that despite variations in imaging conditions, the model's ability to cluster and map the mitochondrial properties remains stable and accurate. The robust performance of MitoSpace across different sessions underscores its reliability for use in varied experimental conditions and highlights its generalizability.

[0118] Table 2: The table quantifies the top-1 and top-3 accuracies for the 5-day dataset. The second column represents the results from the same day (day-1 against day-1). The subsequent columns represent the cross-day accuracies of data from respective days against day-1. Dataset / Accuracy (%) Day-1 / Day-1 Day-2 / Day-1 Day-3 / Day-1 Day-4 / Day-1 Day-5 / Day-1 Top-1 74.39 51.96 47.38 52.50 57.46

[0119] Analyzing the effect of drug treatment on Mitochondrial Biology and MitoSpace – 2.5D MitoSpace. In the experiments, MitoSpace is sensitive to the duration of drug exposure on cells. To investigate this temporal effect, a study was designed that allowed one to image cells at various time intervals. Data was collected by imaging all our drug-treated cells at fixed intervals over a period of time, ensuring that we captured images from multiple time points.Attorney Docket No.00015-429WO1

[0120] MitoSpace was then constructed using images from these different time intervals, color-coding the data according to the time points to study the temporal effects. The analysis revealed that some drugs induced changes in mitochondrial morphology and TMRM potential over time, which were effectively captured by MitoSpace. This was evidenced by the clustering of different colors representing the various time steps.

[0121] Additionally, the findings aligned with existing literature, confirming that drugs known to be time-sensitive displayed distinct temporal clustering in MitoSpace. This validation highlights MitoSpace's capability to detect and represent time-dependent morphological changes induced by drug treatments.

[0122] Robust Prediction Technique for Drug Classification. Given the inherent biological and imaging variations that can impact MitoSpace, it is nearly impossible to completely eliminate these variations. Therefore, a new prediction method was developed for classifying drugs that is more robust to these variations and considers a broader perspective of drug actions on cells.

[0123] It was hypothesized that the mixture of mitochondrial phenotypes induced by a specific drug remains consistent when observed across numerous cells, and this pattern can be utilized to make more accurate predictions. The method constructs two confusion matrices: one from the training data (reference confusion matrix) and one from the testing data against the training data (evaluation confusion matrix). Figure 9 displays both confusion matrices.

[0124] To make predictions or calculate the percentage accuracy, we perform distribution matching for all rows (drugs) from the evaluation confusion matrix against the reference confusion matrix. We employ the Kullback–Leibler divergence to identify similarities in distributions. Using this new evaluation criterion, the results were recalculated; it was observed there was significant improvement in performance (see Table 3).

[0125] Table 3: Results With The Robust Prediction Algorithm. The table quantifies the top-1 and top-3 accuracies for the 5-day dataset. The second column represents the results from the same day (day-1 against day-1). The subsequent columns represent the cross-Attorney Docket No.00015-429WO1 day accuracies of data from respective days against the reference confusion matrix constructed solely from the day-1 data. The consistency in the numbers across both cross-day and same-day scenarios demonstrates the robustness of our model. Dataset / Accuracy (%) Day-1 / Day-1 Day-2 / Day-1 Day-3 / Day-1 Day-4 / Day-1 Day-5 / Day-1 Top-1 100 84.61 69.23 88.46 88.46 Top-3 100 100 100 100 100 eagainst day-to-day variations, demonstrating the effectiveness of the method in providing accurate and reliable drug classifications despite inherent biological and imaging inconsistencies.

[0127] It will be understood that various modifications may be made without departing from the spirit and scope of this disclosure. Accordingly, other embodiments are within the scope of the following claims.

Claims

Attorney Docket No.00015-429WO1 WHAT IS CLAIMED:

1. A method comprising: creating low-dimensional embedding spaces from microscopy images using deep learning by: (i) determining one or a plurality of subcellular structure or organelles of interest; (ii) capturing a plurality of images of the structures or organelle of interest in a cell under the influence of an agent suspected of modifying the phenotype of the structures or organelles of interest; (iii) training a self-supervised deep learning model on the plurality of images such that no labels are provided to the model as to which agent was contacted with the cell; repeating (i) – (iii) for a plurality of cells under different culture conditions; and converting the images obtained to a compact and expressive atlas that places images of cells showing similar visual phenotypes together, thus generating a map that links phenotypes from microscope images to the agent’s function.

2. The method of claim 1, wherein the plurality of cells under different culture conditions comprise cells exposed to different agents suspected of modifying their phenotypes.

3. The method of claim 1, wherein the structure or organelle of interest is a mitochondria.

4. The method of claim 1, wherein the images are obtained from cells stained with a fluorescent or luminescent agent.

5. The method of claim 1, wherein the training of the self- supervised deep learning model comprises analysis of a plurality of visual characteristics obtained using one or more channel imagesAttorney Docket No.00015-429WO1 that highlight different functional and morphological aspects of the structure or organelle of interest.

6. The method of claim 1, wherein the method further comprises obtaining a large number of multi-channel images of single cells treated with different agents.

7. The method of claim 6, further comprising normalizing each channel of the large number of multi-channel images with the 95 percentile of the maximum intensity value of each channel.

8. The method of claim 6 or 7, further comprising performing gamma correction and denoising to highlight the structure or organelle of interest in each image of the large number of multi- channel images.

9. The method of any one of claims 6, 7, or 8, further comprising stochastically performing data augmentation on the large number of multi-channel images.

10. The method of claim 9, wherein data augmentation includes random rotation, flipping, additive noise, multiplicative brightness, additive brightness, random crops and combinations thereof.

11. The method of claim 1, wherein images are applied to a deep learning algorithm selected from the group consisting of SimCLR, transformer-based (DINO), autoencoder based (VQVAE) and bootstrapping-based (VYOL).

12. The method of claim 11, wherein the deep learning algorithm maps images into a low dimensional (128-2048) latent space.

13. The method of claim 12, further comprising mapping cell images to a single vector in the low dimensional latent space.Attorney Docket No.00015-429WO1 14. The method of claim 13, further comprising mapping the biological effect of a test agent on a cell contacted with the test agent comprising generating a low dimensional latent space image of the cell contacted with the test agent and applying an atlas of vectors to the image to map a phenotypic changes of the cell contacted with the test agent.

15. The method of claim 14, further comprising creating a mapping that is robust to day-to-day variations that occur between microscopy images captured on different days.

16. The method of claim 14, further comprising a robust prediction technique to classify the agent acting on the cell from the microscopy image.

17. The method of claim 14, further comprising the ability to not only work on images that have been captured to assess an agent’s effect on the cell for a single time window but also work on temporal series of images that have been captured to assess an agent’s effect on the cell for a longer time period, and thereby consisting of images that capture a subcellular structure or organelle that evolves over time.

18. A method for automated cell analysis, the method comprising: receiving, by a computer apparatus, a first cellular image associated with a first cell treated with a first culture condition or having a first phenotype; determining, by the computer apparatus, one or more cellular structures or organelles in the first cell; creating low-dimensional embedding spaces from the first cellular image using deep learning by: (i) determining a subcellular structure or organelle of interest; (ii) capturing a plurality of images of the structure or organelle of interest; (iii) training a self-supervised deep learning model on the plurality of images such that no labelsAttorney Docket No.00015-429WO1 are provided to the model as to which culture condition or phenotype was used on the first cell; repeating (i) – (iii) for a plurality of cells under different culture conditions or different disease types; and converting the images obtained to a compact and expressive atlas that places images of cells showing similar visual phenotypes together, thus generating a map of phenotypes from cellular images; providing a second cellular image of a second cell to the computer apparatus; determining, by the computer apparatus, one or more structures or cellular organelles in the second cell; measuring a plurality of visual characteristics from the structures or organelles in the second cell; identifying, by the computer apparatus, cellular structures or organelles in the compact and expressive atlas that have the same or similar visual characteristics to the one or more structure or organelles of the second cell; identifying a culture condition or phenotypes associated with the identified cellular structures or organelles in the compact and expressive atlas to a user as being associated with the second cell.

19. The method of claim 18, wherein the plurality of cells under different culture conditions comprise cells exposed to different agents suspected of modifying their phenotypes.

20. The method of claim 18, wherein the structure or organelle is a mitochondria.

21. The method of claim 18, wherein the images are obtained from cells stained with a fluorescent or luminescent agent.

22. The method of claim 18, wherein the plurality of visual characteristics are obtained using one or more channel images that highlight different functional and morphological aspects of the organelle.Attorney Docket No.00015-429WO1 23. The method of claim 18, wherein the method further comprises obtaining a large number of multi-channel images of single cells treated with different agents.

24. The method of claim 23, further comprising normalizing each channel of the large number of multi-channel images with the 95 percentile of the maximum intensity value of each channel.

25. The method of claim 23 or 24, further comprising performing gamma correction and denoising to highlight the organelle in each image of the large number of multi-channel images.

26. The method of any one of claims 23 or 24, further comprising stochastically performing data augmentation on the large number of multi-channel images.

27. The method of claim 26, wherein data augmentation includes random rotation, flipping, additive noise, multiplicative brightness, additive brightness, random crops and combinations thereof.

28. The method of claim 18, wherein image data is applied to a deep learning algorithm selected from the group consisting of SimCLR, transformer-based (DINO), autoencoder based (VQVAE) and bootstrapping-based (VYOL).

29. The method of claim 28, wherein the deep learning algorithm maps each image of the large number of images into a low dimensional (128-2048) latent space.

30. The method of claim 29, further comprising mapping each cell image to a single vector in the low dimensional latent space.

31. The method of claim 30, further comprising mapping the biological effect of a test agent on a cell contacted with the test agent comprising generating a low dimensional latent space image ofAttorney Docket No.00015-429WO1 the cell contacted with the test agent and applying an atlas of vectors to the image to map a phenotypic changes of the cell contacted with the test agent.

32. The method of claim 30, further comprising mapping the biological effect of a test agent on a cell contacted with the test agent comprising generating a low dimensional latent space embedding that relates how the subcellular structure’s or organelle’s morphology once contacted with the test agent is related to the subcellular structure’s or organelle’s morphology contacted with other agents and thereby creating an atlas to map phenotypic changes of the cell contacted with the test agent.

Images