Cell time series imaging, modeling and analysis system

Abstract

The present disclosure relates generally to providing a cell time series imaging, modeling, and analysis platform, and more particularly to acquiring time series image data and using various machine learning models to model and analyze subcellular particle motion as well as cell position and morphological feature variations generated using unsupervised embedding. The platform can be applied to assess various cellular and subcellular processes by generating digest embedding of time-series image data that can analyze time-varying dynamic cellular and subcellular processes (e.g., movement of intracellular particles, neurons on developmental neurons, etc. In some embodiments, methods and systems are provided to enhance the identification of differences between cellular states (e.g., between ill and healthy cells) and generate a disease model that can be used to analyze the effects of various therapeutic interventions, as well as other improvements described herein.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of U.S. Application 18 / 666,672, filed May 16, 2024, and U.S. Provisional Application 63 / 467,582, filed May 18, 2023, the entire contents of which are incorporated by reference herein for all purposes. TECHNICAL FIELD

[0003] The present disclosure relates generally to a platform for cell time series imaging, modeling, and analysis, and more particularly to using machine learning models to analyze time series image data (e.g., studying subcellular particle motion and changes in cell position and morphological features over time). BACKGROUND

[0004] Existing systems and methods for monitoring and analyzing living organisms (e.g., living biological cells) fail to provide adequate mechanisms for analyzing subcellular motion (e.g., motion of intracellular particles, neurites on developing neurons, etc.). Instead, such subcellular particle motion and changes in cell position and morphological features are considered too random, transient, noisy, and / or complex to gain insight therefrom. As a result, such information has not been studied, for example, in the context of cell monitoring, disease modeling, and drug discovery. SUMMARY

[0005] Disclosed herein are methods, systems, electronic devices, non-transitory storage media, and apparatuses relating generally to providing a platform for cell time series imaging, modeling, and analysis, and more particularly to acquiring time series image data and using various machine learning models to model and analyze subcellular particle motion and changes in cell position and morphological features over time. The platform is applicable to studying a variety of cellular and subcellular processes.

[0006] Recently, the assignee herein has developed systems and methods for autonomous cell imaging and modeling. Such systems are described, for example, in U.S. Patent Application No. 18 / 111,405 and PCT Application No. PCT / US2022 / 080200, the entireties of both of which are incorporated herein by reference. The cell time series imaging, modeling, and analysis platform described herein provides numerous practical applications related to studying and controlling cellular and subcellular processes using time series image data. In some embodiments, the cell time series imaging, modeling, and analysis platform can be used to acquire time series image data and generate a summary embedding that provides a low-dimensional representation of the time series image data for downstream analysis. The embedding captures rich semantic information of the imaging data (e.g., microstructural features of the tissue reflected in the images, including cellular substructures), while excluding information that is irrelevant for downstream analysis (e.g., orientation of the images). Moreover, the summary embedding of the time series image data (e.g., determined based on an embedding sequence as described herein) captures the temporal dimension and allows for analysis of dynamic cellular processes that change over time. Thus, the summary embedding is capable of analyzing subcellular motion (e.g., motion of intracellular particles, neurites on developing neurons, etc.) and identifying differences between cellular states (e.g., differences in subcellular motion between diseased and healthy cells). The summary embedding can also be used to generate disease models, thereby enhancing analysis of the effects of various therapeutic interventions.

[0007] The time series image data can include a set of images of the same region of interest (e.g., a cell, a portion of a cell), arranged in chronological order. The time series image data can include a burst of images or a video segment, and can be encoded by one or more machine learning models into an embedding sequence, which in turn can be encoded into a summary embedding having a temporal dimension based on temporal information associated with the embedding sequence, as described throughout this document. In some embodiments, the system can continuously generate time series image data of a set of living biological cells in a non-destructive manner and efficiently analyze the images using machine learning techniques (e.g., self-supervised machine learning models) to assess the effects of physical, chemical, or genetic perturbations (e.g., chemical treatments, therapeutic agents, genetic modifications, media changes, or any combination thereof) on the cells. In some embodiments, the system can generate time series image data of different sets of living biological cells corresponding to different disease states, convert the time series image data into low-dimensional summary embeddings, and generate a disease model in a topological space that can be used to model the progression of the disease. In some embodiments, the system can continuously generate time series image data of a cell culture in a non-destructive manner and analyze the time series image data to study features of interest of the cell culture, such as cell proliferation rate, reversal to a healthy phenotype, etc.

[0008] In some embodiments, the cellular time-lapse imaging, modeling, and analysis platform comprises multiple stages. In some embodiments, the platform comprises a first autonomous imaging stage. In some embodiments, the first autonomous imaging stage provides label-free time-lapse imaging (i.e., does not rely on fluorescent dyes to label different cellular components), such as quantitative phase imaging ("QPI"), to acquire one or more sets of time-lapse image data. In some embodiments, the time-lapse image data comprises time-lapse images. In some embodiments, the time-lapse image data comprises video clips. The imaging stage can acquire time-lapse phase images that depict position and morphological features in specific cellular substructures. In some embodiments, the imaging stage is compatible with low phototoxicity fluorescence and autofluorescence multispectral time-lapse imaging techniques. The imaging stage can acquire time-lapse fluorescence images and / or autofluorescence images of live biological cells from converted brightfield images. In some embodiments, the time-lapse image data (e.g., phase images, fluorescence images, autofluorescence images, etc.) of live biological cells is captured at the imaging stage using a microscope according to optical settings that can be manually and / or automatically configured.

[0009] In some embodiments, the time-lapse image data comprises multiple phase images captured before or after multiple fluorescence images. This acquisition configuration provides a number of technical advantages over acquiring only one of phase or fluorescence images. For example, acquiring both phase and fluorescence time-lapse image data enables cross-registration and validation of phase and immunohistochemistry readouts at the same location (e.g., a region of interest of a cell), which can serve as a tag to confirm the biological origin of the refractive index changes. Moreover, the fluorescence channel can serve as a phenotypic readout in many cases. The time-lapse image data captured using fluorescence imaging and phase imaging can be used as data to train machine learning models, as further described throughout this document.

[0010] Moreover, capturing time-lapse image data comprising time-lapse images and / or video clips provides a number of technical advantages over other imaging techniques. For example, time-lapse image data can depict subcellular granule movements that can appear random, noisy, or transient to a human observer. By acquiring time-lapse image data that depicts such seemingly random movements and processes and inputting the time-lapse images into trained machine learning models, various patterns can be identified that represent relationships between the movements and cellular states, disease progression, etc., which can lead to new discoveries in treating a number of diseases.

[0011] In some embodiments, the platform includes a second machine learning-based stage. At the second stage, the exemplary system (e.g., one or more electronic devices) performs machine learning-based image processing on the time series image data of the living biological cells to obtain a representation (e.g., an embedding). The embedding is a vector representation of the phenotypic state of the living biological cells. In some embodiments, the exemplary system includes a first trained machine learning model that determines one or more embedding sequences based on one or more sets of time series image data. Each embedding of the one or more embedding sequences can represent morphological or positional features of the cell or cellular substructure and / or process at a particular point in time (e.g., a single frame in the set of time series image data), such that the sequence represents multiple morphological or positional features of the cell or cellular substructure and / or process at multiple respective points in time. As described above, each embedding of the embedding sequence captures rich semantic information of the imaging data (e.g., microstructural features of the tissue reflected in the image, including cellular substructures), while excluding information that is irrelevant for downstream analysis (e.g., orientation of the image).

[0012] In some embodiments, one or more summary embeddings are determined based on the one or more embedding sequences. The summary embeddings can be determined by inputting the embedding sequence into a second trained machine learning model. In some embodiments, the summary embeddings are determined directly from the set of time series image data by inputting the set of time series image data into a trained machine learning model that can directly process time series image data (e.g., a model that can directly process video data). The summary embeddings can take into account dynamics (e.g., subcellular motion) between subsequent frames in the set of time series image data. Thus, the summary embeddings contain temporal information of the set of time series image data.

[0013] The combination of time series image data and trained machine learning models for determining summary embeddings based on the time series image data can enable increasingly sophisticated modeling and analysis of cellular and subcellular morphology, motion, and processes that existing systems fail to provide. The systems and methods described herein can enable users (e.g., researchers) to determine relationships between seemingly random particle motion and structural changes at the cellular and subcellular level to observed disease states, metabolic states, and / or kinetic states of the cells, e.g., to better understand biological processes and assess therapeutic interventions in novel ways.

[0014] In some embodiments, the platform can further include a third stage for data analysis. In some embodiments, the summary embeddings generated by the second stage are used for downstream tasks. In some embodiments, the summary embeddings can be used to determine a cell state of one or more cells. The cell state can be indicative of a disease state, a health state, a degree of a disease state, a level of metabolic activity, or a kinetic state. In some embodiments, an exemplary system can determine, based on the cell state of one or more cells and the time series image dataset, a relationship between one or more time-varying morphological features depicted in the time series image dataset and the cell state of the one or more cells, or a relationship between one or more subcellular or cellular motions or processes depicted in the time series image dataset and the cell state of the one or more cells. In some embodiments, a human user of the system can compare the cell state to the time series image data to identify a relationship between a time-varying morphological feature (e.g., a subcellular granule motion) in the time series image and the determined cell state.

[0015] In some embodiments, the summary embeddings can be used to determine the impact of a physical, chemical, or genetic perturbation (e.g., a chemical treatment, a therapeutic agent, a genetic therapy, or a media change, or any combination thereof) in slowing down or reversing disease progression, for example, by detecting a change in the cell state classification. In some embodiments, the summary embeddings can be used to generate a disease model.

[0016] In some embodiments, the disease model can be used to generate a synthetic image of one or more disease states of a cell. The synthetic image can be compared to a real image of the cell to determine the accuracy of the disease state model. In addition, the synthetic image can be used to study morphological features of the disease.

[0017] Overall, the cell time series imaging, modeling, and analysis platform provides an imaging and modeling setup that can be used to analyze subcellular motions and processes on extremely short time scales to identify relationships between seemingly random subcellular motions and biological processes (e.g., cell states, progression of various diseases, and treatments).

[0018] In some aspects, provided herein is a method for determining a cell state of one or more cells, the method comprising: receiving a time series image dataset depicting one or more cells; determining an embedding sequence by inputting the time series image dataset into a first trained machine learning model; determining a summary embedding based on the embedding sequence, the summary embedding comprising a temporal dimension based on temporal information associated with the embedding sequence; and determining the cell state of the one or more cells of the subject by inputting the summary embedding into a second trained machine learning model.

[0019] In some embodiments, the temporal information associated with the embedding sequence comprises a temporal relationship between a first embedding in the embedding sequence and a second embedding in the embedding sequence.

[0020] In some embodiments, the time information associated with the sequence of embeddings includes an order relationship between a first embedding in the sequence of embeddings and a second embedding in the sequence of embeddings.

[0021] In some embodiments, the time information associated with the sequence of embeddings includes a timestamp associated with each embedding in the sequence of embeddings.

[0022] In some embodiments, the time series image dataset includes time series images.

[0023] In some embodiments, the time series image dataset includes a video clip.

[0024] In some embodiments, the time series image dataset includes image data acquired at a frame rate of at least four frames per second or eight frames per second.

[0025] In some embodiments, the time series image dataset includes a plurality of phase images.

[0026] In some embodiments, the plurality of phase images is captured using an imager having a frame rate of at least four frames per second or at least one frame per second. In some embodiments, the frame rate is about 40 frames per second or about 8 frames per second.

[0027] In some embodiments, the time series image dataset includes a plurality of fluorescence images. In some embodiments, the plurality of fluorescence images is captured using an imager having a frame rate of at least four frames per second or at least one frame per second.

[0028] In some embodiments, the cell state is indicative of a disease state, a health state, or a degree of a disease state.

[0029] In some embodiments, the method further includes determining, based on the cell state of the one or more cells and the time series image dataset, a relationship between one or more time-varying morphological features depicted in the time series image dataset and the cell state of the one or more cells.

[0030] In some embodiments, the method further includes determining, based on the cell state of the one or more cells and the time series image dataset, a relationship between one or more subcellular or cellular motions or processes depicted in the time series image dataset and the cell state of the one or more cells.

[0031] In some embodiments, the cell state includes an indication of lipid accumulation.

[0032] In some embodiments, the cell state is indicative of a level of metabolic activity.

[0033] In some embodiments, the cell state is indicative of a kinetic state.

[0034] In some embodiments, the rate of change in the cellular state is indicative of a change in a cellular process, wherein the cellular process comprises any one or more of cargo transport, organelle assembly, and organelle disassembly.

[0035] In some embodiments, the method further comprises mapping a network of one or both of axons and neurites to a cell in the one or more cells based on the time series image dataset and the cellular state of the one or more cells.

[0036] In some embodiments, the first trained machine learning model comprises a self-supervised machine learning model.

[0037] In some embodiments, the first trained machine learning model is trained using an unlabeled training dataset.

[0038] In some embodiments, the unlabeled training dataset comprises a plurality of images of biological samples.

[0039] In some embodiments, the first machine learning model is pre-trained using unlabeled images that do not depict biological samples and retrained using unlabeled images of biological samples.

[0040] In some embodiments, determining the summary embedding based on the sequence of embeddings comprises: inputting the sequence of embeddings into a third trained machine learning model.

[0041] In some embodiments, the third trained machine learning model comprises: a transformer model, a recurrent neural network (“RNN”), a long short-term memory network (“LSTM”), or any combination thereof.

[0042] In some embodiments, the second trained machine learning model comprises a trained classifier.

[0043] In some embodiments, the second trained machine learning model comprises a trained regression model.

[0044] In some embodiments, the time series image dataset depicts a single cell. In some embodiments, the single cell is identified using an image segmentation model.

[0045] In some embodiments, the one or more cells comprise one or more living biological cells.

[0046] In some embodiments, the one or more living biological cells comprise one or more mammalian cells.

[0047] In some embodiments, the one or more living biological cells comprise one or more neurons.

[0048] In some embodiments, the one or more living biological cells comprise healthy cells, pathologic cells, or any combination thereof.

[0049] In some embodiments, the one or more living biological cells comprise one or more genetic mutations.

[0050] In some embodiments, the one or more genetic mutations are selected from the group consisting of a deletion mutation, an insertion mutation, a substitution mutation, a missense mutation, a nonsense mutation, and a frameshift mutation.

[0051] In some embodiments, the one or more living biological cells have a phenotypic difference compared to healthy cells that do not comprise the one or more genetic mutations.

[0052] In some embodiments, the phenotypic difference comprises a difference in metabolic activity, cell kinetics, cell morphology, or any combination thereof.

[0053] In some aspects, provided herein is a non-transitory computer-readable storage medium for determining a cell state of one or more cells, the storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: receive a time series image data set depicting the one or more cells; determine an embedding sequence by inputting the time series image data set into a first trained machine learning model; determine an abstract embedding based on the embedding sequence, the abstract embedding comprising a temporal dimension based on temporal information associated with the embedding sequence; and determine the cell state of the one or more cells of the subject by inputting the abstract embedding into a second trained machine learning model.

[0054] In some aspects, provided herein is a system for determining a cell state of one or more cells, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for: receiving a time series image data set depicting the one or more cells; determining an embedding sequence by inputting the time series image data set into a first trained machine learning model; determining an abstract embedding based on the embedding sequence, the abstract embedding comprising a temporal dimension based on temporal information associated with the embedding sequence; and determining the cell state of the one or more cells of the subject by inputting the abstract embedding into a second trained machine learning model.

[0055] In some aspects, provided herein is a method for modeling progression of a disease, the method comprising: obtaining a first plurality of time series image data sets of a first non-zero concentration of live biological cells expressing a first disease state of a disease; determining a first plurality of sequences of embeddings by inputting the first plurality of time series image data sets into a trained machine learning model; determining a first plurality of summary embeddings based on the first plurality of sequences of embeddings, wherein a summary embedding of the first plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a first sequence of embeddings of the first plurality of sequences of embeddings; obtaining a second plurality of time series image data sets of a second non-zero concentration of live biological cells expressing a second disease state of the disease; determining a second plurality of sequences of embeddings by inputting the second plurality of time series image data sets into the trained machine learning model; determining a second plurality of summary embeddings based on the second plurality of sequences of embeddings, wherein a summary embedding of the second plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a second sequence of embeddings of the second plurality of sequences of embeddings; generating a disease model based on the first plurality of summary embeddings and the second plurality of summary embeddings; and modeling progression of the disease based on the disease model.

[0056] In some embodiments, the temporal information associated with the first sequence of embeddings comprises a temporal relationship between a first embedding of the first sequence of embeddings and a second embedding of the first sequence of embeddings.

[0057] In some embodiments, the temporal information associated with the first sequence of embeddings comprises an order relationship between a first embedding of the first sequence of embeddings and a second embedding of the first sequence of embeddings.

[0058] In some embodiments, the temporal information associated with the first and second sequences of embeddings comprises a timestamp associated with each embedding of the first sequence of embeddings and the second sequence of embeddings.

[0059] In some embodiments, generating the disease model comprises: mapping the first plurality of summary embeddings and the second plurality of summary embeddings into a topological space.

[0060] In some embodiments, the method further comprises: identifying a location of a first cluster of summary embeddings based on the first plurality of summary embeddings in the topological space; generating a representation of the first disease state based on the location of the first cluster; identifying a location of a second cluster of summary embeddings based on the second plurality of summary embeddings in the topological space; and generating a representation of the second disease state based on the location of the second cluster.

[0061] In some embodiments, the method further comprises: generating a disease axis based on the location of the first cluster and the location of the second cluster.

[0062] In some embodiments, the disease axis represents a hyperplane separating a centroid of the first cluster of summary embeddings from a centroid of the second cluster of summary embeddings.

[0063] In some embodiments, the method further comprises identifying one or more embeddings in the topological space based on the disease axis.

[0064] In some embodiments, the method further comprises generating one or more synthetic images based on the identified one or more embeddings using a GAN model.

[0065] In some embodiments, the method further comprises comparing the phenotypes depicted in the one or more synthetic images to one or more real images depicting real cells to validate the phenotypes depicted in the one or more synthetic images.

[0066] In some embodiments, the phenotypes include neurite length, cell size, motion of one or more particles within the cell, or any combination thereof.

[0067] In some embodiments, the method further comprises obtaining a first set of time series image data of one or more critically ill living biological cells; determining a first sequence of embeddings by inputting the first set of time series image data into the trained machine learning model; determining a first summary embedding based on the first sequence of embeddings; applying a perturbation to the one or more critically ill living biological cells; obtaining a second set of time series image data of the one or more critically ill living biological cells after applying the perturbation; determining a second sequence of embeddings by inputting the second set of time series image data into the trained machine learning model; determining a second summary embedding based on the second sequence of embeddings; and determining an effect of the perturbation on reversing the one or more critically ill living biological cells from a disease state based on the first summary embedding, the second summary embedding, and a disease model.

[0068] In some embodiments, determining the effect of the perturbation comprises comparing: a location of the first summary embedding in the topological space; a location of the second summary embedding in the topological space; a location of the first cluster; and a location of the second cluster.

[0069] In some embodiments, the perturbation is a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof.

[0070] In some embodiments, the method further comprises determining a dosage of the perturbation based on the determination of the effect of the perturbation.

[0071] In some embodiments, the method further comprises determining one or more dosing intervals for administering the perturbation based on the determination of the effect of the perturbation.

[0072] In some embodiments, the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets comprises a time series image.

[0073] In some embodiments, the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data includes video clips.

[0074] In some embodiments, the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data includes image data acquired at a frame rate of at least four frames per second or at least eight frames per second.

[0075] In some embodiments, the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data includes a plurality of phase images.

[0076] In some embodiments, the plurality of phase images is captured using an imager having a frame rate of at least four frames per second or at least one frame per second.

[0077] In some embodiments, the frame rate is approximately 40 frames per second or 8 frames per second.

[0078] In some embodiments, the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data includes a plurality of fluorescence images.

[0079] In some embodiments, the plurality of fluorescence images is captured using an imager having a frame rate of at least four frames per second or at least one frame per second.

[0080] In some embodiments, the trained machine learning model includes a self-supervised machine learning model.

[0081] In some embodiments, the trained machine learning model is trained using an unlabeled training dataset.

[0082] In some embodiments, the unlabeled training dataset includes a plurality of images of biological samples.

[0083] In some embodiments, the machine learning model is pre-trained using unlabeled images that do not depict biological samples and retrained using unlabeled images of biological samples.

[0084] In some embodiments, the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data depicts a single cell.

[0085] In some embodiments, the single cell is identified using an image segmentation model.

[0086] In some embodiments, the live biological cell includes one or more mammalian cells.

[0087] In some embodiments, the live biological cell includes one or more neurons.

[0088] In some aspects, provided herein is a non-transitory computer-readable storage medium for modeling progression of a disease, the storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to: obtain a first plurality of time series image data sets of a first non-zero concentration of live biological cells expressing a first disease state of a disease; determine a first plurality of sequence of embeddings by inputting the first plurality of time series image data sets into a trained machine learning model; determine a first plurality of summary embeddings based on the first plurality of sequence of embeddings, wherein a summary embedding of the first plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a first sequence of embeddings of the first plurality of sequence of embeddings; obtain a second plurality of time series image data sets of a second non-zero concentration of live biological cells expressing a second disease state of the disease; determine a second plurality of sequence of embeddings by inputting the second plurality of time series image data sets into the trained machine learning model; determine a second plurality of summary embeddings based on the second plurality of sequence of embeddings, wherein a summary embedding of the second plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a second sequence of embeddings of the second plurality of sequence of embeddings; generate a disease model based on the first plurality of summary embeddings and the second plurality of summary embeddings; and model progression of the disease based on the disease model.

[0089] In some aspects, provided herein is a system for modeling progression of a disease, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for: obtaining a first plurality of time series image data sets of a first non-zero concentration of live biological cells expressing a first disease state of a disease; determining a first plurality of sequence of embeddings by inputting the first plurality of time series image data sets into a trained machine learning model; determining a first plurality of summary embeddings based on the first plurality of sequence of embeddings, wherein a summary embedding of the first plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a first sequence of embeddings of the first plurality of sequence of embeddings; obtaining a second plurality of time series image data sets of a second non-zero concentration of live biological cells expressing a second disease state of the disease; determining a second plurality of sequence of embeddings by inputting the second plurality of time series image data sets into the trained machine learning model; determining a second plurality of summary embeddings based on the second plurality of sequence of embeddings, wherein a summary embedding of the second plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a second sequence of embeddings of the second plurality of sequence of embeddings; generating a disease model based on the first plurality of summary embeddings and the second plurality of summary embeddings; and modeling progression of the disease based on the disease model. BRIEF DESCRIPTION OF DRAWINGS

[0091] The various aspects of the disclosure are set forth in the appended claims. Such features and advantages can be better understood with reference to the drawings and detailed description when considered in connection with the following detailed description, which describes illustrative embodiments in accordance with principles of the present disclosure, and the accompanying drawings, wherein: Figure 1 A cell time series imaging, modeling, and analysis platform is illustrated in accordance with some embodiments.

[0092] Figure 2 An exemplary method for determining cell states is illustrated in accordance with some embodiments.

[0093] Figure 3A An exemplary first machine learning model for determining embedding sequences, an exemplary second machine learning model for determining summary embeddings, and a third trained machine learning model for classifying summary embeddings are illustrated in accordance with some embodiments.

[0094] Figure 3B An exemplary machine learning model for determining summary embeddings based on a time series image dataset is illustrated in accordance with some embodiments.

[0095] Figure 4 Training of an exemplary contrastive learning algorithm is illustrated in accordance with some embodiments.

[0096] Figure 5 An exemplary method for modeling disease progression is illustrated in accordance with some embodiments.

[0097] Figure 6 Exemplary time series image data of IPSC-derived neurons imaged using QPI at 60x magnification is illustrated. The inset (bottom) shows granule movement along an axon, representing a dynamic process detectable with the proposed imaging method.

[0098] Figure 7 Exemplary time series image data of IPSC-derived neurons imaged using QPI at 60x magnification is illustrated, acquired at 10 Hz and rendered at 1 Hz. A 3D median filter and blob detector algorithm applied for computation are overlaid in red to highlight differences between time points in the QPI images.

[0099] Figure 8 An exemplary computing system is illustrated in accordance with some embodiments.

[0100] Figure 9 Exemplary intracellular dynamics are illustrated in accordance with some embodiments.

[0101] Figure 10 Exemplary intracellular dynamics are illustrated in accordance with some embodiments. DETAILED DESCRIPTION

[0102] Methods, systems, electronic devices, non-transitory storage media, and apparatuses generally related to providing a cellular time series imaging, modeling, and analysis platform, and more particularly to acquiring time series image data and modeling and analyzing subcellular particle motion, particle accumulation, and changes in cellular position and morphological features over time using various machine learning models. The platform can be applied to studying various cellular and subcellular processes.

[0103] The cellular time series imaging, modeling, and analysis platform described herein provides numerous practical applications related to studying and controlling cellular and subcellular processes using time series image data. In some embodiments, the cellular time series imaging, modeling, and analysis platform can be used to acquire time series image data and generate a summary embedding that provides a low-dimensional representation of the time series image data for downstream analysis. The embedding captures rich semantic information of the imaging data (e.g., microstructural features of the tissue reflected in the images, including cellular substructures), while excluding information that is irrelevant for downstream analysis (e.g., the orientation of the images). Furthermore, the summary embedding of the time series image data (e.g., determined based on an embedding sequence as described herein) captures the temporal dimension and allows for analysis of dynamic cellular processes that change over time. As such, the summary embedding is capable of analyzing subcellular motion (e.g., motion of particles within a cell, neurites on developing neurons, etc.) and identifying differences between cellular states (e.g., differences in subcellular motion between diseased and healthy cells). The summary embedding can also be used to generate a disease model, thereby enhancing analysis of the effects of various cellular perturbations.

[0104] The time series image data can include a set of images of the same region of interest (e.g., a cell, a portion of a cell), arranged in chronological order. The time series image data can include a burst of images or a video segment, and can be encoded by one or more machine learning models into an embedding sequence, which in turn can be encoded into a summary embedding with a temporal dimension, as described throughout this document. In some embodiments, the system can continuously generate time series image data of a set of living biological cells in a non-destructive manner and efficiently analyze the images using machine learning techniques (e.g., self-supervised machine learning models) to assess the effects of physical, chemical, or genetic perturbations (e.g., chemical treatments, genetic therapies) on the cells. In some embodiments, the system can generate time series image data of different sets of living biological cells corresponding to different disease states, convert the time series image data into low-dimensional summary embeddings, and generate a disease model in a topological space that can be used to model the progression of the disease. In some embodiments, the system can continuously generate time series image data of a cell culture in a non-destructive manner and analyze the time series image data to study features of interest of the cell culture, such as cell proliferation rate, reversal to a healthy phenotype, etc.

[0105] In some embodiments, the cellular time series imaging, modeling, and analysis platform includes multiple stages. In some embodiments, the platform includes a first autonomous imaging stage. In some embodiments, the first autonomous imaging stage provides label-free time series imaging (i.e., does not rely on fluorescent dyes to label different cellular components), such as quantitative phase imaging ("QPI"), to acquire one or more time series image datasets. In some embodiments, the time series image data includes time series images. In some embodiments, the time series image data includes video clips. The imaging stage can acquire time series phase images that depict position and morphological features in particular cellular substructures. In some embodiments, the imaging stage is compatible with low phototoxicity fluorescence and autofluorescence multispectral time series imaging techniques. The imaging stage can acquire time series fluorescence images and / or autofluorescence images of live biological cells from converted brightfield images. In some embodiments, the time series image data (e.g., phase images, fluorescence images, autofluorescence images, etc.) of live biological cells is captured at the imaging stage using a microscope according to optical settings that can be manually and / or automatically configured.

[0106] In some embodiments, the time series image data includes multiple phase images captured before or after multiple fluorescence images. This acquisition configuration provides a number of technical advantages over acquiring only one of phase or fluorescence images. For example, acquiring both phase and fluorescence time series image data enables cross-registration and validation of phase and immunohistochemistry readouts at the same location (e.g., a region of interest of a cell), which can serve as a tag to confirm the biological origin of the refractive index changes. Moreover, the fluorescence channel can serve as a phenotypic readout in many cases. The time series image data captured using fluorescence imaging and phase imaging can be used as data to train machine learning models, as further described throughout this document.

[0107] Moreover, capturing time series image data that includes time series images and / or video clips provides a number of technical advantages over other imaging techniques. For example, time series image data can depict subcellular granule movements that can appear random, noisy, or transient to a human observer. By acquiring time series image data that depicts such seemingly random movements and processes and inputting the time series images into trained machine learning models, various patterns can be identified that represent relationships between the movements and cellular states, disease progression, etc., which can lead to new discoveries in treating a number of diseases.

[0108] In some embodiments, the platform includes a second machine learning-based stage. At the second stage, the exemplary system (e.g., one or more electronic devices) performs machine learning-based image processing on the time series image data of the living biological cells to obtain a representation (e.g., an embedding). The embedding is a vector representation of the phenotypic state of the living biological cells. In some embodiments, the exemplary system includes a first trained machine learning model that determines one or more embedding sequences based on one or more time series image data sets. Each embedding of the one or more embedding sequences can represent morphological or positional features of the cell or cellular substructure and / or process at a particular point in time (e.g., a single frame in the time series image data set), such that the sequence represents multiple morphological or positional features of the cell or cellular substructure and / or process at multiple respective points in time. As described above, each embedding of the embedding sequence captures rich semantic information of the imaging data (e.g., microstructural features of the tissue reflected in the image, including cellular substructures), while excluding information that is irrelevant for downstream analysis (e.g., orientation of the image).

[0109] In some embodiments, one or more summary embeddings are determined based on the one or more embedding sequences. The summary embeddings can be determined by inputting the embedding sequence into a second trained machine learning model. In some embodiments, the summary embeddings are determined directly from the time series image data set by inputting the time series image data set into a trained machine learning model that can directly process time series image data (e.g., a model that can directly process video data). The summary embeddings can take into account dynamics (e.g., subcellular motion) between subsequent frames in the time series image data set. Thus, the summary embeddings contain temporal information of the time series image data set.

[0110] The combination of time series image data and training of machine learning models for determining summary embeddings based on the time series image data can enable increasingly sophisticated modeling and analysis of cellular and subcellular morphology, motion, and processes that existing systems fail to provide. The systems and methods described herein can enable users (e.g., researchers) to determine relationships between seemingly random particle motion and structural changes at the cellular and subcellular level to observed disease states, metabolic states, and / or kinetic states of the cells, e.g., to better understand biological processes and assess therapeutic interventions in novel ways.

[0111] In some embodiments, the platform can further include a third stage for data analysis. In some embodiments, the summary embeddings generated by the second stage are used for downstream tasks. In some embodiments, the summary embeddings can be used to determine a cell state of one or more cells. The cell state can be indicative of a disease state, a health state, a degree of a disease state, a level of metabolic activity, or a kinetic state. In some embodiments, an exemplary system can determine, based on the cell state of one or more cells and the time series image dataset, a relationship between one or more time-varying morphological features depicted in the time series image dataset and the cell state of the one or more cells, or a relationship between one or more subcellular or cellular motions or processes depicted in the time series image dataset and the cell state of the one or more cells. In some embodiments, a human user of the system can compare the cell state to the time series image data to identify a relationship between a time-varying morphological feature (e.g., a subcellular granule motion) in the time series image and the determined cell state.

[0112] In some embodiments, the summary embeddings can be used to determine the impact of a physical, chemical, or genetic perturbation (e.g., a chemical treatment, a therapeutic agent, a genetic therapy, or a media change, or any combination thereof) in slowing or reversing disease progression, for example, by detecting a change in the cell state classification. In some embodiments, the summary embeddings can be used to generate a disease model.

[0113] In some embodiments, the disease model can be used to generate a synthetic image of one or more disease states of a cell. The synthetic image can be compared to a real image of the cell to determine the accuracy of the disease state model. Further, the synthetic image can be used to study morphological features of the disease.

[0114] Overall, the cell time series imaging, modeling, and analysis platform provides an imaging and modeling setup that can be used to analyze subcellular motions and processes on extremely short time scales to identify relationships between seemingly random subcellular motions and biological processes (e.g., cell states, progression of various diseases, and treatments).

[0115] Figure 1 An exemplary autonomous cell imaging and modeling system is shown, in accordance with some embodiments. The process 100 is performed, for example, at least partially using one or more electronic devices. While portions of the process 100 are described herein as being performed by particular devices, it should be understood that the process 100 is not so limited. In the process 100, some blocks can optionally be combined, the order of some blocks can optionally be changed, and some blocks can optionally be omitted. In some examples, additional steps can be performed in connection with the process 100. Accordingly, the operations as shown (and described in more detail below) are exemplary in nature, and, as such, should not be viewed as limiting.

[0116] At block 102, the system obtains time series image data 104 of live biological cells (e.g., in vitro cell cultures) using techniques that do not destroy the imaged live biological cells. Further, the time series images in the time series image data can visualize, model, and analyze subcellular motions and processes over a very small time range (e.g., at a frame rate of at least four frames per second, at least eight frames per second).

[0117] The live biological cells can be mammalian cells. In some embodiments, the cells are healthy cells. In some embodiments, the healthy cells have not been previously treated (e.g., with a perturbation such as a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof). In some embodiments, the healthy cells have been previously treated with a perturbation. In some embodiments, the cells are diseased cells. In some embodiments, the diseased cells have not been previously treated (e.g., with a perturbation such as a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof). In some embodiments, the diseased cells have been previously treated with a perturbation (e.g., by one or more robotic devices that automatically apply the perturbation and / or by a human that applies a genetic, chemical, or physical perturbation), which can be determined in a previous iteration of the workflow. In some embodiments, the diseased cells have been previously treated with both a genetic perturbation and a chemical perturbation. In some embodiments, the perturbation is a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof. The time series image data can be captured at any desired interval. For example, a time-lapse time series image or video data can be captured over any duration of time (e.g., the time series image data can be collected over a period of a few milliseconds, a tenth of a second, a second, a minute, five minutes, ten minutes, thirty minutes, hourly, daily, weekly, etc.).

[0118] In some embodiments, the system performs marker-free time series imaging. In other words, the system images live biological cells without relying on fluorescent dyes to label different cellular components. In some embodiments, the system performs quantitative phase imaging ("QPI") to obtain time series phase images that depict position and morphological features in specific cellular substructures. Since different components in a cell shift differently based on the phase of light passing through them, capturing these shifts through time series QPI allows sub-micron resolution observations. QPI produces time series images with enhanced information compared to ordinary light contrast microscopes, and due to the inherent coherent gating effect, is able to observe live cells' activities in scattering environments such as 3D collagen matrices. Further, QPI is able to observe cells with minimal phototoxicity. Thus, QPI can be used to accelerate in vitro detection development and can provide unique insights into live biological process dynamics.

[0119] In some embodiments, the system uses low phototoxicity fluorescence and autofluorescence multispectral imaging techniques to obtain a time series of fluorescence images and / or a time series of autofluorescence images of live biological cells. In some embodiments, the time series image data (e.g., a time series of phase images, a time series of fluorescence images, and / or a time series of autofluorescence images, etc.) of live biological cells are captured using a microscope according to optical settings. The optical settings can include an illumination mode for illuminating the live biological cells. In some embodiments, the illumination mode can be dynamically determined during process 104.

[0120] At block 106, the system performs machine learning-based image processing on the time series image data 104 of live biological cells to obtain cell representations 108. In some embodiments, the system deploys self-supervised learning (SSL) techniques in which a machine learning model learns from unlabelled sample data, as described in detail herein. For example, the system can input a time series image dataset containing a time series of live biological cell images into one or more trained self-supervised learning models configured to receive a time series dataset and output a summary embedding (i.e., a vector) representing the time series image dataset in a latent space. The summary embedding can be a vector representation of the input time series image dataset in the latent space. Converting the input time series image dataset into a summary embedding can significantly reduce the size and dimensionality of the original data. The lower dimensional summary embedding can be used for downstream processing, as described herein.

[0121] In some embodiments, at block 106, the system determines an embedding sequence by inputting the time series image dataset into a first trained machine learning model, and then determines a summary embedding based on the embedding sequence by inputting the embedding sequence into a second trained machine learning model. In some embodiments, the embedding sequence includes an embedding sequence of individual cells depicted in the time series image dataset, the embeddings can represent morphological or positional features of the cells or cell substructures and / or processes, independent of the time at which the respective cell images in the time series image data were acquired. In other words, each embedding can represent morphological or positional features of the cells or cell substructures and / or processes at a particular point in time (e.g., a single frame in the time series image dataset), such that the sequence represents multiple morphological or positional features of the cells or cell substructures and / or processes at multiple respective points in time. In some embodiments, the summary embedding can take into account dynamics (e.g., subcellular motion) between subsequent frames in the time series image dataset. Thus, the summary embedding takes into account the time axis in the time series image dataset.

[0122] In some embodiments, the machine learning model trained to determine the embedding sequence is a self-supervised machine learning model that is trained. In some embodiments, the machine learning model is trained using an unlabelled training dataset, which can include a plurality of images of biological samples. For example, the self-supervised learning model can be a DINO Vision Transformer, a SimCLR model, or any other model that learns from unlabelled sample data. In some embodiments, the machine learning model is pre-trained using unlabelled images that do not depict biological samples, and retrained using unlabelled images of biological samples. In some embodiments, the machine learning model used to determine the embedding sequence is trained together with the machine learning model used to determine the summary embedding for each embedding sequence, and the machine learning model used to determine the value (e.g., classification) for each summary embedding, e.g., as described with respect to Figure 3A and 3B .

[0123] In some embodiments, the machine learning model trained to determine the summary embedding is a self-supervised machine learning model that is trained. In some embodiments, the model trained to determine the summary embedding can be a transformer model, a recurrent neural network (“RNN”), a long short-term memory network (“LSTM”), or any combination thereof. In some embodiments, the summary embedding is determined directly from the time series image dataset by inputting the time series image dataset into a trained machine learning model that can directly process the set of imaging sequences (e.g., a model that can directly process video data). In such embodiments, the machine learning model can include a video transformer network or a three-dimensional convolutional neural network, as shown below Figure 3B .

[0124] Prior to processing input time series image data (e.g., time series image data 104) using the machine learning models described above, each model needs to be trained. In some embodiments, training the machine learning model used to determine the embedding sequence, the machine learning model used to determine the summary embedding, and the machine learning model used to determine the value (e.g., classification of the summary embedding) based on the summary embedding includes training each model together, e.g., as described with respect to Figure 3A . In some embodiments, training the machine learning model used to determine the summary embedding directly based on input time series image data is trained together with the machine learning model used to determine the value (e.g., classification of the summary embedding) based on the summary embedding, e.g., as described with respect to Figure 3B .

[0125] In some embodiments, the one or more trained machine learning models are self-supervised machine learning models that are pre-trained using unlabelled images that do not depict biological samples. In some embodiments, the models are retrained or fine-tuned using unlabelled images of biological samples, such as phase images. In some embodiments, the models are continuously updated across all imaging experiments to collect information about all existing cell types, perturbations, and imaging conditions. The training of exemplary contrastive learning algorithms for determining embeddings based on input image data is discussed below. Figure 4 The training of exemplary contrastive learning algorithms for determining embeddings based on input image data is discussed below according to some embodiments.

[0126] In some embodiments, the one or more self-supervised models can be trained using non-microscopy images and then used to process Figure 1 live cell time series image data at block 106. In some embodiments, the models are first trained using non-microscopy images and then fine-tuned (e.g., retrained) using live cell images for several rounds before being used to process Figure 1 live cell input time series image data at block 106. In some embodiments, the live cell images used to fine-tune the models can be selected from the images 104 in Figure 1 . In other words, the live cell images can be first used to train the models and then input into the trained models to obtain embeddings and summary embeddings.

[0127] In some embodiments, at block 106, the system performs segmentation (e.g., cell detection, nucleus detection) on the live cell time series image data. In some embodiments, the system performs quality control on the live cell time series image data to obtain quality control measures that can be used to remove artifacts, determine which live cell time series image data to use for downstream processing at 110, etc.

[0128] At block 110, the system performs data analysis. In some embodiments, the system can determine a cell state of one or more cells of the subject by inputting the summary embeddings into a trained machine learning model. In some embodiments, the trained machine learning model comprises a trained classifier. In some embodiments, the trained machine learning model comprises a linear regression classifier configured to receive the summary embeddings and output a cell state classification (e.g., healthy or diseased). In some embodiments, the machine learning model comprises a regression classifier, such as a linear regression classifier or a logistic regression classifier, configured to receive the summary embeddings and output a degree of disease state.

[0129] In some embodiments, the cell state indicates a disease state (i.e., a mutant state), a healthy state (i.e., a wild-type state), or a degree of disease state. The cell state can indicate a level of metabolic activity or a kinetic state. In some embodiments, the cell state indicates an accumulation of lipids.

[0130] In some embodiments, the system is configured to determine one or more relationships between the determined cell state and morphological or positional features of the cells or cellular substructures depicted in, for example, the time series image data. For example, the system can determine, based on the cell state of the one or more cells and the time series image data set, a relationship between one or more time-varying morphological features depicted in the time series image data set and the cell state of the one or more cells. In some embodiments, the system can determine, based on the cell state of the one or more cells and the time series image data set, a relationship between one or more subcellular or cellular motions or processes depicted in the time series image data set and the cell state of the one or more cells.

[0131] In some embodiments, a user of the system can review the time series image data set based on the determined cell state, for example, to interpret processes and / or morphological or positional features of the cells and / or cellular substructures that led to the determined cell state. For example, a relationship between the determined cell state and any one or more of protein / cargo transport along neurites, changes in neurite network geometry, and motion of large lipid aggregates within the cell, particularly rates of diffusion and mass transport, which can serve as a proxy for metabolic activity, can be identified. In some embodiments, an exemplary system can be configured to map networks of one or both of axons and neurites to cells in the one or more cells based on the time series image data set and the cell state of the one or more cells.

[0132] In some embodiments, the system is configured to determine a rate of change of the cell state. In some embodiments, the rate of change of the cell state is indicative of a change in a cellular process, where the cellular process comprises any one or more of cargo transport, organelle assembly, and organelle disassembly. In other words, a change in the cell state or a rate of change of the cell state can serve as a proxy for potential cellular changes.

[0133] In some embodiments, the summary embeddings can be used to determine the impact of a perturbation (e.g., a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof) in slowing or reversing disease progression by detecting changes in the cell morphology classification. In some embodiments, the summary embeddings can be used to generate disease models (e.g., toxicity assessments). In some embodiments, the summary embeddings can be used to study features of interest of a cell culture, such as proliferation rate, cell health, cell development, etc., which can then be used to optimize culture conditions, whether in real-time for the cell culture for which the embeddings are being generated, or in subsequent cell cultures. In some embodiments, the summary embeddings can be used to identify conditions that cause cells to develop towards a desired cell state, and to identify combination therapies and drug synergies in chemical and genetic screens.

[0134] Figure 2An example process for determining a cell state of one or more cells of a subject is illustrated in accordance with some embodiments. Process 200 is performed, for example, using one or more electronic devices implementing a software platform. In some examples, process 200 is performed using one or more electronic devices. In some embodiments, process 200 is performed using a client-server system, with blocks of process 200 allocated between the server and one or more client devices in any manner. Thus, while portions of process 200 are described herein as being performed by particular devices, it is understood that process 200 is not so limited. In process 200, some blocks are optionally combined, the order of some blocks is optionally changed, and some blocks are optionally omitted. In some examples, additional steps can be performed in connection with process 200. Thus, the operations shown (and described in greater detail below) are exemplary in nature, and, thus, should not be considered to be limiting.

[0135] At block 202, an example system (e.g., one or more electronic devices) receives a time series image dataset depicting one or more cells of a subject. In some embodiments, the time series image dataset includes time series images. In some embodiments, the time series image dataset includes a video clip. In some embodiments, the time series image dataset includes image data acquired at a frame rate of at least four frames per second, at least eight frames per second, or the like.

[0136] In some embodiments, the time series image dataset includes a plurality of phase images, which can be captured using an imager with a frame rate greater than at least 4 frames per second. In some embodiments, the plurality of phase images are captured using an imager with a frame rate of approximately 40 frames per second. In some embodiments, the time series image dataset includes a plurality of phase images, which can be captured using an imager with a frame rate greater than at least 1 frame per second. In some embodiments, the plurality of phase images are captured using an imager with a frame rate of approximately 8 frames per second. For example, the plurality of phase images can be captured using a high-speed phase imaging device, such as a Phasics SID4-sC8 camera accessory or a Nanolive 3D Cell Explorer 96 focus, which are capable of recovering quantitative phase retardation (in radians). In some embodiments, the example system receives two or more burst images at a single location using a high-speed phase imaging device. In some embodiments, the phase images provide information about the refractive index of the entire cell by imaging the refractive index of the entire cell. Unbiased measurement thereby providing an unbiased measurement (e.g., as compared to imaging and tracking of intracellular labels described below with respect to fluorescence imaging).

[0137] In some embodiments, the time series image dataset includes a plurality of fluorescence images that can be captured using an imager having a frame rate of at least 4 frames per second. In some embodiments, the plurality of fluorescence images can be captured using an imager having a frame rate of approximately 100 frames per second. In some embodiments, the time series image dataset includes a plurality of fluorescence images that can be captured using an imager having a frame rate of at least 1 frame per second. In some embodiments, the plurality of fluorescence images can be captured using an imager having a frame rate of approximately 8 frames per second. Fluorescence imaging enables immunochemical readouts by applying tags to cells that attach to various substances that aggregate lipid droplets or other substances that aggregate in specific regions of the cell. Thus, the tags are able to track the movement of the target substances within the cell. For example, the plurality of fluorescence images can be captured using a scientific complementary metal-oxide-semiconductor (sCMOS) camera, such as a Hamamatsu ORCA-Fusion digital scientific CMOS camera or an Andor Zyla 5.5. Acquisition of the time series image dataset can be implemented using open source microscope software and custom python layers for implementing specific hardware controls.

[0138] In some embodiments, the time series image dataset includes a plurality of phase images captured prior to the plurality of fluorescence images. This acquisition configuration can enable cross-registration and validation of phase and immuno-histochemical readouts at the same location (e.g., a region of interest of a cell), which can be used as a tag to confirm the biological origin of the refractive index change. Moreover, the fluorescence channel can be used as a phenotypic readout in many cases. Both fluorescence imaging and quantitative phase delay can be used as data to train machine learning classifiers.

[0139] In some embodiments, the one or more cells depicted in the time series image dataset include one or more living biological cells. In some embodiments, the one or more living biological cells include one or more mammalian cells. In some embodiments, the one or more living biological cells include one or more neurons. In some embodiments, the one or more living biological cells can include healthy cells, pathologic cells, or any combination thereof. For example, the one or more cells can include any of the cells described in the Examples section below.

[0140] In some embodiments, the one or more living biological cells include one or more genetic mutations that can be selected from the group consisting of a deletion mutation, an insertion mutation, a substitution mutation, a missense mutation, a nonsense mutation, and a frameshift mutation. The one or more living biological cells including the one or more genetic mutations can have a phenotypic difference compared to a healthy cell that does not include the one or more genetic mutations. The phenotypic difference can include a difference in metabolic activity, cell kinetics, cell morphology, or any combination thereof.

[0141] In some embodiments, the time series image dataset depicts a single cell. In some embodiments, the single cell is identified using an image segmentation model. In some embodiments, the time series image dataset is pre-processed upon receipt according to one or more pre-processing steps, e.g., identifying the single cell in the time series image dataset and centering image tiles around the single cell. In some embodiments, the time series image dataset depicts a single cell and one or more subcellular structures and / or processes. For example, the time series image dataset can depict a single cell and one or more neurites. The time series image dataset can depict a single cell and one or more axons and / or dendrites. Thus, the time series image dataset can depict a single cell and one or more regions of interest associated with the cell.

[0142] At block 204, the system can determine an embedding sequence by inputting the time series image dataset into the trained machine learning model. In some embodiments, the embedding sequence includes an embedding sequence of the single cell depicted in the time series image dataset, the embeddings can represent morphological or positional features of the cell or cell substructure and / or process, independent of the time at which the respective cell images in the time series image data were taken. In other words, each embedding can represent morphological or positional features of the cell or cell substructure and / or process at a particular point in time (e.g., a single frame in the time series image dataset), such that the sequence represents multiple morphological or positional features of the cell or cell substructure and / or process at multiple respective points in time.

[0143] In some embodiments, the trained machine learning model of block 204 includes a self-supervised machine learning model. In some embodiments, the machine learning model is trained using an unlabelled training dataset, which can include multiple images of biological samples. For example, the self-supervised learning model can be a DINO Vision Transformer, a SimCLR model, or any other model that learns from unlabelled sample data. In some embodiments, the machine learning model is pre-trained using unlabelled images that do not depict biological samples, and retrained using unlabelled images of biological samples.

[0144] At block 206, the system can determine an summary embedding based on the sequence of embeddings. In some embodiments, the summary embedding includes a temporal dimension based on temporal information associated with the sequence of embeddings. In some embodiments, the temporal information associated with the sequence of embeddings includes a temporal relationship between a first embedding in the sequence of embeddings and a second embedding in the sequence of embeddings. In some embodiments, the temporal information of a set of embeddings is compared to another set of embeddings in the measurement sequence. For example, the sets can be compared using Fourier or wavelet analysis and can be used to analyze trends occurring at different temporal frequencies. In some embodiments, the temporal information associated with the sequence of embeddings includes a timestamp associated with each embedding in the sequence of embeddings.

[0145] The summary embedding can account for dynamics (e.g., subcellular motion) between subsequent frames in the time series image dataset. Thus, the summary embedding accounts for a time axis in the time series image dataset. In some embodiments, the summary embedding determined based on the sequence of embeddings can be determined by inputting the sequence of embeddings into a trained machine learning model. The trained machine learning model of block 206 can include a transformer model, a recurrent neural network ("RNN"), a long short-term memory network ("LSTM"), or any combination thereof.

[0146] In some embodiments, the summary embedding is determined directly from the time series image dataset by inputting the time series image dataset into a trained machine learning model that can directly process a set of imaging sequences (e.g., a model that can directly process video data). In such embodiments, the machine learning model can include a video transformer network or a three-dimensional convolutional neural network, as shown below in Figure 3B

[0147] At block 208, the system can determine a cell state of one or more cells of the subject by inputting the summary embedding into a trained machine learning model. In some embodiments, the trained machine learning model of block 208 includes a trained classifier. In some embodiments, the trained machine learning model of block 208 includes a linear regression classifier configured to receive the summary embedding and output a cell state classification (e.g., healthy or diseased). In some embodiments, the machine learning model of block 208 includes a regression classifier, such as a linear regression classifier or a logistic regression classifier, configured to receive the summary embedding and output a degree of disease state.

[0148] In some embodiments, the cell state indicates a disease state (i.e., a mutation state), a healthy state (i.e., a wild-type state), or a degree of disease state. The cell state can indicate a level of metabolic activity or a kinetic state. In some embodiments, the cell state indicates an accumulation of lipids. ​

[0149] In some embodiments, the system is configured to determine one or more relationships between the determined cell state and morphological or positional features of cells or subcellular structures depicted in, for example, the time series image data. For example, the system can determine, based on the cell state of the one or more cells and the time series image data set, a relationship between one or more time-varying morphological features depicted in the time series image data set and the cell state of the one or more cells. In some embodiments, the system can determine, based on the cell state of the one or more cells and the time series image data set, a relationship between one or more subcellular or cellular motions or processes depicted in the time series image data set and the cell state of the one or more cells.

[0150] In some embodiments, a user of the system can review the time series image data set based on the determined cell state, for example, to interpret processes and / or morphological or positional features of cells and / or subcellular structures that led to the determined cell state. For example, a relationship between the determined cell state and any one or more of protein / cargo transport along neurites, changes in neurite network geometry, and motion of large lipid aggregates within the cell, particularly rates of diffusion and mass transport, which can serve as a proxy for metabolic activity, can be identified. In some embodiments, an exemplary system can be configured to map networks of one or both of axons and neurites to cells in the one or more cells based on the time series image data set and the cell state of the one or more cells.

[0151] In some embodiments, the system can be configured to determine a rate of change of the cell state. In some embodiments, the rate of change of the cell state is indicative of a change in a cellular process, where the cellular process comprises any one or more of cargo transport, organelle assembly, and organelle disassembly. In other words, a change in the cell state or a rate of change of the cell state can serve as a proxy for potential cellular changes.

[0152] As described above, in some embodiments, the embedding sequence can be determined by inputting the time series image data set into a first trained machine learning model, and the summary embedding can be determined based on the embedding sequence by inputting the embedding sequence into a second machine learning model. In other embodiments, the summary embedding can be determined directly by inputting the time series image data set into a trained machine learning model configured to directly process the time series image data to generate the summary embedding. Figure 3A and 3B Each of the above-described configurations is illustrated.

[0153] Figure 3AExemplary first trained machine learning models for determining embedding sequences, exemplary second trained machine learning models for determining summary embeddings, and third trained machine learning models for classifying summary embeddings determined by the second trained machine learning models are described in accordance with some embodiments.

[0154] In some embodiments, an exemplary system (e.g., one or more electronic devices) receives a time series image dataset 302a depicting one or more cells. In some embodiments, the time series image dataset includes time series images. In some embodiments, the time series image dataset includes a video clip. In some embodiments, the time series image dataset includes image data acquired at a frame rate of at least four frames per second. In some embodiments, the time series image dataset includes image data acquired at a frame rate of at least eight frames per second. The time series image dataset can include a plurality of phase images, which can be captured using an imager with a frame rate of at least four frames per second. In some embodiments, the plurality of phase images are captured using an imager with a frame rate of approximately forty frames per second. In some embodiments, the time series image dataset includes a plurality of phase images, which can be captured using an imager with a frame rate of at least one frame per second. In some embodiments, the plurality of phase images are captured using an imager with a frame rate of approximately eight frames per second. For example, the plurality of phase images can be captured using a high-speed phase imaging device, such as a Phasics SID4-sC8 camera accessory or a Nanolive 3D Cell Explorer 96focus, which are capable of recovering quantitative phase retardation (in radians). In some examples, the exemplary system receives two or more burst images at a single location using a high-speed phase imaging device.

[0155] The time series image dataset can include a plurality of fluorescence images, which are captured using an imager with a frame rate of at least four frames per second. In some embodiments, the plurality of fluorescence images are captured using an imager with a frame rate of approximately one hundred frames per second. In some embodiments, the time series image dataset includes a plurality of fluorescence images, which can be captured using an imager with a frame rate of at least one frame per second. In some embodiments, the plurality of fluorescence images can be captured using an imager with a frame rate of approximately eight frames per second. The plurality of fluorescence images are captured using a scientific complementary metal-oxide-semiconductor (sCMOS) camera, such as a Hamamatsu ORCA-Fusion digital scientific CMOS camera or an Andor Zyla 5.5. Acquisition of the time series image dataset can be implemented using open-source microscope software and custom python layers for implementing specific hardware controls.

[0156] In some embodiments, the time series image dataset includes a plurality of phase images captured prior to the plurality of fluorescence images. As described above with reference to FIG. 3A, the exemplary system can determine a plurality of phase embeddings for the plurality of phase images and a plurality of fluorescence embeddings for the plurality of fluorescence images. Figure 2As noted, such acquisition configurations can enable cross-registration of phase and immunohistochemical readouts at the same location (e.g., a region of interest of a cell), which can serve as a tag to confirm the biological origin of the refractive index change. Moreover, the fluorescence channel can in many cases serve as a phenotypic readout. Both fluorescence imaging and quantitative phase delay can be used as data to train a machine learning classifier.

[0157] In some embodiments, the one or more cells depicted in the time series image dataset comprise one or more living biological cells. The one or more living biological cells can comprise one or more mammalian cells. The one or more living biological cells can comprise one or more neurons. The one or more living biological cells can comprise healthy cells, pathologic cells, or any combination thereof.

[0158] In some embodiments, the one or more living biological cells comprise one or more genetic mutations, which can be selected from the group consisting of a deletion mutation, an insertion mutation, a substitution mutation, a missense mutation, a nonsense mutation, and a frameshift mutation. The one or more living biological cells comprising one or more genetic mutations can have a phenotypic difference compared to a healthy cell that does not comprise one or more genetic mutations. The phenotypic difference can comprise a difference in metabolic activity, cell kinetics, cell morphology, or any combination thereof.

[0159] In some embodiments, the time series image dataset is pre-processed upon receipt according to one or more pre-processing steps, e.g., identifying individual cells in the time series image dataset and centering image tiles around the individual cells. Accordingly, in some embodiments, the time series image dataset depicts individual cells. The individual cells can be identified using an image segmentation model. In some embodiments, the time series image dataset depicts individual cells and one or more subcellular structures and / or processes. For example, the time series image dataset can depict individual cells and one or more neurites. The time series image dataset can depict individual cells and one or more axons and / or dendrites. Accordingly, the time series image dataset can depict individual cells and one or more regions of interest associated with the cells.

[0160] In some embodiments, the system can determine the embedding sequence 306a by inputting the time series image dataset into the first trained machine learning model 304a. In some embodiments, the embedding sequence 306a comprises an embedding sequence of the individual cells depicted in the time series image dataset, the embeddings can represent morphological or positional features of the cells or cell substructures and / or processes, independent of the time at which the respective cell images in the time series image dataset were taken. In other words, each embedding can represent morphological or positional features of the cells or cell substructures and / or processes at a particular point in time (e.g., a single frame in the time series image dataset), such that the sequence represents multiple morphological or positional features of the cells or cell substructures and / or processes at multiple respective points in time.

[0161] In some embodiments, the first trained machine learning model 304a comprises a self-supervised machine learning model. The first trained machine learning model can be trained using an unlabelled training dataset, which can include a plurality of images of biological samples. In some embodiments, the self-supervised learning model is a DINO Vision Transformer, a SimCLR model, or any other model that learns from unlabelled sample data. In some embodiments, the first trained machine learning model is pre-trained using unlabelled images that do not depict biological samples, and retrained using unlabelled images of biological samples.

[0162] The system can determine the summary embedding 310a based on the sequence of embeddings 306a. In some embodiments, the summary embedding includes a temporal dimension based on temporal information associated with the sequence of embeddings. In some embodiments, the temporal information associated with the sequence of embeddings includes a temporal relationship between a first embedding in the sequence of embeddings and a second embedding in the sequence of embeddings. In some embodiments, the temporal information associated with the sequence of embeddings includes an order relationship between a first embedding in the sequence of embeddings and a second embedding in the sequence of embeddings. In some embodiments, the temporal information associated with the sequence of embeddings includes a timestamp associated with each embedding in the sequence of embeddings.

[0163] The summary embedding accounts for dynamics (e.g., subcellular motion) that occur between subsequent frames in the time series image dataset. Thus, the summary embedding accounts for a time axis in the time series image dataset. In some embodiments, the system determines the summary embedding 310a by inputting the sequence of embeddings 306a into a second trained machine learning model 308a. The second trained machine learning model 308a can include a transformer model, a recurrent neural network (“RNN”), a long short-term memory network (“LSTM”), or any combination thereof.

[0164] In some embodiments, the system can determine a value 314a (e.g., a cell state) by inputting the summary embedding 310a into a third trained machine learning model 312a. In some embodiments, the trained machine learning model 312a includes a linear regression classifier configured to receive the summary embedding and output a cell state classification (e.g., healthy or diseased). In some embodiments, the third trained machine learning model includes a regression classifier, such as a linear regression classifier or a logistic regression classifier, configured to receive the summary embedding and output a degree of disease state.

[0165] In some embodiments, training the machine learning model 304a, the machine learning model 308a, and the machine learning model 312a includes training each model together by inputting a time series image dataset (e.g., a time series of images depicting healthy or diseased cells) into the machine learning model 304a to generate an embedding sequence, inputting the embedding sequence into the machine learning model 308a to generate an abstract embedding based on the embedding sequence, and inputting the abstract embedding into the machine learning model 312a to determine a value (e.g., a classification of a cell state or a degree of a disease state). The classification can then be compared to a true value, and classification errors can be backpropagated.

[0166] Figure 3B Exemplary machine learning models for determining an abstract embedding based on a time series image dataset are described in accordance with some embodiments. In some embodiments, an exemplary system (e.g., one or more electronic devices) receives a time series image dataset 302b depicting one or more cells. The time series image dataset can include any of the time series image data described above with reference to Figure 2 and 3A .

[0167] The system can determine an abstract embedding 306b based on the time series image dataset by inputting the time series image dataset into the trained machine learning model 304b. In some embodiments, the abstract embedding includes a temporal dimension based on time information associated with the embedding sequence. In some embodiments, the time information associated with the embedding sequence includes a temporal relationship between a first embedding in the embedding sequence and a second embedding in the embedding sequence. In some embodiments, the time information associated with the embedding sequence includes an order relationship between a first embedding in the embedding sequence and a second embedding in the embedding sequence. In some embodiments, the time information associated with the embedding sequence includes a timestamp associated with each embedding in the embedding sequence.

[0168] In some embodiments, the trained machine learning model 304b includes a video transformer network model or a three-dimensional convolutional neural network trained to directly process a time series image dataset to determine an abstract embedding. As described above with reference to Figure 2 and 3A , the abstract embedding accounts for dynamics (e.g., subcellular motion) that occur between subsequent frames in the time series image dataset. Thus, the abstract embedding accounts for a time axis in the time series image dataset.

[0169] In some embodiments, the system can determine the value 310b (e.g., a cell state) by embedding 306n the summary into a second trained machine learning model 308b. In some embodiments, the trained machine learning model 308b includes a linear regression classifier configured to receive the summary embedding and output a classification of a cell state (e.g., healthy or diseased). In some embodiments, the third trained machine learning model includes a regression classifier, such as a linear regression classifier or a logistic regression classifier, configured to receive the summary embedding and output a degree of a disease state.

[0170] In some embodiments, training the machine learning model 304b and the machine learning model 308b includes training the models together by inputting time series image data (e.g., a time series of images depicting healthy or diseased cells) into the machine learning model 304b to generate a summary embedding based on the time series image data, and inputting the summary embedding into the machine learning model 308b to determine a value (e.g., a classification of a cell state or a degree of a disease state). The classification can then be compared to a true value, and classification errors can be backpropagated.

[0171] Figure 4 Training of an example contrastive learning algorithm is described in accordance with some embodiments. Figure 1 The one or more models used in the system can be Figure 4 one of the encoders in the system. During training, an original image X is obtained. Data transformations or augmentations can be applied to the original image X to obtain two augmented images X i and X j . For example, the system can randomly apply two separate data augmentation operators (e.g., crop, flip, color jitter, grayscale, blur) to obtain X i and X j .

[0172] Each of the two augmented images X i and X j is passed through an encoder to obtain a respective vector representation in a latent space. In the depicted example, the two encoders have shared weights. In some examples, each encoder is implemented as a neural network. For example, the encoders can be implemented using a variant of a residual neural network (“ResNet”) architecture. As shown, the two encoders output h i (the vector output by the encoder from X i ) and h j (the vector output by the encoder from X j ).

[0173] The two vector representations h i and h j are passed through a projection head to obtain two projections z i and z jIn some examples, the projection head includes a series of non-linear layers (e.g., Dense - Relu - Dense layers) to apply a non-linear transformation to the vector representation to obtain the projection. The projection head amplifies invariant features and maximizes the network’s ability to recognize different transformations of the same image.

[0174] During training, the similarity between two projections z i and z j of the same image is maximized. For example, a loss is computed based on z i and z j and the encoder is updated based on the loss to maximize the similarity between the two latent representations. In some examples, to maximize the consistency (i.e., similarity) between the z projections, the system can define the similarity measure as the cosine similarity:

[0175] In some examples, the system trains the network by minimizing a normalized temperature-scaled cross-entropy loss:

[0176] where τ represents an adjustable temperature parameter. Thus, through training, the encoder learns to output vector representations that preserve the input image’s invariant features while minimizing image-specific features (e.g., imaging angle, resolution, artifacts).

[0177] Figure 5 An example process for modeling disease progression is illustrated. Process 500 is performed, for example, using one or more electronic devices implementing a software platform. In some examples, process 500 is performed using one or more electronic devices. In some embodiments, process 500 is performed using a client-server system, with the blocks of process 500 allocated between the server and one or more client devices in any manner. Thus, although portions of process 500 are described herein as being performed by particular devices, it is understood that process 500 is not so limited. In process 500, some blocks can optionally be combined, the order of some blocks can optionally be changed, and some blocks can optionally be omitted. In some examples, additional steps can be performed in connection with process 500. Thus, the operations shown (and described in greater detail below) are exemplary in nature, and, as such, should not be considered to be limiting.

[0178] At block 502, an example system (e.g., one or more electronic devices) obtains a first plurality of time series image data sets of a first non-zero concentration of live biological cells expressing a first disease state of a disease. In some embodiments, a time series image data set of the first plurality of time series image data sets includes a time series image. In some embodiments, a time series image data set of the first plurality of time series image data sets includes a video clip. In some embodiments, a time series image data set of the first plurality of time series image data sets includes image data acquired at a frame rate of at least four frames per second, at least eight frames per second, or the like.

[0179] In some embodiments, a time series image data set of the first plurality of time series image data sets includes a plurality of phase images. In some embodiments, the plurality of phase images are captured using an imager having a frame rate of at least four frames per second. In some embodiments, the frame rate of the imager used to capture the plurality of phase images is approximately forty frames per second. In some embodiments, a time series image data set includes a plurality of phase images that can be captured using an imager having a frame rate of at least one frame per second. In some embodiments, the plurality of phase images are captured using an imager having a frame rate of approximately eight frames per second. For example, the plurality of phase images can be captured using a high-speed phase imaging device, such as a Phasics SID4-sC8 camera accessory or a Nanolive 3D CellExplorer 96 focus, which are capable of recovering quantitative phase retardation (in radians). In some examples, the example system receives two or more burst images at a single location using a high-speed phase imaging device.

[0180] In some embodiments, a time series image data set of the first plurality of time series image data sets includes a plurality of fluorescence images. In some embodiments, the plurality of fluorescence images are captured using an imager having a frame rate of at least four frames per second. In some embodiments, the plurality of fluorescence images are captured using an imager having a frame rate of approximately one hundred frames per second. In some embodiments, a time series image data set includes a plurality of fluorescence images that can be captured using an imager having a frame rate of at least one frame per second. In some embodiments, the plurality of fluorescence images can be captured using an imager having a frame rate of approximately eight frames per second. For example, the plurality of fluorescence images can be captured using a scientific complementary metal-oxide-semiconductor (sCMOS) camera, such as a Hamamatsu ORCA-Fusion digital scientific CMOS camera or an Andor Zyla 5.5. Acquisition of the time series image data sets can be implemented using open-source microscope software and custom python layers for implementing specific hardware controls.

[0181] In some embodiments, the time-series image dataset in the first plurality of time-series image datasets includes multiple phase images captured before or after multiple fluorescence images. This acquisition configuration enables cross-registration of phase and immunohistochemical readings at the same location (e.g., region of interest of a cell), which can be used as labels to confirm the biological origin of refractive index changes. Furthermore, fluorescence channels can be used as phenotypic readings in many cases. Time-series image data captured using both fluorescence imaging and phase imaging can be used as data for training machine learning classifiers.

[0182] In some embodiments, the time-series image datasets in the first plurality of time-series image datasets are preprocessed upon receipt according to one or more preprocessing steps, such as identifying one or more individual cells in the time-series image datasets and centering image tiles around the individual cells. In some embodiments, the time-series image datasets in the first plurality of time-series image datasets depict individual cells. In some embodiments, individual cells are identified using an image segmentation model. In some embodiments, the time-series image datasets in the first plurality of time-series image datasets depict individual cells and one or more subcellular structures and / or processes. For example, the time-series image datasets in the first plurality of time-series image datasets may depict individual cells and one or more neurites. The time-series image datasets in the first plurality of time-series image datasets may depict individual cells and one or more axons and / or dendrites. Therefore, the time-series image datasets in the first plurality of time-series image datasets may depict individual cells and one or more regions of interest associated with that cell.

[0183] In some embodiments, the first non-zero concentration of living biological cells comprises one or more mammalian cells. In some embodiments, the first non-zero concentration of living biological cells comprises one or more neurons. For example, the living biological cells may include any cells described in the Examples section below.

[0184] In block 504, an exemplary system (e.g., one or more electronic devices) determines a first plurality of embedding sequences by feeding a first plurality of time-series image datasets into a trained machine learning model. In some embodiments, the trained machine learning model includes a self-supervised machine learning model. In some embodiments, the trained machine learning model is trained using an unlabeled training dataset. In some embodiments, the self-supervised learning model is a DINO Vision Transformer, a SimCLR model, or any other model learned from unlabeled sample data. In some embodiments, the unlabeled training dataset includes multiple images of biological samples. In some embodiments, the machine learning model is pre-trained using unlabeled images that do not depict biological samples and retrained using unlabeled images of biological samples.

[0185] In some embodiments, the embedded sequences of the first plurality of embedded sequences include embedded sequences of a single cell depicted in the first plurality of time series image data sets, the embeddings can represent morphological or positional features of the cell or cell substructure and processes, independent of the time at which the respective cell images of the time series image data were acquired. In other words, each embedding can represent morphological or positional features of the cell or cell substructure and processes at a particular point in time (e.g., a single frame in the time series image data set), such that the sequence represents a plurality of morphological or positional features of the cell or cell substructure at a plurality of respective points in time.

[0186] At block 506, the example system (e.g., one or more electronic devices) determines a first plurality of summary embeddings based on the first plurality of embedded sequences. In some embodiments, a summary embedding of the first plurality of summary embeddings includes a temporal dimension based on temporal information associated with a first embedded sequence of the first plurality of embedded sequences. In some embodiments, the temporal information associated with the first embedded sequence includes a temporal relationship between a first embedding of the first embedded sequence and a second embedding of the first embedded sequence. In some embodiments, the temporal information associated with the first embedded sequence includes an order relationship between a first embedding of the first embedded sequence and a second embedding of the first embedded sequence. In some embodiments, the temporal information associated with the first embedded sequence includes a timestamp associated with each embedding of the first embedded sequence.

[0187] In some embodiments, a summary embedding of the first plurality of summary embeddings accounts for dynamics (e.g., subcellular motion) that occur between subsequent frames in a time series image data set of the first plurality of time series image data sets. Thus, the summary embeddings account for a time axis in the time series image data sets. In some embodiments, the first plurality of summary embeddings is determined based on the first plurality of embedded sequences by inputting the first plurality of embedded sequences into a trained machine learning model. The trained machine learning model can include a transformer model, a recurrent neural network (“RNN”), a long short-term memory network (“LSTM”), or any combination thereof.

[0188] As noted above, in some embodiments, the summary embeddings can be determined directly from the time series image data by inputting the time series image data into a trained machine learning model that can directly process sets of imaging sequences (e.g., a model that can directly process video data). In such embodiments, the machine learning model can include a video transformer network or a three-dimensional convolutional neural network, as shown in Figure 3B

[0189] ​At block 508, the example system (e.g., one or more electronic devices) obtains a second plurality of time series image data sets of a second non-zero concentration of live biological cells expressing a second disease state of a disease. In some embodiments, a time series image data set of the second plurality of time series image data sets includes a time series image. In some embodiments, a time series image data set of the second plurality of time series image data sets includes a video clip. In some embodiments, a time series image data set of the second plurality of time series image data sets includes image data acquired at a frame rate of at least four frames per second, at least eight frames per second, or the like.

[0190] In some embodiments, a time series image data set of the second plurality of time series image data sets includes a plurality of phase images. In some embodiments, the plurality of phase images are captured using an imager having a frame rate of at least four frames per second. In some embodiments, the frame rate of the imager used to capture the plurality of phase images is approximately forty frames per second. In some embodiments, a time series image data set includes a plurality of phase images that can be captured using an imager having a frame rate of at least one frame per second. In some embodiments, the plurality of phase images are captured using an imager having a frame rate of approximately eight frames per second. For example, the plurality of phase images can be captured using a high-speed phase imaging device, such as a Phasics SID4-sC8 camera accessory or a Nanolive 3D CellExplorer 96 focus, which are capable of recovering quantitative phase retardation (in radians). In some examples, the example system receives two or more burst images at a single location using a high-speed phase imaging device.

[0191] In some embodiments, a time series image data set of the second plurality of time series image data sets includes a plurality of fluorescence images. In some embodiments, the plurality of fluorescence images are captured using an imager having a frame rate of at least four frames per second. In some embodiments, the plurality of fluorescence images are captured using an imager having a frame rate of approximately one hundred frames per second. In some embodiments, a time series image data set includes a plurality of fluorescence images that can be captured using an imager having a frame rate of at least one frame per second. In some embodiments, the plurality of fluorescence images can be captured using an imager having a frame rate of approximately eight frames per second. For example, the plurality of fluorescence images can be captured using a scientific complementary metal-oxide-semiconductor (sCMOS) camera, such as a Hamamatsu ORCA-Fusion digital scientific CMOS camera or an Andor Zyla 5.5. Acquisition of the time series image data sets can be implemented using open-source microscope software and custom python layers for implementing specific hardware controls.

[0192] In some embodiments, the time series image data sets of the second plurality of time series image data sets include a plurality of phase images captured prior to the plurality of fluorescence images. This acquisition configuration can enable cross-registration of phase and immunohistochemistry readouts at the same location (e.g., a region of interest of a cell), which can serve as a tag to confirm the biological origin of the refractive index change. In addition, the fluorescence channels can serve as phenotypic readouts. Both fluorescence imaging and quantitative phase retardation can be used as data to train a machine learning classifier.

[0193] In some embodiments, the time series image data sets of the second plurality of time series image data sets are pre-processed upon receipt according to one or more pre-processing steps, e.g., identifying one or more individual cells in the time series image data sets and centering image tiles around the individual cells. In some embodiments, the time series image data sets of the second plurality of time series image data sets depict individual cells. In some embodiments, the individual cells are identified using an image segmentation model. In some embodiments, the time series image data sets of the second plurality of time series image data sets depict individual cells and one or more subcellular structures and / or processes. For example, the time series image data sets of the second plurality of time series image data sets can depict individual cells and one or more neurites. The time series image data sets of the second plurality of time series image data sets can depict individual cells and one or more axons and / or dendrites. Thus, the time series image data sets of the second plurality of time series image data sets can depict individual cells and one or more regions of interest associated with the cells.

[0194] In some embodiments, the second non-zero concentration of live biological cells includes one or more mammalian cells. In some embodiments, the second non-zero concentration of live biological cells includes one or more neurons. In some embodiments, the second non-zero concentration of live biological cells can include any of the cells described in the Examples section below.

[0195] At block 510, the example system (e.g., one or more electronic devices) determines a second plurality of sequences of embeddings by inputting the second plurality of time series image data sets into a trained machine learning model. In some embodiments, the trained machine learning model includes a self-supervised machine learning model. In some embodiments, the trained machine learning model is trained using an unlabelled training dataset. In some embodiments, the self-supervised learning model is a DINO Vision Transformer, a SimCLR model, or any other model that learns from unlabelled sample data. In some embodiments, the unlabelled training dataset includes a plurality of images of the biological sample. In some embodiments, the machine learning model is pre-trained using unlabelled images that do not depict the biological sample and retrained using unlabelled images of the biological sample.

[0196] In some embodiments, the embedding sequences of the second plurality of embedding sequences include embedding sequences of individual cells depicted in the second plurality of time series image data sets, the embeddings can represent morphological or positional features of the cells or cellular substructures and processes, independent of the times at which the respective cell images were acquired in the time series image data. In other words, each embedding can represent morphological or positional features of the cells or cellular substructures and processes at a particular point in time (e.g., a single frame in the time series image data set), such that the sequence represents a plurality of morphological or positional features of the cells or cellular substructures at a plurality of respective points in time.

[0197] At block 512, the example system (e.g., one or more electronic devices) determines a second plurality of summary embeddings based on the second plurality of embedding sequences. In some embodiments, the summary embeddings of the second plurality of summary embeddings include a temporal dimension based on temporal information associated with the second embedding sequences of the second plurality of embedding sequences. In some embodiments, the temporal information associated with the second embedding sequences includes a temporal relationship between a first embedding of the second embedding sequences and a second embedding of the second embedding sequences. In some embodiments, the temporal information associated with the second embedding sequences includes an order relationship between a first embedding of the second embedding sequences and a second embedding of the second embedding sequences. In some embodiments, the temporal information associated with the second embedding sequences includes a timestamp associated with each embedding of the second embedding sequences.

[0198] In some embodiments, the summary embeddings of the second plurality of summary embeddings account for dynamics (e.g., subcellular motion) that occur between subsequent frames in the time series image data of the second plurality of time series image data sets. Thus, the summary embeddings account for a time axis in the time series image data sets. In some embodiments, the second plurality of summary embeddings is determined based on the first plurality of embedding sequences by inputting the first plurality of embedding sequences into a trained machine learning model. The trained machine learning model can include a transformer model, a recurrent neural network (“RNN”), a long short-term memory network (“LSTM”), or any combination thereof.

[0199] As noted above, in some embodiments, the summary embeddings can be determined directly from the time series image data by inputting the time series image data into a trained machine learning model that can directly process sets of imaging sequences (e.g., models that can directly process video data). In such embodiments, the machine learning model can include a video transformer network or a three-dimensional convolutional neural network, as shown in Figure 3B

[0200] ​At block 514, the example system (e.g., one or more electronic devices) generates a disease model based on the first plurality of summary embeddings and the second plurality of summary embeddings. The disease model can represent various states of the disease in a topological space. As described below, for example, in a topological space, the disease model can include a cluster of summary embeddings corresponding to a first disease state (e.g., a healthy state), a cluster of summary embeddings corresponding to a second disease state (e.g., a less severe disease state), a cluster of summary embeddings corresponding to a third disease state (e.g., a more severe disease state), and so on. In some embodiments, generating the disease model includes mapping the first plurality of summary embeddings and the second plurality of summary embeddings into the topological space. At block 516, the example system (e.g., one or more electronic devices) models progression of the disease based on the disease model.

[0201] In some embodiments, the system can further generate one or more representations of the first and second disease states based on the clusters of embeddings in the topological space. The example system can identify a location of a first cluster of summary embeddings based on the first plurality of summary embeddings in the topological space and generate a representation of the first disease state based on the location of the first cluster. The system can further identify a location of a second cluster of summary embeddings based on the second plurality of summary embeddings in the topological space and generate a representation of the second disease state based on the location of the second cluster.

[0202] In some embodiments, the system can further generate a disease axis based on the location of the first cluster and the location of the second cluster. In some embodiments, the disease axis represents a hyperplane that separates the centroid of the first summary embedding cluster from the centroid of the second summary embedding cluster. In some embodiments, the system can further identify one or more embeddings in the topological space based on the disease axis and generate one or more synthetic images using a generative model. In general, any known generative model including a discriminator and a generator can be used. In some embodiments, the generative model can be an autoregressive model, a variational autoencoder (VAE) model, or a generative adversarial network (GAN) model. For example, the system can further identify one or more embeddings in the topological space based on the disease axis and generate one or more synthetic images using a GAN model based on the identified one or more embeddings. For example, an embedding can be identified by selecting a location in the topological space that represents a disease state of interest. In some embodiments, the system can further compare a phenotype depicted in the one or more synthetic images to one or more real images depicting real cells to validate the phenotype depicted in the one or more synthetic images. The phenotype can include neurite length, cell size, motion of one or more particles within the cell, or any combination thereof. In some embodiments, the imaging stage can generate fluorescent images and / or autofluorescent images of live biological cells from the converted brightfield images. Further information on the image transformation model can be found in U.S. Application No. 17 / 480,047, titled “BIOLOGICAL IMAGE TRANSFORMATION USING MACHINE-LEARNING MODELS,” which issued as U.S. Patent No. 11,423,256 on August 23, 2022, the entirety of which is incorporated by reference herein.

[0203] In some embodiments, the system can further determine an effect of a perturbation (e.g., a perturbation agent, such as a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof) on the reversal of one or more pathologically alive biological cells from a disease state. To determine the effect of the perturbation on the reversal of the one or more pathologically alive biological cells from the disease state, the system can obtain a first set of time series image data of the one or more pathologically alive biological cells, determine a first sequence of embeddings by inputting the first set of time series image data into the trained machine learning model, and determine a first summary embedding based on the first sequence of embeddings. The system can further apply the perturbation to the one or more pathologically alive biological cells, after applying the perturbation, the system can obtain a second set of time series image data of the one or more pathologically alive biological cells, determine a second sequence of embeddings by inputting the second set of time series image data into the trained machine learning model, determine a second summary embedding based on the second sequence of embeddings, and determine the effect of the perturbation on the reversal of the one or more pathologically alive biological cells from the disease state based on the first summary embedding, the second summary embedding, and the disease model (e.g., whether the second summary embedding is closer to the healthy state in the disease model, the rate of movement). Similar techniques can be used to evaluate different doses of the same drug.

[0204] In some embodiments, determining the effect of the perturbation includes comparing a location of the first summary embedding in the topological space; a location of the second summary embedding in the topological space; a location of the first cluster; and a location of the second cluster. For example, before applying the perturbation, the first summary embedding can be located near a centroid of the first cluster. After applying the perturbation, the second summary embedding can be located closer to the second cluster, indicating a reversal of the cell from the first disease state to the second disease state.

[0205] In some embodiments, the system can further determine a dose of the perturbation (e.g., a chemical treatment or a therapeutic agent) based on the determination of the effect of the perturbation. In some embodiments, the system can further determine one or more dosing intervals for administering the perturbation based on the determination of the effect of the perturbation. In some embodiments, the perturbation is a chemical treatment, a therapeutic agent, a genetic therapy, a media change, or any combination thereof.

[0206] Figure 6 Exemplary time series image data of IPSC-derived neurons imaged using QPI at 60x magnification is shown. The inset (bottom) shows granule movement along an axon, representing a dynamic process that can be detected with the proposed imaging method.

[0207] Figure 7 Exemplary time series image data of IPSC-derived neurons imaged using QPI at 60x magnification is shown, acquired at 10 Hz and rendered at 1 Hz. A 3D median filter and blob detector algorithm applied to highlight differences between time points in the QPI images are overlaid in red.

[0208] Figure 9 Exemplary intracellular dynamics are illustrated in accordance with some embodiments. Specifically, Figure 9 Processed hepatocytes (HepG2) are depicted, eliciting a known lipid droplet accumulation phenotype. Arrows indicate intracellular dynamics on 0-1 second and 1-10 second time scales.

[0209] Figure 10 Exemplary intracellular dynamics are illustrated in accordance with some embodiments. Specifically, Figure 10 Neurons differentiated from induced pluripotent stem cells (iPSCs) are depicted, showing mass transport along axons and cellular motion on 0-1 second and 1-10 second time scales.

[0210] Figure 8 An exemplary computing device 800 is depicted in accordance with one or more examples of the present disclosure. Device 800 can be a host computer connected to a network. Device 800 can be a client computer or a server. As Figure 8 shown, device 800 can be any suitable type of microprocessor-based device, such as a personal computer, workstation, server, or handheld computing device (portable electronic device), such as a telephone or tablet. The device can include, for example, one or more processors 802, input devices 806, output devices 808, memory 810, and communication devices 804. Input devices 806 and output devices 808 can generally correspond to those described above and can be connectable or integrated with the computer.

[0211] Input devices 806 can be any suitable device that provides input, such as a touchscreen, keyboard or keypad, mouse, or voice recognition device. Output devices 808 can be any suitable device that provides output, such as a touchscreen, haptic device, or speaker.

[0212] Memory 810 can be any suitable device that provides storage, such as electrical, magnetic, or optical storage including RAM, cache, hard drives, or removable storage disks. Communication devices 804 can include any suitable device that enables transmission and reception of signals over a network, such as a network interface chip or device. The components of the computer can be connected in any suitable manner, such as by a physical bus or wirelessly.

[0213] Software 812 can be stored in memory 810 and executed by processor 802, for example, and can include programming that embodies the functionality of the present disclosure (for example, as embodied in the devices described above).

[0214] The software 812 can also be stored and / or transported within any non-transitory computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above. In this context, a computer-readable storage medium can be any medium, such as the memory 810, that can contain or store program code for use by or in connection with an instruction execution system, apparatus, or device.

[0215] The software 812 can also be propagated within any transport medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above. In this context, a transport medium can be any medium that can communicate, propagate or transport program code for use by or in connection with an instruction execution system, apparatus, or device. The transport readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation medium.

[0216] The device 800 can be connected to a network, which can be any suitable type of interconnected communication system. The network can implement any suitable communication protocol and can be protected by any suitable security protocol. The network can include any suitable arrangement of network links, which can enable transmission and reception of network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0217] The device 800 can implement any suitable operating system for running on the network. The software 812 can be written in any suitable programming language, such as C, C++, Java, or Python. In various embodiments, application software embodying the functionality of the present disclosure can be deployed in different configurations, such as in a client / server arrangement or through a web browser as a web-based application or web service, for example.

[0218] While the present disclosure and examples have been fully described with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as included within the scope of the disclosure and examples as defined by the claims. Finally, all publications mentioned in this application are incorporated herein by reference in their entirety.

[0219] Examples

[0220] The present application can be better understood by reference to the following non-limiting examples, which are provided as illustrative examples of the present application. The following examples are provided to more fully demonstrate embodiments of the application and should not be construed as limiting the broad scope of the present application. While certain embodiments of the application have been shown and described herein, it will be apparent to those skilled in the art that many changes, modifications, and substitutions can be made therein without departing from the spirit and scope of the claims. It is to be understood that in the practice of the methods described herein various alternatives to the embodiments described herein can be employed.

[0221] Example 1. Biological applications of the methods provided.

[0222] This example demonstrates the biological context of the methods for determining the cellular state of one or more cells or the methods for modeling disease progression as described herein. In particular, this example demonstrates an induced pluripotent stem cell (iPSC)-derived motor neuron differentiation program that can be used to study neurite growth and axonal transport.

[0223] Differentiation of motor neurons from iPSCs

[0224] The motor neuron differentiation program was adapted from Fernandopulle et al. . Curr Protoc Cell Biol. 2018 Jun;79(l):el51. Doxycycline-induced expression of three neuronal transcription factors ISL1, LHX3 and NGN2 in iPSC lines leads to rapid and efficient motor neuron differentiation that can be used to study axonal degeneration and motor neuron diseases.

[0225] Briefly, iPSCs were cultured in TeSR TM -E8 TM media and passaged with ReLeSR TM (STEMCELL Technologies). hNIL iPSC lines were generated after G418 selection using PiggyBac TM based plasmid transfection. To generate TDP43 wild-type and TDP43 ANLS overexpression iPSC lines, we transfected PiggyBac TM plasmids encoding these constructs and selected iPSC cells with blasticidin. G4C2 repeats were removed from C9orf72 repeat-expanding iPSC lines using CRISPR-Cas9 gene editing.

[0226] iPSCs were then differentiated into day 2 hNIL motor neuron progenitors. 6-well plates were coated with Matrigel® (Corning, #354277) and 70% of iPSC cultures were dissociated with AccutaseTM (STEMCELL Technologies, Cat #07920) were dissociated into single cells and counted. 0.25-0.5 million cells were plated in each well of a 6-well plate using E8 TM medium supplemented with 1 :5000 Rock inhibitor (Y27632, SelleckChem, S1049, 50 mM stock solution). Twenty-four hours after plating, cells were washed once with phosphate buffered saline (PBS) (Gibco) and induced to differentiate with doxycycline induction medium (DMEM / F-12, 10 mM HEPES, IX N2, IX Non-Essential Amino Acids, IX Gluta-Max, 10 mM Rock inhibitor, 0.2 mM Compound E, and 2 mM doxycycline). Forty-eight hours after doxycycline induction, cells were washed once with PBS and induced to differentiate with doxycycline induction medium without Compound E or doxycycline. Cells were fed with half medium three times per week to complete differentiation. TM Cells were treated at 37°C for 5 to 10 minutes. Cells were then pipetted to remove cell clumps and transferred as single progenitor cells. Day 2 progenitor cells can be cryopreserved (2-3 million cells / vial) with CryoStor CS10 (Sigma-Aldrich).

[0227] Day 2 progenitor cells were then plated in 96-well glass plates (96-1.5H-N) at about 20,000 cells / well or 12-well glass plates (P12-1.5H-N) at 200,000 cells / well. Plates were coated with PLO (Sigma P4957-50 mL) overnight at 37°C. The next day, PLO was removed and plates were dried in a BSC hood for 3 hours. Plates were then coated with diluted laminin-511 (5 mL / mL PBS diluted from stock Biogem, BG iMatrix-511). After 1 hour, NGN2 maintenance medium (Neurobasal Plus, IX B27 plus, IX Glutmax TM , BDNF, 1 pg / mL, NT3, 1 pg / mL, and mouse laminin 0.2 pg / mL; supplemented with 0.2 mM Compound E, 2 mM doxycycline, and 1 pg / mL EdU) was added for plating. After 24 hours, cells were washed once with PBS and cultured in fresh NGN2 maintenance medium without Compound E or doxycycline. To complete motor neuron differentiation, cells were fed with half medium three times per week.

[0228] Imaging of cultured motor neurons

[0229] The resulting motor neurons were imaged using the provided methods. Briefly, live images were collected between day 3 and day 40 of differentiation for the above protocol. Imaging times can be optimized according to phenotypic readouts.

[0230] For example, imaging can be used to determine cell state. Alternatively, imaging can be used to generate a disease model to model progression of a disease.

Claims

1. A method for determining a cell state of one or more cells, the method comprising: receiving a time series image data set depicting one or more cells; determining an embedding sequence by inputting the time series image data set into a first trained machine learning model; determining an abstract embedding based on the embedding sequence, the abstract embedding comprising a temporal dimension based on temporal information associated with the embedding sequence; and determining the cell state of the one or more cells of the subject by inputting the abstract embedding into a second trained machine learning model.

2. The method of claim 1, wherein the temporal information associated with the embedding sequence comprises a temporal relationship between a first embedding in the embedding sequence and a second embedding in the embedding sequence.

3. The method of claim 1 or 2, wherein the temporal information associated with the embedding sequence comprises an order relationship between a first embedding in the embedding sequence and a second embedding in the embedding sequence.

4. The method of any one of claims 1-3, wherein the temporal information associated with the embedding sequence comprises a timestamp associated with each embedding in the embedding sequence.

5. The method of any one of claims 1-4, wherein the time series image data set comprises time series images.

6. The method of any one of claims 1-5, wherein the time series image data set comprises a video clip.

7. The method of any one of claims 1-6, wherein the time series image data set comprises image data acquired at a frame rate of at least eight frames per second.

8. The method of any one of claims 1-7, wherein the time series image data set comprises a plurality of phase images.

9. The method of claim 8, wherein the plurality of phase images are captured using an imager having a frame rate of at least one frame per second.

10. The method of claim 9, wherein the frame rate is about eight frames per second.

11. The method of any one of claims 1-10, wherein the time series image data set comprises a plurality of fluorescence images.

12. The method of claim 11, wherein the plurality of fluorescence images are captured using an imager having a frame rate of at least one frame per second.

13. The method of any one of claims 1-12, wherein the cell state is indicative of a disease state, a health state, or a degree of a disease state.

14. The method of claim 13, further comprising: determining, based on the cell state of the one or more cells and the time series image data set, a relationship between one or more time-varying morphological features depicted in the time series image data set and the cell state of the one or more cells.

15. The method of claim 13 or 14, further comprising: determining, based on the cell state of the one or more cells and the time series image data set, a relationship between one or more subcellular or cellular motions or processes depicted in the time series image data set and the cell state of the one or more cells.

16. The method of any one of claims 1-15, wherein the cell state comprises an indication of lipid accumulation.

17. The method of any one of claims 1-15, wherein the cell state is indicative of a level of metabolic activity.

18. The method of any one of claims 1-15, wherein the cell state is indicative of a kinetic state.

19. The method of any one of claims 1-18, wherein the rate of change in cell state is indicative of a change in a cellular process, wherein the cellular process comprises any one or more of cargo transport, organelle assembly, and organelle disassembly.

20. The method of any one of claims 1-19, further comprising mapping a network of one or both of an axon and a neurite to a cell in the one or more cells based on the time series image dataset and the cell state of the one or more cells.

21. The method of any one of claims 1-20, wherein the first trained machine learning model comprises a self-supervised machine learning model.

22. The method of any one of claims 1-21, wherein the first trained machine learning model is trained using an unlabelled training dataset.

23. The method of claim 22, wherein the unlabelled training dataset comprises a plurality of images of biological samples.

24. The method of any one of claims 1-23, wherein the first machine learning model is pre-trained using unlabelled images that do not depict biological samples and retrained using unlabelled images of biological samples.

25. The method of any one of claims 1-24, wherein determining the summary embedding based on the embedded sequence comprises: inputting the embedding sequence into a third trained machine learning model.

26. The method of claim 25, wherein the third trained machine learning model comprises: transformer model, recurrent neural network ("RNN"), long short-term memory network ("LSTM"), or any combination thereof.

27. The method of any one of claims 1-26, wherein the second trained machine learning model comprises a trained classifier.

28. The method of any one of claims 1-27, wherein the second trained machine learning model comprises a trained regression model.

29. The method of any one of claims 1-28, wherein the time series image dataset depicts a single cell.

30. The method of claim 29, wherein the single cell is identified using an image segmentation model.

31. The method of any one of claims 1-30, wherein the one or more cells comprise one or more living biological cells.

32. The method of claim 31, wherein the one or more living biological cells comprise one or more mammalian cells.

33. The method of claim 31, wherein the one or more living biological cells comprise one or more neurons.

34. The method of claim 31, wherein the one or more living biological cells comprise healthy cells, pathologic cells, or any combination thereof.

35. The method of claim 31, wherein the one or more living biological cells comprise one or more genetic mutations.

36. The method of claim 35, wherein the one or more genetic mutations are selected from the group consisting of a deletion mutation, an insertion mutation, a substitution mutation, a missense mutation, a nonsense mutation, and a frameshift mutation.

37. The method of claim 35, wherein the one or more living biological cells have a phenotypic difference compared to a healthy cell that does not comprise the one or more genetic mutations.

38. The method of claim 37, wherein the phenotypic difference comprises a difference in metabolic activity, cellular dynamics, cell morphology, or any combination thereof.

39. A non-transitory computer-readable storage medium for determining a cellular state of one or more cells, the storage medium storing one or more programs comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to: receive a time-series image dataset depicting one or more cells; determine an embedding sequence by inputting the time-series image dataset into a first trained machine learning model; determine an abstract embedding based on the embedding sequence, the abstract embedding comprising a temporal dimension based on temporal information associated with the embedding sequence; and determine a cellular state of the one or more cells of the subject by inputting the abstract embedding into a second trained machine learning model.

40. A system for determining a cellular state of one or more cells, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for: receiving a time-series image dataset depicting one or more cells; determining an embedding sequence by inputting the time-series image dataset into a first trained machine learning model; determining an abstract embedding based on the embedding sequence, the abstract embedding comprising a temporal dimension based on temporal information associated with the embedding sequence; and determining a cellular state of the one or more cells of the subject by inputting the abstract embedding into a second trained machine learning model.

41. A method for modeling progression of a disease, the method comprising: obtaining a first plurality of time-series image datasets of a first non-zero concentration of live biological cells expressing a first disease state of a disease; determining a first plurality of embedding sequences by inputting the first plurality of time-series image datasets into a trained machine learning model; determining a first plurality of abstract embeddings based on the first plurality of embedding sequences, wherein an abstract embedding of the first plurality of abstract embeddings comprises a temporal dimension based on temporal information associated with a first embedding sequence of the first plurality of embedding sequences; obtaining a second plurality of time-series image datasets of a second non-zero concentration of live biological cells expressing a second disease state of the disease; determining a second plurality of embedding sequences by inputting the second plurality of time-series image datasets into the trained machine learning model; determining a second plurality of abstract embeddings based on the second plurality of embedding sequences, wherein an abstract embedding of the second plurality of abstract embeddings comprises a temporal dimension based on temporal information associated with a second embedding sequence of the second plurality of embedding sequences; generating a disease model based on the first plurality of abstract embeddings and the second plurality of abstract embeddings; and modeling progression of the disease based on the disease model.

42. The method of claim 41, wherein the temporal information associated with the first embedding sequence comprises a temporal relationship between a first embedding in the first embedding sequence and a second embedding in the first embedding sequence.

43. The method of claim 41 or 42, wherein the temporal information associated with the first embedding sequence comprises an order relationship between a first embedding in the first embedding sequence and a second embedding in the first embedding sequence. ​ 44. The method of any one of claims 41-43, wherein the temporal information associated with the first and second sequences of embeddings comprises a timestamp associated with each embedding in the first sequence of embeddings and the second sequence of embeddings.

45. The method of claim 41, wherein generating a disease model comprises: mapping the first plurality of summary embeddings and the second plurality of summary embeddings into a topological space.

46. The method of claim 45, further comprising: identifying a location of a first cluster of summary embeddings based on the first plurality of summary embeddings in the topological space; generating a representation of a first disease state based on the location of the first cluster; identifying a location of a second cluster of summary embeddings based on the second plurality of summary embeddings in the topological space; and generating a representation of a second disease state based on the location of the second cluster.

47. The method of claim 46, further comprising: generating a disease axis based on the location of the first cluster and the location of the second cluster.

48. The method of claim 47, wherein the disease axis represents a hyperplane separating a centroid of the first cluster of summary embeddings from a centroid of the second cluster of summary embeddings.

49. The method of claim 47, further comprising: identifying one or more embeddings in the topological space based on the disease axis.

50. The method of claim 49, further comprising: generating one or more synthetic images based on the identified one or more embeddings using the GAN model.

51. The method of claim 50, further comprising: comparing a phenotype depicted in the one or more synthetic images to one or more real images depicting real cells to validate the phenotype depicted in the one or more synthetic images.

52. The method of claim 51, wherein the phenotype comprises neurite length, cell size, motion of one or more particles within the cell, or any combination thereof.

53. The method of any one of claims 46-52, further comprising: obtaining a first set of time series image data of the one or more morbid living biological cells; determining a first sequence of embeddings by inputting the first set of time series image data into the trained machine learning model; determining a first summary embedding based on the first sequence of embeddings; applying a perturbation to the one or more morbid living biological cells; after applying the perturbation, obtaining a second set of time series image data of the one or more morbid living biological cells; determining a second sequence of embeddings by inputting the second set of time series image data into the trained machine learning model; determining a second summary embedding based on the second sequence of embeddings; and determining an effect of the perturbation on the reversal of the one or more morbid living biological cells from the disease state based on the first summary embedding, the second summary embedding, and the disease model.

54. The method of claim 53, wherein determining the effect of the perturbation comprises: comparing: a location of the first summary embedding in the topological space; a location of the second summary embedding in the topological space; the location of the first cluster; and the location of the second cluster.

55. The method of any one of claims 53-54, wherein the perturbation is a chemical treatment, a therapeutic agent, a genetic modification, a media change, or any combination thereof.

56. The method of any one of claims 53-55, further comprising: determining a dosage of the perturbation based on the determination of the effect of the perturbation.

57. The method of any one of claims 53-56, further comprising: determining one or more dosing intervals for administering the perturbation based on the determination of the effect of the perturbation.

58. The method of any one of claims 41-57, wherein the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets comprises time series images.

59. The method of any one of claims 41-58, wherein the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets comprises a video clip.

60. The method of any one of claims 41-59, wherein the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets comprises image data acquired at a frame rate of at least eight frames per second.

61. The method of any one of claims 41-60, wherein the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets comprises a plurality of phase images.

62. The method of claim 61, wherein the plurality of phase images are captured using an imager having a frame rate of at least one frame per second.

63. The method of claim 62, wherein the frame rate is about eight frames per second.

64. The method of any one of claims 41-63, wherein the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets comprises a plurality of fluorescence images.

65. The method of claim 64, wherein the plurality of fluorescence images are captured using an imager having a frame rate of at least one frame per second.

66. The method of any one of claims 41-65, wherein the trained machine learning model comprises a self-supervised machine learning model.

67. The method of any one of claims 41-66, wherein the trained machine learning model is trained using an unlabeled training data set.

68. The method of claim 67, wherein the unlabeled training data set comprises a plurality of images of biological samples.

69. The method of any one of claims 41-68, wherein the machine learning model is pre-trained using unlabeled images that do not depict biological samples and retrained using unlabeled images of biological samples.

70. The method of any one of claims 41-69, wherein the set of time series image data in the first plurality of time series image data sets and the second plurality of time series image data sets depicts a single cell.

71. The method of claim 70, wherein the single cell is identified using an image segmentation model.

72. The method of any one of claims 41-71, wherein the live biological cells comprise one or more mammalian cells.

73. The method of any one of claims 41-72, wherein the live biological cells comprise one or more neurons.

74. A non-transitory computer-readable storage medium for modeling disease progression, the storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to: obtain a first plurality of time series image data sets of a first non-zero concentration of live biological cells expressing a first disease state of a disease; determine a first plurality of embedding sequences by inputting the first plurality of time series image data sets into a trained machine learning model; ​ determine a first plurality of summary embeddings based on the first plurality of embedding sequences, wherein a summary embedding of the first plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a first embedding sequence of the first plurality of embedding sequences; obtain a second plurality of time series image datasets of a second non-zero concentration of live biological cells expressing a second disease state of the disease; determine a second plurality of embedding sequences by inputting the second plurality of time series image datasets into the trained machine learning model; determine a second plurality of summary embeddings based on the second plurality of embedding sequences, wherein a summary embedding of the second plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a second embedding sequence of the second plurality of embedding sequences; generate a disease model based on the first plurality of summary embeddings and the second plurality of summary embeddings; and model progression of the disease based on the disease model.

75. A system for modeling progression of a disease, comprising: one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: obtaining a first plurality of time series image datasets of a first non-zero concentration of live biological cells expressing a first disease state of the disease; determining a first plurality of embedding sequences by inputting the first plurality of time series image datasets into the trained machine learning model; determine a first plurality of summary embeddings based on the first plurality of embedding sequences, wherein a summary embedding of the first plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a first embedding sequence of the first plurality of embedding sequences; obtaining a second plurality of time series image datasets of a second non-zero concentration of live biological cells expressing a second disease state of the disease; determining a second plurality of embedding sequences by inputting the second plurality of time series image datasets into the trained machine learning model; determine a second plurality of summary embeddings based on the second plurality of embedding sequences, wherein a summary embedding of the second plurality of summary embeddings comprises a temporal dimension based on temporal information associated with a second embedding sequence of the second plurality of embedding sequences; generate a disease model based on the first plurality of summary embeddings and the second plurality of summary embeddings; and model progression of the disease based on the disease model.

Images