Systems and methods for image-based cell segmentation, cell division detection, and cell tracking

Abstract

The present application provides methods and systems for image-based cell segmentation detection, image-based cell division detection, and image based cell tracking comprising receiving image data and applying machine learning algorithms to extract cell segments, division events, and cell tracks through images.

Description

Docket No.78106-20008.40 SYSTEMS AND METHODS FOR IMAGE-BASED CELL SEGMENTATION, CELL DIVISION DETECTION, AND CELL TRACKING CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No.63 / 516,782 filed July 31, 2023, the entire contents of which are incorporated herein by reference. FIELD

[0002] The present disclosure relates generally to computer-imaging based analysis of cells, and more specifically to computer-imaging based cell segmentation, detection of cell division events, and tracking of live cells between images to generate single cell trajectories that collectively enable downstream analysis and forecasting of cell behaviors from time course datasets. BACKGROUND

[0003] High-content screening (HCS) with computer-based image analysis is widely used in drug development to profile drug mechanism of action and target biology in scalable cell- based assay systems. HCS uses carefully selected fluorescent labels (FLs) to tag cellular components and biological information of interest. Fluorescence microscopy imaging is then used to visualize the FLs in the populations of cells imaged in the HCS assay. Cell instance segmentation is a key part of HCS analyses because it enables the quantification of assay readouts at the single-cell rather than at the population level. Cell instance segmentation relies on the use of FLs with a primary function to label a specific cell compartment, typically the nucleus and / or cytoplasm. Such FLs are often referred to as structural FLs, meaning they mark proteins with constant expression and localization. In comparison, non-structural FLs highlight variable-expression proteins that are often found in signaling pathways and may be incorporated into biosensors of their activity and that are of interest in studies to understand cell perturbations that impact health, disease and drug mechanisms of action.

[0004] Extending HCS to incorporate the study of individual cells over time through timecourse imaging of live cells followed by computer-based image analysis of the resulting assay data presents the opportunity to interrogate the dynamic cellular behaviors in a range of contexts relevant to health, disease and drug discovery. The resulting time-resolved single cell profiles can offer critical insights into the mechanisms of action (MoA) of drugs; toxicity liabilities associated with a drug or drug target and the dynamic cellular adaptations that underpin disease progression and therapeutic resistance, thus enabling a deeper 1sf-6002883Docket No.78106-20008.40 comprehension of cellular and signaling events in disease responses to treatment. Single-cell behavioral data from live-cell imaging will unlock the identification of novel biomarkers, actionable drug targets and therapeutics through more accurate and comprehensive characterization of underlying dynamic disease biology. Accumulation of these time course datasets will ultimately permit modeling and forecasting of treatment responses and failures (e.g. due to treatment resistant adaptations). These approaches are applicable across therapeutic areas and may be integrated with other datasets, e.g. genetic sequencing or proteomic analysis.

[0005] The analysis of HCS and live-cell microscopy data relies on computer based image analysis of cells. However, extraction of meaningful temporal information from sequences of microscopy images represents a major challenge to characterizing biological processes due to the absence of robust automated analysis approaches to quantify single cell trajectories. Computer based image analysis for these studies mandates improved approaches for cell instance segmentation and new approaches to identify cell division events and to track cellular dynamics over time. Current methods for image-based cell detection rely on structural FLs which limit available channels that can be used to monitor other biologically relevant non-structural labels. In addition, current methods for image based cell division detection and cell tracking rely on images taken at short temporal resolutions that lead to phototoxic effects, impacting cell viability and causing non-physiological behaviors when sustained throughout extended time course experiments. Thus, current methods suffer by placing substantial limitations on experimental design. In order to maximize the biological information that can be captured from HCS and live drug screening, improvements to image- based cell segmentation, cell division detection, and cell tracking are necessary.

[0006] All references cited herein, including patent applications and publications, are incorporated by reference. BRIEF SUMMARY

[0007] The methods provided herein allow for analysis of single cell trajectories from live cell high content screening data of live data and the quantification of cellular dynamics. Such dynamics may include cellular growth, division, and cell to cell interactions or other temporally modulated aspects of cell behavior. Further, the methods can be used to offer a detailed look at how they respond to various perturbations automatically and in detail. By observing these processes in real time, valuable insights into the natural progression of cellular activities and the nuanced mechanisms of drug targets and drug treatment effects, 2sf-6002883Docket No.78106-20008.40 including their efficacy and potential toxicity can be gained. Detailed imaging and long-term live-cell imaging helps to track the activation of signaling pathways in response to stimuli, revealing how cells communicate and react at the molecular level. It also sheds light on the diversity within cell populations, uncovering different heterogeneous responses to perturbations. The resulting insights that can be gained using the methods described herein is crucial for understanding why drugs might work effectively in some cellular contexts but not in others. Furthermore, such detailed observation helps to explore how these responses may be modulated by individual or combinations of cellular processes including cell cycle, apoptosis, intracellular trafficking, protein synthesis and degradation, metabolic modulation, and other core cell machinery - processes that are inherently dynamic.

[0008] Provided herein are methods that can be used for cell based image analysis of HCS and live-cell microscopy data. The methods described herein have significant impact on: (1) reducing cost, as the methods result in reducing the number of assays or time points needed to extract equivalent biological information, (2) increasing possible discoveries from downstream analysis without increasing perturbations to the cells that may decrease the biological relevance of the experiment; (3) unlocking exclusive insights into disease biology targets and drug mechanism action that are encoded within time-dependent cell trajectories, including the ability to recognize patterns that enable prediction or forecasting of therapeutic response or failure.

[0009] The methods comprise image-based cell segmentation comprising fine tuning cell- segmentation models using image data representing cells that have been fluorescently labeled with non-structural fluorescence labels. The finetuned models can be aggregated into an aggregate finetuned model and then one or more of the finetuned models can be applied to perform image based cell segmentation on additional image data without needing to re-fine tune models for the new data.

[0010] The methods further comprise methods that can be used for image-based cell division detection comprising a dual input classifier model to feature vectors and embedding vectors generated using a trained image detection model. The feature vectors can be computed based on the embedding vectors for a set of candidate cell triplets. The candidate cell triplets can be selected by identifying a cell triplet in multiple images wherein the first cell in the first image is likely a mother cell that divided into the two daughter cells in the second image. The models can allow for identifying cell division with images taken at longer time intervals creating a lessened risk of photobleaching of the cells. 3sf-6002883Docket No.78106-20008.40

[0011] The methods further comprise methods that can be used for image-based cell tracking using image data depicting a population of cells at a plurality of time points that have been annotated to designate mother-daughter cell divisions. The cell tracking method comprises applying a first model to the set of images to generate a plurality of embedding vectors and using the vectors to generate an acyclic graph structure. Subgraphs and tracklets can be extracted and used to update the acyclic graph structure. A set of tracks indicative of respective cells in the set of images can be extracted iteratively from the updated acyclic graph data. Like the cell division detection methods, the cell tracking methods can be applied to data with longer intervals between images and thus less photobleaching effects than previous methods.

[0012] Provided herein are method for image-based cell segmentation, comprising: receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generating a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receiving a cell-segmentation model configured to predict instance segmentation for images of cells; and generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and applying one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image.

[0013] In some embodiments, the methods further comprise generating an evaluation metric that compares a predicted segmentation generated by the aggregated finetuned model to a ground truth segmentation. In some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model is based on selection of the aggregated finetuned model based on the generated evaluation metric.

[0014] In some embodiments, the methods further comprise: generating, for each of the plurality of finetuned models, an evaluation metric that compares a predicted segmentation generated by the respective finetuned model to a ground truth segmentation; and selecting, 4sf-6002883Docket No.78106-20008.40 based on the evaluation metrics, a best-performing model; wherein applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the selected best-performing model. In some embodiments, generating a first finetuned model of the plurality of finetuned models comprises: training the first finetuned model based on a first subset of the training image data for a first combination of cellular compartment, cell line, and a single imaging channel. In some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the first finetuned model based on channel data from the second image data for the single imaging channel.

[0015] In some embodiments, the methods further comprise aggregating the first finetuned model with another model trained based on an imaging channel different from the single imaging channel. In some embodiments, generating a second finetuned model of the plurality of finetuned models comprises: training the second respective finetuned model based on a second subset of the training image data for a second combination of cellular compartment, cell line, and a set of multiple imaging channels. In some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the second finetuned model based on channel data from the second image data for one or more imaging channels of the set of multiple imaging channels.

[0016] In some embodiments, the methods further comprise fusing the segmentation map of the image with a second segmentation map of the image, wherein the second segmentation map of the image is generated by a second finetuned model.

[0017] In some embodiments, the plurality of fluorescent labels comprise one or more non- structural fluorescence labels. In some embodiments, the one or more non-structural fluorescence labels have a primary function of highlighting a variable expression protein. In some embodiments, the primary function is labelling a protein in a signaling pathway or that changes expression in response to a stimulus.

[0018] In some embodiments, the at least a subset of the plurality of fluorescence imaging channels comprises between 1 and 35 of fluorescence imaging channels.

[0019] In some embodiments, the cell-segmentation model is a model trained to predict instance segmentation for images of cells using image data comprising images of cells, wherein the images comprise a plurality of structural fluorescent labels detected by a plurality of fluorescence imaging channels. In some embodiments, the plurality of structural fluorescent labels comprise one or more structural fluorescent labels with a primary function of labeling a cellular compartment. In some embodiments, the cell-segmentation model is selected from a group consisting of CellPose, CellSam, and nucleAlzer. 5sf-6002883Docket No.78106-20008.40

[0020] Also disclosed herein are systems for image-based cell segmentation, the system comprising one or more processors configured to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image.

[0021] Also provided herein are non-transitory computer-readable storage mediums storing instructions for image-based cell segmentation, the instructions configured to be executed by one or more processors of a system to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and apply one or more of the finetuned models from the 6sf-6002883Docket No.78106-20008.40 aggregated finetuned model to an image of the second image data to generate a segmentation map of the image.

[0022] Also provided herein are methods for image-based cell division detection, comprising: receiving image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; applying a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process.

[0023] In some embodiments, the set of images comprises a third image depicting the population of cells at a third time point. In some embodiments, the set of candidate cells are selected by identifying a cell triplet representing a possible first mother cell in the first image that divided into a set of two possible daughter cells in the second image.

[0024] In some embodiments, identifying a cell triplet comprises: calculating, based on the second image, a second distance between two cells in the population of cells at the second timepoint; identifying based on the first image and the second image, a corresponding set of one or more cells in the population of cells at the first time point; calculating, based on the first image, a first distance representing a distance based on the set of one or more cells in the population of cells in the first image; identifying the cell triplet as the two cells and the corresponding two cells if the second distance is less than or equal to the first distance. In some embodiments, the methods further comprise determining based on the second image that the second image is less than a predefined maximum distance.

[0025] In some embodiments, the feature vector comprises a plurality of relational characteristics between cells in the set of candidate cells. In some embodiments, the plurality of relational characteristics comprise one or more Spatial Euclidean distance between cells in the set of candidate cells, one or more size differences between cells in the set of candidate cells, one or more aspect ratio differences between cells in the set of candidate cells, one or more pairwise cosine distances from the embedding vectors associated with the candidate cells. In some embodiments, the feature vector comprises at least 41 dimensions. 7sf-6002883Docket No.78106-20008.40

[0026] In some embodiments, the trained first model comprises contrastive learning model. In some embodiments, the trained first model comprises a simCLR model, a SimSiam model, or a DINO model.

[0027] In some embodiments, the dual input classifier model comprises a recurrent encoder, wherein the recurrent encoder condenses the dimensionality of the embedding vectors associated with the candidate cells to produce a concatenated embedding vector.

[0028] In some embodiments, the concatenated embedding vector encodes a temporal relationships between respective cells in the candidate cells.

[0029] In some embodiments, the dual input classifier model further comprises a binary classifier model, wherein the binary classifier model integrates the concatenated embedding vector and the features vector to generate the classification output.

[0030] Also provided herein are systems for image-based cell division detection, the system comprising one or more processors configured to cause the system to: receive image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; apply a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process.

[0031] Also provided herein are non-transitory computer-readable storage mediums storing instructions for image-based cell division detection, the instructions configured to be executed by one or more processors of a system to cause the system to: receive image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; apply a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and apply a dual input classifier model to the features vector and the 8sf-6002883Docket No.78106-20008.40 embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process.

[0032] Also provided herein are methods for image-based cell tracking, comprising: receiving image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; applying a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extracting a plurality of subgraphs from the acyclic graph data structure G; extracting, from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

[0033] In some embodiments, receiving the image data comprises receiving image data based on an image based cell division detection method. In some embodiments, the image based cell division detection method is an image based cell detection method as described herein.

[0034] In some embodiments, generating the acyclic graph data structure G comprises applying one or more constraints based on the relationship between the embedding vectors associated with the plurality of annotated images. In some embodiments, generating the acyclic graph data structure G comprises applying one or more constraints based on division relationships of mother-daughter cell divisions indicated by the annotated images. In some embodiments, generating the acyclic graph data structure G comprises define edge weights for the acyclic graph data structure G based on a computed composite distance between mother-daughter cells.

[0035] In some embodiments, extracting the plurality of subgraphs from the acyclic graph data structure G comprises using a temporal rolling window. In some embodiments, extracting, from the one or more of the plurality of subgraphs, the plurality of tracklets comprises extracting the plurality of tracklets based on the plurality of tracklets having a minimal average cost along a path.

[0036] In some embodiments, generating the updated acyclic graph data structure G’ comprises constraining the updated acyclic graph data structure G’ based on occurrence of an edge in the plurality of tracklets. In some embodiments, generating the updated acyclic graph 9sf-6002883Docket No.78106-20008.40 data structure G’ comprises weighting one or more edges of the updated acyclic graph data structure G’ based on a frequency of occurrences of a link in the tracklets.

[0037] In some embodiments, the plurality of tracklets represent possible temporal linkages between cell in the population of cells at the plurality of respective time points. In some embodiments, extracting the set of tracks from the updated acyclic graph data structure G’ comprises using a Markov decision process. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than about 4 hour intervals.

[0038] Also provided herein are systems for image-based cell tracking, the system comprising one or more processors configured to cause the system to: receive image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; apply a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

[0039] Also provided herein are non-transitory computer-readable storage mediums storing instructions for image-based cell tracking, the instructions configured to be executed by one or more processors of a system to cause the system to: receive image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother- daughter cell divisions for cells across the plurality of time points; apply a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; 10sf-6002883Docket No.78106-20008.40 and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

[0040] Also provided herein are methods for image-based cell analysis, comprising: receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generating a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receiving a cell-segmentation model configured to predict instance segmentation for images of cells; and generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; applying one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; applying a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generating a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extracting a plurality of 11sf-6002883Docket No.78106-20008.40 subgraphs from the acyclic graph data structure G; extracting, from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

[0041] In some embodiments, the first image data comprise images of live cells. In some embodiments, wherein the second image data comprise images live cells.

[0042] In some embodiments, the set of tracks are for use in analyzing cellular behavior in response to one or more perturbations. In some embodiments, the one or more perturbations are chosen for their potential to treat a disease. In some embodiments, cellular behavior in response to the one or more perturbations is used to choose a perturbation of the one or more perturbations that can be used to treat a disease.

[0043] In some embodiments, the method is for use in a drug discovery pipeline or use in a diagnostic pipeline. In some embodiments, wherein the method is for use in forecasting a response to a perturbation in an assay system, a preclinical model, or an individual. In some embodiments, the perturbation is treatment with a therapeutic compound.

[0044] Also provided herein are systems for image-based cell analysis, the system comprising one or more processors configured to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; apply a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second 12sf-6002883Docket No.78106-20008.40 image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generate a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

[0045] Also provided herein are non-transitory computer-readable storage mediums storing instructions for image-based cell analysis, the instructions configured to be executed by one or more processors of a system to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging 13sf-6002883Docket No.78106-20008.40 channels; apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; apply a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generate a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG.1 depicts a flow-chart for an image-based cell segmentation detection method according to some of the embodiments described herein.

[0047] FIG.2 provides a non-limiting example of a channel-wise (CW) method for finetuning cell segmentation models as described herein. In the CW approach, individual finetuned models may be generated for each FL by finetuning the model on each imaging channel.

[0048] FIG.3 provides non-limiting example of a multi-channel (MC) method for finetuning cell segmentation models as described herein. In the MC approach, finetuned models may be 14sf-6002883Docket No.78106-20008.40 generated for multiple FLs by finetuning the model with data from multiple imaging channels.

[0049] FIGs.4A-4E provide 5-fold cross validated F1-scores for cytoplasm segmentation on all 5 cell-lines. FIG.4A displays scores for CL1. FIG.4B displays scores for CL2. FIG.4C displays scores for CL3. FIG.4D displays scores for CL4. FIG.4E displays scores for CL5. As used in FIGs.4A-4E, the evaluations compare vanilla Cellpose (V), the channel-wise (CW) strategy, and the multi-channel (MC) strategy, as described herein, as columns, on the powerset of channels as rows, aggregated together using the flow average (FA) method.

[0050] FIGs.5A-5E provide 5-fold cross validated F1-scores for nuclei segmentation on all 5 cell-lines. FIG.5A displays scores for CL1. FIG.5B displays scores for CL2. FIG.5C displays scores for CL3. FIG.5D displays scores for CL4. FIG.5E displays scores for CL5. As used in FIGs.5A-5E, the evaluations compare the V, the CW, and the MC strategies as described herein, as columns, on the powerset of channels as rows, aggregated together using the FA method.

[0051] FIGs 6A-6B provide F1-scores for the CL1 cell-line imaged with the A2 acquisition method. FIG.6A represents results for cytoplasmic segmentation. FIG.6B represents results or nuclei segmentation. As used in FIGs.6A-6B, the evaluations compare the V, the CW and the MC strategies, as described herein, as columns, on the powerset of channels as rows, aggregated together using the FA method.

[0052] FIG.7 depicts a flow-chart for an image-based cell division detection method according to some of the embodiments described herein.

[0053] FIG.8 provides non-limiting examples of the training and inference methods described herein that can be used for an image based cell division detection model. The method’s training phase (top “training” section) comprises two stages: (1) Pretext task training involves training an embedding model (e.g., a simCLR model) with patches extracted from individual cells’ Regions of Interest and (2) Target task training involves training a classification model that utilizes features from the embedding model to detect division events. In the inference phase on a new video (bottom “inference” section), three steps are involved: (1) Selection of mother / daughter candidates from the video based on predefined constraints. (2) Generation of learned (e.g.,simCLR) representations for each candidate triplet using the trained model. (3) Classification of each candidate into negative or positive division events using the trained classifier.

[0054] FIGs.9A-9B provide non-limiting examples of the application of simCLR for the division and tracking methods. In both these diagrams, the simCLR architecture is 15sf-6002883Docket No.78106-20008.40 represented as a Convolutional Neural Network (CNN) from which features are extracted as well as a Multi-Layer Perceptron (MLP) used for the training only. FIG 9A depicts Division- simCLR training wherein for each individual cell in the training videos, an image patch corresponding to the cell’s Region of Interest and scale is extracted and scaled to a constant size. At training, for each sample image patch, stochastic augmentations are applied, the individual images are propagated through the model and subsequently the model is trained with the objective of matching augmented images issued from the same sample and repelling them in the embedding space from the other samples’ augmented images. FIG.9B depicts Tracking-simCLR training workflow wherein given two frames with weakly supervised tracking labels, for each pair of corresponding cells within a track, image patches corresponding to the cell’s region of interest in their respective frames are extracted. At training, stochastic augmentations are applied on both patches and propagated through the model. The model was trained with the objective of matching image patches issued from the same track and to repel them from other pairs in the embedding space.

[0055] FIG.10 provides a nonlimiting example of a dual input classification model as described herein. The diagram illustrates the architecture of a division detection classifier. The classifier utilizes division-simCLR features and hand-crafted features via two sequential models: (1) a recurrent classifier (LSTM), where the input consists of concatenated cell triplets’ division-simCLR embeddings along the time dimension. The LSTM generates a latent state, which is subsequently concatenated with the hand-crafted features. (2) The resulting vector is then fed into a binary classifier (MLP), which outputs the class probability of the division event.

[0056] FIG.11 depicts a flow-chart for an image-based cell tracking method according to some of the embodiments described herein.

[0057] FIG.12 provides a non-limiting example of a weakly-supervised cell tracking method. The two components comprise (1) a Pretext Task involving training a simCLR embedding model, trained using time-based augmentations with weakly supervised tracks annotated by a segmentation-based tracker algorithm; (2) a Target Task - which is applied without changes at inference - utilizing features extracted from the simCLR embedding model trained in the pretext task in conjunction with additional hand-crafted and deep features, to optimize cell assignment over a graph structure and generate complete tracks.

[0058] FIGs.13A-13C provides radial dendrograms facilitating the analysis of track reconstruction from video footage, providing a clear evaluation of reconstruction quality. Each ring in the dendrogram corresponds to a frame, starting from the center, and these 16sf-6002883Docket No.78106-20008.40 visualizations distinctly highlight the duration, lineage, and entry / exit points of individual cells. A legend accompanies each dendrogram, explaining the types of nodes shown. FIG. 13A displays the ground truth. FIG.13B displays the tracks from Lineage Mapper. FIG.13C displays results from method described herein using the Sliding Window Assignment Graph.

[0059] FIG.14 depicts a computer, in accordance with some embodiments. DETAILED DESCRIPTION

[0060] Described herein are methods and data demonstrating a workflow to train and infer segmentations of cell identifying cell compartments such as nuclei, organelles or cytoplasm that are typically used for cell instance segmentation, without having to acquire images in which these compartments are labeled with structural FLs. The methods described herein are compatible with a variety of cell line / FLs configurations. The methods described herein can be used for a range of assays and removing the requirement to include structural FLs expands the range of experiment-specific FLs (non-structural FLs) that can be incorporated in a given assay. The data provided herein show satisfactory segmentation performance can be achieved and replicated on various assays by leveraging multiple non-structural FLs. Thus, using the methods described herein, the resulting experiments could provide richer descriptions of each cell’s response to the perturbation, while limiting costs of assays needed to obtain the same biological information. The methods are easily adaptable to fit a generalist image processing pipeline and can be applied on various assays by aggregating segmentation models trained on multiple cell lines and FLs into a model zoo as described herein. Such a zoo of finetuned models will greatly support microscopy based cellular assays, and HCS allowing for its application to new HCS experiments without the need to re-fine-tune models.

[0061] Additionally provided herein are methods and data demonstrating a workflow for cell division detection and cell tracking in fluorescence video microscopy, specifically designed for low temporal resolution datasets. Low temporal resolution is necessary because current methods suffer from requiring live-cell imaging studies with prolonged exposure to fluorescent light that can induce phototoxic effects, impacting cell viability and causing non- physiological behaviors. Current deep learning methods to analyze low resolution time course images require large quantity of annotations for training, which hinders the ability of a computational platform to adapt to changes in experimental set-up, which typically arise in live cell assay experiments. The models described herein may significantly reduce the requirement for manual annotation. In some embodiments, the division detection utilizes a 17sf-6002883Docket No.78106-20008.40 semi-supervised approach incorporating elements of self-supervised learning. In some embodiments, the tracking algorithm is unsupervised, harnessing similarity estimations derived from both hand-crafted and annotation-free weakly-supervised contrastive deep learning features, coupled with dynamic programming. In some embodiments, the dual approach is uniquely tailored to accommodate the rapid morphological and phenotypic changes characteristic of low temporal resolution microscopy, demonstrating a novel solution to a longstanding challenge in the field. By addressing the trade-off between experiment duration and temporal resolution, the methods provided herein are flexible and robust tools for comprehensive cell analysis in long-term studies, opening new avenues for understanding complex biological processes. 1) Definitions

[0062] Reference to “about” a value or parameter herein includes (and describes) embodiments that are directed to that value or parameter per se or value within a 10% range of the value. For example, description referring to “about “ included description of “X.”

[0063] As used herein, the singular form of the articles, “a”, “an”, and “the” includes the plural references unless indicated otherwise.

[0064] It is understood that aspects and embodiments of the invention described herein include “comprising”, “consisting”, and / or “consisting essentially of” aspects and embodiments. 2) Methods of cell segmentation detection

[0065] Described herein are methods that can be used to perform image based cell segmentation using images of cell that are labeled with fluorescent markers that are not primarily used for identifying structural cellular components, i.e. non-structural FLs. The methods provided herein allow for identification of cellular components in images from experiments while leaving fluorescence imaging channels open for nonstructural labels. The methods comprise generating finetuned machine learning models that can be applied to image data from subsequent experiments.

[0066] Image-based cellular assays allow for the investigation of cellular and population phenotypes and signaling and thus an understanding of biological phenomena with high precision. Specific tailoring of fluorescent labels (FL) such as immunofluorescence staining or fluorescent proteins can be used to investigate biological process and pathways. Cellular and subcellular features can then be detected and quantified by measuring fluorescence signal 18sf-6002883Docket No.78106-20008.40 intensity and localization or by using multi-parametric measurements in a machine-learning framework. Cell instance segmentation is a key part of such bioimage analysis pipelines as it allows for the study of cells at a single-cell level rather than at the population level.

[0067] Each FL has a function in an assay design. FL with primary functions to label a specific cellular compartment are structural FL; otherwise, it is a non-structural FL. Structural FLs highlight a constant expression protein. A non-limiting example of a structural FL is DAPI. Non-structural FLs highlight variable-expression proteins. Some non-structural FLs may localize to a cellular compartments, even if it is not their primary function, for example, even if their expression is not always constant. Such non-structural FLs can be referred to as structurally strong. Indeed, a FL can also, on top of its primary role (e.g., to label a protein associated with a signaling pathway), highlight cellular structures useful for segmentation, for example the Bcl-2 family of prosurvival signaling proteins that localize to the mitochondrial outer membrane. If they do not consistently do so and exhibit variable expression, they can be labelled as structurally weak, for example the ERK protein that may localize to the cytoplasm or nucleus depending on the inactivation or activation, respectively, of the MAPK signaling pathway. For example, due to the unique behavior of individual cells when exposed to chemical compounds, signaling pathways highlighted by non-structural FLs can lead to translocation to different cellular compartments, increased / decreased expression, or altered distribution. As indicated by the examples above, non-structural FLs lie on a continuum between both categories: the structural and morphological information they carry can vary significantly.

[0068] Deep learning models for cell instance segmentation have recently reached the quality of manual annotations, especially thanks to the emergence of models like U-Net. Cellpose is a notable U-Net based approach, which uses a multi-modal training dataset spanning several cell types and cell lines imaged under a variety of different imaging methods. It also benefits from being associated with a large community which increases the size and diversity of the dataset, in turn improving the performance of the model. Cellpose approaches the problem of multiple instance segmentation by predicting spatial gradient maps from the images, from which individual cell segmentations can be inferred. Other recent deep learning segmentation methods for cell biology include StarDist and NucleAIzer which also make use of the U-Net architecture but with different representation for their images, and Mask-RCNN which directly segments regions of interests (ROIs) in images with deep learning methods. Cellpose may be selected as a foundational cell segmentation model for the methods described herein, 19sf-6002883Docket No.78106-20008.40 as it is a widely used and well-designed framework which offers to be the most generalist with its community driven, ever-expanding training dataset.

[0069] However, a limitation of Cellpose is its heavy reliance on structural FLs for cellular compartments such as nucleic and cytoplasmic segmentation. Cellpose was trained on several datasets, most of which relied on active staining of organelles’ structural proteins (e.g. cytoskeleton for cytoplasm). Indeed, only 15% of the Cellpose training dataset contain other fluorescent FLs. While such structural FLs are generally integrated in assay designs, being able to segment cells without using them allows incorporation of an expanded number of more biologically informative non-structural FLs into the assay design. Since the total number of available microscopy fluorescence detection channels is typically limited to 4 by standard filter selection and detector geometry, freeing two additional channels by removing the need for structural FLs to segment cell nuclei and cytoplasm doubles the biologically relevant information that can be delivered by an assay enabling more rapid and detailed characterization cellular processes.

[0070] The methods provided herein come from the unexpected finding that non-structural FLs may contain information about cell and nuclear morphology that can be leveraged to support cell instance segmentation, even though they are not optimized in this regard, in contrast with structural FLs. Further the methods provided herein come from the finding that a collection of non-structural FLs are a sufficient substitute for structural FLs with regard to segmentation. Thus, the methods provided herein provide methods for “experitization” of general cell segmentation models to provide prediction of a wider range of cellular image types and styles, with very small additional training from humans in the loop. The “experitization” methods provided herein rely on non-structural FLs.

[0071] Provided herein are methods that can be used to generate a generic framework for cellular compartment segmentation without the need to include corresponding structural FLs in the assay design. Cellular compartments may be the nucleus, other organelles, or the cytoplasm. The framework requires few annotations to finetune a pre-trained generalist deep learning segmentation base-model, for example Cellpose, on each FL with a small set of annotated images. Also provided herein are data from multiple datasets that show that by combining the predictions from multiple non-structural FLs with various structural characteristics, the methods described herein can reach segmentation performance comparable to the state of the art without relying on structural FLs.

[0072] FIG.1 provides a schematically depiction of steps that can be used in methods for image-based cell detection according to some of the embodiments described herein. The 20sf-6002883Docket No.78106-20008.40 methods comprise generating finetuned cell segmentation models for image data of cells labeled with fluorescence labels. By finetuning the cell segmentation models according to the embodiments of the method, an aggregated finetuned model can be created and deployed for use in subsequent imaging datasets. As described herein, one or more finetuned model from the aggregated fine-tuned model can be applied to the subsequent imaging set to generate a segmentation map of the cell in the image. As shown herein, the finetuned models are able to generate segmentation maps of cells in an image that have not been marked with structural fluorescence labels.

[0073] In 102, first image data comprising images of cells marked with fluorescent labels are received. In some embodiments, the first image data comprises images of live cells. In some embodiments the first image data is live cell image data. In some embodiments, the first image data comprise images of cells from a high content screening experiments. In some embodiments, the fluorescent label may be a genetically encoded fluorescent labels fused to a protein of interest or one or more genetically encoded fluorescent labels incorporated into a single color or multicolor fluorescent biosensor. In some embodiments the fluorescent label may be GFP, Cerulean, mCherry, venus, or another fluorescent protein or genetically encoded fluorescent label known in the art. In some embodiments, the fluorescent label may be a chemical dye or chemical dye conjugate. In some embodiments the fluorescent label may be a chemical dye from the cytoTracker, Mitotracker, Hoechst 3342, DAPI, AlexaFluor, CellMask, MitoTracker or SYTO families of dyes or other chemical dyes known in the art. In some embodiments the chemical dye may be conjugated to phalloidin, wheat germ agglutinin (WGA), a plasma membrane or mitochondrial membrane targeting sequence, concanavalin or other conjugates know in the art that localize to cell structure compartments. In some embodiments the fluorescent label may be one or more of those used for Cell Painting. In some embodiments the fluorescent label may be a chemical dye conjugated to an antibody for immunofluorescence staining. In some embodiments, the fluorescent labels comprise one or more non-structural fluorescent labels. In some embodiments, the non-structural fluorescent labels have a primary function of highlighting a variable expression protein. In some embodiments the primary function of the non-structural fluorescent label is labeling a protein in a signaling pathway or that changes expression in response to a stimulus. An exemplary non-structural fluorescent label may be a fluorescent tag fused to ERK, that localizes to the nucleus when MAPK signaling is activated, but localizes to the cytoplasm when it is not. Although the ERK protein localizes to structural components and can thus be used to segment them, its primary function is as a readout of pathway activation, not a structural marker 21sf-6002883Docket No.78106-20008.40 because its localization is variable. A further example of a non-structural fluorescent label is a tandem biosensor of autophagy encoding RFP, GFP and the LC3 autophagy marker. Although the ratio of GFP to RFP signal is primarily a non-structural readout of autophagic flux, it also indicates the formation of autophagosome and lysosome structures. Similarly, a biosensor of cell cycle encoding spectrally distinct fluorescent proteins fused to proteins that fluctuate throughout the cell cycle such as Cdt1 and Geminin is primarily a readout of cell cycle phase, but these proteins also localize to the nucleus. Additionally, a proximity biosensor that measures fluorescence resonance energy transfer between a pair of spectrally distinct but co-localized fluorescent protein tags is primarily a readout of signaling activity, but may also indicate the localization of certain proteins.

[0074] In some embodiments, the plurality of fluorescent labels are detected by a plurality of fluorescence imaging channels. In some embodiments, the plurality of fluorescence labeling channels comprise between 1 and 35 fluorescence imaging channels. In some embodiments, the plurality of fluorescence labeling channels comprise greater than 1 channel, greater than 5 channels, greater than 10 channels, greater than 15 channels, greater than 20 channels, greater than 25 channel, greater than 30 or greater than 35 fluorescence imaging channels. In some embodiments, the plurality of fluorescence labeling channels comprise less than 35 channels, less than 30 channels, less than 25 channels, less than 20 channels, less than 15 channel, less than 10 or less than 5 fluorescence imaging channels.

[0075] In some embodiments, the plurality of fluorescence labeling channels are configured in microscopes with multichannel or multispectral detectors that allow up to 8 fluorophores from a 32 channel detector.

[0076] The first image data from 102 are used in 104 wherein a training image data set is generated by annotating the image data to indicate cellular compartments and cell boundaries. In some embodiments, the annotation is done by human. In some embodiments, the annotation is done by a machine learning algorithm.

[0077] In 106 a cell-segmentation model configured to predict instance segmentation for the images of cells. In some embodiments, the cell-segmentation model is a model trained to predict instance segmentation for images of cells using image data comprising images of cells, wherein the images comprise a plurality of structural fluorescent labels detected by a plurality of fluorescence imaging channels. In some embodiments, the plurality of structural fluorescent labels comprise one or more structural fluorescent labels with a primary function of labeling a cellular component. In some embodiments, the plurality of structural fluorescent labels comprise a genetically encoded fluorescent tag fused to a structural protein. 22sf-6002883Docket No.78106-20008.40 In some embodiments, the structural component is a protein unique to the cytoskeleton or an organelle compartment. In some embodiments, the plurality of structural fluorescent labels comprise immunofluorescence staining or fluorescent protein expression markers. In some embodiments, the cell segmentation model is selected from a group consisting of CellPose, CellSam, and nucleAlzer.

[0078] The training image data set from 104 and the cell-segmentation model from 106 are used as inputs to 108 wherein a plurality of finetuned models are generated. In some embodiments, generating a plurality of finetuned models comprises generating a first finetuned model. In some embodiments, generating a first finetuned model comprises training the first finetuned model based on a first subset of the training image data for a first combination of cellular compartment, cell line, and a single imaging channel. In some embodiments, generating a plurality of finetuned models comprises generating a second finetuned model. In some embodiments, generating a second finetuned models comprises training the second respective finetuned model based on a second subset of the training image data for a second combination of cellular compartment, cell line, and a set of multiple imaging channels. In some embodiments, training a plurality of finetuned models comprises applying a channel-wise approach as described herein. In some embodiments, training a plurality of finetuned models comprises applying a multichannel approach as described herein.

[0079] In 110, the finetuned modes are aggregated to generate an aggregated finetuned model. The aggregated fine tuned model may be referred to as a model zoo. In some embodiments, the methods comprise aggregating the first finetuned model trained based on an imaging channel different from a single imaging channel. In some embodiments, the methods comprise generating an evaluation metric, as described herein, that compares a predicted segmentation generated by the aggregated finetuned model to a ground truth segmentation.

[0080] In 112, second image data are received comprising images of cells marked with fluorescence labels. In some embodiments, second image data comprises images of live cells. In some embodiments the second image data is live cell image data. In some embodiments, the second image data comprise images of cells from a high content screening experiments. Using the aggregated finetuned model from 110, and the second image data from 112, in 114, one or more of the finetuned models are applied to the second image data to generate a segmentation maps of the image date. The choice of finetuned models may be made according to the cell line and / or fluorescence labels depicted in the second image data. In 23sf-6002883Docket No.78106-20008.40 some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model is based on selection of the aggregated finetuned model based on the generated evaluation metric. In some embodiments, choosing the finetuned model may comprise generating, for each of the plurality of finetuned models, an evaluation metric that compares a predicted segmentation generated by the respective finetuned model to a ground truth segmentation; and selecting, based on the evaluation metrics, a best-performing model; wherein applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the selected best-performing model. In some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the first finetuned model based on channel data from the second image data for the single imaging channel. In some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the second finetuned model based on channel data from the second image data for one or more imaging channels of the set of multiple imaging channels. In some embodiments, the generated segmentation map can be fused with a second segmentation map of the image, wherein the second segmentation map of the image is generated by a second finetuned model. A. Training data

[0081] The methods provided herein comprise receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels. In some embodiments, the image data comprise data on five multi-color reporter cell lines derived from commonly used U2OS and A375 cancer cell lines, that are designated as CL1, ..., CL5. The image data may comprise live-cell imaging data where fluorescent labeling is done by tagging proteins of interest with fluorophores, the combination of which can be referred to as fluorescent reporter proteins (FRPs). Each reporter cell line can express 3 or 4 spectrally distinct FRPs. In some instances, all images are acquired with a Nikon A1R confocal microscope as a live video microscopy imaging sequence. For each experimental condition, images may be acquired on 3 xy positions every 3 hours for 72 hours, at various optical zooms. Exemplary characteristics of the cell lines used to generate the image data according to embodiments of the current application are provided in Table 1. 24sf-6002883Docket No.78106-20008.40 Table 1: Cell line characteristics for cell lines used to generate image data used in the cell segmentation methods. MTS - mitochondria targeting signal; palm - palmitoylation signal; 53BP1trunc - 53BP1trunc(1220-1711); KTR - kinase translocation reporter. For RAS and RAF proteins different isoforms and / or mutations were tagged.

[0082] The methods provided herein comprise generating a training data set by annotating the received image data to indicate cellular compartments such as nuclei and cytoplasm and cell boundaries represented in the images. Generating the training data may comprise annotating the image data set as described herein. Nuclei and cell boundary of 50 (resp.28 and 22) images for each cell line / FL combination of cell lines CL1, CL4, CL5 (resp. cell lines CL2 and CL3) may be manually annotated. Images from different experimental conditions and evenly distributed over time can be used in order to capture the dynamic localization of certain proteins (e.g. due to experimental treatments and levels of expression at different stages of cell cycle) as well as possible variations in population size caused by cell division or cell death were randomly selected. The number of cells per image may range from 20 cells to more than 100 in some images. For each set of annotated images, 80% may be used for training, 10% for validation and the last 10% for evaluation. In some embodiments an average of 50 cells per images, about 250 cells per validation / test set and approximately 2000 individual cells 25sf-6002883Docket No.78106-20008.40 are used in the training set, an appropriate number for training and evaluation. The annotations may be carried out and validated by multiple biologists using the different channels available in combination. Additional training data may be generated by annotating 5 additional images of CL1 that were acquired with a different microscope (widefield) and higher temporal resolution, resulting in noisier images. B. Model fine tuning and aggregation

[0083] The methods described herein comprise receiving a cell-segmentation model configured to predict instance segmentation for images of cells. In some instances, the cell- segmentation model is selected from a group consisting of CellPose, CellSam, and nucleAlzer. The data provided herein refers to an embodiment using CellPose, however the approach provided herein is model-agnostic and therefore alternative models can be used.

[0084] Cellpose segments images using a three-part pipeline: Firstly, it resizes the images so that the average cell diameter of the dataset conform to the model original training cells’ diameter. Secondly, for a cell object o ∈ {nuclei, cyto}, a Cellpose model Momaps a rescaled image Î intrinsic intensity space to a flow and probability space (FX, FY, P). The flow maps FX, FYare the derivatives (along the X and Y axes) of a spatial diffusion representation of individual cell pixels from the cell’s center of mass to its extremities. Thirdly, Cellpose combines the (FX, FY, P) flow and probability maps to predict instance segmentations Sousing flow analysis and thresholding on all three maps combined. First, the flows FX and FY are interpolated and consolidated where the pixel-wise probabilities P are above a pre-set threshold. The instance masks may then be generated by analyzing the flows histogram from their peak. In some embodiments, the Cellpose overall segmentation process can be summarized in the following equation: ^^^^0�Î� →(^^^ ^^^^^, ^^^^ ^^^^, ^^^^)�^^�^�^ ^�^^�^ ^�^^�^�^�^�^�^^^�^^ ^�^�^�^ ^�^^�^�^^�^^�^�^�^^�^�^^�^�^^�^�^ ^�^�^^�^^�^^�^^�^^�^^�^�^ ^�^^^ ^^^^ ^^^^, ^^^^ ∈{^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^ ^^^^ ^^^^ ^^^^}

[0085] The methods described herein comprise generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels and aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model. Generating finetuned models and aggregating the finetuned model may comprise building a model zoo as described herein. 26sf-6002883Docket No.78106-20008.40

[0086] The generalization powers of Cellpose may be leveraged to segment images with non- structural FLs. A model zoo of pre-trained models finetuned on each cell line’s FLs may be generated using the annotated data. To segment images with non-structural FLs, the generalization powers of Cellpose are leveraged. The base Cellpose model may be used as the cell-segmentation model in the methods described herein and may be described as Vanilla Cellpose. For use in Vanilla Cellpose, the training images may be rescaled to the average diameter of images in the training set.

[0087] The methods may comprise data augmentation. Augmentation methods may be used during training the Vanilla Cellpose model to both virtually increase the size of the dataset as well as offer better generalization. They may be performed iteratively from scratch on each image batch. For methods involving random distributions, the parameters may be uniformly sampled from a pre-defined parameter range. Each augmentation may have an application probability paugment= 0.5, adding more variability across epochs and samples. Example data augmentation models according to some embodiments are described in Table 2. Table 2: Data augmentation for cell segmentation

[0088] To finetune Vanilla Cellpose, a finetuned train for each organelle and cell line (o, cl) combination (with o ∈ {nuclei, cyto} and cl one of the dataset cell lines)may be trained as well several Cellpose models on subsets of channels ^^^^ ⊂taken from the powerset of channels (excluding the empty set), with K the total number of channels. The powerset may be denoted as {Co, ... CK}. When |C| > 1, each individual channel C from the same image sample may be imputed as independent training samples. Each finetuned model Mo,cl,Cmay then be used to predict segmentation flow by evaluating each channel from C individually. Those segmentations may then be fused together at inference to produce a single 27sf-6002883Docket No.78106-20008.40 segmentation m†ap. To segment an image for an (o, cl) combination, the model trained using the channels Ctrainfor which the highest score is achieved on the evaluation set using the channels Cevalmay be used. Mo,cl,C({ Îc}c∈C) → {( ^^^^^^^^^^^^, ^^^^^^^^^^^^, ^^^^^^^^)}c∈C, C ⊂ {c0, ... , cK}

[0089] FIG.2 and FIG.3 illustrate exemplary embodiments of model finetuning and inference using the finetuned models. First for training, a training set of multi-modal fluorescent images (e.g.3 channels) and a training set for annotations of the organelles segmentations can be used to train an Out-of-the-box pre-trained Cellpose model (Vanilla Cellpose) to generate finetuned models. The finetuned models can be aggregated into a model zoo. For inference, multi-modal fluorescent image data (e.g.3 channels) are obtained and models are selected from the model zoo corresponding to the image’s cell line and FL channel combination. (c) Spatial flows and probability maps are outputted by the finetuned models for each of the channels, Channel-wise averaging of the maps is performed, and the results are integrated into segmentation labels.

[0090] When |C | = 1, the finetuning method can be referred to as channel-wise (CW), such that a model is trained for each individual channel (FIG.2). The respective CW models can be used on their respective training channels or can be aggregated together to produce a channel- wise segmentation for several channels at once. In some embodiments, generating a finetuned model comprised applying a channel-wise approach. In some embodiments, a CW approach may be used when the non-structural FLs highlight (either directly or by contrast) one or multiple organelles. The approach may be used because the CW approach is powerful enough to segment the organelles of interest, while also enabling a higher modularity of the segmentation workflow by storing in the model zoo a model associated with each single fluorescent reporter, that could be reused in another dataset even when the other fluorescent reporters vary.

[0091] When |C | > 1, the finetuning method can be referred to as multi-channel (MC), such that a model is trained on several channels at once (FIG.3). The MC segmentation models can be evaluated on any subset of the channel set C they were trained with. In some embodiments, generating a finetuned model comprised applying a multi-channel approach. In some embodiments, a MC approach may be used when the non-structural FLs do not consistently highlight (either directly or by contrast) one or multiple organelles. The approach may be used because the MC approach might allow to accurately segment the organelles 28sf-6002883Docket No.78106-20008.40 without non-structural FLs that consistently highlight one ore multiple organelles. In some embodiments, the MC approach comprises training a segmentation model to learn jointly from multiple FLs so that it can identify complex cues and patterns within each of them that, when combined, enable the identification of the organelles’ outline.

[0092] The model’s hyper-parameters can be retrained from the original Vanilla Cellpose training with the following two exceptions: (1) as detailed herein, non-deterministic augmentations on each of the training samples may be used; (2) the training may be stopped using early stopping on the validation set, with a patience of 50. Both additions may be efficient regularization methods limiting over-fitting and contributing to the overall robustness of the segmentation methods with respect to changes in the imaging setting.

[0093] Once segmentations are generated for each channel using finetuned models, a final image segmentation may be generated by fusing the individual channel segmentation maps. The method can be implemented as Flow Averaging (FA). In some embodiments, fusion methods may comprise FA, Selective and Iterative Method for Performance Level Estimation (SIMPLE), Simultaneous Truth and Performance Level Estimation (STAPLE), Voting (V), or Majority Voting (MV).

[0094] The FA method may use Cellpose internal representations to aggregate the segmentation maps. FA averaged the segmentation probability maps and flow maps obtained by running finetuned models on each channel individually, may be used to obtain a final aggregated segmentation map.

[0095] Given |C| channels selected for an image, the approach may yield |C| segmentation tuples { ^^^^^^^^^^^^, ^^^^^^^^^^^^ , ^^^^^^^^}^^^^∈ ^^^^generated by the Cellpose-U-Net(s). From these tuples each individualmaps may be averaged along the channel dimensions, yielding maps ( ^^�^^ ^^^^ , ^^�^^ ^^^^ , ^^�^^). The averagemaps may be transformed into instance segmentation masks using Cellpose’s integration method.C. Applying finetuned models

[0096] The methods provided herein comprise receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and applying one or more of 29sf-6002883Docket No.78106-20008.40 the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image. The methods may comprise generating an evaluation metric that compares a predicted segmentation generated by the augmented finetuned model to ground truth segmentation. In some embodiments, applying the one or more of the finetuned models from the aggregated finetuned model is based on selection of the aggregated finetuned model based on the generated evaluation metric. The method may comprise generating, for each of the plurality of finetuned models, an evaluation metric that compares a predicted segmentation generated by the respective finetuned model to a ground truth segmentation; and selecting, based on the evaluation metrics, a best-performing model; wherein applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the selected best-performing model.

[0097] Performance validation and evaluation of the finetuned models described herein are provided to demonstrative how the finetuned models can be applied. In some instances, the evaluation metrics may comprise one or more segmentation evaluation metrics, such as but not limited to, precision, recall F1-score, Jaccard similarity, the aggregated Jaccard index, and the average precision. A predicted segmentation may be considered as a true positive if the intersection-over-union (IoU) between this segmentation and a ground truth segmentation is above a threshold. In some instances, the threshold may be 0.5. Recall is the percentage of cells detected, precision the probability that a detected cell is really a cell, and F1-score is the harmonic mean of the two. F1-score was used as a principal metric of evaluation for models’ performance, as it measures accuracy through both precision and recall.

[0098] FIGs.4A-4E and FIGs.5A-5E provide the performance of the exemplary methods described herein using the F1-score metric. Those results are computed using 5-fold cross validation on annotated datasets. The performance of Vanilla Cellpose can be compared to the methods described herein across every combination of training channels (channel-wise and multi-channel) upon every combination of evaluation channel aggregated together using the FA fusion method. The Vanilla Cellpose scores may be evaluated on several channels at once using this fusion method as well.

[0099] The results indicate that fine-tuning is an essential step when dealing with datasets which do not contain cytoplasmic and nucleic structural FLs, as indicated in Table 2. Fine- tuning may provide state-of-the-art level results on individual channels trained independently (CW), the fusion of channels evaluated through independently trained models (CW) and on the fusion of channels trained together (MC). Furthermore, it may be observed that 30sf-6002883Docket No.78106-20008.40 combining the different segmentations together outperforms not only the Vanilla Cellpose results but also the results of those finetuned models on their respective channels in all cases (the only exception being CL3 on cytoplasm). These results are particularly striking for structurally weak FLs which, when aggregated together using either the CW or MC approach, reach the segmentation quality of structurally strong FLs (e.g. static FLs in a single organelle). Additionally, it can be noted that when trained on a set of channels which include both strong and weak FLs, the model performs well upon evaluation on the subset of its training channels which excluded the structurally strong FLs.

[0100] The generalization capabilities of the models described herein to image data acquired by additional imaging acquisition procedures may be evaluated by evaluating the performance drift of a model trained on images with acquisitions parameters A1(confocal microscope, image size 512x512 pixels, and image scale of 1.24µM per pixel) when applied to images acquired with parameters A2(widefield microscope, image size 2044x2044 pixels, and image scale of 0.32µm per pixel). In some embodiments, 50 annotated images acquired under conditions A1 of cell line CL1 are used for training and testing a model, and 5 annotated images acquired under condition A2 of the same cell line are used to evaluate the aforementioned model performance drift. The segmentation evaluation results according to some embodiments (FIGs 6A-6B) demonstrate the ability of the models described herein to generalize to acquisition method A2 by producing similar segmentation scores and out- performing Vanilla Cellpose. These results indicate that the approaches described herein do not merely finetune Cellpose to the dataset described herein, but rather to a specific set of FLs of a cell line, successfully generalizing to other datasets with the same assay-related conditions. These results suggest that finetuned models from the model zoo can be used for image data received that represents FLs and cell lines that have previously been seen by the model without additional finetuning.

[0101] The method provided herein can be used to generate a reliable, re-usable and scalable method for the segmentation of cell images without structural FLs, with manageable annotation effort. The data provided herein demonstrates the proposed methods lead to models outperforming Vanilla Cellpose on datasets with only non-structural FLs, while requiring few annotated examples by leveraging Cellpose extended pre-training.

[0102] Specifically, the data showed that leveraging non-structural (and even structurally weak) FLs in concert improves segmentation, even when the signal is very heterogeneous between cells, and some cells do not appear at all in some channels. Indeed, each channel 31sf-6002883Docket No.78106-20008.40 provides some useful and potentially complementary information on nucleus and cytoplasm, which can be combined by segmentation fusion. Thus, aggregating channels together allows for benefiting from complementary non-structural FLs to outline individual cell objects. These observations may be especially salient for cell lines which do not contain any structurally strong FLs (such as CL2 and CL3 for cytoplasm, and CL4 for nuclei) or for cell lines which segmentation models are trained and evaluated without their structurally strong FLs (e.g. CL1 without channel 0 for both nuclei and cytoplasm, or CL2 and CL3 without channel 0 and 2 for nuclei). For example, while cell line CL5 only expressed structurally unreliable FLs, the methods are able to leverage its different FLs together to produce cytoplasmic segmentation with an F1-score of 0.81 when evaluated on the fusion of channels 0 and 2 using a model finetuned on all of its channels together. Excluding a structurally strong channel from the evaluation results in the same conclusion. For example, nucleic segmentation on CL2 scored an F1 of 0.7 when trained using channels 1 and 3 in MC, with FLs that highlight the endoplasmic reticulum and the mitochondria. Using Vanilla Cellpose on the same evaluation channels yielded an F1 score of 0.4. Similar results can be observed for CL3 on the same channels.

[0103] Furthermore, it is significant to note that the use of a structurally strong FL influences the segmentation quality, even when that FL channel was not used at inference. For example, cell line CL3, which has two structurally strong FLs highlighting the nucleic structure (channels 0 and 2), performs nonetheless very well on segmenting nuclei using only the fusion of channels 1 and 3 when trained using all 4 channels (F1 = 0.82). The same can be observed for CL4 on cytoplasmic segmentation: excluding channels 0 and 3 during the evaluation yields an F1-score of 0.85 with a model finetuned on all 4 available channels. This is particularly interesting for the segmentation of cell lines containing subsets of the FLs trained for in the model zoo described herein. Future cell lines could benefit from the multi- channel models trained on some of their FLs as well as stronger – although possibly absent – FLs which would improve the segmentation quality.

[0104] The methods may comprise selecting non-structural FLs for finetuning. In some embodiments, one may select non-structural FLs carefully when applying the proposed workflow. Indeed, some FLs by nature or under the influence of a compound introduced in the assay regimen may be too unreliable for the segmentation task. This is exemplified in the results with channel 1 of CL2 on cytoplasmic segmentation. That FL which is dynamic in the endoplasmic reticulum carries almost no information relevant to the cytoplasmic 32sf-6002883Docket No.78106-20008.40 segmentation by itself. Although models finetuned using this channel benefit from its presence (reaching a 0.91 F1-score on evaluation of the fusion of channel 0 and 2 trained using all 4 channels), models evaluated on it or trained with it in over-proportions perform poorly. If a cell-line was constituted only of FLs of similarly unreliable structural information, the workflow may not be able to segment cells. It cannot – like any segmentation method – segment any organelles out of thin air, but it can leverage structurally unreliable FLs together - with structurally medium or strong FLs when they are available - to reach the quality of segmentation one would get by including functionally structural FLs in the cell lines.

[0105] For instance, CL3 is only constituted of structurally weak FLs with regards to cytoplasmic segmentation. The presence of those FLs explains the low F1-score provided herein. However, as shown in the data provided herein, it nevertheless outperformed Vanilla Cellpose in terms of recall and detection of each individual cell. In this specific instance – with highly dynamic and unreliable FLs – the methods provided herein generated segmentation masks in the likes of a Voronoi diagram. While not optimal in its boundary detections, it translates to a better segmentation than Vanilla Cellpose, especially in the context of single-cell phenotypic analysis.

[0106] In some embodiments, one or more additional techniques or modifications may be applied to adapt the methods described herein to handle occluded or overlapping cells. In some embodiments in which merging or splitting of individual cell instances is present, a robust post-processing step may be applied to correct for this. While the disclosure herein has contemplated using Cellpose as an exemplary model backbone, in some embodiments, a different model backbone aside from Cellpose may be used. Using a different model backbone aside from Cellpose (e.g., one that does not detect nuclei and cytoplasm independently, or that applies one or more corrections for detecting nuclei and cytoplasm independently) may help to provide for accurate and robust building of cell objects. ) Methods of cell division detection

[0107] The methods described herein can be used to identify mother-daughter cell triplets in videos while requiring minimal human annotations. The methods comprise developing a robust representation for these triplets, enabling a classifier to learn effectively from few annotations and distinguish positive division events in highly imbalanced datasets. 33 sf-6002883Docket No.78106-20008.40

[0108] FIG 7 schematically depicts steps that can be used in methods for image-based cell division detection. The methods described herein can be used to annotate mother-daughter cells in image data with images depicting a population of cells at multiple time points. The methods comprise generating embedding vectors for images then using the embedding vectors and the images to create a feature vector for candidate mother-daughter cells for use in a dual input classifier model. The dual classifier provides a classification for whether the candidate cells represent a cell division process.

[0109] In 702, a set of images with an image of a population cells at a first time point and a second image depicting the population of cells are a second time point are received. In some embodiments the set of images comprises a third image depicting the population of cells at a third timepoint. In some embodiments, the set of images comprises images of live cells. In some embodiments, the set of images comprise images from live cell image data. In some embodiments, the set of images comprise images of cells from a high content screening experiment.

[0110] In 704, a trained first model is applied to generate a plurality of embedding vectors for the plurality of cells depicted in the set of images from 702. In some embodiments, the trained first model comprises contrastive learning model. In some embodiments, the trained first model comprises a simCLR model, a SimSiam model, or a DINO model.

[0111] The embedding vectors are used in 706 to compute a features vector on the set of images associated with a set of candidate cells and / or the embedding vectors associated with the candidate cells. In some embodiments, the set of candidate cells are selected by identifying a cell triplet representing a possible first mother cell in the first image that divided into a set of two possible daughter cells in the second image. In some embodiments, identifying a cell triplet comprises: calculating, based on the second image, a second distance between two cells in the population of cells at the second timepoint; identifying based on the first image and the second image, a corresponding set of one or more cells in the population of cells at the first time point; calculating, based on the first image, a first distance representing a distance based on the set of one or more cells in the population of cells in the first image; identifying the cell triplet as the two cells and the corresponding two cells if the second distance is less than or equal to the first distance. In some embodiments, identifying a cell triplet further comprises determining based on the second image that the second image is less than a predefined maximum distance. In some embodiments, the features vector comprises a plurality of relational characteristics between cells in the set of candidate cells. In some embodiments, the plurality of relational characteristics the plurality of relational 34sf-6002883Docket No.78106-20008.40 characteristics comprise metrics described herein such as but not limited to one or more Spatial Euclidean distance between cells in the set of candidate cells, one or more size differences between cells in the set of candidate cells, one or more aspect ratio differences between cells in the set of candidate cells, one or more pairwise cosine distances from the embedding vectors associated with the candidate cells. In some embodiments, the feature vector comprises at least 41 dimensions.

[0112] In 708, a dual input classifier is applied to the embedding vectors from 704 and the featured vector from 706 to generate a classification output indicating whether the set of candidate cells represents a cell division process. In some embodiments, the dual input classifier model comprises a recurrent encoder, wherein the recurrent encoder condenses the dimensionality of the embedding vectors associated with the candidate cells to produce a concatenated embedding vector. In some embodiments, the concatenated embedding vector encodes a temporal relationships between respective cells in the candidate cells. In some embodiments, the dual input classifier model further comprises a binary classifier model, wherein the binary classifier model integrates the concatenated embedding vector and the features vector to generate the classification output. In some embodiments, the results the classification output are used to annotate mother-daughter cells in the image data.

[0113] As further illustrated in FIG.8, an embodiment of the training process may comprise three stages: (1) Train a dedicated self-supervised learning model, designed to learn representations for cell image patches and their temporal context; (2) Identify mother- daughter candidate triplets in videos by selecting triplets of cells which conform to the criteria described herein. Using the annotated videos, are labeled corresponding to division events as positive samples and the others as negatives for subsequent training, (3) Train a classification model to detect division events, using the candidate representations from the annotated videos. As illustrated in the embodiment in FIG.8, the pretext task may be an exemplary embodiment of FIG.7, 704 and the target task may be an exemplary embodiment of FIG.7, 706.

[0114] The inference for a previously unseen video according to some embodiments (FIG.8, bottom row) may comprise three steps: (1) Build a graph structure of its cells, adhering to the same constraints employed during the training phase; (2) Generate representations for every cell within this graph, and then classify each triplet’s representations using the classifier to distinguish division events and accordingly label the graph; (3) Extract division events from the graph to ensure the coherence and uniqueness of each division event and mother- 35sf-6002883Docket No.78106-20008.40 daughters relationship. As illustrated with the embodiment in FIG.8, the inference task may be an exemplary embodiment of FIG.7, 708.

[0115] Data provided herein demonstrates the employment of the methods with a simCLR model with a Resnet-50 architecture, trained on sequential image patches, enabling precise identification of morphological changes indicative of cell division through our classification framework. The benefits of the methods described herein are substantiated by improved F1- scores, underscoring the effectiveness of incorporating temporal information into the division detection process. A. Receiving Image data

[0116] Provided herein are methods for image-based cell division detection that comprise receiving image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at the second point. In some embodiments, the image data comprises images of live cells. In some embodiments, the image data is live cell image data. In some embodiments, the image data is image data comprise images of cells from a high content screening experiment.

[0117] In some embodiments, the first time point and the second time point may be taken at greater than about 3 hours apart, greater than about 4 hours apart, greater than about 5 hours apart, greater than about 6 hours apart, greater than about 7 hours apart, or greater than about 8 hours apart. In some embodiments, the first time point and the second time point may be taken at less than about 8 hours apart, less than about 7 hours apart, less than about 6 hours apart, less than about 5 hours apart, less than about 4 hours apart, or less than about 3 hours apart. In some embodiments, the first time point and the second time point may be taken between 2 hours and 8 hours apart, between 2 hours and 7 hours apart, between 2 hours and 6 hours apart, between 2 hours and 5 hours apart, between 2 hours and 4 hours apart, or between 2 hours and 3 hours apart. In some embodiments, the first time point and the second time point may be taken between 2 hours and 8 hours apart, between 3 hours and 8 hours apart, between 4 hours and 8 hours apart, between 5 hours and 8 hours apart, between 6 hours and 8 hours apart, or between 7 hours and 8 hours apart.

[0118] Provided herein are methods for image-based cell division detection comprise receiving image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells 36sf-6002883Docket No.78106-20008.40 at the second point. The methods described herein may allow for more time between the first time point and the second time point than with other methods known in the art.

[0119] One of skill in the art may appreciate the time between the first time point and second time point to depend on the cellular system being sampled. Factors which may impact the time between time points include but are not limited to the level of cell motility, the rate of cellular division and the rate of other adaptations to a given cellular perturbation, for example an adaptation that causes treatment resistance. Changes in cell motility and morphology may occur on timescales of seconds to minutes. Division events in yeast cells occur on time scales of minutes to hours and in mammalian cells on time scales of approximately 1 day. Adaptations to treatment may occur on timescales of days to weeks or even months. When the cells depicted in the cell images are highly motile and divide rapidly (short cell cycle) the time between the first time point and the second time point may be shorter than when the cells depicted in the cell images are less motile and have divide slowly (long cell cycle). In some embodiments, the first time point and the second time point may be taken at time intervals that are appropriate for the experimental system. The appropriateness of the time intervals may be related to the variability of cell dynamics and the time between divisions for the cells in the experimental system. For example, when detecting cell division events in a cell population comprising cells that divide less frequently and more uniformly, the timepoints may be further apart.

[0120] In some embodiments, the cells depicted in the images of cells have a cell cycle interval of about 24 hours. In some embodiments, the first time point and the second time point may be taken at greater than about 3 hours apart, greater than about 4 hours apart, greater than about 5 hours apart, greater than about 6 hours apart, greater than about 7 hours apart, or greater than about 8 hours apart. In some embodiments, the first time point and the second time point may be taken at less than about 8 hours apart, less than about 7 hours apart, less than about 6 hours apart, less than about 5 hours apart, less than about 4 hours apart, or less than about 3 hours apart. In some embodiments, the first time point and the second time point may be taken between 2 hours and 8 hours apart, between 2 hours and 7 hours apart, between 2 hours and 6 hours apart, between 2 hours and 5 hours apart, between 2 hours and 4 hours apart, or between 2 hours and 3 hours apart. In some embodiments, the first time point and the second time point may be taken between 2 hours and 8 hours apart, between 3 hours and 8 hours apart, between 4 hours and 8 hours apart, between 5 hours and 8 hours apart, between 6 hours and 8 hours apart, or between 7 hours and 8 hours apart.

[0121] In some embodiments, the cells depicted in the images of cells are yeast cells. In some embodiments, the yeast cells have cell cycle interval of about 90 min. In some embodiments, the 37sf-6002883Docket No.78106-20008.40 first time point and the second time point may be taken at greater than about 3 minutes apart, greater than about 4 minutes apart, greater than about 5 minutes apart, greater than about 6 minutes apart, greater than about 7 minutes apart, or greater than about 8 minutes apart. In some embodiments, the first time point and the second time point may be taken at less than about 8 minutes apart, less than about 7 minutes apart, less than about 6 minutes apart, less than about 5 minutes apart, less than about 4 minutes apart, or less than about 3 minutes apart. In some embodiments, the first time point and the second time point may be taken between 2 hours and 8 minutes apart, between 2 minutes and 7 minutes apart, between 2 minutes and 6 minutes apart, between 2 minutes and 5 minutes apart, between 2 minutes and 4 minutes apart, or between 2 minutes and 3 minutes apart. In some embodiments, the first time point and the second time point may be taken between 2 minutes and 8 minutes apart, between 3 minutes and 8 minutes apart, between 4 minutes and 8 minutes apart, between 5 minutes and 8 minutes apart, between 6 minutes and 8 minutes apart, or between 7 minutes and 8 minutes apart.

[0122] In some embodiments, the image data comprises live-cell video microscopy data derived from High Content Screening (HCS) assays on a A375 cell line. The A375 cell line, a human malignant melanoma line, is extensively used in cancer research due to its well- characterized nature and responsiveness to various treatments. For the methods and data described herein, this cell line may be genetically modified to express several fluorescent reporters of the MAPK pathway: mCerulean-RAF, Venus-RAS, mCherry-ERK, and miRFP670-MEK. These reporters may allow for the detailed observation of key signaling pathways involved in cell proliferation and response to drug treatments. The cells may be plated 1000 cells / well in a 384 well plate, and the cells may be treated with 9 point half-log dilution of 6 compounds and DMSO control. The cell populations may be subjected to diverse perturbations, including but not limited to changes in drug types and concentrations, providing a comprehensive dataset to observe a range of cellular behaviors and responses.

[0123] The cells may be imaged at 4 hours intervals for 72 hours starting 10 minutes after initial treatment. The extended intervals may be used to minimize phototoxicity in order to study the cells over extended period of time without inducing the effects of photobleaching and to adapt to the constraints of long-term imaging in high content screening assays.

[0124] Some of the videos in the A375 dataset are annotated to train and evaluate the division detection methods as well as to evaluate the cell tracking methods disclosed herein. Specifically, the division dataset comprises 261 annotated division events across 10 videos. The annotations may capture a diverse array of division scenarios, providing a 38sf-6002883Docket No.78106-20008.40 comprehensive basis for training and testing a division detection algorithm as described herein. B. Apply a trained first model

[0125] Provided herein are methods for image-based cell division detection which comprise applying a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells. In some embodiments, the trained first model comprises a contrastive learning model. In some embodiments, the trained first models comprises a simCLR model, a SimSiam model, or a DINO model. In some embodiments, the trained first model is simCLR. At its core, simCLR operates by generating multiple augmented versions of a given image and then training a neural network to identify which augmented views originate from the same base image. The key idea is to pull the representations of positive pairs (different augmentations of the same image) closer together while pushing the representations of negative pairs (augmentations of different images) apart. This is achieved through a contrastive loss function, known as the NT-Xent (Normalized Temperature-scaled Cross- Entropy Loss), which for a positive pair of examples may be given by:with ziand zjthe representations of two augmented versions of the same image, sim(zi, zj) is the cosine similarity between these representations, τ the temperature scaling parameter, and N the number of images in the batch. The indicator function ⊮[ ^^^^≠ ^^^^]equals 1 when ^^^^^^^^ and 0 otherwise. This formula may essentially measure the probability that a pair of transformed images are recognized as originating from the same base image, normalized by the similarity of all other possible pairings in the batch.

[0126] In the context of weakly and self-supervised learning, one can categorize tasks into two types: pretext and target. Pretext tasks may be solved during the initial training phase: the model may learns representations from data by solving a task that is not ultimately useful but for which data may be available in large quantities. One can then use these learned representations to solve the target tasks, e.g. the one wanted to be solved using the method but for which only limited expert-annotated data is available (e.g. specific predictive or 39sf-6002883Docket No.78106-20008.40 classification tasks). As used herein, these may include identifying cell division events and tracking cell movements over time.

[0127] In this regard, the choice of the pretext task and the applied augmentations in the simCLR framework can be seen as a form of supervision, as they directly influence the properties of the obtained representation space, which in turn influence its efficiency in helping to solve the downstream task . For example, having rotations within the augmentations may be useful to devise representations ultimately invariant to such rotations.

[0128] In some embodiments, applying the first trained model comprises employing the contrastive learning model by employing the workflow presented in the exemplary embodiment illustrated in FIG.9A.

[0129] In applying the first trained model, each image within a candidate triplet ^^^^^3^^^, identified as described herein, can be treated independently. The image data may be duplicated and augmentations may be applied. In some embodiments, the augmentations comprise augmentations from simCLR. In some embodiments, the augmentations comprise applying Poisson noise to emulate the noise generated by microscopy imaging, Additive White Gaussian Noice (AWGN) to emulate the variation in the expression of the lit-up fluorescent pixels and random background noise, Salt and Pepper Noise to emulate a variation in the location and expression of the fluorescent reporters by simulating activation or inhibitions of fluorescent proteins. In some embodiments, the augmentations comprise novel augmentation techniques. In some embodiments, the novel augmentation technique is channel switching. Channel switching may involve the random permutation of fluorescent channels. This method served as a substantial color augmentation, enhancing the strength of Self-Supervised Learning (SSL) features. Without being held to this theory the rationale may be that more extensive augmentations generate more robust features. Additionally, this augmentation may diminish the model’s reliance on the expression of individual fluorescent reporters, leading to a more generalized and morphology-oriented representation. The resulting features may be denoted as division-simCLR, and the distance between representations of two given cells can be calculated using the cosine distance metric, for which the simCLR model was trained with.

[0130] In some embodiments, candidate cells are selected by identifying a cell triplet representing a possible first mother cells in the first image that divided into a set of two possible daughter cells in the second image. In some embodiments, calculating, based on the second image, a second distance between two cells in the population of cells at the second timepoint; identifying based on the first image and the second image, a corresponding set of one or more cells in the population of cells at the first time point; calculating, based on the 40sf-6002883Docket No.78106-20008.40 first image, a first distance representing a distance based on the set of one or more cells in the population of cells in the first image; identifying the cell triplet as the two cells and the corresponding two cells if the second distance is less than or equal to the first distance. In some embodiments, identifying a cell triplet further comprises determining based on the second image that the second image is less than a predefined maximum distance.

[0131] For the data disclosed herein, the dataset comprises 128x128 pixel patches associated with individually segmented cells. For each cell c, its time-context may be considered as a triplet of cell patches, ^^^^^^^^In some embodiments, ^^^^^^^^captures both information about their immediate neighborhood as well as their motion. Each patch may be a one zero-padded square patch centered around the cell located at a time (t). Patches ^^^^^^^^^^^^−1^^^^ ^^^^ ^^^^ ^^^^^^^^^^^^+1may share the same spatial coordinates but situated in the preceding and the succeeding frame, respectively. For cells in the first and last frames of the time-series, thepatch may be repeated in place of the predecessor or successor patch. For instances, if a cell remained stationary, the patches at t and t+1 may center around the cell. Conversely, if the cells underwent significant movement, the patches at t and t+2 may reflect a spatial shift.

[0132] Candidate cell triplets may be selected from the video. A candidate cell triplet may be constituted by a possible mother cell at frame t and associated two daughter cells at frame t + 1. These triplets may be subsequently classified as either positive or negative division events. This classification problem is typically highly imbalanced as the number of division events is very low compared with the number of potential mother-daughters candidate in a video. The selection of candidates may involve generating a graph representation of the cell population in the video over time, aiming to match potential mother cells with their potential daughters. For a video spanning T frames, where each frame contains Nt cells ^^^^^^^^^^^^, a graph G may be constructed consisting of a vertex set V and an edge set E. The vertex ^^^^^^^^, ^^^^=� ^^^^^^^^ ^^^^^^^^, ^^^^^^^^�, may be created for every pair of cells within the same frame (except the first frame) which have a centroid distance less the Dmax. Subsequently for each pair of cells in v, the preceding frame can be searched for cells that maintain a mean centroid distance to the cells ^^^^^^^^^^^^and ^^^^^^^^^^^^smaller than Dmax. If a cell ^^^^^^^^^^^−^1satisfies the criterion, it can be incorporated into the vertex set as vm= ^^^^^^^^^^^−^1, and an edge em,i,j= (vm, vi,j) can be created. Thus, each edge in E may represent a mother-daughter candidate triplet. In some embodiments, Dmax is selected to match 95% of the annotated division events and not 100% so as to remove (and deliberately ignore) outlier triplets. Reducing the number of candidates selected overall may be key to not only make 41sf-6002883Docket No.78106-20008.40 sure true positives are included, but that negatives are filtered to reduce the class imbalance in the classification dataset. C. Compute a features vector

[0133] Provided herein are methods for image-based cell division detection comprising for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells.

[0134] In some embodiments, the feature vector comprises a set of hand crafted features. In some embodiments, the hand crafted features represent a plurality of relational characteristics between cells in the set of candidate cells. In some embodiments, the feature vector comprises a plurality of relational characteristics between cells in the set of candidate cells. In some embodiments, the plurality of relational characteristics comprise one or more Spatial Euclidean distance between cells in the set of candidate cells, one or more size differences between cells in the set of candidate cells, one or more aspect ratio differences between cells in the set of candidate cells, one or more pairwise cosine distances from the embedding vectors associated with the candidate cells. In some embodiments, the feature vector comprises at least 41 dimensions.

[0135] In some embodiments, the features vector can be generated according to the methods disclosed herein. For every edge in the graph, representing a potential division event triplet, the time-context division-simCLR representations for the mother cell and its two potential daughter cells can be computed. Given a candidate mother cell cmalongside its who prospective daughters cd0and cd1, triplet image patches ^^^^^3^^^, ^^^^^3^^^0, ^^^^ ^^^^ ^^^^centered in the respective cells’ centroid can be obtained. For each potential division a set of hand crafted vectors is generated to evaluate the candidacy of a mother cells cmand its two potential daughter cells cd0and cd1.In some embodiments, the handcrafted vectors may include: • Spatial Euclidean distances between the mother candidate and each daughter candidate, as well as between the two daughter candidates: DL2(cm, cd), DL2(cm, cd), DL2(cd, cd). • Size Euclidean distances between the daughter candidates, based on their areas (sd0,and sd1), calculated as:• Aspect ratio difference between daughter candidates, using their aspect ratios (ARd0, ARd1), computed42sf-6002883Docket No.78106-20008.40 • Pairwise cosine distances within the SimCLR embedding space for the candidates cm, cd0, and cd1with their embeddings D ^^^^^3^^^, D ^^^^^3^^^0, D ^^^^^3^^^1, by calculating the cosine distance between each pair DS^^^^ ^^^^i,DSj, of 9 total embeddings as: ^^^^^^^^ ∙ ^^^^ ^^^^ ^^^^^^^^ ^^^^ ^^^^= 1 − This mayresult in 36 unique pairwise cosine distances for the 9 embeddings Thus, for any edge em,d0,d1in E, a feature vector may be constructed as HCm,d0,d1, encapsulating 41 distances. D. Apply a Dual input classifier model

[0136] Provided herein are methods for image-based cell division detection comprising applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process. In some embodiments, the dual input classifier model comprises a recurrent encoder, wherein the recurrent encoder condenses the dimensionality of the embedding vectors associated with the candidate cells to produce a concatenated embedding vector. In some embodiments, the concatenated embedding vector encodes a temporal relationships between respective cells in the candidate cells. In some embodiments, the dual input classifier model further comprises a binary classifier model, wherein the binary classifier model integrates the concatenated embedding vector and the features vector to generate the classification output.

[0137] A dual-input classifier specifically tailored for identifying division events may be constructed. It can be configured to process both the division-simCLR embeddings and the hand-crafted feature vectors simultaneously to perform the classification task. The architecture of the model can be made of two main components: (1) a recurrent encoder that condenses the dimensionality of the embeddings, where the input consists of concatenated cell triplets’ simCLR embeddings along the time dimension, capturing the temporal relationship between mother and daughter samples, and (2) a binary classifier that integrates the recurrent encoder’s hidden state with the hand-crafted feature vector to make its classification. FIG.10 shows an exemplary model according to some embodiments.

[0138] The recurrent encoder may employ a Long Short-Term Memory (LSTM) network with a hidden layer size of 64 across two layers, including a dropout rate of 0.2 to prevent overfitting. The Binary classifier Multi-Layer Perceptron (MLP) may utilize four linear layers with intermediary sizes of 64, 32, 16, and 8, activated by ReLU functions, leading to a final classification layer activated by a Sigmoid function. 43sf-6002883Docket No.78106-20008.40

[0139] The model may be trained using a weighted Binary Cross-Entropy loss and optimized with ADAM gradient descent, with sample weights computed to balance the dataset’s class imbalance among candidate division triplets. A learning rate of 1x10-5may be adopted, which can gradually be adjusted using cosine annealing over a period of 50 epochs to a minimum learning rate of 1x10-7. To address dataset imbalances and avoid bias from any specific annotated video - and thus its experimental conditions - examples can be up sampled from videos with fewer division events to equalize the number of division events across the training dataset.

[0140] Aggregated ensemble learning can be used. With k, the total number of videos used for the training and p, the number of videos used for training of each fold. For each fold, (k-p) videos can be used for training and evaluation was completed on the remaining p unseen videos. For each fold, the (k-p) training-split videos can themselves be divided into a training set (80%) and a validation set (20%). The latter can be used for early stopping. This approach can result in k / p models, each trained on (k-p) training videos. During inference, the predictions may be aggregated from all models, averaging the probabilities for assignment to the graph’s edge.

[0141] After classifying all graph edges, division consistency can be ensured by iteratively choosing edges with the highest probability scores and removing their corresponding nodes until no positively classified edge remains. This process may result in the identification of mother- daughter sets indicative of confirmed division events, using minimal annotations for the classifier by leveraging the self-supervised contrastive features.

[0142] Various metrics may be employed to evaluate the methods for image-based cell division detection. For the cell division classifier, in some embodiments, a F1-score, a harmonic mean of precision and recall, alongside the area under the precision-recall curve (AUC), and the model’s Accuracy can be employed. These measures may provide insights into the model’s performance, especially in dealing with class imbalances, with scores ranging from 0.0 to 1.0—where scores closer to 1.0 indicate better performance. In the context of learning outcomes, these metrics may be calculated through cross-validation, ensuring a unified evaluation across all test folds for a direct comparison of trainable and non-trainable models. This approach aggregates the predictions to produce a single, comprehensive performance score.

[0143] Some of the methods provided herein are benchmarked using datasets from the Cell Tracking Challenge (CTC), a well-established resource providing a variety of cell tracking scenarios and datasets. While the CTC datasets typically feature much shorter time intervals between frames, the datasets may be adjusted to match the 3-hour time interval of the A375 44sf-6002883Docket No.78106-20008.40 cell line dataset. This adjustment ensures a consistent basis for evaluating the effectiveness of the tracking and division detection methods described herein under similar temporal resolution constraints, thus allowing a direct comparison and validation of the methodologies against established benchmarks. CTC ground truths for cell segmentation may be used for evaluation purposes.

[0144] Deep feature analysis: A comparative evaluation of the different deep learning backends presented herein may be conducted. This evaluation may oppose the division- simCLR features with analogous formats based on AlexNet and ResNet backends. In combination, two types of representations can be explored: the use of temporal-context, denoted as = [ ^^^^^^^^^^^^−1, ^^^^^^^^^^^^, ^^^^^^^^^^^^+1], and a single-framework, ^^^^^1^^^= [ ^^^^^^^^^^^^], within the classification framework. The results of this analysis according to some embodiments are presented in Table 3. The presented results are computed using leave-p-video-out cross-validation, with 5 folds each with testing set size of 2 videos. Iteratively 20% of the annotated dataset are considered as test data and the model is trained on the remaining data, until the entire dataset is evaluated. Table 3: Evaluation of the performance of the division detection method with and without our contrastive features and the inclusion of the time-context. These variations of the tracking algorithm on the dataset (A375 with a temporal resolution of 240min) are assessed using the AUC, the F1-score, the Precision, the Recall, and the Accuracy.

[0145] The results indicate that deep features, trained on ImageNet, achieves an F1-score of up to 0.36 when time-context is applied to resnet features. In contrast, relying solely on single time-point data decreases performance, with the F1-score dropping to a maximum of 0.30 when using alexnet features. The introduction of the simCLR feature significantly improves outcomes, reaching an F1-score of 0.42 without time-context and rising to 0.48 with its addition. Correspondingly, the Precision in division detection with division-simCLR features increases from 0.47 to 0.63 with the inclusion of time-context. These findings affirm the efficacy of the approach of the embodiments described herein, highlighting the benefits of using contrastively trained features and integrating time-context. 45sf-6002883Docket No.78106-20008.40

[0146] Cell division detection method benchmarking: a comparative analysis of the method disclosed herein, Contrastive Division Detection, against the Lineage Mapper’s (LM) division detection algorithm, an unsupervised tracking tool notably integrated into Fiji may be conducted. The widely used LM for the benchmark may be chosen, as it is - like the methods provided herein - a semantic cell division method, and comes with few annotation requirements. This comparative assessment can be undertaken across four distinct datasets: the A375 cell line dataset and three datasets from the Cell Tracking Challenge (CTC) that presented a sufficient number of division events for effective training of the methodologies described herein.

[0147] A noteworthy point in this comparison is the adaptation necessitated by the CTC datasets. Due to constraints related to the size of these datasets, as well as the diversity in cell lines and imaging parameters within the CTC data, training a division-simCLR model may not be feasible. As a result, the division-alexnet features may be employed for analyses involving the CTC datasets.

[0148] The outcomes of this benchmarking exercise according to embodiments described herein are detailed in Table 4. The results underscore the performance of the methods across both the A375 and CTC datasets. Particularly in the context of the A375 dataset, the method demonstrates a pronounced efficacy, underscoring its potential in advancing division detection capabilities within diverse cellular datasets. Precision may be prioritized over recall to minimize the impact of false positives, which can introduce significant errors into the tracking process. This approach may reduce the risk of mistakenly identifying non-existent division events, which could complicate tracking accuracy and integrity. By focusing on precision, it may be ensured that a high likelihood of correct division detection, maintaining the overall quality and effectiveness of the tracking system. Table 4. Benchmarking of the proposed division detection method against the Lineage Mapper algorithm on the A375 and CTC datasets using their native time resolution. These methods using the AUC, the F1-score, the Precision, the Recall and the Accuracy were evaluated.46sf-6002883Docket No.78106-20008.40

[0149] On the A375 dataset, the Contrastive Division Detection method significantly outperforms Lineage Mapper, achieving an F1-score of 0.47 versus Lineage Mapper’s 0.11. However, the comparison yields mixed results across the CTC datasets, each with distinct characteristics. For the Fluo-N2DH-SIM+ dataset, which features simulated nuclei of HL60 cells stained with Hoechst, the simulated nature seemingly impacts the performance of the method negatively, leading to Lineage Mapper outperforming the approach. The lack of significant evolution in cell appearance and phenotype within this dataset render the Contrastive Division Detection method less effective than Lineage Mapper.

[0150] Conversely, in the PhC-C2DL-PSC dataset, which comprises pancreatic stem cells imaged with Phase Contrast microscopy on a polystyrene substrate, the Contrastive Division Detection method surpasses Lineage Mapper, registering an F1-score of 0.7 against Lineage Mapper’s 0.18. Similarly, on the Fluo-N2DL-HeLa dataset, which closely resembles the A375 dataset, the Contrastive Division Detection method again outperformed Lineage Mapper, achieving an F1-score of 0.77 compared to Lineage Mapper’s 0.11. These notable improvements on the latter two datasets can be attributed to the distinct cell characteristics captured by morphology or fluorescent expression, enabling the Contrastive Division Detection model to more effectively discern division events even under challenging conditions.

[0151] It is important to note that the CTC dataset results are achieved using AlexNet features instead of division-simCLR features, as the CTC datasets does not provide a sufficient data volume for effective simCLR model training. Additionally, the Contrastive Division Detection method demonstrated here has the highest performance on the HeLa dataset, which is not only the most similar to the A375 dataset but also had the lowest temporal resolution (30 minutes) among the three evaluated CTC datasets.

[0152] The Contrastive Division Detection method may be explored as a function of the temporal resolution. To this end, an experimental time subsampling on the Cell Tracking Challenge (CTC) datasets may be conducted and utilized in the benchmarking. This process may involve adjusting the temporal resolution of the dataset to time-step sizes of 60, 120, 180 minutes and 240. The objective may be to observe the variation in Contrastive Division Detection algorithm’s performance with changing temporal resolutions and to align these conditions with those of the dataset described herein. The analysis of these results may focus however mostly on time intervals up to 180min, as beyond this time resolutions, results are difficult to consider due to the resulting number of frames in the videos after the time subsampling. 47sf-6002883Docket No.78106-20008.40

[0153] The results of this temporal resolution simulation according to embodiments of the invention are presented in Table 5. This analysis reveals that Contrastive Division Detection method maintains a coherent and satisfactory level of performance when subjected to significant reductions in temporal resolution. In the case of the HeLa dataset which is most similar to the one these method are developed for in some embodiments, classification scores well above commonly-used unsupervised methods are produced. This evaluation provides insights into the performance of the methods described herein, especially at low temporal resolutions on benchmarking datasets. It must however be noted that these results assess the Contrastive Division Detection method when using alexnet features due to lack of available training data, and thus a better performance could be expected if using simCLR features. Table 5: Benchmarking of the division detection method disclosed herein against the Lineage Mapper algorithm on the CTC datasets using artificial time resolutions of 60min, 180min and 240min. The two methods were evaluated at these time-steps using the F1-score, the Precision, the Recall.

[0154] For the Fluo-N2DH-SIM+ dataset, Lineage Mapper continue to outperform the Contrastive Division Detection method, which can be attributed to the limited motion in the simulated dataset and the previously noted static cell appearance. On the Fluo-N2DL-HeLa and PhC-C2DL-PSC datasets, Lineage Mapper’s performance yields F1-scores ranging between 0.08 and 0.18, displaying significantly low recalls across all temporal resolutions. Conversely, the Contrastive Division Detection method experiences a performance decrease from the native time resolution results, which correlates with the decreased temporal resolutions. Nonetheless, the performance remains satisfactory, with F1-scores of 0.69 at ∆ = 60min, 0.64 at ∆ = 120min, and 0.46 at ∆ = 180min, alongside precision scores of 0.85, 0.79, and 0.53 respectively for the Fluo-N2DL-HeLa dataset. For the PhC-C2DL-PSC dataset, the F1-score is relatively stable across various time resolutions, ranging between 0.25 and 0.30. 48sf-6002883Docket No.78106-20008.40

[0155] The Contrastive Division Detection method effectively combines self-supervised learning with custom augmentations to enhance cell division detection and tracking across various datasets. This integration significantly outperforms traditional methods, highlighting the importance of tailored augmentations for this specific challenge. Initially, using standard simCLR models without these augmentations led to modest results. However, introducing specific augmentations to better capture cell division complexities marked a major improvement, increasing both the accuracy and reliability of our detection algorithm. ) Methods of cell tracking

[0156] Provided herein are methods that can be used for image-based cell tracking. FIG.11 schematically depicts steps that can be used in methods for image-based cell tracking. The methods described herein can produce cell tracks using image data annotated to designate mother-daughter cell divisions. The methods comprise producing and updating acyclic graph structures that are generated using embedding vectors. The tracks can be extracted from updated acyclic graph structures methods such as a Markov Decision Process (MDP).

[0157] In 1102, image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points are received, wherein the images have been annotated to designate mother-daughter cell divisions. The images may have been annotated using the method outlined in FIG.7. In some embodiments, receiving the image data comprises receiving image data based on an image based cell division detection method.

[0158] The image data from 1102 is used as input to 1104, wherein a trained first model is applied to generate a plurality of embedding vectors for the plurality of images. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 4 hour intervals. In some embodiments, the image data comprise images of live cells. In some embodiments, the image data comprise images from live cell image data. In some embodiments, the image data comprises images of cells from a high content screening experiments.

[0159] The image data from 1102 and the embedding vectors of 1104 are used in 1106 to generate an acyclic graph data structure G. In some embodiments, generating the acyclic graph data structure G comprises applying one or more constraints based on the relationship between the embedding vectors associated with the plurality of annotated images. In some embodiments, generating the acyclic graph data structure G comprises applying one or more 49 sf-6002883Docket No.78106-20008.40 constraints based on division relationships of mother-daughter cell divisions indicated by the annotated images. In some embodiments, generating the acyclic graph data structure G comprises define edge weights for the acyclic graph data structure G based on a computed composite distance between mother-daughter cells.

[0160] In 1108 subgraphs are extracted from acyclic graph structure G and in 1110 tracklets are iteratively from the subgraphs generated in 1108. In some embodiment, extracting the plurality of subgraphs from the acyclic graph data structure G comprises using a temporal rolling window. In some embodiments, extracting, from the one or more of the plurality of subgraphs, the plurality of tracklets comprises extracting the plurality of tracklets based on the plurality of tracklets having a minimal average cost along a path.

[0161] The subgraphs from 1108 and the tracklets from 1110 are used in 1112 to generate an updated acyclic graph structure G’. In some embodiments, generating the updated acyclic graph data structure G’ comprises weighting one or more edges of the updated acyclic graph data structure G’ based on a frequency of occurrences of a link in the tracklets.

[0162] In some embodiments, the method further comprises generating a single cell trajectory from the set of tracks. In some embodiments, the single cell trajectory is based on tracking the trajectory of a cellular compartment. In some embodiments, the single cell trajectory is based on tracking an interpretable feature. In some embodiments, an interpretable feature relates to cell morphology, texture, fluorescence intensity of a cellular marker, cell localization, cellular compartment localization, readouts from a biosensor, or other features known in the art to be tracked in response to a cellular perturbation. In some embodiments, the single cell trajectory is based on tracking a non-interpretable feature. In some embodiments, a non-interpretable feature may relate to a computationally defined feature. In some embodiments, a computationally defined feature may be related to image embeddings. In some embodiments, non-interpretable features can be tracked to analyze how cells respond to perturbation. In some embodiments, single cell trajectories can be combined to inform cellular adaptation in response to a perturbation. In some embodiments, analysis of cellular adaptation may be used to predict disease progression. In some embodiments, analysis of cellular adaptation can be used for drug discovery.

[0163] In 1114, a set of tracks that are indicative of respective cells in the images from the updated acyclic graph data structure G’ are extracted. In some embodiments, the plurality of tracklets represent possible temporal linkages between cell in the population of cells at the plurality of respective time points. In some embodiments, iteratively extracting the set of 50sf-6002883Docket No.78106-20008.40 tracks from the updated acyclic graph data structure G’ comprises using a Markov decision process.

[0164] As described herein, the cell tracking module aims at following individual cells over time, even at low temporal resolution, where spatial proximity is not the only relevant feature to consider. To do so, it uses weakly-supervised contrastive representation learning to generate robust features and graph assignment optimization. This module’s workflow is illustrated in FIG.12.

[0165] The weakly-supervised contrastive model training according to some embodments may comprise two main steps: (1) Generate “weak” preliminary tracks with an unsupervised algorithm, providing useful cell match pairs for further training despite their imperfections; (2) Leverage these weak examples to train a contrastive model and learn cell representations over time. Inference is then a two-step process: (1) Use the weakly-supervised contrastive model to extract features for each cell, enabling precise cell identification and tracking across frames; (2) Use graph optimization techniques that integrate spatial and other relevant features with our tracking contrastive features to construct accurate cell sequences over time.

[0166] The methods disclosed herein describes effectively utilizing contrastive features, trained using label-free weakly supervised tracks, through a graph optimization technique. This allows for accurate tracking of cells in images captured at greater than a 4-hour intervals without requiring human annotations, enabling effective downstream analysis at the individual cell level over time.

[0167] The data provided herein shows that development of a tracking-simCLR feature marks a notable improvement to previously developed tracking methods. This feature, derived from weakly supervised learning and temporal augmentations, facilitates the robust representation of cellular dynamics. Coupled with a Sliding Window Assignment Graph technique, it demonstrates superior performance in key tracking metrics, including Track Reconstruction Accuracy, Complete Tracks, and Track Fraction. The methodology’s adaptability to varied temporal resolutions may be particularly advantageous for extended High Content Screening studies, offering a means to circumvent the challenges of photo- toxicity for extended studies. A. Receiving Image data

[0168] In some embodiments, the methods disclosed herein comprise receiving image data comprising a set of images comprising a plurality of images depicting a population of cells at a 51sf-6002883Docket No.78106-20008.40 plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points.

[0169] In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 4 hour intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 3 hour intervals, greater than 4 hour intervals, greater than 5 hour intervals, greater than 6 hour intervals, greater than 7 hour intervals, or greater than 8 hour intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at less than 8 hour intervals, less than 7 hour intervals, less than 6 hour intervals, less than 5 hour intervals, less than 4 hour intervals, or less than 8 hour intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at intervals between 2 hours and 8 hours, 2 hours and 7 hours, 2 and 6 hours, 2 and 5 hours, 2 hours and 4 hours or 2 hours and 3 hours. In some embodiments, the plurality of respective time points comprises a plurality of time points at intervals between 2 hours and 8 hours, 3 hours and 8 hours, 4 and 8 hours, 5 and 8 hours, 6 hours and 8 hours or 7 hours and 8 hours.

[0170] As described above, one of skill in the art may appreciate the plurality of respective time points comprises a plurality of time points at intervals that depend on the cellular system being sampled. Factors which may impact the time between time points include but are not limited to the level of cell motility and the rate of cellular division and the rate of other adaptations to a given cellular perturbation, for example an adaptation that causes treatment resistance. Changes in cell motility and morphology may occur on timescales of seconds to minutes. Division events in yeast cells occur on time scales of minutes to hours and in mammalian cells on time scales of approximately 1 day. Adaptations to treatment may occur on timescales of days to weeks or even months. When the cells depicted in the cell images are highly motile, divide rapidly (short cell cycle), the time between the first time point and the second time point may be shorter than when the cells depicted in the cell images are less motile and divide slowly (long cell cycle). In some embodiments, the time intervals are chosen as appropriate for the experimental system. The appropriateness of the time intervals may be related to the variability of cell dynamics and the time between divisions for the cells in the experimental system.

[0171] In some embodiments, the cells depicted in the images of cells have a cell cycle interval of about 24 hours. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 4 hour intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 3 hour intervals, greater than 4 hour intervals, greater than 5 hour intervals, greater than 6 hour intervals, 52sf-6002883Docket No.78106-20008.40 greater than 7 hour intervals, or greater than 8 hour intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at less than 8 hour intervals, less than 7 hour intervals, less than 6 hour intervals, less than 5 hour intervals, less than 4 hour intervals, or less than 8 hour intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at intervals between 2 hours and 8 hours, 2 hours and 7 hours, 2 hours and 6 hours, 2 and 5 hours, 2 hours and 4 hours or 2 hours and 3 hours. In some embodiments, the plurality of respective time points comprises a plurality of time points at intervals between 2 hours and 8 hours, 3 hours and 8 hours, 4 hours and 8 hours, 5 and 8 hours, 6 hours and 8 hours or 7 hours and 8 hours.

[0172] In some embodiments, the cells depicted in the images of cells are yeast cells. In some embodiments, the yeast cells have cell cycle interval of about 90 min. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 4 minute intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at greater than 3 minute intervals, greater than 4 minute intervals, greater than 5 minute intervals, greater than 6 minute intervals, greater than 7 minute intervals, or greater than 8 minute intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at less than 8 minute intervals, less than 7 minute intervals, less than 6 minute intervals, less than 5 minute intervals, less than 4 minute intervals, or less than 8 minute intervals. In some embodiments, the plurality of respective time points comprises a plurality of time points at intervals between 2 minutes and 8 minutes, 2 minutes and 7 minutes, 2 minutes and 6 minutes, 2 minutes and 5 minutes, 2 minutes and 4 minutes or 2 minutes and 3 minutes. In some embodiments, the plurality of respective time points comprises a plurality of time points at intervals between 2 minutes and 8 minutes, 3 minutes and 8 minutes, 4 minutes and 8 minutes, 5 and 8 minutes, 6 minutes and 8 minutes or 7 minutes and 8 minutes.

[0173] In some embodiments, the image data comprises live-cell microscopy data derived from an HCS assay. In some embodiments, the image data comprises live-cell video microscopy data derived from High Content Screening (HCS) assays on a A375 cell line as described herein.

[0174] In some embodiments, the image data comprises a tracking dataset. According to the data described herein the tracking dataset may encompass 528 distinct cell tracks, for a total of 1538 individual cell observations across time within 3 videos for which we also have annotated division events. The dataset may be defined to reflect the dynamic nature of cell movement and interaction over extended periods, which may be critical for assessing the 53sf-6002883Docket No.78106-20008.40 performance and accuracy of an unsupervised tracking method. The image sequences may be annotated using previously acquired segmentation performed using the finetuned models described herein. B. Apply a trained first model

[0175] Provided herein are methods for image-based cell tracking comprising applying a trained first model to a set of images to generate a plurality of embedding vector associated with a plurality of images.

[0176] To generate cell representations useful for cell tracking, a weakly-supervised contrastive learning approach may be used. It consists of using an unsupervised (although sub- optimal) tracking algorithm to generate cell tracks from video-microscopy datasets, and then training a trained first model (e.g. simCLR model) to generate representations such that consecutive cells in tracks are close to one-another in the resulting representation space. This approach is illustrated in one embodiment in FIG 9B. A Lineage Mapper was used as unsupervised tracking algorithm to generate a weakly-supervised dataset of pairs of consecutive cells (e.g. cell matches). It uses auxiliary information (not directly contained within the images) about cells to generate tracks, namely the cell’s spatial distance, overlap and size ratio. While the resulting set of tracks does not attain the same quality as human annotations or supervised tracking approaches, it can be amassed at scale without requiring human intervention. In the data provided herein, validation upon annotated tracks confirm that the method achieves a Tracking Fraction (TF) score of 0.75, representing the average correctness of links within tracks. This level of performance may be considered adequate for enabling the representation model to acquire generalized embeddings pertinent to the tracking task. The resulting dataset may comprise pairs of cell matches, each patch being of size 128 × 128 pixels and centered around the cell.

[0177] Subsequently, a simCLR model can used to generate representations of cell matches based on the weakly-supervised dataset of cell tracks. However, instead of training the model to match different augmentations of the same image, the approach can be adapted to match the same cell at two consecutive time-points from within the tracks. This adaptation effectively incorporates time-shifting as an augmentation technique within the standard simCLR framework, allowing the model to learn cell representations that are optimized to remain constant over time. Furthermore, a classical simCLR augmentations can be applied along with microscopy-specific augmentations. In some embodiments, the resulting features 54sf-6002883Docket No.78106-20008.40 can be denoted as tracking-simCLR, and the distance between representations of two given cells is calculated using the cosine distance metric.

[0178] In some embodiments, generating the embedding vectors further comprises integrating features including spatial, hand-crafted, and deep learning-based features. In some embodiments, the features are paired with distance metrics. The features paired with distance metrics may contribute to a comprehensive deposit distance metric facilitating nuanced comparisons between cells in the plurality of images.

[0179] In some embodiments, the spatial features comprise features known in the art for tracking applications. In some embodiments, the spatial features comprise a centroid distance and an overlap metric. As used herein, a centroid distance (Dcentroid) may be computed as: the Euclidean distance between two cells ( ^^^^^^^^^^^^and ^^^^^^^^^^^^), ( ^^^^^^^^ ^^^^^^^^, ^^^^^^^^) and ( ^^^^^^^^^^^^, ^^^^^^^^^^^^). Dcentroid is a direct measure of cellular spatial separation. As used herein, an overlap metric measures the pixel area overlap between two cells ^^^^^^^^^^^^and ^^^^^^^^^^^^. The overlap metric, Doverlapmay be measured as the degree of spatial intersection between cells. D may be defined as Doverlap= 1- whereiand ajrepresent the pixel areas of the respect cells and oi,jtheir intersection.

[0180] In some embodiments, the distance in cell size of candidates may be calculated to compare candidate morphologies. The Size distance may be calculated as the size difference between two cells, ^^^^^^^^^^^^and ^^^^^^^^^^^^, in terms of their pixel areas, ai, and aj. The distance metric, Dsize, reflects the relative size variation. D can be defined as Dsize =max ( ^^^^ ^^^^, ^^^^ ^^^^)

[0181] For the data provided herein, several Deep-Learning feature sets with respect to their performance in the workflow from both standard pre-trained classification models and custom trained models may be used. The Pretrained features may include those extracted using pre- trained models trained on extensive Image-Net dataset. The ResNet-50 model and AlexNet model can be utilized. Features can be collected by extracting centered 128x 128 bounded boxes around individual cells and pass them through the selected model, capturing the internal representation just before the classification layer. In some embodiments, the process yields a 2048-dimensional vector for ResNet-50 and a 4096-dimensional vector for AlexNet. When considering two cell, e.g. ^^^^^^^^^^^^and ^^^^^^^^^^^^, the representative embeddings can be extracted, alexne ^^^^^^^^^^^^and alexne ^^^^^^^^^^^^, and the distance between these cells an be computed as L2 norm: ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^^^^^^^^^�. An alternative time-context feature extraction technique can be embedded for each image triplet for each cell. To quantify the distance between two 55sf-6002883Docket No.78106-20008.40 cells using this approach, the L2 norms of their temporally aligned embeddings can be summed. In some embodiments, these features are referred to as division-alexnet and division-resnet, depending on the model employed.

[0182] Deep features can be generated from models trained on the dataset. A key component of these features can be derived from a custom-trained simCLR model, as elaborated herein. For the purpose of tracking, the simCLR model can be leveraged to analyze the ^^^^^^3^^time-lapse representations, capturing the temporal evolution of cellular features. The distances between cell pairs can be determined by synchronizing the embeddings with respect to their temporal stamps, a process that can be denoted as division-simCLR. Additionally, for purposes of comparative evaluation, the distances using the simCLR embeddings from individual frames can be assessed, and can be referred to as division-simCLR. In both scenarios, the similarity between cells can be quantified using the cosine distance metric, ensuring a consistent and robust measure of feature similarity, as per simCLR loss function.

[0183] In some embodiments, generating the embedding vectors comprised weakly-supervised contrastive learning features. In such embodiments, embeddings for individual cells in time were extracted using the previously described tracking-simCLR model. Those features can be optimized for the tracking of cells using weakly supervised contrastive learning as described herein.

[0184] It should be noted that the methods described herein are not confined to specific features or distance metrics. The methods are designed as a flexible framework, adaptable to various feature types suitable for the dataset at hand. Table 5 summarizes various exemplary features used in tracking methodology, along with their corresponding distance metrics: Table 5: Exemplary embedding features for tracking56sf-6002883Docket No.78106-20008.40 C. Generating an acyclic graph structure

[0185] Provided herein are methods for image-based cell tracking comprising, generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G. In some embodiments, generating the acyclic graph data structure G comprises applying one or more constraints based on the relationship between the embedding vectors associated with the plurality of annotated images. In some embodiments, generating the acyclic graph data structure G comprises applying one or more constraints based on division relationships of mother-daughter cell divisions indicated by the annotated images. In some embodiments, generating the acyclic graph data structure G comprises define edge weights for the acyclic graph data structure G based on a computed composite distance between mother-daughter cells.

[0186] The approach to cell tracking described herein capitalizes on the features generated to characterize individual cells over time, utilizing a graph structure to represent cellular data efficiently across temporal sequences. This graph structure facilitates the matching of cells, now vertices in the graph, and the decision-making regarding cell linkages, which are influenced by constraints such as time, localization, and division events. These constraints are represented by the presence of an edge, while the similarity between cells in the feature space is denoted by the edge weights. By identifying optimal paths within this graph, delineate cell tracks can be identified throughout the video, strictly adhering to the predefined conditions - e.g. a cell without a successor marks the end of a track, while one with successors indicates an ongoing track.

[0187] In some embodiments, the methods describ ied herein introduce a methodology for representing cellular movements within a video through video through a constrained weighted graph. Specifically, a weighted directed graph, denoted as G = (V, E ) can be constructed, where V represents the set of vertices corresponding to individual cells, and E comprises the edges indicating potential connections between these cells. Each vertex v in V represents a cell object within an image frame, identified as ^^^^^^^^^^^^where i is the cell’s index in the frame, and t is the frame’s relative timestamp. The edges E may connect vertices from V based on specific criteria: • An edge e can only connect cell objects from one frame to the next, e.g., between times t and T= t+1. • A vertex v modeling a cell ^^^^^^^^^^^^which is a mother in a division event cannot have outgoing edges. 57sf-6002883Docket No.78106-20008.40 • Similarly, a vertex v modeling a cell ^^^^^^^^^^^^which is a daughter in a division event cannot have ingoing edges. • An edge is established between two vertices viand vj, corresponding to cells ^^^^^^^^^^^^and ^^^^^^^^^^^^, if the spatial distance between them, calculated�, is below the maximum distance threshold Dmax. The weights of edges in E can be determined by composite distance derived from features extracted from the associated cell objects. For a pair of vertices viand vj, representing cells ^^^^^^^^^^^^and ^^^^^^^^^^^^, the edges weight wijcan be calculated using the following formula: ^^^^^^^^ ^^^^= where K is the set of features related to the cellobjects, Dkis the distance function applied to these features, and akrepresents the weight assigned to each feature’s distance. The features can be normalized to have a mean of 0 and a standard deviation of 1 over the entire video. C. Extracting subgraphs and tracklets to update G

[0188] Provided herein are methods for image-based cell tracking comprising, extracting a plurality of subgraphs from the acyclic graph data structure G; extracting from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’. In some embodiments, extracting the plurality of subgraphs from the acyclic graph data structure G comprises using a temporal rolling window. In some embodiments, extracting, from the one or more of the plurality of subgraphs, the plurality of tracklets comprises extracting the plurality of tracklets based on the plurality of tracklets having a minimal average cost along a path.

[0189] In some embodiments, generating the updated acyclic graph data structure G’ comprises constraining the updated acyclic graph data structure G’ based on occurrence of an edge in the plurality of tracklets. In some embodiments, generating the updated acyclic graph data structure G’ comprises weighting one or more edges of the updated acyclic graph data structure G’ based on a frequency of occurrences of a link in the tracklets. In some embodiments, the plurality of tracklets represent possible temporal linkages between cell in the population of cells at the plurality of respective time points. In some embodiments, extracting a plurality of subgraphs from the acyclic graph data structure G; extracting from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the 58sf-6002883Docket No.78106-20008.40 acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’ comprises adopting a sliding window assignment graph (SWAG) approach.

[0190] In order to assess cell linkages under varying multiple initial conditions (start, end, and length of video), tracklets may be computed over sub-graphs. This approach may allow for a probabilistic rather than a deterministic track evaluation, which may enable several evaluation of each cell’s role within a track but also allows for the consideration of various track dynamics influencing cell matching within the framework of the sub-graphs.

[0191] According to the data presented herein, a rolling-window method can be implemented to extract subgraph and tracklet and to facilitate a structured and detailed examination of the temporal dynamics and lineage relationships of cells throughout the time series, all the while allowing each cell to be considered at least once as the potential start or end of a track. This approach, involving the definition and analysis of rolling window sub-graphs, can be characterized by two parameters: the window size l and the stride s.

[0192] Using the rolling window, sub-graph Gt+lcan be defined as the subset of th te larger graph G that includes only the nodes corresponding to cell objects in frames from timestamp t to t + l. Sub-graphs from G were extracted systematically, beginning at t = 0 and incrementing t by s until we cover the entire duration of the image time series, denoted by T.

[0193] Furthermore, in each of these sub-graphs, ^^^^^^^^^^^^+1two new verti tces can be introduced, vSand vT, which serve as the source and sink, respectively. These two vertices may serve as placeholders for the beginning and end of tracks: A track in the graph can be any path between vSand vT. These vertices can be connected to the existing vertices in the sub-graph using directional edges weighted at 0. An edge from vSto a vertex vican be created if vihas no incoming edges, indicating the absence of predecessors for viwithin the sub-graphConversely, an edge from a vertex vjto vTcan be established if vjlacks outgoing edges, implying no successors for vjwithin the same sub-graph. In each subgraph, tracklets can be computed using a method comprising the following steps: The shortest average path between the source node vSand the sink node vTwithin the sub-graph was ^^^^^^^^^^^^+1calculated. The calculation can denote a tracklet aswhich includes all vertices present along the identified path. The vertices that are part of the tracklet, can be removed from the sub-graph, effectively updating the graph’s structure. The process can be repeated: if the sub- graph still contained vertices other than vSand vTremain the in the sub-graph. The shortest average path algorithm may be defined according to the algorithm in Table 6. 59sf-6002883Docket No.78106-20008.40Table 6: Shortest average path algorithm according to an embodiment.Algorithm 1 Algorithm for Shortest Average Pathprocedure AVGSHORTESTPATH( ^^^^ = ( ^^^^,ℰ), ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^)^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^[] ←{∞,∞,⋯ ,∞}^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^[ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^for ^^^^ ← 1for all^^^^ ^^^^ + ^^^^ ^^^^ ^^^^ ^^^^( ^^^^, ^^^^) ^^^^ ^^^^ ^^^^[^^^^]+ 1^^^^ ^^^^ if ^^^^then ^^^^ ^^^^ ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ℎend ifend forend forreturn ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^end procedure

[0194] The extracted tracklets represented possible linkage between cells in multiple initial video conditions (duration, start and end). The tracklets can then be combined to form entire tracks spanning throughout the videos. To combine the tracks, after computing tracklets for all sub-graphs, a comprehensive set of tracklets can be created that spanned the entire graph G. This set enables for the determination of the frequency of selection for each edge between two nodes in the graph. Here, the edge frequency can indicate how often a link between two nodes appears across the entire set of trackless. Using this data, a new directed graph G=( V’, E’) can be constructed as explained hereafter.

[0195] The vertex set V’ = V ∪ (v’s, v’T) may supplement the original graph vertices by the addition of source and sink vertices v’sand v’T. Thus, these vertices may correspond to cells ^^^^^^^^^^^^and ^^^^^^^^^^^^or represent the start and terminal states of a tracklet. Then an edge e’i,jin E’ can connect two vertices v’iand v’jif and only if there is at least one connection between these vertices in the set of previously computed tracklets.

[0196] The occurrence of each edge e’i,jcan then be transformed in the set of tracklets into a transition probability, represented as pi,j. This can be defined in some embodiments as:60sf-6002883Docket No.78106-20008.40 where K denotes the set of successor vertices from the node vi, and oi,jrefers to the frequency of the edge e’i,jlinking vertices v’iand v’jin the set of tracklets. This calculation of transition probabilities can provide a quantitative basis for determining the likelihood of cell transition within the graph, grounding in the observed occurrences within the tracklet dataset. D. Extracting a set of tracks

[0197] Provided herein are methods for image-based cell tracking comprising iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. In some embodiments, extracting the set of tracks from the updated acyclic graph data structure G’ comprises using a Markov decision process.

[0198] In some embodiments, the graph G’ can be employed to establish a Markov Decision Process (MDP) using the following parameters: • The set of states, denoted as S, is defined as S=V’ • The set of available actions in a given state s, represented as As, is defined as Avi=e’i,kK’,where K is the set of successor vertices of v’iin G’. • The transition probability, p(s,s’), quantifies the likelihood that action a in state s at time t leads to state s’ at time t+1. Formally, this was expressed as p(s,s’) = p(v’i,v’j)= pi,j. • The immediate reward for transitioning from state s to state s’ due to action a, Ra(s,s’), is calculated as Ra(s, s’) where K represents the sets ofpredecessor vertices of v’j. This reward model is applicable to all transitions, except for the transition to the terminal state v’T, where the reward is set to zero. • The policy function Π maps actions from A to states in S.

[0199] To optimize the policy Π, the policy iteration algorithm can be applied. This may involve iteratively calculating a value function over the state space using the formula:

[0200] Here, ϒ is the reward horizon parameter, set at 0.9. The policy may then be optimized as follows:

[0201] This iterative process can continue until convergence is achieved, yielding the optimal policy Π*. To extract a track from G’, it can be initiated at the start vertex vs, following the optimal policy Π* to the terminal vertex vT. Vertices along the track can then be removed 61sf-6002883Docket No.78106-20008.40 from G’, and the process can be repeated until only the start and the terminal nodes remaine in the graph. The method may effectively return a set of tracks, each denoted Tm,such that Tm=[ ^^^^^^^^ ^^^^+1^^^^, ... ^^^^^^^^] for a track of length l.

[0202] Furthermore, an alternative method can be introduced to enhance the extraction of tracklets by varying the size of the sliding window. By executing the sliding window technique multiple times with different window sizes and aggregating the resultant tracklets from sub-graphs of these varying sizes, an augmented graph G′ can be created through this pyramidal strategy. This approach may enable the consideration of a broader array of sub- graph states, each of a different size, thereby influencing the transition probabilities between cells in the Markov Decision Process (MDP).

[0203] To evaluate the methods for image-based cell tracking, a set of metrics to precisely measure the accuracy of the tracking algorithms described herein can be employed. These metrics may enable an in-depth analysis of how individual features, or their combinations perform in correctly matching cells across frames.

[0204] Evaluation metrics for the cell tracking methods may comprise: Top-1 (Whole Frame), Top-3 (Whole Frame), Top-1 (In Range), Top-3 (In Range), and Mean Rank (Whole Frame / In Range). Top-1 (Whole Frame) measures the accuracy of the algorithm in identifying the best tracking match in the subsequent frame, highlighting the precision of the tracking feature. Top-3 (Whole Frame) expands this assessment to whether the correct match falls within the top three selections in the next frame. Both Top-1 (In Range) and Top-3 (In Range) limit the evaluation to cells within a 100-pixel radius, focusing on the performance of the feature in tracking spatially proximal cells. Scores for these metrics range from 0.0 to 1.0, where a score closer to 1.0 denotes effective tracking capability. Mean Rank (Whole Frame / In Range) calculates the average rank of the correct match, either globally or within the specified proximity, offering an overarching view of the feature’s tracking accuracy. A lower mean rank indicates higher efficiency in identifying correct tracking matches.

[0205] The method for image-based cell tracking can be evaluated using established metrics from cell tracking research, ensuring a thorough assessment of tracking performance in cell screening applications. Such metrics may comprise: Track Reconstruction Accuracy (TRA), Complete Tracks (CT), Track Fraction (TF), Partially Reconstructed at 70% (PR@0.7). Track Reconstruction Accuracy (TRA) may follow Matula et al.’s methodology, using the normalized Acyclic Oriented Graph Matching (AOGM) measure for evaluating the precision of tracking algorithms in reconstructing cell trajectories. Matula et al, Cell Tracking 62sf-6002883Docket No.78106-20008.40 Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs.2015 PLOS ONE 10(12):e0144959.

[0206] Additionally, Complete Tracks (CT) assesses the percentage of cell trajectories fully reconstructed by the tracking algorithm, with any error disqualifying the entire track. Track Fraction (TF) evaluates the average completeness of the reconstructed tracks, providing an aggregate measure of tracking success across the dataset. The Partially Reconstructed at 70% (PR@0.7) metric, considers the portion of cell tracks reconstructed to at least 70% completeness, offering insights into the algorithm’s capability to accurately follow significant segments of cell trajectories. Scores for these metrics fall between 0 and 1, where a score of 1 represented optimal tracking performance.

[0207] The efficacy of both individual and combined feature sets within the cell tracking methodology described herein can be evaluated. The results are presented in Table 7. Table 7: Evaluation of the performance of individual tracking features and combination of tracking features. Each individual features and various feature combination using the A375 at a time resolution of 240min dataset using matching cell pairs from the ground truth. With each feature - when used – indicated in between parenthesis the weight used for this features in the linear combination. The performance of a feature or feature set using the Top-1, Top-3 and Mean Rank metrics both across the whole frame and within a limited distance range can be assessed.

[0208] The analysis reveals that the centroid feature, a commonly acknowledged leading feature in tracking literature, may be the most effective in our dataset, achieving a top-1 match in 77% of cases. The performance may underscore the need for additional features to 63sf-6002883Docket No.78106-20008.40 enhance match accuracy in the graph-based tracking algorithm described herein. In contrast, the size feature, a key component of the Lineage Mapper algorithm, demonstrates limited effectiveness, particularly in whole-frame scenarios (top-1 score of 0.11) and improves slightly within the distance range (top-1 score of 0.43). This suggests rapid morphological changes between frames and a diversity of cell morphologies within a cell’s proximity, challenging the efficacy of the size feature. Deep features, particularly those incorporating time-context, outperform the size feature in all metrics. Notably, when combined with spatial features (as in the ’ImageNet’ Features set), these deep features achieve a top-1 match of 78% in whole-frame scenarios and 80% within the distance range, surpassing both the spatial and Lineage Mapper feature sets. In terms of metrics relevant to the approach of some embodiments (with constraints applied to a graph) like top-3 in range, these combined features maintain similar high scores, around the mid 90%. The custom deep features, including division-simCLR and tracking-simCLR, demonstrat robust performance on their own. The tracking-simCLR feature achieves a top-1 score of 63% in the whole frame and 67% within the distance range, with top-3 scores reaching 85% and 92%, respectively. These results may position it as the most effective non-spatial feature. When integrated with both spatial and deep features, this combination yields the highest top-1 score of 84% across both whole frame and distance range scenarios.

[0209] The assessment provided herein underscores the notable performance the features according to the methods described herein, particularly those developed using the simCLR framework and notably the one using the weakly-supervised approach described herein. The feature sets can be categorized and evaluated based on their characteristics and the data availability in evaluation datasets. Two primary sets of features may emerge as focal points for an in-depth evaluation: Standard Features: These are the hand-crafted features, which are formulated based on methodologies described in the Lineage Mapper study. Deep Features: This set comprises a blend of deep learning-based features and hand-crafted features. It includes simCLR features, contingent upon the availability of adequate model. In scenarios where sufficient data was present to train a model, a combination of deep and hand-crafted features can be utilized, as detailed in the ’Deep and Hand-Crafted Feature Combination’ in Table 5. In cases with more limited training availability, the ’ImageNet Feature set’ from the same table can be employed, replacing the novel trained features with pretrained ImageNet features. 64sf-6002883Docket No.78106-20008.40

[0210] This structured evaluation approach allows for the discernment of the individual and aggregated strengths of various feature sets, providing a comprehensive understanding of their contribution to the overall effectiveness of the cell tracking methodology described herein and deciding inclusion of features for further evaluation.

[0211] The performance of various configurations of the cell tracking algorithm described herein are evaluated, each employing distinct sets of features. This comparative analysis spans multiple methodologies: Lineage Mapper: The method described in Chalfoun et al., Lineage mapper: A versatile cell and particle Tracker.2016. Scientific reports 6(1):36984, may serves as benchmark for unsupervised tracking, providing a baseline against which to compare the developed algorithms described herein. Global Tracking: In this embodiment, the graph-based tracking algorithm devoid of the rolling window and Markov Decision Process (MDP) elements are deployed. Here, tracking is conducted on the entire graph G simultaneously, utilizing the shortest mean path algorithm for path determination. Sliding Window Assignment Graph: The configuration of the method described here using experiments with different window sizes to exploring their impact on tracking efficacy. Pyramidal Sliding Window Algorithm: This embodiment combines multiple sliding window sizes concurrently. The resultant tracklets can then be integrated into the MDP algorithm, aiming to convey the algorithm’s tracking ability by leveraging a multi-scale perspective.

[0212] The methodologies are evaluated in Table 8. For the evaluation of these methodologies, the tracking performance metrics as defined herein may be computed and can be illustrated as results such as those presented in Table 8. Table 8. Comparison of the tracking performance methods introduced in this work on A375 at a time resolution of 240min dataset in combination with different feature sets (Standard and Deep). For the Sliding Window tracking algorithm different window size are showcased. For each model configuration performance is assessed using the TRA, the Complete Tracks (CT), the Track Fraction (TF) and the Partial Reconstruction at 70% (PR@0.7).65sf-6002883Docket No.78106-20008.40

[0213] Table 8 shows the methods provided herein outperforming Lineage Mapper across all configurations on the A375 dataset, with the Pyramidal sliding window approach achieving the highest TRA of 0.95. Yet, metrics like Complete Tracks (CT), Track Fraction (TF), and Partially Reconstructed at 70% (PR@0.7) provide more relevant insights for Drug Discovery and High Content Screening (HCS) applications, emphasizing accuracy in track reconstruction. The novel deep feature set consistently outperformed the standard set, and the pyramidal sliding window method proved best, showing stable performance across various window sizes.

[0214] Incorporating deep features into Lineage Mapper may improve its scores, e.g., TF from 0.88 to 0.92 and PR@0.7 from 0.78 to 0.83. However, the methods described herein, especially with the sliding window approach, surpass Lineage Mapper under every configuration, highlighting the robustness of our tracking strategy even with basic features.

[0215] Upon these results, the Sliding Window Assignment Graph method, particularly with a pyramidal sliding window and incorporating deep features, may stand out as an effective configuration. This approach, distinguished by its specific window size and feature utilization, demonstrates superior performance in accurately tracking cellular movements and interactions.

[0216] A comparative assessment of the tracking methodology against the Lineage Mapper algorithm can also be performed. Table 9 shows results of a comparison between the A375 dataset and the CTC dataset’s tracking using Lineage Mapper and the novel method described herein. Table 9. Benchmarking of the tracking method with Lineage Mapper on two sets of tracking features (Standard Feature) on A375 dataset and the CTC datasets in their native time resolution. For each dataset and model the performance using the TRA can be assessed, the Complete Tracks (CT), the Track Fraction (TF) and the Partial Reconstruction at 70% (PR@0.7).66sf-6002883Docket No.78106-20008.40

[0217] The evaluation indicates that the methods described herein generally exceed the performance of the Lineage Mapper across different datasets. Notably, instances where the performance appears equivalent are often those where both methods achieve near-optimal results.

[0218] The methods disclosed herein consistently outperform Lineage Mapper, particularly when deep features are used, though there were exceptions. For instance, in the Fluo-N2 DH- GOWT1 dataset, deep features didn’t improve outcomes, likely due to the dataset’s short 5- minute time steps and minimal frame-to-frame changes. Similarly, in the PhC-C2DH-U373 dataset, neither the disclosed approach nor deep features significantly outdid Lineage Mapper, possibly due to its phase contrast imaging and high time-resolution, where deep features might be less effective.

[0219] A critical aspect of the evaluation involved addressing the discrepancy in temporal resolution between the A375 dataset and the CTC dataset. To this end, performance assessments over a range of time-step sizes, from 60 to 240 minutes may be conducted, using time-subsampling methods. This aspect of the evaluation may be pivotal in demonstrating the flexibility and efficiency of the methods according to embodiments presented herein under conditions of low temporal resolution. The results indicate that the methods described herein generally perform satisfyingly, especially in scenarios characterized by altered time resolutions.

[0220] The temporal resolution of the CTC datasets (60 to 180 minutes increments) can also be adjusted, as detailed in Table 10 to assess the tracking metrics’ resilience. Table 10 provides a summary of the results for adjusted time intervals of length 60min, 120min, 180min according to some embodiments. As shown in Table 10, the two methods at these time-steps using the TRA can be evaluated, the Complete Tracks (CT), the Track Fraction (TF) and the Partial Reconstruction at 70% (PR@0.7). Table 10: Benchmarking results of the tracking methods of the present disclosure against the Lineage Mapper algorithm on the CTC datasets.67sf-6002883Docket No.78106-20008.40

[0221] The methods described herein maintain superior performance over Lineage Mapper, particularly in challenging conditions like the PhC-C2DH-U373 dataset, showing the disclosed method’s robustness at various temporal resolutions. Notably, performance of the methods disclosed herein remain high (TRA = 0.995, CT = 0.81, TF = 0.97, PR@0.7 = 0.95 at ∆ = 180min) even when Lineage Mapper’s declines. However, the simulated Fluo- N2DH-SIM+ dataset may not show benefits from deep features, similar to the findings in division detection described herein. These results affirm the disclosed method’s effectiveness and adaptability across different datasets and conditions, especially at lower temporal resolutions, showcasing its potential for reliable cell tracking, making it well-suited to address the challenges posed by diverse tracking scenarios in low temporal resolution microscopy.

[0222] For a qualitative analysis, dendrograms visualizations were constructed. The dendrograms presented in FIGs.13A-13C highlight the differences in tracking outcomes between the methods described herein (Sliding Window Graph Assignment) and the Lineage Mapper method. They reveal that Sliding Window Graph Assignment more precisely reconstructs track durations and points of entry and exit, aligning more closely with the actual observed data. 5) Methods of image-based cell analysis

[0223] In some embodiments, the methods described herein can be used together for live image-based cell analysis. The method can be used to annotate one or more cellular compartments and track the compartments over time at a single cell level. As such, the method offers the ability to follow the dynamics of cellular components as described herein. These methods may be used to track the morphological or physiological effects of a perturbation on a cell. For example, the methods can be used to track cellular compartment such as organelles in 68sf-6002883Docket No.78106-20008.40 response to a treatment of the cells with a potentially therapeutic compound. The methods described herein can be used to generate single cell trajectories as described herein.

[0224] In some embodiments, the image-based cell segmentation methods as described herein can be used to annotate cell compartments within an image. In some embodiments, the images annotated with cell compartments can further be annotated with the image-based cell division detection methods as described herein to label mother-daughter cell division events. In some embodiments, the images annotated through the image-based cell segmentation method and the image-based cell division detection methods described herein can be used as input for the image- based cell tracking methods described herein. In some embodiments, the methods can be used to track cell compartment dynamics. In some embodiments, the methods can be used to generate a single cell trajectory.

[0225] As demonstrated herein, image data from a HCS experiment may be used as input for all of the methods described herein. Combining the method may have the benefit of improving analysis of single cell time course data in drug screening studies such as HCS and live-drug screening.

[0226] In some embodiments, the methods described herein can be used for image-based cell analysis. In some embodiments, the image-based cell analysis comprises image-based cell segmentation, image-based cell division detection, and image-based cell tracking. In some embodiments, image data from a single experiment on a population of cells comprising a set of images taken at a plurality of time points can be used as input for the image-based cell analysis. The image data may comprise a plurality of fluorescent labels detected by fluorescence imaging channels. Using the methods described herein, one or more finetuned models corresponding to the fluorescence imaging channels and cell lines used in the experiment can be applied to each image in the image data to generate a segmentation map of the image. The segmented images can be used as input to a trained first model to generate a plurality of embedding vectors for the segmented images that represent cells at a first and second timepoint. Feature vectors, as described herein, can be computed using the segmented images the corresponding embedding vectors. A dual input classifier as described herein can be used to generate candidate classifications for the candidate cells and the classifications can be used to annotate mother- daughter cell divisions across the cells in the image data. Using the annotated images and the associated embedding vectors, acyclic graphs can be generated and updated according to the methods described herein. Tracks can then be extracted representing the cell dynamics of the cells in the image data. 69sf-6002883Docket No.78106-20008.40

[0227] The methods provided herein can be used to analyze cellular behavior in response to a perturbation for drug discovery. In some embodiments, analyzing cellular behavior comprises analyzing the trajectory of a cellular compartment. In some embodiments, the set of tracks are for use in analyzing cellular behavior in response to one or more perturbations. In some embodiments, the one or more perturbations are chosen for their potential to treat a disease. In some embodiments, the one or more perturbations is treatment with one or more therapeutic compound. In some embodiments, the methods described herein are for use in a drug discovery pipeline, a diagnostic pipeline or for forecasting a response to a perturbation in an individual. 6) Systems and non-transitory computer readable storage medium

[0228] FIG.14 depicts a computer, in accordance with some embodiments. Computer 1400 can be a component of a any of the systems described herein. In some embodiments, computer 1400 is configured to execute a method image-based cell segmentation, image-based cell division detection, and / or image based cell tracking, such as any of the methods and / or techniques described herein with reference to FIGS.1, 7, and 11. In some embodiments, computer 1400 may be configured to control, monitor, or otherwise send and / or receive electronic signals to and / or from any systems described herein. In some embodiments, computer 1400 may be a microprocessing device configured to be disposed on a substrate, layer, or chip included in or provided in association with any one or more of the systems, devices, modules, layers, and / or components described herein.

[0229] Computer 1400 can be a host computer connected to a network. Computer 1400 can be a client computer or a server. As shown in FIG.14, computer 1400 can be any suitable type of microprocessor-based device, such as a personal computer; workstation; server; or handheld computing device, such as a phone or tablet. The computer can include, for example, one or more of processor 1410, input device 1420, output device 1430, storage 1440, and communication device 1460.

[0230] Input device 1420 can be any suitable device that provides input, such as a touch screen or monitor, keyboard, mouse, or voice-recognition device. Output device 1430 can be any suitable device that provides output, such as a touch screen, monitor, printer, disk drive, or speaker.

[0231] Storage 1440 can be any suitable device that provides storage, such as an electrical, magnetic, or optical memory, including a RAM, cache, hard drive, CD-ROM drive, tape drive, or removable storage disk. Communication device 1460 can include any suitable device capable of 70sf-6002883Docket No.78106-20008.40 transmitting and receiving signals over a network, such as a network interface chip or card. The components of the computer can be connected in any suitable manner, such as via a physical bus or wirelessly. Storage 1440 can be a non-transitory computer-readable storage medium comprising one or more programs, which, when executed by one or more processors, such as processor 1410, cause the one or more processors to execute methods and / or techniques described herein, such as, but not limited to: all or part of any methods for image-based cell segmentation, image-based cell division detection, and / or image based cell tracking, such as any of the methods and / or techniques described herein with reference to FIGs.1, 7, and 11 and the accompanying embodiments.

[0232] Software 1450, which can be stored in storage 1440 and executed by processor 1410, can include, for example, the programming that embodies the functionality of the present disclosure (e.g., as embodied in the systems, computers, servers, and / or devices as described above). In some embodiments, software 1450 can be implemented and executed on a combination of servers such as application servers and database servers.

[0233] Software 1450 can also be stored and / or transported within any computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch and execute instructions associated with the software from the instruction execution system, apparatus, or device. In the context of this disclosure, a computer-readable storage medium can be any medium, such as storage 1440, that can contain or store programming for use by or in connection with an instruction execution system, apparatus, or device.

[0234] Software 1450 can also be propagated within any transport medium for use by or in connection with an instruction execution system, apparatus, or device, such as those described above, that can fetch and execute instructions associated with the software from the instruction execution system, apparatus, or device. In the context of this disclosure, a transport medium can be any medium that can communicate, propagate, or transport programming for use by or in connection with an instruction execution system, apparatus, or device. The transport-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, or infrared wired or wireless propagation medium.

[0235] Computer 1400 may be connected to a network, which can be any suitable type of interconnected communication system. The network can implement any suitable communications protocol and can be secured by any suitable security protocol. The network can comprise network links of any suitable arrangement that can implement the transmission and reception of network 71sf-6002883Docket No.78106-20008.40 signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0236] Computer 1400 can implement any operating system suitable for operating on the network. Software 1450 can be written in any suitable programming language, such as C, C++, Java, or Python. In various embodiments, application software embodying the functionality of the present disclosure can be deployed in different configurations, such as in a client / server arrangement or through a Web browser as a Web-based application or Web service, for example. EXEMPLARY EMBODIMENTS Embodiments disclose herein may include: 1. A method for image-based cell segmentation, comprising: receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generating a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receiving a cell-segmentation model configured to predict instance segmentation for images of cells; and generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and applying one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image. 2. The method of embodiment 1, further comprising generating an evaluation metric that compares a predicted segmentation generated by the aggregated finetuned model to a ground truth segmentation. 72sf-6002883Docket No.78106-20008.40 3. The method of embodiment 2, wherein applying the one or more of the finetuned models from the aggregated finetuned model is based on selection of the aggregated finetuned model based on the generated evaluation metric. 4. The method of any one of embodiment 1-3, further comprising: generating, for each of the plurality of finetuned models, an evaluation metric that compares a predicted segmentation generated by the respective finetuned model to a ground truth segmentation; and selecting, based on the evaluation metrics, a best-performing model; wherein applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the selected best-performing model. 5. The method of any one of embodiment 1-4, generating a first finetuned model of the plurality of finetuned models comprises: training the first finetuned model based on a first subset of the training image data for a first combination of cellular compartment, cell line, and a single imaging channel. 6. The method of embodiment 5, wherein: applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the first finetuned model based on channel data from the second image data for the single imaging channel. 7. The method of any one of embodiment 5-6, further comprising aggregating the first finetuned model with another model trained based on an imaging channel different from the single imaging channel. 8. The method of any one of embodiment 1-7, generating a second finetuned model of the plurality of finetuned models comprises: training the second respective finetuned model based on a second subset of the training image data for a second combination of cellular compartment, cell line, and a set of multiple imaging channels. 9. The method of embodiment 8, wherein: 73sf-6002883Docket No.78106-20008.40 applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the second finetuned model based on channel data from the second image data for one or more imaging channels of the set of multiple imaging channels. 10. The method of any one of embodiment 1-9, further comprising fusing the segmentation map of the image with a second segmentation map of the image, wherein the second segmentation map of the image is generated by a second finetuned model. 11. The method of any of embodiment 1-10, wherein the plurality of fluorescent labels comprise one or more non-structural fluorescence labels. 12. The method of embodiment 11, wherein the one or more non-structural fluorescence labels have a primary function of highlighting a variable expression protein. 13. The method of embodiment 12, wherein the primary function is labelling a protein in a signaling pathway or that changes expression in response to a stimulus. 14. The method of embodiment 1-13, wherein the at least a subset of the plurality of fluorescence imaging channels comprises between 1 and 35 of fluorescence imaging channels. 15. The method of embodiment 1-14, wherein the cell-segmentation model is a model trained to predict instance segmentation for images of cells using image data comprising images of cells, wherein the images comprise a plurality of structural fluorescent labels detected by a plurality of fluorescence imaging channels. 16. The method of embodiment 15, wherein the plurality of structural fluorescent labels comprise one or more structural fluorescent labels with a primary function of labeling a cellular compartment. 17. The method of embodiment 1-16, wherein the cell-segmentation model is selected from a group consisting of CellPose, CellSam, and nucleAlzer. 74sf-6002883Docket No.78106-20008.40 18. A system for image-based cell segmentation, the system comprising one or more processors configured to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image. 19. A non-transitory computer-readable storage medium storing instructions for image- based cell segmentation, the instructions configured to be executed by one or more processors of a system to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image 75sf-6002883Docket No.78106-20008.40 data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image. 20. A method for image-based cell division detection, comprising: receiving image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; applying a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process. 21. The method of embodiment 20, wherein the set of images comprises a third image depicting the population of cells at a third time point. 22. The method of embodiment 20 or embodiment 21, wherein the set of candidate cells are selected by identifying a cell triplet representing a possible first mother cell in the first image that divided into a set of two possible daughter cells in the second image. 23. The method of embodiment 22, wherein identifying a cell triplet comprises: calculating, based on the second image, a second distance between two cells in the population of cells at the second timepoint; 76sf-6002883Docket No.78106-20008.40 identifying based on the first image and the second image, a corresponding set of one or more cells in the population of cells at the first time point; calculating, based on the first image, a first distance representing a distance based on the set of one or more cells in the population of cells in the first image; identifying the cell triplet as the two cells and the corresponding two cells if the second distance is less than or equal to the first distance. 24. The method of embodiment 23, further comprising determining based on the second image that the second image is less than a predefined maximum distance. 25. The method of any one of embodiment 20-24, wherein the feature vector comprises a plurality of relational characteristics between cells in the set of candidate cells. 26. The method of any one of embodiment 25, wherein the plurality of relational characteristics comprise one or more Spatial Euclidean distance between cells in the set of candidate cells, one or more size differences between cells in the set of candidate cells, one or more aspect ratio differences between cells in the set of candidate cells, one or more pairwise cosine distances from the embedding vectors associated with the candidate cells. 27. The method of any one of embodiment 20-26, wherein the feature vector comprises at least 41 dimensions. 28. The method of any one of embodiment 20-27, wherein the trained first model comprises contrastive learning model. 29. The method of any one of embodiment 20-28, wherein the trained first model comprises a simCLR model, a SimSiam model, or a DINO model. 30. The method of any one of embodiment 20-29, wherein the dual input classifier model comprises a recurrent encoder, wherein the recurrent encoder condenses the dimensionality of the embedding vectors associated with the candidate cells to produce a concatenated embedding vector. 77sf-6002883Docket No.78106-20008.40 31. The method of embodiment 30, wherein the concatenated embedding vector encodes a temporal relationships between respective cells in the candidate cells. 32. The method of any one of embodiment 20-31, wherein the dual input classifier model further comprises a binary classifier model, wherein the binary classifier model integrates the concatenated embedding vector and the features vector to generate the classification output. 33. A system for image-based cell division detection, the system comprising one or more processors configured to cause the system to: receive image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; apply a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process. 34. A non-transitory computer-readable storage medium storing instructions for image- based cell division detection, the instructions configured to be executed by one or more processors of a system to cause the system to: receive image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; apply a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and 78sf-6002883Docket No.78106-20008.40 apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process. 35. A method for image-based cell tracking, comprising: receiving image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; applying a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extracting a plurality of subgraphs from the acyclic graph data structure G; extracting, from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 36. The method of embodiment 35, wherein receiving the image data comprises receiving image data based on an image based cell division detection method. 37. The method of embodiment 36, wherein the image based cell division detection method is the method of any of embodiments 20-32. 38. The method of any one of embodiment 35-37, wherein generating the acyclic graph data structure G comprises applying one or more constraints based on the relationship between the embedding vectors associated with the plurality of annotated images. 39. The method of any one of embodiment 35-38, wherein generating the acyclic graph data structure G comprises applying one or more constraints based on division relationships of mother-daughter cell divisions indicated by the annotated images. 79sf-6002883Docket No.78106-20008.40 40. The method of any one of embodiment 35-39, wherein generating the acyclic graph data structure G comprises define edge weights for the acyclic graph data structure G based on a computed composite distance between mother-daughter cells. 41. The method of any one of embodiment 35-40, wherein extracting the plurality of subgraphs from the acyclic graph data structure G comprises using a temporal rolling window. 42. The method of any one of embodiment 35-41, wherein extracting, from the one or more of the plurality of subgraphs, the plurality of tracklets comprises extracting the plurality of tracklets based on the plurality of tracklets having a minimal average cost along a path. 43. The method of any one of embodiment 35-42, wherein generating the updated acyclic graph data structure G’ comprises constraining the updated acyclic graph data structure G’ based on occurrence of an edge in the plurality of tracklets. 44. The method of any one of embodiment 35-43, wherein generating the updated acyclic graph data structure G’ comprises weighting one or more edges of the updated acyclic graph data structure G’ based on a frequency of occurrences of a link in the tracklets. 45. The method of any one of embodiment 35-44, wherein the plurality of tracklets represent possible temporal linkages between cell in the population of cells at the plurality of respective time points. 46. The method of any one of embodiment 35-45, wherein extracting the set of tracks from the updated acyclic graph data structure G’ comprises using a Markov decision process. 47. The method of any one of embodiment 35-46, wherein the plurality of respective time points comprises a plurality of time points at greater than about 4 hour intervals. 48. A system for image-based cell tracking, the system comprising one or more processors configured to cause the system to: receive image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of 80sf-6002883Docket No.78106-20008.40 images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; apply a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 49. A non-transitory computer-readable storage medium storing instructions for image- based cell tracking, the instructions configured to be executed by one or more processors of a system to cause the system to: receive image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; apply a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 50. A method for image-based cell analysis, comprising: receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; 81sf-6002883Docket No.78106-20008.40 generating a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receiving a cell-segmentation model configured to predict instance segmentation for images of cells; and generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; applying one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; applying a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generating a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extracting a plurality of subgraphs from the acyclic graph data structure G; 82sf-6002883Docket No.78106-20008.40 extracting, from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 51. The method of embodiment 50, wherein the first image data comprise images of live cells. 52. The method of embodiment 50 or embodiment 51, wherein the second image data comprise images live cells. 53. The method of any one of embodiment 50-52, wherein the set of tracks are for use in analyzing cellular behavior in response to one or more perturbations. 54. The method of embodiment 53, wherein the one or more perturbations are chosen for their potential to treat a disease. 55. The method of embodiment 53 or embodiment 54, wherein cellular behavior in response to the one or more perturbations is used to choose a perturbation of the one or more perturbations that can be used to treat a disease. 56. The method of any of embodiment 50-55, wherein the method is for use in a drug discovery pipeline or use in a diagnostic pipeline. 57. The method of any one of embodiment 50-55, wherein the method is for use in forecasting a response to a perturbation in an assay system, a preclinical model, or an individual. 58. The method of embodiment 57, wherein the perturbation is treatment with a therapeutic compound. 59. A system for image-based cell analysis, the system comprising one or more processors configured to cause the system to: 83sf-6002883Docket No.78106-20008.40 receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; apply a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generate a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; 84sf-6002883Docket No.78106-20008.40 generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 60. A non-transitory computer-readable storage medium storing instructions for image- based cell analysis, the instructions configured to be executed by one or more processors of a system to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; apply a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; 85sf-6002883Docket No.78106-20008.40 for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generate a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 86sf-6002883

Claims

Docket No.78106-20008.40 CLAIMS What is claimed is:

1. A method for image-based cell segmentation, comprising: receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generating a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receiving a cell-segmentation model configured to predict instance segmentation for images of cells; and generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and applying one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image.

2. The method of claim 1, further comprising generating an evaluation metric that compares a predicted segmentation generated by the aggregated finetuned model to a ground truth segmentation.

3. The method of claim 2, wherein applying the one or more of the finetuned models from the aggregated finetuned model is based on selection of the aggregated finetuned model based on the generated evaluation metric.

4. The method of any one of claims 1-3, further comprising: 87sf-6002883Docket No.78106-20008.40 generating, for each of the plurality of finetuned models, an evaluation metric that compares a predicted segmentation generated by the respective finetuned model to a ground truth segmentation; and selecting, based on the evaluation metrics, a best-performing model; wherein applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the selected best-performing model.

5. The method of any one of claims 1-4, generating a first finetuned model of the plurality of finetuned models comprises: training the first finetuned model based on a first subset of the training image data for a first combination of cellular compartment, cell line, and a single imaging channel.

6. The method of claim 5, wherein: applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the first finetuned model based on channel data from the second image data for the single imaging channel.

7. The method of any one of claims 5-6, further comprising aggregating the first finetuned model with another model trained based on an imaging channel different from the single imaging channel.

8. The method of any one of claims 1-7, generating a second finetuned model of the plurality of finetuned models comprises: training the second respective finetuned model based on a second subset of the training image data for a second combination of cellular compartment, cell line, and a set of multiple imaging channels.

9. The method of claim 8, wherein: applying the one or more of the finetuned models from the aggregated finetuned model comprises applying the second finetuned model based on channel data from the second image data for one or more imaging channels of the set of multiple imaging channels. 88sf-6002883Docket No.78106-20008.40 10. The method of any one of claims 1-9, further comprising fusing the segmentation map of the image with a second segmentation map of the image, wherein the second segmentation map of the image is generated by a second finetuned model.

11. The method of any of claims 1-10, wherein the plurality of fluorescent labels comprise one or more non-structural fluorescence labels.

12. The method of claim 11, wherein the one or more non-structural fluorescence labels have a primary function of highlighting a variable expression protein.

13. The method of claim 12, wherein the primary function is labelling a protein in a signaling pathway or that changes expression in response to a stimulus.

14. The method of claims 1-13, wherein the at least a subset of the plurality of fluorescence imaging channels comprises between 1 and 35 of fluorescence imaging channels.

15. The method of claims 1-14, wherein the cell-segmentation model is a model trained to predict instance segmentation for images of cells using image data comprising images of cells, wherein the images comprise a plurality of structural fluorescent labels detected by a plurality of fluorescence imaging channels.

16. The method of claim 15, wherein the plurality of structural fluorescent labels comprise one or more structural fluorescent labels with a primary function of labeling a cellular compartment.

17. The method of claims 1-16, wherein the cell-segmentation model is selected from a group consisting of CellPose, CellSam, and nucleAlzer.

18. A system for image-based cell segmentation, the system comprising one or more processors configured to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; 89sf-6002883Docket No.78106-20008.40 generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image.

19. A non-transitory computer-readable storage medium storing instructions for image- based cell segmentation, the instructions configured to be executed by one or more processors of a system to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; 90sf-6002883Docket No.78106-20008.40 receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; and apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image.

20. A method for image-based cell division detection, comprising: receiving image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; applying a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process.

21. The method of claim 20, wherein the set of images comprises a third image depicting the population of cells at a third time point.

22. The method of claim 20 or claim 21, wherein the set of candidate cells are selected by identifying a cell triplet representing a possible first mother cell in the first image that divided into a set of two possible daughter cells in the second image.

23. The method of claim 22, wherein identifying a cell triplet comprises: calculating, based on the second image, a second distance between two cells in the population of cells at the second timepoint; identifying based on the first image and the second image, a corresponding set of one or more cells in the population of cells at the first time point; calculating, based on the first image, a first distance representing a distance based on the set of one or more cells in the population of cells in the first image; 91sf-6002883Docket No.78106-20008.40 identifying the cell triplet as the two cells and the corresponding two cells if the second distance is less than or equal to the first distance.

24. The method of claim 23, further comprising determining based on the second image that the second image is less than a predefined maximum distance.

25. The method of any one of claims 20-24, wherein the feature vector comprises a plurality of relational characteristics between cells in the set of candidate cells.

26. The method of any one of claims 25, wherein the plurality of relational characteristics comprise one or more Spatial Euclidean distance between cells in the set of candidate cells, one or more size differences between cells in the set of candidate cells, one or more aspect ratio differences between cells in the set of candidate cells, one or more pairwise cosine distances from the embedding vectors associated with the candidate cells.

27. The method of any one of claims 20-26, wherein the feature vector comprises at least 41 dimensions.

28. The method of any one of claims 20-27, wherein the trained first model comprises contrastive learning model.

29. The method of any one of claims 20-28, wherein the trained first model comprises a simCLR model, a SimSiam model, or a DINO model.

30. The method of any one of claims 20-29, wherein the dual input classifier model comprises a recurrent encoder, wherein the recurrent encoder condenses the dimensionality of the embedding vectors associated with the candidate cells to produce a concatenated embedding vector.

31. The method of claim 30, wherein the concatenated embedding vector encodes a temporal relationships between respective cells in the candidate cells. 92sf-6002883Docket No.78106-20008.40 32. The method of any one of claims 20-31, wherein the dual input classifier model further comprises a binary classifier model, wherein the binary classifier model integrates the concatenated embedding vector and the features vector to generate the classification output.

33. A system for image-based cell division detection, the system comprising one or more processors configured to cause the system to: receive image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; apply a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process.

34. A non-transitory computer-readable storage medium storing instructions for image- based cell division detection, the instructions configured to be executed by one or more processors of a system to cause the system to: receive image data comprising a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; apply a trained first model to the set of images to generate a plurality of embedding vectors for the plurality of cells; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; and apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process. 93sf-6002883Docket No.78106-20008.40 35. A method for image-based cell tracking, comprising: receiving image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; applying a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extracting a plurality of subgraphs from the acyclic graph data structure G; extracting, from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

36. The method of claim 35, wherein receiving the image data comprises receiving image data based on an image based cell division detection method.

37. The method of claim 36, wherein the image based cell division detection method is the method claims 20-32.

38. The method of any one of claims 35-37, wherein generating the acyclic graph data structure G comprises applying one or more constraints based on the relationship between the embedding vectors associated with the plurality of annotated images.

39. The method of any one of claims 35-38, wherein generating the acyclic graph data structure G comprises applying one or more constraints based on division relationships of mother-daughter cell divisions indicated by the annotated images.

40. The method of any one of claims 35-39, wherein generating the acyclic graph data structure G comprises define edge weights for the acyclic graph data structure G based on a computed composite distance between mother-daughter cells. 94sf-6002883Docket No.78106-20008.40 41. The method of any one of claims 35-40, wherein extracting the plurality of subgraphs from the acyclic graph data structure G comprises using a temporal rolling window.

42. The method of any one of claims 35-41, wherein extracting, from the one or more of the plurality of subgraphs, the plurality of tracklets comprises extracting the plurality of tracklets based on the plurality of tracklets having a minimal average cost along a path.

43. The method of any one of claims 35-42, wherein generating the updated acyclic graph data structure G’ comprises constraining the updated acyclic graph data structure G’ based on occurrence of an edge in the plurality of tracklets.

44. The method of any one of claims 35-43, wherein generating the updated acyclic graph data structure G’ comprises weighting one or more edges of the updated acyclic graph data structure G’ based on a frequency of occurrences of a link in the tracklets.

45. The method of any one of claims 35-44, wherein the plurality of tracklets represent possible temporal linkages between cell in the population of cells at the plurality of respective time points.

46. The method of any one of claims 35-45, wherein extracting the set of tracks from the updated acyclic graph data structure G’ comprises using a Markov decision process.

47. The method of any one of claims 35-46, wherein the plurality of respective time points comprises a plurality of time points at greater than about 4 hour intervals.

48. A system for image-based cell tracking, the system comprising one or more processors configured to cause the system to: receive image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; apply a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; 95sf-6002883Docket No.78106-20008.40 generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

49. A non-transitory computer-readable storage medium storing instructions for image- based cell tracking, the instructions configured to be executed by one or more processors of a system to cause the system to: receive image data comprising a set of images comprising a plurality of images depicting a population of cells at a plurality of respective time points, wherein the plurality of images are annotated to designate mother-daughter cell divisions for cells across the plurality of time points; apply a trained first model to the set of images to generate a plurality of embedding vectors associated with the plurality of images; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

50. A method for image-based cell analysis, comprising: receiving first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generating a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receiving a cell-segmentation model configured to predict instance segmentation for images of cells; and 96sf-6002883Docket No.78106-20008.40 generating, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregating one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receiving second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; applying one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; applying a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, computing a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; applying a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generating a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generating, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extracting a plurality of subgraphs from the acyclic graph data structure G; extracting, from one or more of the plurality of subgraphs, a plurality of tracklets; generating, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and 97sf-6002883Docket No.78106-20008.40 iteratively extracting a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

51. The method of claim 50, wherein the first image data comprise images of live cells.

52. The method of claim 50 or claim 51, wherein the second image data comprise images live cells.

53. The method of any one of claims 50-52, wherein the set of tracks are for use in analyzing cellular behavior in response to one or more perturbations.

54. The method of claim 53, wherein the one or more perturbations are chosen for their potential to treat a disease.

55. The method of claim 53 or claim 54, wherein cellular behavior in response to the one or more perturbations is used to choose a perturbation of the one or more perturbations that can be used to treat a disease.

56. The method of any of claims 50-55, wherein the method is for use in a drug discovery pipeline or use in a diagnostic pipeline.

57. The method of any one of claims 50-55, wherein the method is for use in forecasting a response to a perturbation in an assay system, a preclinical model, or an individual.

58. The method of claim 57, wherein the perturbation is treatment with a therapeutic compound.

59. A system for image-based cell analysis, the system comprising one or more processors configured to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; 98sf-6002883Docket No.78106-20008.40 receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; apply a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generate a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; 99sf-6002883Docket No.78106-20008.40 generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images.

60. A non-transitory computer-readable storage medium storing instructions for image- based cell analysis, the instructions configured to be executed by one or more processors of a system to cause the system to: receive first image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by a plurality of fluorescence imaging channels; generate a training image data set by annotating the received image data to indicate cell compartments and cell boundaries represented in the images; receive a cell-segmentation model configured to predict instance segmentation for images of cells; and generate, based on the cell-segmentation model and based on the training image data set, a plurality of finetuned models, wherein generating a respective finetuned model comprises training the respective finetuned model based on a subset of the training image data for a respective combination of cellular compartment, cell line, and a set of one or more of the plurality of imaging channels; aggregate one or more of the finetuned models from the plurality of finetuned models. to generate an aggregated finetuned model; receive second image data comprising images of cells, wherein the images comprise a plurality of fluorescent labels detected by at least a subset of the plurality of fluorescence imaging channels; apply one or more of the finetuned models from the aggregated finetuned model to an image of the second image data to generate a segmentation map of the image; apply a trained first model to a set of images from the second image data to generate a plurality of embedding vectors for the plurality of cells, wherein the second image data comprises a set of images comprising a first image depicting a population of cells at a first time point, and a second image depicting the population of cells at a second time point; for a set of candidate cells from the plurality of cells, wherein one or more of the cells is represented at the first time point and one or more of the cells is represented at the second time point, compute a features vector based on the set of images associated with the candidate cells and / or the embedding vectors associated with the candidate cells; 100sf-6002883Docket No.78106-20008.40 apply a dual input classifier model to the features vector and the embedding vectors associated with the candidate cells to generate a classification output indicating whether the set of candidate cells represents a cell division process; using the classification output, generate a plurality of annotated images from the set of images from the second image data annotated to designate mother-daughter cell divisions for cells across images in the set of images, wherein the set of images comprise a plurality of images depicting the population of cells at a plurality of time points; generate, based on the plurality of annotated images and the embedding vectors associated with the plurality of annotated images an acyclic graph data structure G; extract a plurality of subgraphs from the acyclic graph data structure G; extract, from one or more of the plurality of subgraphs, a plurality of tracklets; generate, based on the acyclic graph data structure G and based on the plurality of tracklets, an updated acyclic graph data structure G’; and iteratively extract a set of tracks from the updated acyclic graph data structure G’, wherein tracks in the set of tracks are indicative of respective cells in the set of images. 101sf-6002883

Images